refactor: monorepo structure cleanup

[raaymax]

poetry install
npm i
poetry run npm test

[ramedina86]

@mmikita95 @FabienArcellier Please share your thoughts

[mmikita95]

I don't feel entirely confident about src/ not being the top-level directory; in my experience, it was always an indicator for an "entry-point" of the codebase, something like "go here to start working!".

Personally, this kind of structure would probably feel more "traditional":

- src
    └ cli
    └ ui
    └ tests
        └ unit (?)
        └ e2e (?)

I'm not sure about "tests" here though (i.e. how feasible it is).

Also, I think that cli nomenclature feels slightly out of place here, because it is more like "some CLI-features included" rather than a "full-scale CLI", intended for usage as one most of the time. Maybe backend, svc ("service-layer") or engine would be better for the Python-code directory?

[raaymax]

Totally agree that the name CLI is out of place, but I didn't have a better idea. The idea behind the my monorepo approach is that each package is in a separate directory with a package.json file; it doesn't need to be a JS/TS package. It only needs a package.json to define relationships between packages and assign them scripts like lint checks.

Notice that ui have it's own src folder and this would create path such as src/ui/src.
In my opinion unit tests are highly bounded to the code and should be kept as close as possible to code they testing.
integration and e2e can be kept somewhere else because they depend on whole packages but unit tests are bound to internal code of a package.

In turbo.json, I defined caching and job dependencies. Packages have defined dependencies, and this way, for example, to run e2e tests with npm run e2e, the required packages will be built first because the e2e tests need to have the UI built first. If you run many jobs at the same time (like in npm test), Turbo will determine which can be parallelized and which are dependent.

Additionally, Turbo has caching. It will run only things that have changed. For example, if the backend depends on the UI and if only the backend code changed, then it won't run the UI build because the UI code didn't change, but it will show outputs from the previous build.

You can try running the snippet from the description; you will see how it works.

poetry run npm test

Notice that if you run this command multiple times, it will be much faster after the first run.

[raaymax]

Just to be clear, I'm not trying to force anything, nor am I particularly attached to this specific solution. I'm open to discussion 😉

[FabienArcellier]

I agree with the remark of mikita. Is there a way to split this PR in two ? One about repository organisation, one about the implementation of turborepo ?

I have no opinion about the usage of turborepo as make-based build system (trigger only necessary build step based on code change). I let @raaymax decide what is needing on that part.

By the way, I design alfred to handle monorepo and cross platform build for python. I haven't yet implemented the primitive to trigger command on code change. I don't plan to do it in the short future.

[FabienArcellier]

that looks good for me

[ramedina86]

As expected after Fabien's changes, getting conflicts here, please merge dev into your branch.

alfred/ci.py
alfred/npm.py
e2e_tests/tests/components.spec.ts
e2e_tests/tests/lowCode.spec.ts
package-lock.json
package.json
ui/package.json
ui/src/core_components/content/CoreMessage.vue
ui/src/core_components/embed/CoreMapbox.vue
ui/src/core_components/input/CoreTextInput.vue
ui/src/core_components/layout/CoreColumn.vue
ui/tools/generator.js
ui/viteStreamsyncPlugin.ts