This file should serve as an entrypoint for learning about and contributing to Rill Developer.
This is a full list of development dependencies:
- Docker
- Node.js 18 (we recommend installing it with nvm)
- Go 1.19 (on macOS, install with
brew install go) - GraalVM and Maven for Java (we recommend installing both through sdkman)
- Protocol Buffers (
protoc) (on macOS, install withbrew install protobuf)
Here's a high-level overview of the technologies we use for different parts of the project:
- Typescript and SvelteKit for all frontend code
- Go for most of the backend functionality
- Java and Apache Calcite for the SQL engine, which gets compiled to a native library using GraalVM
- DuckDB for OLAP on small data
- Apache Druid for OLAP on big data
- Postgres for handling metadata in cloud deployments
- OpenAPI and/or gRPC for most APIs
- Docker for running Postgres and Druid in local development
Rill uses a monorepo and you can expect to find all its code in this repository. This allows us to move faster as we can coordinate changes across multiple components in a single PR. It also gives people a single place to learn about the project and follow its development.
We want the codebase to be easy to understand and contribute to. To achieve that, every directory that contains code of non-trivial complexity should include a README.md file that provides details about the module, such as its purpose, how to run and test it, links to relevant tutorials or docs, etc. Only the root README.md file should be considered user-facing.
The project uses NPM for Node.js (specifically, NPM workspaces), Go modules for Go, and Maven for Java. They function well alongside each other and we have not yet found a need for a cross-language build tool.
Here's a guide to the top-level structure of the repository:
.githuband.travis.ymlcontain CI/CD workflows. We allow both, but the goal is to move fully to Github Actions.cliis an implementation of a new CLI in Go (in progress, not launched yet).docscontains the user-facing documentation that we deploy to docs.rilldata.com.runtimeis our data plane, responsible for querying and orchestrating data infra. It currently supports DuckDB and Druid.server-cloudcontains the backend control plane for a multi-user version of Rill (in progress, not launched yet).sqlcontains our SQL parser and transpiler. It's based on Apache Calcite and is used heavily by theruntime.web-cloudcontains the frontend control plane for a multi-user version of Rill (in progress, not launched yet).web-commoncontains common functionality shared across the local and cloud frontends.web-localcontains the local Rill Developer application, including the data modeller and current CLI.