Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ Pull requests should be done on the `dev` branch. When the release is finalised,
## Setting up a development environment

Whether you're interested in contributing to the repository, creating a fork, or just improving your understanding of Streamsync, these are the suggested steps for setting up a development environment.
- Install streamsync[test] or streamsync[build].
- You can install the package in editable mode using `pip install -e .`, which will make it more convenient if you intend to tweak the backend.
- You can install the package in editable mode using `poetry install --with build`, which will make it more convenient if you intend to tweak the backend.
- Enable the virtual environment with `poetry shell`
- Run streamsync in port 5000. For example, `streamsync edit hello --port 5000`.
- Install dependencies and run `npm run dev` in `/ui`. This runs the frontend for Streamsync in development mode while proxying requests to port 5000.
- A breakdown of the steps required for packaging can be found in `./build.sh`. Notably, it includes compiling the frontend and taking it from `/ui` and into the Python package.
3 changes: 0 additions & 3 deletions MANIFEST.in

This file was deleted.

6 changes: 3 additions & 3 deletions build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,12 @@ cd ..
# PYTEST

cd ./tests
pytest
poetry run pytest
cd ..

# TYPE CHECKS (MYPY)

mypy ./src/streamsync/*.py
poetry run mypy ./src/streamsync/*.py

# APP PROVISIONING

Expand All @@ -28,4 +28,4 @@ cp -r ./apps/hello ./src/streamsync/app_templates
# PYTHON PACKAGE BUILD

rm -rf ./dist/*
python -m build
poetry build
1,175 changes: 1,175 additions & 0 deletions poetry.lock

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions poetry.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[virtualenvs]
in-project = true
88 changes: 41 additions & 47 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,59 +1,53 @@
[build-system]
requires = ["setuptools", "setuptools-scm"]
build-backend = "setuptools.build_meta"
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

[project]
[tool.poetry]
name = "streamsync"
authors = [{ name = "Ramiro Medina", email = "ramiro.a.medina@gmail.com" }]
version = "0.3.0"
description = "Streamsync helps you create performant data apps, via Python code and its built-in visual UI editor."
authors = ["Ramiro Medina <ramiro.a.medina@gmail.com>"]
license = "Apache 2.0"
readme = "README.md"
requires-python = ">=3.9.2"
homepage = "https://www.streamsync.cloud"
repository = "https://www.github.com/streamsync-cloud/streamsync"
documentation = "https://www.streamsync.cloud/getting-started.html"
keywords = ["data apps", "gui", "ui"]
license = { text = "Apache 2.0" }
classifiers = ["Development Status :: 4 - Beta"]
dependencies = [
"pydantic >= 2.6.0, < 3",
"fastapi >= 0.89.1, < 1",
"websockets >= 12, < 13",
"uvicorn >= 0.20.0, < 1",
"watchdog >= 3.0.0, < 4"
classifiers = [
"Development Status :: 4 - Beta"
]
packages = [
{ include = "streamsync", from = "src" }
]

dynamic = ["version"]
[tool.poetry.dependencies]
python = ">=3.9.2, <4.0"
pydantic = ">= 2.6.0, < 3"
fastapi = ">= 0.89.1, < 1"
websockets = ">= 12, < 13"
uvicorn = ">= 0.20.0, < 1"
watchdog = ">= 3.0.0, < 4"
pandas = {version = ">= 2.2.0, < 3", optional = true}
pyarrow = {version = ">= 15.0.0, < 16.0.0",optional = true}
plotly = {version = ">= 5.18.0, < 6", optional = true}

[project.optional-dependencies]
ds = [
"pandas >= 2.2.0, < 3",
"pyarrow >= 15.0.0, < 16.0.0",
"plotly >= 5.18.0, < 6"
]
test = [
"pandas >= 2.2.0, < 3",
"pyarrow >= 15.0.0, < 16.0.0",
"plotly >= 5.18.0, < 6",
"pytest-asyncio >= 0.23.4, < 1",
"pytest >= 7.0.0, < 8",
"altair >= 5.2.0, < 6",
"httpx >= 0.26.0, < 1"
]
build = [
"pandas >= 2.2.0, < 3",
"pyarrow >= 15.0.0, < 16.0.0",
"plotly >= 5.18.0, < 6",
"build >= 1.0.3, < 2",
"mypy >= 1.8.0, < 2",
"pandas-stubs >= 2.0.0, <3",
"pytest >= 8.0.0, < 9",
"pytest-asyncio >= 0.23.4, < 1",
"altair >= 5.2.0, < 6",
"httpx >= 0.26.0, < 1"
]

[project.urls]
Home = "https://www.github.com/streamsync-cloud/streamsync"
[tool.poetry.group.build]
optional = true

[tool.poetry.group.build.dependencies]
pandas = ">= 2.2.0, < 3"
pyarrow = ">= 15.0.0, < 16.0.0"
plotly = ">= 5.18.0, < 6"
mypy = ">= 1.8.0, < 2"
pandas-stubs = ">= 2.0.0, <3"
pytest-asyncio = ">= 0.23.4, < 1"
pytest = ">= 7.0.0, < 8"
altair = ">= 5.2.0, < 6"
httpx = ">=0.26.0, < 1"

[project.scripts]
streamsync = "streamsync.command_line:main"
[tool.poetry.extras]
ds = ["pandas", "pyarrow", "plotly"]

[tool.setuptools.dynamic]
version = { attr = "streamsync.VERSION" }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No way to get version dynamically?

Copy link
Contributor Author

@FabienArcellier FabienArcellier Feb 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the design choices of poetry, there is a strong choice not to read a python module dynamically like in setuptools. I understand this choice but it leads to a frustrating limit regards to version number. I prefer the setuptools strategy but impossible to use it properly with poetry.

A plugin written by the creator of fastapi go over this limit but this plugin is incompatible with src layout and the PR to fix it has been open for 3 months and hasn't been merged.

[tool.poetry.scripts]
streamsync = 'streamsync.command_line:main'
3 changes: 2 additions & 1 deletion src/streamsync/__init__.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
import importlib.metadata
from typing import Union, Optional, Dict, Any
from streamsync.core import Readable, FileWrapper, BytesWrapper, Config
from streamsync.core import initial_state, component_manager, session_manager, session_verifier

VERSION = "0.3.0"
VERSION = importlib.metadata.version("streamsync")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the idea is to switch to defining version in the package and then fetching it from the code? Makes more sense I guess

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the recommended way to manage this situation. Another one is to have a utility as bumpversion to update both location. I would avoid this one.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like this option.


component_manager
session_manager
Expand Down
2 changes: 1 addition & 1 deletion src/streamsync/app_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -539,7 +539,7 @@ def __init__(self, app_path: str, mode: str):
self.is_app_process_server_ready = multiprocessing.Event()
self.is_app_process_server_failed = multiprocessing.Event()
self.app_process_listener: Optional[AppProcessListener] = None
self.observer: Optional[watchdog.observers.Observer] = None
self.observer: Optional[PollingObserver] = None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this intentional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is, otherwise mypy was complaining in build.sh

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah nice, must be related to the dependencies I upgraded, thanks for fixing.

self.app_path: str = app_path
self.response_events: Dict[int, ThreadSafeAsyncEvent] = {}
self.response_packets: Dict[int, AppProcessServerResponsePacket] = {}
Expand Down