Skip to content

Commit

Permalink
Merge pull request #434 from streamsync-cloud/dev
Browse files Browse the repository at this point in the history
chore: Merge for release
  • Loading branch information
ramedina86 authored May 20, 2024
2 parents e0319f1 + 5ef6ecc commit d342b6e
Show file tree
Hide file tree
Showing 185 changed files with 2,410 additions and 1,279 deletions.
10 changes: 5 additions & 5 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,13 @@ docs/docs/.vitepress/temp/
ui/dist/
build/
dist/
src/streamsync.egg-info/
src/writer.egg-info/
*:Zone.Identifier
*.DS_Store
src/streamsync/static
src/streamsync/app_templates/*
!src/streamsync/app_templates/.gitkeep
src/streamsync/ui.py
src/writer/static
src/writer/app_templates/*
!src/writer/app_templates/.gitkeep
src/writer/ui.py
src/ui/src/stories/**
src/ui/components.codegen.json
playground/
Expand Down
23 changes: 12 additions & 11 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
# Contributing to Streamsync
# Contributing to Writer Framework

Thank you for your interest in contributing to Streamsync.
Thank you for your interest in contributing to Writer Framework.

## Ways to contribute

Beyond contributing to the repository, some ways to contribute to this project include:
Beyond contributing to the repository, some ways to contribute to this project include:

- *Reporting bugs*. Bug reports are relatively easy to write, but have a big impact. Please include the steps required to reproduce the bug. Use "Issues" on GitHub. This is an example of a [wonderful bug report](https://github.com/streamsync-cloud/streamsync/issues/24).
- *Creating content*. Think articles or tutorials. It doesn't have to be overwhelmingly positive; constructive criticism is appreciated. A great example is [this review](https://jreyesr.github.io/posts/streamsync-review/). A YouTube tutorial would be fantastic!
- *Browse Issues and Discussions*. Browse these sections on GitHub and see if you can help.
- *Suggesting valuable enhancements*. If you think of a feature that can have a positive impact, suggest it. Please use the "Discussions" on GitHub.
- *Sponsoring the project*. Helps offset hosting and other expenses.
- *Promoting the project*. Star it, share on LinkedIn or other social media.
- _Reporting bugs_. Bug reports are relatively easy to write, but have a big impact. Please include the steps required to reproduce the bug. Use "Issues" on GitHub. This is an example of a [wonderful bug report](https://github.com/streamsync-cloud/streamsync/issues/24).
- _Creating content_. Think articles or tutorials. It doesn't have to be overwhelmingly positive; constructive criticism is appreciated. A great example is [this review](https://jreyesr.github.io/posts/streamsync-review/). A YouTube tutorial would be fantastic!
- _Browse Issues and Discussions_. Browse these sections on GitHub and see if you can help.
- _Suggesting valuable enhancements_. If you think of a feature that can have a positive impact, suggest it. Please use the "Discussions" on GitHub.
- _Sponsoring the project_. Helps offset hosting and other expenses.
- _Promoting the project_. Star it, share on LinkedIn or other social media.

## Contributing to the repository

Expand All @@ -23,8 +23,9 @@ Pull requests should be done on the `dev` branch. When the release is finalised,

## Setting up a development environment

Whether you're interested in contributing to the repository, creating a fork, or just improving your understanding of Streamsync, these are the suggested steps for setting up a development environment.
Whether you're interested in contributing to the repository, creating a fork, or just improving your understanding of Writer Framework, these are the suggested steps for setting up a development environment.

- You can install the package in editable mode using `poetry install`
- Enable the virtual environment with `poetry shell`
- Install all the dev dependencies with `alfred install.dev`
- Run streamsync in port 5000. For example, `streamsync edit apps/hello --port 5000`.
- Run Writer Framework on port 5000. For example, `writer edit apps/hello --port 5000`.
27 changes: 15 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
## What is Streamsync?

[![PyPi](https://img.shields.io/pypi/v/streamsync.svg?label=Version)](https://pypi.org/project/streamsync/)
[![CI](https://github.com/streamsync-cloud/streamsync/actions/workflows/ci.yml/badge.svg)](https://github.com/streamsync-cloud/streamsync/actions/workflows/ci.yml)
[![CI](https://github.com/streamsync-cloud/streamsync/actions/workflows/ci.yml/badge.svg)](https://github.com/streamsync-cloud/streamsync/actions/workflows/ci.yml)
[![Discord](https://img.shields.io/badge/discord-streamsync-sn677E3Pd3?logo=discord&logoColor=white)](https://discord.gg/sn677E3Pd3)
[![License](https://img.shields.io/pypi/l/streamsync)](LICENSE)

Streamsync is an open-source framework for creating data apps. Build user interfaces using a visual editor; write the backend code in Python.
Writer Framework is an open-source framework for creating data apps. Build user interfaces using a visual editor; write the backend code in Python.

![Streamsync Builder screenshot](https://raw.githubusercontent.com/streamsync-cloud/streamsync/master/docs/docs/public/sc1.png "Streamsync Builder screenshot")
![Writer Framework Builder screenshot](https://raw.githubusercontent.com/streamsync-cloud/streamsync/master/docs/docs/public/sc1.png "Writer Framework Builder screenshot")

- [Live demo](https://hello.streamsync.cloud/) of an app. [Source code](https://github.com/streamsync-cloud/streamsync/blob/master/apps/hello/main.py).
- [1 minute introduction video](https://youtu.be/XBAPBy_zf8s) on YouTube
Expand All @@ -20,52 +20,55 @@ It aims to be as simple as Streamlit, but faster, more flexible and with a clean

### Reactive and state-driven

Streamsync is **fully state-driven** and provides **separation of concerns** between user interface and business logic.
Writer Framework is **fully state-driven** and provides **separation of concerns** between user interface and business logic.

```py
import streamsync as ss
import writer as wf

def handle_increment(state):
state["counter"] += 1

ss.init_state({
wf.init_state({
"counter": 0
})
```

The user interface is a template, which is defined visually. The template contains reactive references to state, e.g. `@{counter}`, and references to event handlers, e.g. when _Button_ is clicked, trigger `handle_increment`.

### Flexible

- Elements are highly customisable with no CSS required, allowing for shadows, button icons, background colours, etc.
- HTML elements with custom CSS can be included using the _HTML Element_ component. They can serve as containers for built-in components.

### Fast
- Event handling adds minimal overhead to your Python code (~1-2ms*).

- Event handling adds minimal overhead to your Python code (~1-2ms\*).
- Streaming (WebSockets) is used to synchronise frontend and backend states.
- The script only runs once.
- Non-blocking by default. Events are handled asynchronously in a thread pool running in a dedicated process.

*End-to-end figure, including DOM mutation. Tested locally on a Macbook Air M2. [Measurement methodology](https://medium.com/@ramiromedina/measuring-time-elapsed-between-an-event-and-its-associated-dom-mutation-80431ad576e1).
\*End-to-end figure, including DOM mutation. Tested locally on a Macbook Air M2. [Measurement methodology](https://medium.com/@ramiromedina/measuring-time-elapsed-between-an-event-and-its-associated-dom-mutation-80431ad576e1).

### Developer-friendly

- It's all contained in a standard Python package, just one `pip install` away.
- User interfaces are saved as JSON, so they can be version controlled together with the rest of the application.
- Use your local code editor and get instant refreshes when you save your code. Alternatively, use the provided web-based editor.
- You edit the UI while your app is running. No hitting "Preview" and seeing something completely different to what you expected.

## Installation and Quickstart

Getting started with Streamsync is easy. It works on Linux, Mac and Windows.
Getting started with Writer Framework is easy. It works on Linux, Mac and Windows.

```sh
pip install "streamsync[ds]"
streamsync hello
```

- The first command will install Streamsync using `pip` and include the optional data science dependencies.
- The second command will create a demo application in the subfolder "hello" and start Streamsync Builder, the framework's visual editor, which will be accessible via a local URL.
- The first command will install Writer Framework using `pip` and include the optional data science dependencies.
- The second command will create a demo application in the subfolder "hello" and start Writer Framework Builder, the framework's visual editor, which will be accessible via a local URL.

The following commands can be used to create, launch Streamsync Builder and run an application.
The following commands can be used to create, launch Writer Framework Builder and run an application.

```sh
streamsync create my_app
Expand Down
7 changes: 3 additions & 4 deletions alfred/build.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,10 @@ def build(ignore_ci: bool = False):

@alfred.command("build.app_provisionning", help="update app templates using ./apps", hidden=True)
def build_app_provisionning():
if os.path.isdir('src/streamsync/app_templates'):
shutil.rmtree('src/streamsync/app_templates')
if os.path.isdir('src/writer/app_templates'):
shutil.rmtree('src/writer/app_templates')

shutil.copytree( 'apps/default', 'src/streamsync/app_templates/default')
shutil.copytree( 'apps/hello', 'src/streamsync/app_templates/hello')
shutil.copytree( 'apps', 'src/writer/app_templates')

@alfred.command("build.poetry", help="build python packages with poetry", hidden=True)
def build_poetry():
Expand Down
24 changes: 2 additions & 22 deletions alfred/ci.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,9 @@ def ci(front, back, e2e, docs):
if e2e:
alfred.invoke_command("npm.e2e", browser=e2e)

@alfred.command("ci.mypy", help="typing checking with mypy on ./src/streamsync")
@alfred.command("ci.mypy", help="typing checking with mypy on ./src/writer")
def ci_mypy():
alfred.run("mypy ./src/streamsync --exclude app_templates/*")
alfred.run("mypy ./src/writer --exclude app_templates/*")

@alfred.command("ci.ruff", help="linting with ruff")
def ci_ruff():
Expand All @@ -41,24 +41,4 @@ def ci_test():
os.chdir("tests")
alfred.run("pytest")

@contextlib.contextmanager
def _preserve_files(path: List[str]):
"""
Preserve files in a temporary directory and restore them after the context
:param path: list of files to preserve
"""
tmpdir = tempfile.mkdtemp()
try:
for p in path:
os.makedirs(os.path.dirname(os.path.join(tmpdir, p)), exist_ok=True)
shutil.copy(p, os.path.join(tmpdir, p))

yield
finally:
for p in path:
shutil.copy(os.path.join(tmpdir, p), p)

shutil.rmtree(tmpdir)


4 changes: 2 additions & 2 deletions alfred/publish.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ def publish():
>>> $ alfred publish
"""
import streamsync
VERSION = f"v{streamsync.VERSION}"
import writer
VERSION = f"v{writer.VERSION}"

git = alfred.sh("git", "git should be present")
os.chdir(ROOT_DIR)
Expand Down
4 changes: 2 additions & 2 deletions apps/default/main.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import streamsync as ss
import writer as wf

# This is a placeholder to get you started or refresh your memory.
# Delete it or adapt it as necessary.
Expand Down Expand Up @@ -28,7 +28,7 @@ def increment(state):
# "_my_private_element" won't be serialised or sent to the frontend,
# because it starts with an underscore

initial_state = ss.init_state({
initial_state = wf.init_state({
"my_app": {
"title": "MY APP"
},
Expand Down
2 changes: 1 addition & 1 deletion apps/default/ui.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"metadata": {
"streamsync_version": "0.5.0"
"writer_version": "0.5.0"
},
"components": {
"root": {
Expand Down
4 changes: 2 additions & 2 deletions apps/hello/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@ RUN apt-get update -y && mkdir /app && python3 -m venv /app/venv
ENV PATH="/app/venv/bin:$PATH"
COPY . /app
WORKDIR /app
RUN pip3 install 'streamsync[ds]' && pip3 install -r requirements.txt
RUN pip3 install 'writer' && pip3 install -r requirements.txt

FROM python:3.10-slim AS run-image
RUN apt-get update -y && mkdir /app
COPY --from=compile-image /app /app
ENV PATH="/app/venv/bin:$PATH"
WORKDIR /app

ENTRYPOINT [ "streamsync", "run" ]
ENTRYPOINT [ "writer", "run" ]
EXPOSE 5000
CMD [ ".", "--port", "5000", "--host", "0.0.0.0" ]
10 changes: 5 additions & 5 deletions apps/hello/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@
import numpy as np
import pandas as pd
import plotly.express as px
import streamsync as ss
from streamsync.core import StreamsyncState
import writer as wf
from writer.core import WriterState

# EVENT HANDLERS

def handle_timer_tick(state: StreamsyncState):
def handle_timer_tick(state: WriterState):
df = state["random_df"]
for i in range(5):
df[f'pgcf_{i+1}'] = np.around(np.random.rand(10, 1), decimals=9)
Expand All @@ -30,7 +30,7 @@ def update(state, session):


def handle_story_download(state):
data = ss.pack_file("assets/story.txt", "text/plain")
data = wf.pack_file("assets/story.txt", "text/plain")
file_name = "hacker_pigeons.txt"
state.file_download(data, file_name)

Expand Down Expand Up @@ -129,7 +129,7 @@ def _update_scatter_chart(state):
# STATE INIT


initial_state = ss.init_state({
initial_state = wf.init_state({
"main_df": _get_main_df(),
"highlighted_members": _get_highlighted_members(),
"random_df": _generate_random_df(),
Expand Down
22 changes: 11 additions & 11 deletions apps/hello/ui.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"metadata": {
"streamsync_version": "0.5.0"
"writer_version": "0.5.0"
},
"components": {
"root": {
Expand Down Expand Up @@ -170,10 +170,10 @@
"position": 1,
"parentId": "7e625201-20c2-4b05-951c-d825de28b216",
"handlers": {
"ss-number-change": "update"
"wf-number-change": "update"
},
"binding": {
"eventType": "ss-number-change",
"eventType": "wf-number-change",
"stateRef": "filter.min_weight"
}
},
Expand All @@ -190,10 +190,10 @@
"position": 0,
"parentId": "7e625201-20c2-4b05-951c-d825de28b216",
"handlers": {
"ss-number-change": "update"
"wf-number-change": "update"
},
"binding": {
"eventType": "ss-number-change",
"eventType": "wf-number-change",
"stateRef": "filter.min_length"
}
},
Expand Down Expand Up @@ -363,7 +363,7 @@
"id": "5da5e007-d60a-4313-9d21-885deae7b37d",
"type": "text",
"content": {
"text": "You can put other Streamsync components inside HTML Elements."
"text": "You can put other Writer Framework components inside HTML Elements."
},
"isCodeManaged": false,
"position": 1,
Expand Down Expand Up @@ -413,7 +413,7 @@
"position": 0,
"parentId": "09ddb2da-6fa3-4157-8da3-4d5d44a6a58d",
"handlers": {
"ss-tick": "handle_timer_tick"
"wf-tick": "handle_timer_tick"
}
},
"e296866a-75d2-4677-b55d-3c1456113b89": {
Expand Down Expand Up @@ -475,7 +475,7 @@
"position": 2,
"parentId": "b1ee642e-f2e7-453b-a6ef-3d96eea37140",
"binding": {
"eventType": "ss-number-change",
"eventType": "wf-number-change",
"stateRef": "hue_rotation"
}
},
Expand Down Expand Up @@ -950,7 +950,7 @@
"id": "77cb256b-ef12-4a55-a051-500497f41302",
"type": "text",
"content": {
"text": "It's easy to switch between pages and it can be done from the frontend (via Streamsync Builder) and from the backend (via Python)."
"text": "It's easy to switch between pages and it can be done from the frontend (via Writer Framework Builder) and from the backend (via Python)."
},
"isCodeManaged": false,
"position": 2,
Expand Down Expand Up @@ -1039,8 +1039,8 @@
"position": 1,
"parentId": "e1ax8ctt8lrao0e4",
"handlers": {
"ss-change-page": "handle_paginated_members_page_change",
"ss-change-page-size": "handle_paginated_members_page_size_change"
"wf-change-page": "handle_paginated_members_page_change",
"wf-change-page-size": "handle_paginated_members_page_size_change"
},
"visible": true
},
Expand Down
4 changes: 2 additions & 2 deletions apps/quickstart/main.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import plotly.graph_objects as go
import streamsync as ss
import writer as wf
from sklearn.datasets import make_blobs
from sklearn.linear_model import LogisticRegression

Expand Down Expand Up @@ -98,7 +98,7 @@ def update(state):
state['figure'] = fig


initial_state = ss.init_state({
initial_state = wf.init_state({
"my_app": {
"title": "Logistic regression visualizer"
},
Expand Down
Loading

0 comments on commit d342b6e

Please sign in to comment.