Skip to content

Commit

Permalink
Chris1/514 236 doc updates and testing of initial moose js flow (#280)
Browse files Browse the repository at this point in the history
* update github link from nextra to moose

* install and getting started updates

* messaging and structure updates

* minor changes to nic comments
  • Loading branch information
03cranec authored Jan 30, 2024
1 parent 519f101 commit 52050ec
Show file tree
Hide file tree
Showing 8 changed files with 96 additions and 75 deletions.
2 changes: 1 addition & 1 deletion apps/framework-docs/src/pages/_meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,6 @@
"title": "Getting Started"
},
"building": {
"title": "Building a Data Service"
"title": "Building Your App"
}
}
32 changes: 2 additions & 30 deletions apps/framework-docs/src/pages/building.mdx
Original file line number Diff line number Diff line change
@@ -1,35 +1,7 @@
import { Cards, Card } from 'nextra/components'

# Building a Data Service
MooseJS helps you build data intensive applications and service.

## What is a data service?
With the MooseJS's current functionality a data service can be thought of as a microservice that's used to capture, store and serve data. It interfaces with best in-class data infrastructure and allows you to build a data warehouse wihtout any data engineering skills. A service is composed of:

- A series of data models which define the structure of the data you want to caputre
- An ingestion API to capture data from other applications or services
- A storage layer that queues up data on ingestion and stores it for later processing

With the Moose toolchain, you get to use the same best practices you use for your application layer in your analytics and data warehousing layer.

**We'd recommend at least the following:**
- Using version control. We like managing our data services along side our main user applications in a mono-repo
- Using a CI/CD pipeline to deploy your service to production ensuring that tests are run before deployment

## What can data services be used for?
Data services can be used to capture data from your user applications and store it in a data warehouse. That means you can replace dedicated data capture tooling such as Segment or Mixpanel with your own data service and generic BI solution. This greatly reduces the cost of running your data infrastructure and gives you more control over your data.

Data services can be used for a variety of other use cases, including:

- Telemetry applications: Capture and analyze data from user interactions to gain insights and improve user experiences.
- Transaction statistics: Track and analyze transaction data to monitor performance, identify trends, and optimize processes.
- Dashboards: Build interactive dashboards to visualize and explore data in real-time.
- Ad hoc analytics: Perform on-demand analysis and generate custom reports for specific business needs.
- Machine learning: Use data services as a data source for training machine learning models and making predictions.
- Personalization: Leverage data services to deliver personalized experiences and recommendations to users.

## Next steps
Let's get started by building a data service.
# Building Your App
Let's get started building your MooseJS application.

<Cards>
<Card title="Model Data" href="/building/model-data" />
Expand Down
46 changes: 39 additions & 7 deletions apps/framework-docs/src/pages/getting-started.mdx
Original file line number Diff line number Diff line change
@@ -1,9 +1,41 @@
# Getting Started
MooseJS is still in active development. We're working on stabilizing our APIs and adding more features. If you'd like to take it for a spin, we'd love to hear your feedback.
# Introduction
Welcome to the MooseJS documentation! MooseJS is still in early development, and we're talking to users and potential users all the time to make sure we're creating something valuable. If you're at all interested in what we're building here, then we'd love to talk to you! Please join our [Slack community](https://join.slack.com/t/igloocommunity/shared_invite/zt-25gsnx2x2-9ttVTt4L9LYFrRcM6jimcg) and reach out. We can't wait to hear from you!

## Installation
MooseJS is available on [npm](https://www.npmjs.com/package/moosejs). You can install it with the following command:
---

```txt filename="Terminal" copy
npx create-moose-app my-app
```
## What is MooseJS?
MooseJS is a JavaScript/Typescript framework for building data-intensive applications and services. Just like you use frontend application frameworks like NextJS to build your frontend applications, you can use MooseJS to build your data/analytics stack. MooseJS designed to be intuitive and scalable for software engineers who do not have a background in data engineering. You can combine MooseJS and your favorite frontend framework to build feature-rich, interactive data applications for your users and stakeholders.

MooseJS is designed to be a batteries-included framework that helps you build your application without having to worry about the underlying data infrastructure. Under the hood, it's built with a Rust based core that helps you package, build and deploy your application to reliably take your apps from your local machine to production.

## What types of applications can be built with MooseJS?
MooseJS is great for any application that is based around a modern data/analytics stack. Any application where collecting data and extracting insights from that data is at the heart of the application. Common examples include:

1. A data lake or data warehouse
2. An LLM-powered application
3. An AI/ML model platform
4. A customer data analytics platform
5. A monitoring / observability application

## What's the key functionality?

### Intuitive primatives
The MooseJS framework offers three simple code-based primitives through which you can architect your entire application:
- Data models, which define the structure of your data
- Flows (coming soon), which move and transform your data
- Insights (coming soon), which surface data to your users and downstream applications

### Automated infrastructure
Based on these primatives, MooseJS will automatically build and manage your underlying data infrastructure. The default infrastructure of a MooseJS application includes:
- An ingestion API to capture data from other applications or services
- A storage layer that queues up data on ingestion and stores it for later processing - based on open source (RedPanda) [https://redpanda.com/] and (Clickhouse) [https://clickhouse.com/]
- An orchestration layer for managing data movement and transformations

### Data engineering best practices built in
MooseJS is a typed framework, so your entire infrastructure automatically has type checking built in on build + run. MooseJS also offers intuitive migration and change management workflows, for when your data structure inevitably grows and evolves.

### Developer-first
MooseJS is designed by software developers, for software developers. It's built from the ground up to enable software developement best practices, that are often hard to implement in the data engineering world.
- Run your entire application locally in Dev mode, allowing you to develop and preview changes you make in real time
- Utilize git-based version control and CI/CD friendly workflows
- Built-in testing framework to ensure tests are run before deployment (coming soon)
2 changes: 1 addition & 1 deletion apps/framework-docs/src/pages/getting-started/_meta.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"installation": "Installation",
"project-structure": "Project Structure",
"using-moose-cli": "Using the CLI"
"moose-cli": "Moose CLI"
}
19 changes: 15 additions & 4 deletions apps/framework-docs/src/pages/getting-started/installation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,28 @@ You will need the following dependencies installed on your machine before you ca
---
## Quick Start

The easiest way to install MooseJS is to use the NPX command. This will download the latest version of MooseJS and run the initialization command for your project.
#### Start a new MooseJS project

The easiest way to get started with a new MooseJS application is to use `create-moose-app`. This will automatically create a project folder for you, and initialize a skeleton MooseJS app with the entire project structure you need to get started.
```txt filename="Terminal" copy
npx create-moose-app my-app
```

---
## Manual Installation
#### Run your new MooseJS app in Dev Mode

If you'd like more flexibility, you can also also install MooseJS globally using NPM. This will allow you to run the MooseJS CLI from anywhere on your machine and initialize new projects.
MooseJS comes with a handy CLI for managing your MooseJS application. You can run the CLI via NPX, or see below for how to install the CLI. Use the CLI now to start your new application in Development mode -- navigate to your project folder and run:
```txt filename="Terminal" copy
npx @514labs/moose-cli dev
```

Now you can visit [http://localhost:3000](http://localhost:3000) to view your new MooseJS application. As you develop your MooseJS application, changes you make to your code base will be automatically reflected here in real time. Now try [creating your first data model](/building/model-data)!

---
## Install the Moose CLI

Instead of running the MooseJS CLI via NPX, you can also also install globally using NPM. This will get you a cleaner command interface, and better performance on the CLI.
```txt filename="Terminal" copy
npm install -g @514labs/moose-cli
```

Now instead of `npx @514labs/moose-cli [command]`, you can just use `moose [command]`.
37 changes: 37 additions & 0 deletions apps/framework-docs/src/pages/getting-started/moose-cli.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Moose CLI

MooseJS comes with a handy CLI for managing your MooseJS application. Navigate to your project's directory to run commands. There are two ways to run the CLI:

Without installing Moose, you can run CLI commands via NPX: `npx @514labs/moose-cli [command]`

If you've installed Moose, you can run CLI commands directly: `moose [command]`

## Moose CLI commands

### Init

Instead of using `create-moose-app`, you can inialize your project manually with the CLI, if you'd like more flexibility and control. Either option will initialize a new MooseJS project for you, with all the necessary directories including temporary ones for data storage.

```txt filename="Terminal" copy
moose init <name> --language <language> --location <location>
```

- `<name>`: Name of your app or service.
- `--language <language>`: Language of your app or service. We currently only support `Typescript`.
- `--location <location>`: Location of your app or service. Default is the current directory (`.`).

### Dev

Starts a development environment running the full stack of your MooseJS application locally. As you develop your MooseJS application locally, changes you make to your code base will be automatically reflected here in real time (when you save changes).

```txt filename="Terminal" copy
moose dev
```

### Clean

Clears all temporary data and stops development infrastructure.

```txt filename="Terminal" copy
moose clean
```
31 changes: 0 additions & 31 deletions apps/framework-docs/src/pages/getting-started/using-moose-cli.mdx

This file was deleted.

2 changes: 1 addition & 1 deletion apps/framework-docs/theme.config.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import Image from 'next/image'
export default {
logo: <svg width="100%" height="30" viewBox="0 0 369 107" fill="none" xmlns="http://www.w3.org/2000/svg"><path id="logo-path" d="M1.96523 85V35H10.8652V46.8L10.2652 44.4C10.5986 43.0667 11.0652 41.8667 11.6652 40.8C12.2652 39.7333 13.2319 38.6667 14.5652 37.6C15.8986 36.4667 17.4986 35.5667 19.3652 34.9C21.2986 34.1667 23.5986 33.8 26.2652 33.8C30.7319 33.8 34.4319 35 37.3652 37.4C40.2986 39.8 42.1986 43.1 43.0652 47.3L41.3652 47.4C41.8986 45.2667 42.5986 43.4333 43.4652 41.9C44.3319 40.3667 45.5319 38.9333 47.0652 37.6C48.4652 36.4 50.0652 35.4667 51.8652 34.8C53.6652 34.1333 55.9652 33.8 58.7652 33.8C62.4986 33.8 65.6319 34.5 68.1652 35.9C70.7652 37.3 72.7319 39.4 74.0652 42.2C75.3986 45 76.0652 48.4667 76.0652 52.6V85H66.8652V55.6C66.8652 52.5333 66.6986 50.1667 66.3652 48.5C66.0319 46.7667 65.1986 45.2667 63.8652 44C62.6652 42.8 61.3319 42.0333 59.8652 41.7C58.3986 41.3667 56.9319 41.2 55.4652 41.2C52.5319 41.2 50.0652 42.1 48.0652 43.9C46.6652 45.2333 45.5319 46.9 44.6652 48.9C43.8652 50.9 43.4652 53.7333 43.4652 57.4V85H34.2652V55.6V55.2C34.2652 52.3333 34.0986 50.0667 33.7652 48.4C33.4319 46.6667 32.5986 45.2 31.2652 44C30.0652 42.8 28.7319 42.0333 27.2652 41.7C25.7986 41.3667 24.3319 41.2 22.8652 41.2C21.3986 41.2 20.0319 41.4333 18.7652 41.9C17.4986 42.3667 16.3986 43.0333 15.4652 43.9C14.0652 45.2333 12.9319 46.9 12.0652 48.9C11.2652 50.9 10.8652 53.7333 10.8652 57.4V85H1.96523ZM110.782 86.2C105.648 86.2 101.182 85.1667 97.3816 83.1C93.6483 80.9667 90.7816 77.9333 88.7816 74C86.7816 70.0667 85.7816 65.4 85.7816 60C85.7816 54.9333 86.7816 50.4333 88.7816 46.5C90.8483 42.5 93.7483 39.4 97.4816 37.2C101.282 34.9333 105.715 33.8 110.782 33.8C115.782 33.8 120.148 34.9333 123.882 37.2C127.682 39.4 130.582 42.5 132.582 46.5C134.648 50.5 135.682 55 135.682 60C135.682 65.2 134.648 69.8 132.582 73.8C130.515 77.7333 127.582 80.8 123.782 83C120.048 85.1333 115.715 86.2 110.782 86.2ZM110.782 79.1C115.715 79.1 119.415 77.7 121.882 74.9C123.282 73.3 124.315 71.2667 124.982 68.8C125.648 66.2667 125.982 63.3333 125.982 60C125.982 53.0667 124.582 48.0667 121.782 45C119.248 42.2667 115.582 40.9 110.782 40.9C108.315 40.9 106.182 41.2333 104.382 41.9C102.582 42.5 101.082 43.4333 99.8816 44.7C98.3483 46.3 97.215 48.4 96.4816 51C95.815 53.5333 95.4816 56.5333 95.4816 60C95.4816 63.4667 95.815 66.5 96.4816 69.1C97.215 71.6333 98.3483 73.7 99.8816 75.3C102.282 77.8333 105.915 79.1 110.782 79.1ZM167.911 86.2C162.777 86.2 158.311 85.1667 154.511 83.1C150.777 80.9667 147.911 77.9333 145.911 74C143.911 70.0667 142.911 65.4 142.911 60C142.911 54.9333 143.911 50.4333 145.911 46.5C147.977 42.5 150.877 39.4 154.611 37.2C158.411 34.9333 162.844 33.8 167.911 33.8C172.911 33.8 177.277 34.9333 181.011 37.2C184.811 39.4 187.711 42.5 189.711 46.5C191.777 50.5 192.811 55 192.811 60C192.811 65.2 191.777 69.8 189.711 73.8C187.644 77.7333 184.711 80.8 180.911 83C177.177 85.1333 172.844 86.2 167.911 86.2ZM167.911 79.1C172.844 79.1 176.544 77.7 179.011 74.9C180.411 73.3 181.444 71.2667 182.111 68.8C182.777 66.2667 183.111 63.3333 183.111 60C183.111 53.0667 181.711 48.0667 178.911 45C176.377 42.2667 172.711 40.9 167.911 40.9C165.444 40.9 163.311 41.2333 161.511 41.9C159.711 42.5 158.211 43.4333 157.011 44.7C155.477 46.3 154.344 48.4 153.611 51C152.944 53.5333 152.611 56.5333 152.611 60C152.611 63.4667 152.944 66.5 153.611 69.1C154.344 71.6333 155.477 73.7 157.011 75.3C159.411 77.8333 163.044 79.1 167.911 79.1ZM221.039 86.2C214.973 86.2 209.906 84.7333 205.839 81.8C201.773 78.8667 199.773 74.4667 199.839 68.6L208.039 67.8C207.973 71.2667 209.106 74.0333 211.439 76.1C213.773 78.1 216.906 79.1 220.839 79.1C224.906 79.1 227.973 78.4333 230.039 77.1C232.106 75.7 233.139 73.6667 233.139 71C233.139 69.6 232.573 68.4333 231.439 67.5C230.373 66.5667 229.006 65.8667 227.339 65.4C225.739 64.8667 223.639 64.3 221.039 63.7C220.373 63.5667 219.706 63.4333 219.039 63.3C218.373 63.1 217.706 62.9333 217.039 62.8C213.573 61.9333 210.739 61.0333 208.539 60.1C206.406 59.1 204.606 57.6667 203.139 55.8C201.673 53.9333 200.939 51.4667 200.939 48.4C200.939 45.1333 201.806 42.4 203.539 40.2C205.339 38 207.639 36.4 210.439 35.4C213.306 34.3333 216.306 33.8 219.439 33.8C225.239 33.8 230.039 35.1 233.839 37.7C237.639 40.3 239.806 44.4667 240.339 50.2L231.939 51.5C231.873 48.0333 230.739 45.4 228.539 43.6C226.339 41.8 223.273 40.9 219.339 40.9C216.673 40.9 214.339 41.5333 212.339 42.8C210.339 44 209.339 45.6667 209.339 47.8C209.339 49.7333 210.273 51.1667 212.139 52.1C214.006 53.0333 216.739 53.9333 220.339 54.8C220.473 54.8667 220.639 54.9 220.839 54.9C221.039 54.9 221.206 54.9333 221.339 55C221.539 55.0667 221.706 55.1333 221.839 55.2C222.039 55.2 222.239 55.2 222.439 55.2C226.706 56.0667 230.106 56.9667 232.639 57.9C235.173 58.7667 237.306 60.2 239.039 62.2C240.773 64.2 241.639 66.9667 241.639 70.5C241.639 76.1 239.673 80.1333 235.739 82.6C231.806 85 226.906 86.2 221.039 86.2ZM272.97 86.2C269.103 86.2 265.67 85.6 262.67 84.4C259.67 83.2 257.103 81.3667 254.97 78.9C252.903 76.5667 251.37 73.8333 250.37 70.7C249.37 67.5 248.87 63.9 248.87 59.9C248.87 56.1667 249.403 52.7333 250.47 49.6C251.537 46.4 253.103 43.6333 255.17 41.3C257.37 38.8333 259.97 36.9667 262.97 35.7C265.97 34.4333 269.27 33.8 272.87 33.8C276.937 33.8 280.703 34.6667 284.17 36.4C287.637 38.0667 290.37 40.5 292.37 43.7C293.903 46.1 295.037 49 295.77 52.4C296.503 55.7333 296.77 59.1667 296.57 62.7H258.67C258.803 68.2333 260.037 72.3333 262.37 75C264.703 77.6667 268.203 79 272.87 79C277.137 79 280.437 78.1 282.77 76.3C285.103 74.4333 286.57 71.6 287.17 67.8L295.87 68.5C294.803 74.3 292.303 78.7 288.37 81.7C284.503 84.7 279.37 86.2 272.97 86.2ZM258.77 55.7L257.47 55.4H287.97L287.37 55.7C287.37 54.3 287.103 52.7333 286.57 51C286.103 49.2 285.47 47.7 284.67 46.5C283.403 44.6333 281.803 43.2667 279.87 42.4C278.003 41.5333 275.703 41.1 272.97 41.1C270.237 41.1 267.837 41.6 265.77 42.6C263.703 43.5333 262.17 44.8333 261.17 46.5C259.703 48.9 258.903 51.7333 258.77 55V55.7ZM307.35 25.6V15H316.35V25.6H307.35ZM304.05 105.5C302.384 105.5 300.75 105.333 299.15 105L299.35 96.6C300.75 97 302.15 97.2 303.55 97.2C305.084 97.2 306.084 96.6333 306.55 95.5C307.084 94.4333 307.35 92.6 307.35 90V35H316.35V91.1C316.35 95.7667 315.284 99.3333 313.15 101.8C311.017 104.267 307.984 105.5 304.05 105.5ZM307.35 42.6V35H311.95V42.6H307.35ZM347.504 86.2C341.438 86.2 336.371 84.7333 332.304 81.8C328.238 78.8667 326.238 74.4667 326.304 68.6L334.504 67.8C334.438 71.2667 335.571 74.0333 337.904 76.1C340.238 78.1 343.371 79.1 347.304 79.1C351.371 79.1 354.438 78.4333 356.504 77.1C358.571 75.7 359.604 73.6667 359.604 71C359.604 69.6 359.038 68.4333 357.904 67.5C356.838 66.5667 355.471 65.8667 353.804 65.4C352.204 64.8667 350.104 64.3 347.504 63.7C346.838 63.5667 346.171 63.4333 345.504 63.3C344.838 63.1 344.171 62.9333 343.504 62.8C340.038 61.9333 337.204 61.0333 335.004 60.1C332.871 59.1 331.071 57.6667 329.604 55.8C328.138 53.9333 327.404 51.4667 327.404 48.4C327.404 45.1333 328.271 42.4 330.004 40.2C331.804 38 334.104 36.4 336.904 35.4C339.771 34.3333 342.771 33.8 345.904 33.8C351.704 33.8 356.504 35.1 360.304 37.7C364.104 40.3 366.271 44.4667 366.804 50.2L358.404 51.5C358.338 48.0333 357.204 45.4 355.004 43.6C352.804 41.8 349.738 40.9 345.804 40.9C343.138 40.9 340.804 41.5333 338.804 42.8C336.804 44 335.804 45.6667 335.804 47.8C335.804 49.7333 336.738 51.1667 338.604 52.1C340.471 53.0333 343.204 53.9333 346.804 54.8C346.938 54.8667 347.104 54.9 347.304 54.9C347.504 54.9 347.671 54.9333 347.804 55C348.004 55.0667 348.171 55.1333 348.304 55.2C348.504 55.2 348.704 55.2 348.904 55.2C353.171 56.0667 356.571 56.9667 359.104 57.9C361.638 58.7667 363.771 60.2 365.504 62.2C367.238 64.2 368.104 66.9667 368.104 70.5C368.104 76.1 366.138 80.1333 362.204 82.6C358.271 85 353.371 86.2 347.504 86.2Z" fill="currentColor"/></svg>,
project: {
link: 'https://github.com/shuding/nextra'
link: 'https://github.com/514-labs/moose'
},
useNextSeoProps() {
return {
Expand Down

0 comments on commit 52050ec

Please sign in to comment.