Initial draft of linux instructions and removal of internal instructions

README.md

@@ -1,4 +1,7 @@
-## Setup
+## What is Gooey Server?
+Gooey Server is a no-code/low-code AI orchestration engine. It allows users to discover, customize, and deploy AI "recipes" using the best of private and open source AI, all using a single API with a single auth token. Recipes are workflows that incorporate various models to accomplish a task; they are designed to be highly customizable and shareable. 
+
+## Setup (Mac)
 
 * Install [pyenv](https://github.com/pyenv/pyenv) & install the same python version as in our [Dockerfile](Dockerfile)
 * Install [poetry](https://python-poetry.org/docs/)
@@ -41,13 +44,46 @@
   ```
   (If you run into issues with the number of open files, you can remove the limit with `ulimit -n unlimited`)
 
+## Setup (Linux)
+* Install [pyenv](https://github.com/pyenv/pyenv) & install the same python version as in our [Dockerfile](Dockerfile) (currently Python 3.10)
+  - `curl https://pyenv.run | bash`
+* Install [poetry](https://python-poetry.org/docs/)
+  - This is likely available in your distro's package repos. 
+* Clone the gooey-server repository:
+  - `git clone https://github.com/GooeyAI/gooey-server.git`
+* If you want to use the web application frontend, you must clone that repo as well, in the same directory as gooey-server:
+  - `git clone https://github.com/GooeyAI/gooey-ui`
+* Create and activate a virtualenv using `poetry shell`
+* Install dependencies using `poetry install --with dev`
+  - Note: you may have to remove `package-mode=false` on line 7 of `pyproject.toml`
+* Install redis, rabbitmq-server, and postgresql 15 using your distro's package manager. 
+* Enable these services as background services using `sudo systemctl enable --now redis rabbitmq-server postgresql`
+* Configure Postgres to ensure that password authentication is enabled for the gooey user
+    - open the pg_hba.conf file in a text editor. On Linux, by default, it is usually located either at ```/etc/postgresql/<version>/main/``` or ```/var/lib/pgsql/<version>/data/```
+    - add/edit the file so that there are lines at the bottom that looks like this:
+    ```
+    local    all        gooey                    md5
+    host     all        gooey                    md5
+    ```
+    - restart postgresql using ```sudo systemctl restart postgresql```
+* Use the manage.py script to set up the Postgres database:
+  - To create the user and database for gooey: `./manage.py sqlcreate | sudo -u postgres psql postgres `
+  - Test your setup to ensure that gooey-server can access the database by running `psql -W -U gooey gooey` and supplying "gooey" as the password
+* Create a .env file from `.env.example`
+* Install the zbar library using your distro's package manager. 
+
 ## Run
 
+Note: The gooey-server project is not currently set up to be run without support from Gooey. This software requires access to a Google Cloud instance as well as business data loaded in the database. If you are interested in running this software totally independently, reach out to [email protected] to communicate with our enterprise team. 
+
 You can start all required processes in one command with Honcho:
 
 ```shell
 $ poetry run honcho start
 ```
+This will spin up the API server at `http://localhost:8080`. To view the autogenerated API documentation, navigate to `http://localhost:8080/docs`
+
+If you installed the gooey-ui server, you can navigate to 'http://localhost:3000' to access the web application.
 
 The processes that it starts are defined in [`Procfile`](Procfile).
 Currently they are these:
@@ -91,24 +127,6 @@ docker run \
 ./manage.py runscript setup_vespa_db
 ```
 
-## To connect to our GPU cluster 
-
-* Connect to k8s cluster -
-```bash
-gcloud container clusters get-credentials cluster-5 --zone us-central1-a
-```
-
-* Port-forward the rabbitmq and redis services -
-```bash
-kubectl port-forward rabbitmq-1-rabbitmq-0 15674:15672 5674:5672 & kubectl port-forward redis-ha-1-server-0 6374:6379
-```
-
-* Add the following to `.env` file -
-```
-GPU_CELERY_BROKER_URL="amqp://rabbit:<password>@localhost:5674"
-GPU_CELERY_RESULT_BACKEND="redis://:<password>@localhost:6374"
-```
-
 ### Install imagemagick
 
 Needed for HEIC image support - https://docs.wand-py.org/en/0.5.7/guide/install.html
@@ -125,60 +143,8 @@ Use black - https://pypi.org/project/black
 **Recommended**: Black IDE integration Guide: [Pycharm](https://black.readthedocs.io/en/stable/integrations/editors.html#pycharm-intellij-idea)
 
 
-## Running test Whatsapp bot
-
-We use the following facebook app for testing - 
-```
-gooey.ai (dev)
-App ID: 228027632918921
-```
-
-Create a [meta developer account](https://developers.facebook.com/docs/development/register/) & send admin your **facebook ID** to add you to the test app [here](https://developers.facebook.com/apps/228027632918921/roles/roles/?business_id=549319917267066)
-
-1. start ngrok
-
-```
-ngrok http 8080
-```
-
-2. set env var `FB_WEBHOOK_TOKEN = asdf1234`
-
-
-3. Open [WhatsApp Configuration](https://developers.facebook.com/apps/228027632918921/whatsapp-business/wa-settings/?business_id=549319917267066), set the Callback URL and Verify Token
-<img width="500" alt="image" src="https://github.com/GooeyAI/gooey-server/assets/19492893/95bb3a87-ae4f-4f6b-a04e-583ee51b85de">
-
-4. Open [WhatsApp API Setup](https://developers.facebook.com/apps/228027632918921/whatsapp-business/wa-dev-console/?business_id=549319917267066), send yourself a message from the test number.
-<img width="500" alt="image" src="https://github.com/GooeyAI/gooey-server/assets/19492893/f9417723-77c0-4be5-9814-778662215d9c">
-
-5. Copy the temporary access token there and set env var `WHATSAPP_ACCESS_TOKEN = XXXX`
-
-
-**(Optional) Use the test script to send yourself messages** 
-
-```bash
-python manage.py runscript test_wa_msg_send --script-args 104696745926402 +918764022384
-```
-Replace `+918764022384` with your number and `104696745926402` with the test number ID
-
-## Dangerous postgres commands
-
 ### backup & restore postgres db
 
-**on server**
-```bash
-# select a running container
-cid=$(docker ps | grep gooey-api-prod | cut -d " " -f 1 | head -1)
-# give it a nice name 
-fname=gooey_db_$(date +"%Y-%m-%d_%I-%M-%S_%p").dump
-# exec the script to create the fixture
-docker exec -it $cid pg_dump --dbname $PGDATABASE --format c -f "$fname"
-# copy the fixture outside container
-docker cp $cid:/app/$fname .
-# print the absolute path
-echo $PWD/$fname
-```
-
-**on local**
 ```bash
 # reset the database
 ./manage.py reset_db -c
@@ -190,14 +156,8 @@ pg_restore --no-privileges --no-owner -d $PGDATABASE $fname
 
 ### create & load fixtures
 
-**on server**
 ```bash
-# select a running container
-cid=$(docker ps  | grep gooey-api-prod | cut -d " " -f 1 | head -1)
-# exec the script to create the fixture
-docker exec -it $cid poetry run ./manage.py runscript create_fixture
-# upload the fixture
-docker exec -it $cid poetry run ./manage.py runscript upload_fixture
+./scripts/run-tests.sh
 ```
 
 To load the fixture on local db -
@@ -217,7 +177,6 @@ To load the fixture on local db -
 
 ### copy one postgres db to another
 
-**on server**
 ```bash
 ./manage.py reset_db
 createdb -T template0 $PGDATABASE