docs: improve general and development documentatation

.gitignore

@@ -1,10 +1,14 @@
 .cog
-checkpoints
 cache
 __pycache__
 input
 output
+
+# AI Configuration files and model storage folders.
+aiModels.json
 models
+checkpoints
+
+# IDE
 .vscode
 *.code-workspace
-aiModels.json

README.md

@@ -1,19 +1,38 @@
 # ai-worker
 
-Note: This repo contains prototype code that is changing quickly and is not intended for production use at this point.
+> [!WARNING]
+> This is a prototype repository undergoing rapid changes. It's not intended for production use yet.
 
-## Build
+This repository hosts the AI worker and runner for processing inference requests on the Livepeer AI subnet.
+
+## Overview
+
+The AI worker repository includes:
+
+- **Runner**: The [AI runner](https://github.com/livepeer/ai-worker/tree/main/runner), a containerized Python application, processes inference requests on Livepeer AI's Pipelines and models, providing a REST API for model interaction.
+
+- **Worker**: The [AI worker](https://github.com/livepeer/ai-worker) allows the [ai-video](https://github.com/livepeer/go-livepeer/tree/ai-video) branch of [go-livepeer](https://github.com/livepeer/go-livepeer/tree/ai-video) to interact with the AI runner. It includes golang API bindings, a worker for routing inference requests, and a Docker manager for AI runner containers.
+
+### Runner
 
-Follow the README instructions in the [runner](https://github.com/livepeer/ai-worker/tree/main/runner) directory to download model checkpoints and build the runner image.
+The AI runner's code is in the [runner](https://github.com/livepeer/ai-worker/tree/main/runner) directory. For more details, see the [AI runner README](./runner/README.md).
+
+### Worker
+
+The AI worker's code is in the [worker](https://github.com/livepeer/ai-worker/tree/main/worker) directory. It includes:
+
+- **Golang API Bindings**: Generated from the AI runner's OpenAPI spec using `make codegen`.
+- **Worker**: Listens for inference requests from the Livepeer AI subnet and routes them to the AI runner.
+- **Docker Manager**: Manages AI runner containers.
+
+## Build
 
-Generate Go code for the runner OpenAPI spec:
+The AI worker and runner are designed to work with the [ai-video](https://github.com/livepeer/go-livepeer/tree/ai-video) branch of [go-livepeer](https://github.com/livepeer/go-livepeer/tree/ai-video). You can run both independently for testing. To build the AI worker locally and run examples, follow these steps:
 
-```
-make codegen
-```
+1. Follow the README instructions in the [runner](./runner/README.md) directory to download model checkpoints and build the runner image.
+2. Generate Go bindings for the runner OpenAPI spec with `make codegen`.
+3. Run any examples in the `cmd/examples` directory, e.g., `go run cmd/examples/text-to-image/main.go <RUNS> <PROMPT>`.
 
-## Run examples
+## Development documentation
 
-```
-go run cmd/examples/text-to-image/main.go <RUNS> <PROMPT>
-```
+For more on developing and debugging the AI runner, see the [development documentation](./dev/README.md).

cmd/examples/image-to-image/main.go

@@ -1,3 +1,4 @@
+// Package main provides a small example on how to run the 'image-to-image' pipeline using the AI worker package.
 package main
 
 import (

cmd/examples/image-to-video/main.go

@@ -1,3 +1,4 @@
+// Package main provides a small example on how to run the 'text-to-video' pipeline using the AI worker package.
 package main
 
 import (

cmd/examples/text-to-image/main.go

@@ -1,3 +1,4 @@
+// Package main provides a small example on how to run the 'text-to-image' pipeline using the AI worker package.
 package main
 
 import (

dev/README.md

@@ -0,0 +1,19 @@
+# Development Guide
+
+This guide provides instructions for setting up the development environment and debugging the [AI worker](https://github.com/livepeer/ai-worker) repository.
+
+## Debugging
+
+### Local Debugging
+
+To directly debug the AI worker, use the go scripts in the [cmd/examples](https://github.com/livepeer/go-livepeer/tree/ai-video/cmd) folder to test and debug the AI worker. Run these scripts with [Golang](https://go.dev/) or use [Vscode](https://code.visualstudio.com/) with the [golang extension](https://code.visualstudio.com/docs/languages/go) for debugging. Future updates will include tests to enhance the development pipeline.
+
+### Debugging within go-livepeer
+
+To debug the AI worker within the context of the [go-livepeer](https://github.com/livepeer/go-livepeer/tree/ai-video) project, replace the go module reference in the go.mod file of the go-livepeer project with the path to your local AI worker repository:
+
+```bash
+go mod edit -replace github.com/livepeer/ai-worker=../path/to/ai-worker
+```
+
+This setup allows you to debug the AI worker package directly when running the [go-livepeer](https://github.com/livepeer/go-livepeer/tree/ai-video) software.

runner/.devcontainer/devcontainer.json

@@ -12,7 +12,6 @@
 	],
 
 	// Features to add to the dev container. More info: https://containers.dev/features.
-	// "features": {},
 
 	// Configure tool-specific properties.
 	"customizations": {
@@ -29,19 +28,10 @@
 	"forwardPorts": [
 		8000
 	],
-	
+
 	// Use 'mounts' to make a list of local folders available inside the container.
 	"mounts": [
 		// "source=${localWorkspaceFolder}/models,target=/models,type=bind"
 		"source=${localEnv:HOME}/.lpData/models,target=/models,type=bind"
     ]
-
-	// Use 'postCreateCommand' to run commands after the container is created.
-	// "postCreateCommand": ""
-
-	// Configure tool-specific properties.
-	// "customizations": {},
-
-	// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
-	// "remoteUser": "root"
 }