Merge pull request #5676 from EnterpriseDB/docs/edb-postgres-ai/initial

Initial import of edb-postgres-ai

advocacy_docs/edb-postgres-ai/ai-ml/index.mdx

@@ -0,0 +1,22 @@
+---
+title: EDB Postgres AI - AI/ML
+navTitle: AI/ML
+indexCards: simple
+iconName: BrainCircuit
+navigation:
+- overview
+- install-tech-preview
+- using-tech-preview
+---
+
+Postgres AI Database is designed to solve all AI data management needs, including storing, searching, and retrieving of AI data. This uplevels Postgres to a database that manages and serves all types of data modalities directly and combines it with its battle-proof strengths as an established Enterprise system of record that manages high-value business data. 
+
+In this tech preview, you will be able to use the pgai extension to build a simple retrieval augmented generation (RAG) application in Postgres. 
+
+An [overview](overview) of the pgai extension gives you a high-level understanding of the major functionality available to date.
+
+To get started, you will need to [install the pgai tech preview](install-tech-preview) and then you can start [using the pgai tech preview](using-tech-preview) to build your RAG application.
+
+
+
+

advocacy_docs/edb-postgres-ai/ai-ml/install-tech-preview.mdx

@@ -0,0 +1,105 @@
+---
+title: EDB Postgres AI AI/ML - Installing the pgai tech preview
+navTitle: Installing
+description: How to install the EDB Postgres AI AI/ML pgai tech preview and run the container image.
+prevNext: true
+---
+
+The preview release of pgai is distributed as a self-contained Docker container that runs PostgreSQL and includes all of the pgai dependencies.
+
+## Configuring and running the container image
+
+If you haven’t already, sign up for an EDB account and log in to the EDB container registry.
+Login to docker with your the username tech-preview and your EDB Repo 2.0 Subscription Token as your password:
+
+```
+docker login docker.enterprisedb.com -u tech-preview -p <your_EDB_repo_token>
+__OUTPUT__
+Login Succeeded
+```
+
+Download the pgai container image:
+
+```
+docker pull docker.enterprisedb.com/tech-preview/pgai
+__OUTPUT__
+...
+Status: Downloaded newer image for docker.enterprisedb.com/tech-preview/pgai:latest
+docker.enterprisedb.com/tech-preview/pgai:latest
+```
+
+Specify a password to use for Postgres in the environment variable PGPASSWORD. The tech preview container will set up Postgres with this password and use it to connect to it. In bash or zsh set it as follows:
+
+```
+export PGPASSWORD=<your_password>
+```
+
+You can use the pgai extension with encoder LLMs in Open AI or with open encoder LLMs from HuggingFace. If you want to use Open AI you also must provide your API key for that in the OPENAI_API_KEY environment variable:
+
+```
+export OPENAI_API_KEY=<your_openai_key>
+```
+
+You can use the pgai extension with AI data stored in Postgres tables or on S3 compatible object storage. To work with object storage you need to specify the ACCESS_KEY and SECRET_KEY environment variables:.
+
+```
+export ACCESS_KEY=<your_access_key>
+export SECRET_KEY=<your_secret_key>
+```
+
+Start the pgai tech preview container with the following command. It makes the tech preview PostgreSQL database available on local port 15432:
+
+```
+docker run -d --name pgai \
+       -e ACCESS_KEY=$ACCESS_KEY \
+       -e SECRET_KEY=$SECRET_KEY \
+       -e OPENAI_API_KEY=$OPENAI_API_KEY \
+       -e POSTGRES_PASSWORD=$PGPASSWORD \
+       -e PGDATA=/var/lib/postgresql/data/pgdata \
+       -p 15432:5432 \
+       docker.enterprisedb.com/tech-preview/pgai:latest
+```
+
+
+## Connect to Postgres
+
+If you haven’t yet, install the Postgres command-line tools. If you’re on a Mac, using Homebrew, you can install it as follows:
+
+```
+brew install libpq
+```
+
+Connect to the tech preview PostgreSQL running in the container. Note that this relies on $PGPASSWORD being set - if you’re using a different terminal for this part, make sure you re-export the password:
+
+```
+$ psql -h localhost -p 15432 -U postgres postgres
+__OUTPUT__
+psql (16.1, server 16.3 (Debian 16.3-1.pgdg120+1))
+Type "help" for help.
+
+postgres=#
+```
+
+
+Install the pgai extension:
+
+```sql
+postgres=# create extension pgai cascade;
+__OUTPUT__
+NOTICE:  installing required extension "plpython3u"
+NOTICE:  installing required extension "vector"
+CREATE EXTENSION
+```
+
+```sql
+postgres=# \dx
+__OUTPUT__
+                               List of installed extensions
+    Name    | Version |   Schema   |                     Description
+------------+---------+------------+------------------------------------------------------
+ pgai       | 0.0.1   | public     | An extension to do the AIs
+ plpgsql    | 1.0     | pg_catalog | PL/pgSQL procedural language
+ plpython3u | 1.0     | pg_catalog | PL/Python3U untrusted procedural language
+ vector     | 0.6.0   | public     | vector data type and ivfflat and hnsw access methods
+(4 rows)
+```

advocacy_docs/edb-postgres-ai/ai-ml/overview.mdx

@@ -0,0 +1,28 @@
+---
+title: EDB Postgres AI AI/ML - Overview
+navTitle: Overview
+description: Where to start with EDB Postgres AI AI/ML and the pgai tech preview.
+prevNext: True
+---
+
+At the heart of EDB Postgres AI is the EDB Postgres AI database (pgai). This builds on Postgres's flexibility and extends its capability to include store the vector data of embeddings.
+
+The pgai extension is currently available as a tech preview. It will be continuously extended with new functions. Here comes a description of the major functionality available to date.
+
+![PGAI Overview](images/pgai-overview.png)
+
+pgai introduced the concept of a “retriever” that you can create for a given type and location of AI data. Currently pgai supports unstructured plain text documents as well as a set of image formats. This data can either reside in regular columns of a Postgres table or it can reside in an S3 compatible object storage bucket.
+
+A retriever encapsulates all processing that is needed to make the AI data in the provided source location searchable and retrievable via similarity. The application just needs to create a retriever via the pgai.create_retriever() function. When auto_embedding=TRUE is specified the pgai extension will automatically generate embeddings for all the data in the source location. Otherwise it will be up to the application to request a bulk generation of embeddings via pgai.refresh_retriever(). 
+
+Auto embedding is currently supported for AI data stored in Postgres tables and it automates the embedding updates using Postgres triggers. You can also combine the two options by using pgai.refresh_retriever() to embed all previously existing data and also setting `auto_embedding=TRUE` to generate embeddings for all new and changed data from now on.
+
+All embedding generation, storage, indexing and management is handled by the pgai extension internally. The application just has to specify the encoder LLM that the retriever should be using for this specific data and use case.
+
+Once a retriever is created and all embeddings are up to date, the application can just use pgai.retrieve() to run a similarity search and retrieval by providing a query input. When the retriever is created for text data, the query input is also a text term. For image retrievers the query input is an image. The pgai retriever makes sure to use the same encoder LLM for the query input, conducts a similarity search and finally returns the ranked list of similar data from the source location.
+
+pgai currently supports a broad list of open encoder LLMs from HuggingFace as well as a set of OpenAI encoders. Just consult the list of supported encoder LLMs in the pgai.encoders meta table. HuggingFace LLMs are running locally on the Postgres node, while OpenAI encoders involve a call out to the OpenAI cloud service.
+
+
+
+

advocacy_docs/edb-postgres-ai/ai-ml/using-tech-preview/index.mdx

@@ -0,0 +1,16 @@
+---
+title: EDB Postgres AI AI/ML - Using the pgai tech preview
+navTitle: Using
+description: Using the EDB Postgres AI AI/ML tech preview to build a simple retrieval augmented generation (RAG) application in Postgres.
+navigation:
+- working-with-ai-data-in-postgres
+- working-with-ai-data-in-s3
+- standard-encoders
+---
+
+This section shows how you can use your [newly installed pgai tech preview](install-tech-preview) to retrieve and generate AI data in Postgres.
+
+* [Working with AI data in Postgres](working-with-ai-data-in-postgres) details how to use the pgai extension to work with AI data stored in Postgres tables.
+* [Working with AI data in S3](working-with-ai-data-in-s3) covers how to use the pgai extension to work with AI data stored in S3 compatible object storage.
+* [Standard encoders](standard-encoders) goes through the standard encoder LLMs that are supported by the pgai extension.
+

advocacy_docs/edb-postgres-ai/ai-ml/using-tech-preview/standalone-embedding-functions.mdx

@@ -0,0 +1,53 @@
+---
+title: Stand-alone Embedding Functions in pgai
+navTitle: Stand-alone Embedding Functions
+description: Use the pgai extension to generate embeddings for images and text.
+---
+
+Use generate_single_image_embedding function to get embeddings for the given image. Currently, model_provider can only be openai or huggingface. You can check the list of valid embedding models and model providers from the Encoders Supported PGAI section. 
+
+```sql
+SELECT pgai.generate_single_image_embedding(
+    'clip-vit-base-patch32', -- embedding model name
+    'openai', -- model provider
+    'https://s3.us-south.cloud-object-storage.appdomain.cloud', -- S3 endpoint
+    'torsten', -- S3 bucket name
+    'foto.jpg' -- img object to generate embeddings
+);
+__OUTPUT__
+ generate_single_image_embedding
+--------------------------------
+
+(1 row)    
+```
+
+Use the `generate_text_embedding` function to get embeddings for the given image. Currently, the `model_provider` can only be `openai` or `huggingface`. 
+
+```sql
+SELECT pgai.generate_text_embedding(
+'text-embedding-3-small', -- embedding model name
+'openai', -- model provider
+ 0, -- dimensions, setting 0 will replace with the default value in encoder's table
+'Veggie Burger' -- text to generate embeddings
+);
+__OUTPUT__
+ generate_text_embedding
+------------------------
+
+(1 row)    
+```
+
+
+You can check the list of valid embedding models and model providers from pgai.encoders table
+
+```sql
+SELECT provider, count(*) encoder_model_count FROM pgai.encoders group by (provider);
+__OUTPUT__
+  provider   | encoder_model_count
+-------------+---------------------
+ huggingface |                  36
+ openai      |                   4
+(2 rows)
+```
+
+

advocacy_docs/edb-postgres-ai/ai-ml/using-tech-preview/working-with-ai-data-in-S3.mdx

@@ -0,0 +1,89 @@
+---
+title: Working with AI data stored in S3-compatible object storage
+navTitle: Working with AI data in S3
+description: How to work with AI data stored in S3-compatible object storage using the pgai extension.
+---
+
+We recommend you to prepare your own S3 compatible object storage bucket with some test data and try the steps in this section with that. But it is possible to simply use the example S3 bucket data as is in the examples here even with your custom access key and secret key credentials because these have been configured for public access.
+
+In addition we use image data and an according image encoder LLM in this example instead of text data. But you could also use plain text data on object storage similar to the examples in the previous section.
+
+First let’s create a retriever for images stored on s3-compatible object storage as the source.  We specify torsten as the bucket name and an endpoint URL where the bucket is created. We specify an empty string as prefix because we want all the objects in that bucket. We use the [`clip-vit-base-patch32`](https://huggingface.co/openai/clip-vit-base-patch32) open encoder model for image data from HuggingFace. We provide a name for the retriever so that we can identify and reference it subsequent operations:
+
+```sql
+SELECT pgai.create_s3_retriever(
+    'image_embeddings', -- Name of the similarity retrieval setup
+    'public', -- Schema of the source table
+    'img_id', -- Primary key
+    'clip-vit-base-patch32', -- Embeddings encoder model for similarity data
+    'img', -- data type, could be either img or text
+    'torsten', -- S3 bucket name
+    'https://s3.us-south.cloud-object-storage.appdomain.cloud', -- s3 endpoint address
+    '' -- prefix
+);
+__OUTPUT__
+ create_s3_retriever 
+---------------------
+
+(1 row)
+```
+
+
+Next, run the refresh_retriever function.
+
+```sql
+SELECT pgai.refresh_retriever('image_embeddings');
+__OUTPUT__
+ refresh_retriever
+-------------------
+
+(1 row)
+```
+
+Finally, run the retrieve_via_s3 function with the required parameters to retrieve the top K most relevant (most similar) AI data items. Please be aware that the object type is currently limited to image and text files.
+
+```sql 
+SELECT data from pgai.retrieve_via_s3(
+    'image_embeddings', -- retriever's name
+     1, -- top K
+    'torsten', -- S3 bucket name
+    'foto.jpg', -- object name
+    'https://s3.us-south.cloud-object-storage.appdomain.cloud'
+);
+__OUTPUT__
+     data        
+--------------------
+ {'img_id': 'foto'}
+ (1 row)
+```
+
+If you set the `ACCESS_KEY` and `SECRET_KEY` you can use the following queries to run without an `s3_endpoint` setting using a command like this:
+
+```sql
+SELECT pgai.create_s3_retriever(
+   'img_file_embeddings', -- Name of the similarity retrieval setup
+   'demo', -- Schema of the source table
+   'img_id', -- Primary key
+   'clip-vit-base-patch32', -- Embeddings encoder model for similarity data
+   'img', -- data type
+   'bilge-ince-test' -- S3 bucket name
+);
+__OUTPUT__
+ create_s3_retriever
+---------------------
+
+(1 row)
+```
+
+```sql
+SELECT pgai.refresh_retriever('img_file_embeddings');
+```
+
+```sql
+SELECT data from pgai.retrieve_via_s3('img_file_embeddings',
+     1, 
+     'bilge-ince-test', 
+     'kirpis_small.jpg', 
+     'http://s3.eu-central-1.amazonaws.com'
+     );
+```