Project DocEx

This is NOT an official Google product.

Development environment

Before you start, make sure you have the following installed:

• Google Cloud SDK
• Node JS
• Angular JS
• Python 2

Test your development environment.

gcloud version
python2 --version
node --version
ng version

Setup

First, select the project you want to use or create a new project.

Option A: create a new project create a new Google Cloud project

PROJECT_ID=your-project-id
gcloud projects create $PROJECT_ID

Option B: use your currently selected project

PROJECT_ID=$(gcloud config get-value project)

Option C: select an existing project

PROJECT_ID=your-project-id
gcloud config set project $PROJECT_ID

Then enable the Cloud Functions, AutoML, Cloud Pub/Sub, Cloud Storage API.

Update components

gcloud components update
gcloud services enable cloudfunctions.googleapis.com 
gcloud services enable automl.googleapis.com
gcloud services enable pubsub.googleapis.com
gcloud services enable storage-component.googleapis.com

Once you have a project, you will also need to create two Cloud Storage buckets.

BUCKET_NAME=your-bucket-name
gsutil mb gs://$BUCKET_NAME
TRAIN_BUCKET_NAME=your-train-bucket-name
gsutil mb gs://$TRAIN_BUCKET_NAME

Create the Pub/Sub topic and subscription to train model.

ML_ENGINE_TOPIC=your-automl-topic-name
gcloud pubsub topics create projects/$PROJECT_ID/topics/$ML_ENGINE_TOPIC
gcloud pubsub subscriptions create projects/$PROJECT_ID/subscriptions/$ML_ENGINE_TOPIC

Create the Pub/Sub topic and subscription to convert pdf to image.

PDF_CONVERTER_TOPIC=your-pdf-converter-topic-name
gcloud pubsub topics create projects/$PROJECT_ID/topics/$PDF_CONVERTER_TOPIC
gcloud pubsub subscriptions create projects/$PROJECT_ID/subscriptions/$PDF_CONVERTER_TOPIC

• Inside the VM's shell, clone repo by following command prompts (Clone with https)

git clone https://github.com/davidcavazos/project-dragon.git

Deploying PDF to Image converter Colud Function

Follow below steps to create trigger on input bucket to convert pdf file to image.

1. Change to the directory that contains the Cloud Functions sample code:

cd pdf-converter

2. Deploy cloud function

gcloud functions deploy pdf-converter \
	--runtime nodejs10 \
	--trigger-resource $BUCKET_NAME \
	--trigger-event google.storage.object.finalize

Deploying PDF to Image on Compute Engine

1. Create compute engine and SHH it

2. Follow README.md steps to setup compute engine environment and dependencies.

3. Clone this project and change to directory pdf-conveter

4. Install dependencies and run project.

screen
npm install
npm install pm2 --g
pm2 pdf-to-image.js

Deploying AutoML Train Model to Compute Engine

1. Create compute engine and SHH it

2. Clone this project and change to directory auto-ml-train

3. On the TRAIN-BUCKET-NAME upload the static documents with subfolders as label names that contains respective documents corresponding to labels

4. Follow README.md steps to setup compute engine environment and dependencies.

5. Install dependencies and run project.

screen
bash setup.sh

Deploying to App Engine

Deploying will take a couple minutes, but after that the application will autoscale to match the current load of the application.

# Build the Angular web application.
cd ui
npm install
ng build --prod

# Build the Middleware node application
cd ../middleware
npm install
gcloud app deploy