Create a podcast downloader using OpenWhisk

In this Code Pattern, we will demonstrate the use of OpenWhish by creating a podcast downloader. A light weight Flask application is set up to request and download the content, which is then retained in OpenStack Swift object storage.

When the reader has completed this Code Pattern, they will understand how to:

• compose a microservices style app
• utilize the power of OpenWhisk
• make use of OpenStack Swift object storage

Flow

1. A local python app with MySQL backend is set up.
2. User packages Python Actions with a virtual environment in a zip file and uploads to IBM Cloud Functions.
3. User interacts with local app to invoke Action.
4. Data is stored in Object Storage OpenStack Swift.

Included components

• IBM Cloud Object Storage: Build and deliver cost effective apps and services with high reliability and fast speed to market in an unstructured cloud data store.
• OpenWhisk: Execute code on demand in a highly scalable, serverless environment.

Featured technologies

• Microservices: Collection of fine-grained, loosely coupled services using a lightweight protocol to provide building blocks in modern application composition in the cloud.

Watch the Video

Steps

Run locally

1. Clone the repo
2. Create database
3. Server side
4. Create user, podcast, and subscription
5. Create a zip python action
6. Deploy Downloader action to OpenWhisk platform
7. Invoke downloader action from podcast manager

Prerequisite

Make sure you have a Bluemix account. It also helpful if you are slightly familiar with basic OpenWhisk commands Model and idea and have installed both the Bluemix CLI and the Cloud Functions Plugin.

Installation requirements:

• Python2.7 Installed.
• pip install -r requirements.txt
• Mysql DB server instance installed locally or provisioned on remote cloud server.
• curl command tool or similar tools available.

1. Clone the repo

Clone the PodcastDownloader locally. In a terminal, run:

git clone https://github.com/IBM/PodcastDownloader
cd PodcastDownloader

2. Create database

Provision a mysql dabatase instance, using the provided podcasts_downloader.sql to create the database tables.

mysql -u user -p < podcasts_downloader.sql

3. Server side

Start the podcasts manager api server.

export FLASK_APP=podcasts_manager.py
export FLASK_DEBUG=1
flask run

4. Create user, podcast, and subscription

Use curl to create the users, subscriptions, and podcasts. For example, suppose your podcast manager api server runs locally and listen on port 5000.

• create a user.

curl -X POST "http://localhost:5000/customer?name=liu"

• create a podcast.

curl -X POST "http://localhost:5000/podcast?podname=google&url=https://www.ted.com/talks/rss"

• create a subscription.

curl -X POST "http://localhost:5000/sub?customerid=1&podid=1"

5. create a zip python action.

Make sure you have docker installed locally and you have openwhisk/python2action docker images already built successfully.

cp DownloaderAction.py  __main__.py

Integrate Downloader action with the OpenStack swift client.

In order to persist the downloaded podcast content on storage, we now support integration with OpenStack swift client, to persist the downloaded content on IBM Object Storage.

First, you need to provision an object storage service, and copy all your storage service authentication information to VCAP_SERVICES.json, the Downloader Action will read the VCAP_SERVICES.json file, to pass the storage services authentication.

Make the zip python action.

docker run --rm -v "$PWD:/tmp" openwhisk/python2action sh -c "cd tmp; virtualenv virtualenv; source virtualenv/bin/activate; pip install -r requirements.txt;"

zip -r wgetPython.zip virtualenv __main__.py VCAP_SERVICES.json

6. Deploy Downloader action to OpenWhisk platform

bx wsk action create wgetPython --kind python:2 wgetPython.zip

7. Invoke downloader action from podcast manager

curl -X POST  -m 50 "http://localhost:5000/download?customername=liu&downloader_url=https://openwhisk.ng.bluemix.net/api/v1/namespaces/<cf_org>_<cf_space>/actions/wgetPython"

Links

• OpenWhisk
• OpenStack Swift

License

This code pattern is licensed under the Apache Software License, Version 2. Separate third party code objects invoked within this code pattern are licensed by their respective providers pursuant to their own separate licenses. Contributions are subject to the Developer Certificate of Origin, Version 1.1 (DCO) and the Apache Software License, Version 2.

Apache Software License (ASL) FAQ