**Please note that support questions will not be monitored during the summer period of July and August. **
CEF Context Broker integration with the European Data Portal.
This Integration Solution generates from the parameters established in a configuration file an RDF/XML file containing the datasets representing Context Broker Data Models chosen to integrate. The output is available at the location where the solution is deployed.
The Python module to install contains the following main components:
- Integration Solution core component
cb_edp/: Command line interface (CLI) application that offers the options needed to work with the integration. - Integration Solution API
cb_edp/api: The scope of this Flask developed API is to allow the accessing to the data from a dataset in the RDF file using a custom call. It also provides generated RDF through a static URL in order to have it always available on the Internet.
The following instructions will allow you to get a completely functional CB-EDP environment. This is just a guideline about how to install and deploy the solution. Adapt it to your needs.
CB-EDP has some requirements that should be accomplished before starting deploying it.
- Ubuntu 18.04.1 LTS 64-bit or later
- Python 3.6 or later
- pip3 9.0.1 or later
Update packages list in case you didn't do before (recommended):
sudo apt update
Python 3 is already installed in Ubuntu 18 distributions. However, in case you want to use Python 3.7, follow the next steps to install it.
First update packages list and install the prerequisites:
sudo apt install software-properties-common
Then add the deadsnakes PPA to your sources list:
sudo add-apt-repository ppa:deadsnakes/ppa
Last, install Python 3.7 with:
sudo apt install python3.7
You can verify if everything is alright just typing (it should print Python version number):
$ python3.7 --version
Python 3.7.3
pip3 will be used as the package manager for Python. It will be used for CB-EDP installation, so must be installed before starting the deployment.
After packages list update, install pip for Python 3:
sudo apt install python3-pip
You can verify the installation typing:
$ pip3 --version
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.7)Before installing the package, it must first be generated by following these steps:
If there is a virtual environment where the package is to be installed, first of all, activate it.
source <path/to/the/file/activate>
Once activated, install pyBuilder:
pip install pybuilder
When pybuilder finishes installing, we proceed to create the package.
First of all, download this repository as a zip file. Unzip and then enter the path where the core has been unzipped.
cd cb_edp
Once inside the folder and with the virtualenv previously activated, execute the command:
pyb publish -E production
Once the above command has been executed, our package is ready. A target folder will have been created and inside that folder, we will find the package ready to install in the following path:
target/dist/cb-edp-1.1.0/dist/cb-edp-1.1.0.tar.gz
Once you got it, install it using pip:
sudo pip3 install /path/to/cb-edp-1.1.0.tar.gz
It should have installed too every dependency (Click, configobj, Flask,
Gunicorn, requests and time-uuid) of the CB-EDP. In case it didn't or
you aren't sure of it, install them directly using
requirements.txt file. First unzip it cause it's
on downloaded ZIP file:
unzip /path/to/cb_edp.zip
pip3 install -r /path/to/requirements.txt
You can check it's installed launching show pip3 command:
$ pip3 show cb-edp
---
Metadata-Version: 1.0
Name: cb-edp
Version: 1.1
Summary: FIWARE Context Broker instance integration with the EDP
Home-page: https://github.com/ConnectingEurope/ContextBroker-EDP
Location: /usr/local/lib/python3.7/dist-packages
Requires: Click, configobj, Flask, gunicorn, requests, time-uuid
Classifiers:
Entry-points:
[console_scripts]
cb-edp=cb_edp.commands:cli
The Integration Solution includes an API for:
- Doing requests to the CB configured in order to get responses from browsable HTTP requests
- Publishing the RDF/XML file generated for harvesting by the EDP
The deployment of this API will need something serving the solution and other thing that grants access to this server from the Internet. The technologies suggested to do so are Gunicorn and Nginx.
Gunicorn should be installed by pip when installing CB-EDP. If not, please launch:
sudo pip3 install gunicorn==19.9.0
Install required components:
sudo apt-get install python3-pip python3-dev build-essential libssl-dev libffi-dev python3-setuptools
Create a service to assign Gunicorn to it with nano (or any other
editor you like):
sudo nano /etc/systemd/system/cb-edp.service
And paste the following contents replacing the values:
{sudoer-user}for machine’s sudoer user. It is root by default.{user-group}for a group from the user specified before. You can check it using ll command with “Location” value copied before to see which groups have the other files and directories. It is staff by default.{solution-location}for "Location" value copied before.
[Unit]
Description=Gunicorn instance to serve CB-EDP
After=network.target
[Service]
User={sudoer_user}
Group={user_group}
WorkingDirectory={solution-location}/cb_edp/api
ExecStart=/usr/local/bin/gunicorn --worker-class gthread --workers 3 --threads 1 --bind unix:cb-edp.sock -m 704 wsgi:app
[Install]
WantedBy=multi-user.target
Gunicorn accepts custom configuration for number of workers and threads that these workers will use. The values set in the text above are the recommended ones, but can be modified following these equations:
Now Gunicorn service can be started:
sudo systemctl start cb-edp
To enable Gunicorn launching on boot, launch this command:
sudo systemctl enable cb-edp
You can check application service status executing:
sudo systemctl status cb-edp
Install Nginx for Ubuntu:
sudo apt-get install nginx
You can check Nginx service status running:
sudo systemctl status nginx
Now it's turn to configure Nginx to proxy requests to the API. To do so
create new server block configuration for the already created Gunicorn
service with nano (or any other editor you like):
sudo nano /etc/nginx/sites-available/cb-edp
And paste the following contents replacing the values:
{your-public-ip-or-dns}for the public IP of the server or the DNS.{your-custom-route}for the relative path where the API service will be available. It accepts any value (context-data, open-data, cb, etc.) and many levels (as many as the Integration Admin wants) but always has to include the /api text at the end. Some valid examples:- /context-data/api
- /open-data/cb/catalogue/api
- /api
{solution-location}for "Location" value copied before.
server {
listen 80;
server_name {your-public-ip-or-dns};
location /{your-custom-route}/api {
include proxy_params;
proxy_pass http://unix:{solution-location}/cb_edp/api/cb-edp.sock;
}
}
Link the file to the sites-enabled directory to enable Nginx server block configuration:
sudo ln -s /etc/nginx/sites-available/cb-edp /etc/nginx/sites-enabled
Check that there are no errors:
sudo nginx -t
If it returns no issues, then restart Nginx to load the new configuration:
sudo systemctl restart nginx
Now the application should be available on the Internet. Try browsing to
http://{your-public-ip-or-dns}/{your-custom-route}/api/status
This project is licensed under the European Union Public License 1.2 -see the LICENSE file for details.
The Context Broker - European Data Portal enabler has been carried out by: