From 911cf6876e2022a934183b6327a4802774b1d43b Mon Sep 17 00:00:00 2001 From: Jason Fox Date: Thu, 16 Jan 2025 13:27:07 +0100 Subject: [PATCH] Update Stellio --- .env | 2 +- NGSI-v2 Linked Data.postman_collection.json | 8 ++-- README.md | 4 +- docker-compose/stellio.yml | 42 +++++++++++++++------ docker-compose/stellio/kafka/update_run.sh | 11 ------ import-data | 8 ++-- import-ngsi-v2 | 10 ++--- services | 2 +- 8 files changed, 48 insertions(+), 39 deletions(-) delete mode 100755 docker-compose/stellio/kafka/update_run.sh diff --git a/.env b/.env index d063e75..783a698 100644 --- a/.env +++ b/.env @@ -8,7 +8,7 @@ ORION_VERSION=4.0.0 # Orion variables ORION_LD_PORT=1026 -ORION_LD_VERSION=1.7.1 +ORION_LD_VERSION=1.8.0 # Scorpio variables SCORPIO_PORT=9090 diff --git a/NGSI-v2 Linked Data.postman_collection.json b/NGSI-v2 Linked Data.postman_collection.json index ef54a15..0209db0 100644 --- a/NGSI-v2 Linked Data.postman_collection.json +++ b/NGSI-v2 Linked Data.postman_collection.json @@ -2,7 +2,7 @@ "info": { "_postman_id": "16121a01-e085-4afd-8689-fea63a0d7246", "name": "NGSI-v2 Linked Data", - "description": "This tutorial introduces the idea of using an existing **NGSI-v2** Context Broker as a Context Source within an\n**NGSI-LD** data space, and how to combine both JSON-based and JSON-LD based context data into a unified structure\nthrough introducing a simple **NGSI-LD** façade pattern with a fixed `@context`. The tutorial re-uses the data from the\noriginal **NGSI-v2** getting started tutorial but uses API calls from the **NGSI-LD** interface.\n\n\n> This tutorial is designed for exisiting **NGSI-v2** developers looking to attach existing **NGSI-v2** systems to an\n> **NGSI-LD** federation. If you are building a linked data system from scratch you should start from the beginnning of\n> the [NGSI-LD developers tutorial](https://ngsi-ld-tutorials.readthedocs.io/) documentation.\n>\n> Similarly, if you an existing **NGSI-v2** developer and you just want to understand linked data concepts in general,\n> checkout the equivalent [**NGSI-v2** tutorial](https://github.com/FIWARE/tutorials.Linked-Data/tree/NGSI-v2) on\n> upgrading **NGSI-v2** data sources to **NGSI-LD**\n\n## Contents\n\n# Adding Linked Data concepts to FIWARE Data Entities.\n\n> “Always be a first rate version of yourself and not a second rate version of someone else.”\n>\n> ― Judy Garland\n\nThe first introduction to FIWARE [Getting Started tutorial](https://github.com/FIWARE/tutorials.Getting-Started)\nintroduced the [NGSI v2](https://fiware.github.io/specifications/OpenAPI/ngsiv2) JSON-based interface that is commonly\nused to create and manipulate context data entities.\n[NGSI-LD](https://forge.etsi.org/swagger/ui/?url=https://forge.etsi.org/rep/NGSI-LD/NGSI-LD/raw/master/spec/updated/generated/full_api.json)\nis an evolution of that interface which enhances context data entities through adding the concept of **linked data**.\n\nThere is a need for two separate interfaces since:\n\n- **NGSI-v2** offers [JSON](https://www.json.org/json-en.html) based interoperability used in individual Smart Systems\n- **NGSI-LD** offers [JSON-LD](https://json-ld.org/) based interoperability used for Federations and Data Spaces\n\n**NGSI-v2** is ideal for creating individual applications offering interoperable interfaces for web services or IoT\ndevices. It is easier to understand than NGSI-LD and does not require a JSON-LD `@context`, However **NGSI-v2** without\nlinked data falls short when attempting to offer federated data across a data space. Once multiple apps and\norganisations are involved each individual data owner is no longer in control of the structure of the data used\ninternally by the other participants within the data space. This is where the `@context` of **NGSI-LD** comes in, acting\nas a mapping mechanism for attributes allowing the each local system to continue to use its own preferred terminology\nwithin the data it holds and for federated data from other users within the data space to be renamed using a standard\nexpansion/compaction operation allowing each individual system understand data holistically from across the whole data\nspace.\n\n## Creating a common data space\n\nFor example, imagine a simple \"Buildings\" data space consisting of two participants pooling their data together:\n\n- Details of a series of [supermarkets](https://fiware-tutorials.readthedocs.io/en/latest/) from the **NGSI-v2**\n tutorials.\n- Details of a series of [farms](https://ngsi-ld-tutorials.readthedocs.io/en/latest/) from the **NGSI-LD** tutorials.\n\nAlthough the two participants have agreed to use a common\n[data model](https://ngsi-ld-tutorials.readthedocs.io/en/latest/datamodels.html#building) between them, internally they\nhold their data in a slightly different structure.\n\n#### Farm (**NGSI-LD** Data)\n\nWithin the **NGSI-LD** Smart Farm, all of the Building Entities are marked using `\"type\":\"Building\"` as a shortname\nwhich can be expanded to a full URI https://uri.fiware.org/ns/dataModels#Building` using JSON-LD expansion rules. All\nthe attributes of each Building entity (such as `name`, `category` etc are defined using the\n[User `@context`](./data-models/ngsi-context.jsonld) and are structured as NGSI-LD attributes. The standard NGSI-LD\nKeywords are used to define the nature of each attribute - e.g. `Property`, `GeoProperty`, `VocabularyProperty`,\n`Relationship` and each of these terms is also defined within the\n[core `@context`](https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context-v1.8.jsonld).\n\n```\n{\n \"id\": \"urn:ngsi-ld:Building:farm001\",\n \"type\": \"Building\",\n \"category\": {\n \"type\": \"VocabularyProperty\",\n \"vocab\": \"farm\"\n },\n \"address\": {\n \"type\": \"Property\",\n \"value\": {\n \"streetAddress\": \"Großer Stern 1\",\n \"addressRegion\": \"Berlin\",\n \"addressLocality\": \"Tiergarten\",\n \"postalCode\": \"10557\"\n }\n },\n \"location\": {\n \"type\": \"GeoProperty\",\n \"value\": {\n \"type\": \"Point\",\n \"coordinates\": [13.3505, 52.5144]\n }\n },\n \"name\": {\n \"type\": \"Property\",\n \"value\": \"Victory Farm\"\n },\n \"owner\": {\n \"type\": \"Relationship\",\n \"object\": \"urn:ngsi-ld:Person:person001\"\n }\n}\n```\n\n#### Supermarket (**NGSI-v2** Data)\n\nWithin the **NGSI-v2** Smart Supermarket, every entity must have a `id` and `type` - this is part of the data model and\na prerequisite for joining the data space. However due to the backend systems used, the internal short name is\n`\"type\":\"Store\"`. Within an **NGSI-v2** system there is no concept of `@context` - every attribute has a `type` and a\n`value` and the attribute `type` is usually aligned with the datatype (e.g. `Text`, `PostalAddress`) although since\n**NGSI-LD** keyword types such as `Relationship`, `VocabularyProperty` are also permissible and set by convention.\n\n```\n{\n \"id\": \"urn:ngsi-ld:Store:001\",\n \"type\": \"Store\",\n \"address\": {\n \"type\": \"PostalAddress\",\n \"value\": {\n \"streetAddress\": \"Bornholmer Straße 65\",\n \"addressRegion\": \"Berlin\",\n \"addressLocality\": \"Prenzlauer Berg\",\n \"postalCode\": \"10439\"\n }\n },\n \"category\": {\n \"type\": \"VocabularyProperty\",\n \"value\": \"supermarket\"\n },\n \"location\": {\n \"type\": \"geo:json\",\n \"value\": {\n \"type\": \"Point\",\n \"coordinates\": [13.3986, 52.5547]\n }\n },\n \"name\": {\n \"type\": \"Text\",\n \"value\": \"Bösebrücke Einkauf\"\n },\n \"owner\": {\n \"type\": \"Relationship\",\n \"value\": \"urn:ngsi-ld:Person:person001\"\n }\n}\n```\n\n### Types of data space\n\nWhen joining a data space, a participant must abide by the rules that govern that data space. One of the first decisions\na common data space must make is to defined the nature of the data space itself. There are three primary approaches to\nthis, which can broadly be defined as follows:\n\n- An **Integrated** data space requires that every participant uses exactly the same payloads and infrastructure - for\n example _\"Use Scorpio 4.1.15 only\"_ . This could be a requirement for a lottery ticketing system where every\n terminal sends ticket data in the same format.\n- A **unified** data space defines a common data format, but allows for the underlying infrastructure to differ\n between participants - for example _\"Use NGSI-LD only for all payloads\"_\n- A **federated** data space is even looser and defines a common meta structure so each participants has more\n flexibility regarding it underlying technologies for example \"All payloads must be translatable as GeoJSON\"\\_ for a\n combined GIS, NGSI-LD data space.\n\nUsing this terminology, in this example we are creating a **unified** data space in this example, since participants are\nusing NGSI-LD in common for data exchange, but their underlying systems are different.\n\n# Prerequisites\n\n## Docker\n\nTo keep things simple all components will be run using [Docker](https://www.docker.com). **Docker** is a container\ntechnology which allows to different components isolated into their respective environments.\n\n- To install Docker on Windows follow the instructions [here](https://docs.docker.com/docker-for-windows/)\n- To install Docker on Mac follow the instructions [here](https://docs.docker.com/docker-for-mac/)\n- To install Docker on Linux follow the instructions [here](https://docs.docker.com/install/)\n\n**Docker Compose** is a tool for defining and running multi-container Docker applications. A\n[YAML file](https://raw.githubusercontent.com/Fiware/tutorials.Linked-Data/master/docker-compose/orion-ld.yml) is used\nconfigure the required services for the application. This means all container services can be brought up in a single\ncommand. Docker Compose is installed by default as part of Docker for Windows and Docker for Mac, however Linux users\nwill need to follow the instructions found [here](https://docs.docker.com/compose/install/)\n\n# Architecture\n\nThe demo application will send and receive NGSI-LD calls within a data space. This demo application will only make use\nof three FIWARE components.\n\nCurrently, both Orion and Orion-LD Context Brokers rely on open source [MongoDB](https://www.mongodb.com/) technology to\nkeep persistence of the context data they hold. Therefore, the architecture will consist of two elements:\n\n- The [Smart Farm Context Broker](https://fiware-orion.readthedocs.io/en/latest/) which will receive requests using\n [NGSI-LD](https://forge.etsi.org/swagger/ui/?url=https://forge.etsi.org/rep/NGSI-LD/NGSI-LD/raw/master/spec/updated/generated/full_api.json)\n- The [Smart Supermarket Context Broker](https://fiware-orion.readthedocs.io/en/latest/) which will receive requests\n using\n [NGSI-v2](https://forge.etsi.org/swagger/ui/?url=https://forge.etsi.org/rep/NGSI-LD/NGSI-LD/raw/master/spec/updated/generated/full_api.json)\n- Two instances of underlying [MongoDB](https://www.mongodb.com/) database\n - Used by both NGSI-v2 and NGSI-LD Context Brokers to hold context data information such as data entities,\n subscriptions and registrations\n- The FIWARE [Lepus](https://github.com/jason-fox/lepus) proxy which will translate NGSI-LD to NGSI-v2 and vice-versa\n- An HTTP **Web-Server** which offers static `@context` files defining the context entities within the system.\n\nSince all interactions between the two elements are initiated by HTTP requests, the elements can be containerized and\nrun from exposed ports.\n\n![](https://fiware.github.io/tutorials.Linked-Data/img/architecture-lepus.png)\n\nThe necessary configuration information can be seen in the services section of the associated `common.yml` and\n`orion-ld.yml` files:\n\n#### Orion-LD (NGSI-LD)\n\n```\norion-ld:\n image: quay.io/fiware/orion-ld\n hostname: orion\n container_name: fiware-orion\n depends_on:\n - mongo-db\n networks:\n - default\n ports:\n - \"1026:1026\"\n command: -dbhost mongo-db -logLevel DEBUG\n healthcheck:\n test: curl --fail -s http://orion:1026/version || exit 1\n```\n\n#### Orion (NGSI-v2)\n\n```\norion:\n image: quay.io/fiware/orion\n hostname: orion2\n container_name: fiware-orion-v2\n depends_on:\n - mongo-for-orion-v2\n networks:\n - default\n ports:\n - \"1027:1026\" # localhost:1026\n command: -dbURI mongodb://mongo-db2 -logLevel DEBUG\n healthcheck:\n test: curl --fail -s http://orion2:1026/version || exit 1\n```\n\n#### Lepus\n\n```\nlepus:\n image: quay.io/fiware/lepus\n hostname: adapter\n container_name: fiware-lepus\n networks:\n - default\n expose:\n - \"3005\"\n ports:\n - \"3005:3000\"\n environment:\n - DEBUG=adapter:*\n - NGSI_V2_CONTEXT_BROKER=http://orion2:1026\n - USER_CONTEXT_URL=http://context/fixed-context.jsonld\n - CORE_CONTEXT_URL=https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context-v1.8.jsonld\n - NOTIFICATION_RELAY_URL=http://adapter:3000/notify\n```\n\nAll containers are residing on the same network - the Orion-LD Context Broker is listening on Port `1026` and the first\nMongoDB is listening on the default port `27017`. The Orion NGSI-v2 Context Broker is listening on Port `1027` and\nsecond MongoDB is listening on port `27018`. Lepus uses port `3000`, but since that clashes with the tutorial\napplication, it has been amended externally to `3005`.\n\nLepus is driven by the environment variables as shown:\n\n| Key | Value | Description |\n| ---------------------- | ------------------------------------------------------------------ | ---------------------------------------------------------------------------------- |\n| DEBUG | `adapter:*` | Debug flag used for logging |\n| NGSI_V2_CONTEXT_BROKER | `http://orion2:1026` | Hostname and port of the underlying NGSI-v2 context broker |\n| USER_CONTEXT_URL | `http://context/fixed-context.jsonld` | The location of the User-defined `@context` file used to define the data models |\n| CORE_CONTEXT_URL | `https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context-v1.8.jsonld` | The location of the core `@context` file used to the structure of NGSI-LD entities |\n| NOTIFICATION_RELAY_URL | `http://adapter:3000/notify` | Callback URL used for converting notification payloads from NGSI-v2 to NGSI-LD |\n\n# Start Up\n\nAll services can be initialised from the command-line by running the\n[services](https://github.com/FIWARE/tutorials.Linked-Data/blob/NGSI-LD/services) Bash script provided within the\nrepository. Please clone the repository and create the necessary images by running the commands as shown:\n\n```\ngit clone https://github.com/FIWARE/tutorials.Linked-Data.git\ncd tutorials.Linked-Data\ngit checkout NGSI-LD\n\n./services orion|scorpio|stellio\n```\n\n>\n> If you want to clean up and start over again you can do so with the following command:\n>\n> ```\n> ./services stop\n> ```\n\n---", + "description": "This tutorial introduces the idea of using an existing **NGSI-v2** Context Broker as a Context Source within an\n**NGSI-LD** data space, and how to combine both JSON-based and JSON-LD based context data into a unified structure\nthrough introducing a simple **NGSI-LD** façade pattern with a fixed `@context`. The tutorial re-uses the data from the\noriginal **NGSI-v2** getting started tutorial but uses API calls from the **NGSI-LD** interface.\n\n\n> This tutorial is designed for exisiting **NGSI-v2** developers looking to attach existing **NGSI-v2** systems to an\n> **NGSI-LD** federation. If you are building a linked data system from scratch you should start from the beginnning of\n> the [NGSI-LD developers tutorial](https://ngsi-ld-tutorials.readthedocs.io/) documentation.\n>\n> Similarly, if you an existing **NGSI-v2** developer and you just want to understand linked data concepts in general,\n> checkout the equivalent [**NGSI-v2** tutorial](https://github.com/FIWARE/tutorials.Linked-Data/tree/NGSI-v2) on\n> upgrading **NGSI-v2** data sources to **NGSI-LD**\n\n## Contents\n\n# Adding Linked Data concepts to FIWARE Data Entities.\n\n> “Always be a first rate version of yourself and not a second rate version of someone else.”\n>\n> ― Judy Garland\n\nThe first introduction to FIWARE [Getting Started tutorial](https://github.com/FIWARE/tutorials.Getting-Started)\nintroduced the [NGSI v2](https://fiware.github.io/specifications/OpenAPI/ngsiv2) JSON-based interface that is commonly\nused to create and manipulate context data entities.\n[NGSI-LD](https://forge.etsi.org/swagger/ui/?url=https://forge.etsi.org/rep/NGSI-LD/NGSI-LD/raw/master/spec/updated/generated/full_api.json)\nis an evolution of that interface which enhances context data entities through adding the concept of **linked data**.\n\nThere is a need for two separate interfaces since:\n\n- **NGSI-v2** offers [JSON](https://www.json.org/json-en.html) based interoperability used in individual Smart Systems\n- **NGSI-LD** offers [JSON-LD](https://json-ld.org/) based interoperability used for Federations and Data Spaces\n\n**NGSI-v2** is ideal for creating individual applications offering interoperable interfaces for web services or IoT\ndevices. It is easier to understand than NGSI-LD and does not require a JSON-LD `@context`, However **NGSI-v2** without\nlinked data falls short when attempting to offer federated data across a data space. Once multiple apps and\norganisations are involved each individual data owner is no longer in control of the structure of the data used\ninternally by the other participants within the data space. This is where the `@context` of **NGSI-LD** comes in, acting\nas a mapping mechanism for attributes allowing the each local system to continue to use its own preferred terminology\nwithin the data it holds and for federated data from other users within the data space to be renamed using a standard\nexpansion/compaction operation allowing each individual system understand data holistically from across the whole data\nspace.\n\n## Creating a common data space\n\nFor example, imagine a simple \"Buildings\" data space consisting of two participants pooling their data together:\n\n- Details of a series of [supermarkets](https://fiware-tutorials.readthedocs.io/en/latest/) from the **NGSI-v2**\n tutorials.\n- Details of a series of [farms](https://ngsi-ld-tutorials.readthedocs.io/en/latest/) from the **NGSI-LD** tutorials.\n\nAlthough the two participants have agreed to use a common\n[data model](https://ngsi-ld-tutorials.readthedocs.io/en/latest/datamodels.html#building) between them, internally they\nhold their data in a slightly different structure.\n\n#### Farm (**NGSI-LD** Data)\n\nWithin the **NGSI-LD** Smart Farm, all of the Building Entities are marked using `\"type\":\"Building\"` as a shortname\nwhich can be expanded to a full URI https://uri.fiware.org/ns/dataModels#Building` using JSON-LD expansion rules. All\nthe attributes of each Building entity (such as `name`, `category` etc are defined using the\n[User `@context`](./data-models/ngsi-context.jsonld) and are structured as NGSI-LD attributes. The standard NGSI-LD\nKeywords are used to define the nature of each attribute - e.g. `Property`, `GeoProperty`, `VocabProperty`,\n`Relationship` and each of these terms is also defined within the\n[core `@context`](https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context-v1.8.jsonld).\n\n```\n{\n \"id\": \"urn:ngsi-ld:Building:farm001\",\n \"type\": \"Building\",\n \"category\": {\n \"type\": \"VocabProperty\",\n \"vocab\": \"farm\"\n },\n \"address\": {\n \"type\": \"Property\",\n \"value\": {\n \"streetAddress\": \"Großer Stern 1\",\n \"addressRegion\": \"Berlin\",\n \"addressLocality\": \"Tiergarten\",\n \"postalCode\": \"10557\"\n }\n },\n \"location\": {\n \"type\": \"GeoProperty\",\n \"value\": {\n \"type\": \"Point\",\n \"coordinates\": [13.3505, 52.5144]\n }\n },\n \"name\": {\n \"type\": \"Property\",\n \"value\": \"Victory Farm\"\n },\n \"owner\": {\n \"type\": \"Relationship\",\n \"object\": \"urn:ngsi-ld:Person:person001\"\n }\n}\n```\n\n#### Supermarket (**NGSI-v2** Data)\n\nWithin the **NGSI-v2** Smart Supermarket, every entity must have a `id` and `type` - this is part of the data model and\na prerequisite for joining the data space. However due to the backend systems used, the internal short name is\n`\"type\":\"Store\"`. Within an **NGSI-v2** system there is no concept of `@context` - every attribute has a `type` and a\n`value` and the attribute `type` is usually aligned with the datatype (e.g. `Text`, `PostalAddress`) although since\n**NGSI-LD** keyword types such as `Relationship`, `VocabProperty` are also permissible and set by convention.\n\n```\n{\n \"id\": \"urn:ngsi-ld:Store:001\",\n \"type\": \"Store\",\n \"address\": {\n \"type\": \"PostalAddress\",\n \"value\": {\n \"streetAddress\": \"Bornholmer Straße 65\",\n \"addressRegion\": \"Berlin\",\n \"addressLocality\": \"Prenzlauer Berg\",\n \"postalCode\": \"10439\"\n }\n },\n \"category\": {\n \"type\": \"VocabProperty\",\n \"value\": \"supermarket\"\n },\n \"location\": {\n \"type\": \"geo:json\",\n \"value\": {\n \"type\": \"Point\",\n \"coordinates\": [13.3986, 52.5547]\n }\n },\n \"name\": {\n \"type\": \"Text\",\n \"value\": \"Bösebrücke Einkauf\"\n },\n \"owner\": {\n \"type\": \"Relationship\",\n \"value\": \"urn:ngsi-ld:Person:person001\"\n }\n}\n```\n\n### Types of data space\n\nWhen joining a data space, a participant must abide by the rules that govern that data space. One of the first decisions\na common data space must make is to defined the nature of the data space itself. There are three primary approaches to\nthis, which can broadly be defined as follows:\n\n- An **Integrated** data space requires that every participant uses exactly the same payloads and infrastructure - for\n example _\"Use Scorpio 4.1.15 only\"_ . This could be a requirement for a lottery ticketing system where every\n terminal sends ticket data in the same format.\n- A **unified** data space defines a common data format, but allows for the underlying infrastructure to differ\n between participants - for example _\"Use NGSI-LD only for all payloads\"_\n- A **federated** data space is even looser and defines a common meta structure so each participants has more\n flexibility regarding it underlying technologies for example \"All payloads must be translatable as GeoJSON\"\\_ for a\n combined GIS, NGSI-LD data space.\n\nUsing this terminology, in this example we are creating a **unified** data space in this example, since participants are\nusing NGSI-LD in common for data exchange, but their underlying systems are different.\n\n# Prerequisites\n\n## Docker\n\nTo keep things simple all components will be run using [Docker](https://www.docker.com). **Docker** is a container\ntechnology which allows to different components isolated into their respective environments.\n\n- To install Docker on Windows follow the instructions [here](https://docs.docker.com/docker-for-windows/)\n- To install Docker on Mac follow the instructions [here](https://docs.docker.com/docker-for-mac/)\n- To install Docker on Linux follow the instructions [here](https://docs.docker.com/install/)\n\n**Docker Compose** is a tool for defining and running multi-container Docker applications. A\n[YAML file](https://raw.githubusercontent.com/Fiware/tutorials.Linked-Data/master/docker-compose/orion-ld.yml) is used\nconfigure the required services for the application. This means all container services can be brought up in a single\ncommand. Docker Compose is installed by default as part of Docker for Windows and Docker for Mac, however Linux users\nwill need to follow the instructions found [here](https://docs.docker.com/compose/install/)\n\n# Architecture\n\nThe demo application will send and receive NGSI-LD calls within a data space. This demo application will only make use\nof three FIWARE components.\n\nCurrently, both Orion and Orion-LD Context Brokers rely on open source [MongoDB](https://www.mongodb.com/) technology to\nkeep persistence of the context data they hold. Therefore, the architecture will consist of two elements:\n\n- The [Smart Farm Context Broker](https://fiware-orion.readthedocs.io/en/latest/) which will receive requests using\n [NGSI-LD](https://forge.etsi.org/swagger/ui/?url=https://forge.etsi.org/rep/NGSI-LD/NGSI-LD/raw/master/spec/updated/generated/full_api.json)\n- The [Smart Supermarket Context Broker](https://fiware-orion.readthedocs.io/en/latest/) which will receive requests\n using\n [NGSI-v2](https://forge.etsi.org/swagger/ui/?url=https://forge.etsi.org/rep/NGSI-LD/NGSI-LD/raw/master/spec/updated/generated/full_api.json)\n- Two instances of underlying [MongoDB](https://www.mongodb.com/) database\n - Used by both NGSI-v2 and NGSI-LD Context Brokers to hold context data information such as data entities,\n subscriptions and registrations\n- The FIWARE [Lepus](https://github.com/jason-fox/lepus) proxy which will translate NGSI-LD to NGSI-v2 and vice-versa\n- An HTTP **Web-Server** which offers static `@context` files defining the context entities within the system.\n\nSince all interactions between the two elements are initiated by HTTP requests, the elements can be containerized and\nrun from exposed ports.\n\n![](https://fiware.github.io/tutorials.Linked-Data/img/architecture-lepus.png)\n\nThe necessary configuration information can be seen in the services section of the associated `common.yml` and\n`orion-ld.yml` files:\n\n#### Orion-LD (NGSI-LD)\n\n```\norion-ld:\n image: quay.io/fiware/orion-ld\n hostname: orion\n container_name: fiware-orion\n depends_on:\n - mongo-db\n networks:\n - default\n ports:\n - \"1026:1026\"\n command: -dbhost mongo-db -logLevel DEBUG\n healthcheck:\n test: curl --fail -s http://orion:1026/version || exit 1\n```\n\n#### Orion (NGSI-v2)\n\n```\norion:\n image: quay.io/fiware/orion\n hostname: orion2\n container_name: fiware-orion-v2\n depends_on:\n - mongo-for-orion-v2\n networks:\n - default\n ports:\n - \"1027:1026\" # localhost:1026\n command: -dbURI mongodb://mongo-db2 -logLevel DEBUG\n healthcheck:\n test: curl --fail -s http://orion2:1026/version || exit 1\n```\n\n#### Lepus\n\n```\nlepus:\n image: quay.io/fiware/lepus\n hostname: adapter\n container_name: fiware-lepus\n networks:\n - default\n expose:\n - \"3005\"\n ports:\n - \"3005:3000\"\n environment:\n - DEBUG=adapter:*\n - NGSI_V2_CONTEXT_BROKER=http://orion2:1026\n - USER_CONTEXT_URL=http://context/fixed-context.jsonld\n - CORE_CONTEXT_URL=https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context-v1.8.jsonld\n - NOTIFICATION_RELAY_URL=http://adapter:3000/notify\n```\n\nAll containers are residing on the same network - the Orion-LD Context Broker is listening on Port `1026` and the first\nMongoDB is listening on the default port `27017`. The Orion NGSI-v2 Context Broker is listening on Port `1027` and\nsecond MongoDB is listening on port `27018`. Lepus uses port `3000`, but since that clashes with the tutorial\napplication, it has been amended externally to `3005`.\n\nLepus is driven by the environment variables as shown:\n\n| Key | Value | Description |\n| ---------------------- | ------------------------------------------------------------------ | ---------------------------------------------------------------------------------- |\n| DEBUG | `adapter:*` | Debug flag used for logging |\n| NGSI_V2_CONTEXT_BROKER | `http://orion2:1026` | Hostname and port of the underlying NGSI-v2 context broker |\n| USER_CONTEXT_URL | `http://context/fixed-context.jsonld` | The location of the User-defined `@context` file used to define the data models |\n| CORE_CONTEXT_URL | `https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context-v1.8.jsonld` | The location of the core `@context` file used to the structure of NGSI-LD entities |\n| NOTIFICATION_RELAY_URL | `http://adapter:3000/notify` | Callback URL used for converting notification payloads from NGSI-v2 to NGSI-LD |\n\n# Start Up\n\nAll services can be initialised from the command-line by running the\n[services](https://github.com/FIWARE/tutorials.Linked-Data/blob/NGSI-LD/services) Bash script provided within the\nrepository. Please clone the repository and create the necessary images by running the commands as shown:\n\n```\ngit clone https://github.com/FIWARE/tutorials.Linked-Data.git\ncd tutorials.Linked-Data\ngit checkout NGSI-LD\n\n./services orion|scorpio|stellio\n```\n\n>\n> If you want to clean up and start over again you can do so with the following command:\n>\n> ```\n> ./services stop\n> ```\n\n---", "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json", "_exporter_id": "513743", "_collection_link": "https://fiware.postman.co/workspace/NGSI-LD-tutorials~60157bf9-a93f-4c6e-90d1-0917133c1872/collection/513743-16121a01-e085-4afd-8689-fea63a0d7246?action=share&source=collection_link&creator=513743" @@ -28,7 +28,7 @@ "urn:ngsi-ld:Store:001" ] }, - "description": "As usual, you can check if NGSI-v2 context broker holding the supermarket data is running by making an HTTP request to the exposed port, the request is in NGSI-v2 format. It does not require a `Link` header for the `@context`\n\nThe response will be in NGSI-v2 format.\n\nThe `type` attribute in NGSI-v2 is loosely defined, but in this case, with the exception of ordinary properties, we are using `type` to correspond to the terms used in NGSI-LD such as `Relationship` or `VocabularyProperty`. For ordinary NGSI-v2 properties, the `type` corresponds to a datatype such as `Text` or `PostalAddress`, each of these datatypes will need to be mapped to a JSON-LD `@context` if the data is to be understood in an NGSI-LD system." + "description": "As usual, you can check if NGSI-v2 context broker holding the supermarket data is running by making an HTTP request to the exposed port, the request is in NGSI-v2 format. It does not require a `Link` header for the `@context`\n\nThe response will be in NGSI-v2 format.\n\nThe `type` attribute in NGSI-v2 is loosely defined, but in this case, with the exception of ordinary properties, we are using `type` to correspond to the terms used in NGSI-LD such as `Relationship` or `VocabProperty`. For ordinary NGSI-v2 properties, the `type` corresponds to a datatype such as `Text` or `PostalAddress`, each of these datatypes will need to be mapped to a JSON-LD `@context` if the data is to be understood in an NGSI-LD system." }, "response": [] }, @@ -61,7 +61,7 @@ "urn:ngsi-ld:Store:001" ] }, - "description": "To read the data in NGSI-LD format, make an NGSI-LD request via the Lepus adaptor listening on port `` 3005` ``\n\nThe response will be in NGSI-LD format.\n\nIt should be noted that the `@context` supplied in the `Link` header has been totally ignored, and a fixed `@context` used configured for the adaptor has been returned instead. The Adaptor doesn't fully understand NGSI-LD, it merely formats the underlying NGSI-v2 as NGSI-LD. You will notice that the NGSI-v2 `\"type\": \"VocabularyProperty\", \"value\": \"supermarket\"` has been amended to a valid **VocabularyProperty** - `\"type\": \"VocabularyProperty\", \"vocab\": \"supermarket\"`, and similarly the **Relationship** is now using `object` and `objectType` as defined in the core `@context`" + "description": "To read the data in NGSI-LD format, make an NGSI-LD request via the Lepus adaptor listening on port `` 3005` ``\n\nThe response will be in NGSI-LD format.\n\nIt should be noted that the `@context` supplied in the `Link` header has been totally ignored, and a fixed `@context` used configured for the adaptor has been returned instead. The Adaptor doesn't fully understand NGSI-LD, it merely formats the underlying NGSI-v2 as NGSI-LD. You will notice that the NGSI-v2 `\"type\": \"VocabProperty\", \"value\": \"supermarket\"` has been amended to a valid **VocabProperty** - `\"type\": \"VocabProperty\", \"vocab\": \"supermarket\"`, and similarly the **Relationship** is now using `object` and `objectType` as defined in the core `@context`" }, "response": [] }, @@ -263,7 +263,7 @@ "urn:ngsi-ld:Store:001" ] }, - "description": "Once a registration is in place, it is possible to read information about the NGSI-v2 Stores by querying the **NGSI-LD** FMIS system.\n\nBecause of `Prefer: ngsi-ld=1.6` had been set in the registration, the retrieved entity does not a **VocabularyProperty** and `objectType` has been quietly dropped from the response. This ensures backwards compatibility to context brokers conformant to an earlier version of the NGSI-LD specification - in this case version 1.6.\n" + "description": "Once a registration is in place, it is possible to read information about the NGSI-v2 Stores by querying the **NGSI-LD** FMIS system.\n\nBecause of `Prefer: ngsi-ld=1.6` had been set in the registration, the retrieved entity does not a **VocabProperty** and `objectType` has been quietly dropped from the response. This ensures backwards compatibility to context brokers conformant to an earlier version of the NGSI-LD specification - in this case version 1.6.\n" }, "response": [] }, diff --git a/README.md b/README.md index 63c0016..8ddece4 100644 --- a/README.md +++ b/README.md @@ -586,7 +586,7 @@ Because of `Prefer: ngsi-ld=1.6` had been set in the registration, the retrieved "postalCode": "10439" } }, - "category": {"type": "Property", "value": "supermarket"}, + "category": {"type": "VocabProperty", "vocab": "supermarket"}, "location": { "type": "GeoProperty", "value": { @@ -705,4 +705,4 @@ The response is returned in JSON format with short form attribute names in Germa ## License -[MIT](LICENSE) © 2024 FIWARE Foundation e.V. +[MIT](LICENSE) © 2024-2025 FIWARE Foundation e.V. diff --git a/docker-compose/stellio.yml b/docker-compose/stellio.yml index ae71473..e160553 100644 --- a/docker-compose/stellio.yml +++ b/docker-compose/stellio.yml @@ -22,11 +22,11 @@ services: hostname: stellio labels: org.fiware: 'tutorial' - image: stellio/stellio-api-gateway:${STELLIO_DOCKER_TAG} + image: quay.io/fiware/stellio-api-gateway:${STELLIO_DOCKER_TAG} environment: - SPRING_PROFILES_ACTIVE=docker ports: - - ${EXPOSED_PORT:-1026}:${STELLIO_PORT:-9090} + - "${EXPOSED_PORT}:${STELLIO_PORT}" networks: - default @@ -34,7 +34,7 @@ services: container_name: stellio-search-service labels: org.fiware: 'tutorial' - image: stellio/stellio-search-service:${STELLIO_DOCKER_TAG} + image: quay.io/fiware/stellio-search-service:${STELLIO_DOCKER_TAG} environment: - SPRING_PROFILES_ACTIVE=docker - SPRING_R2DBC_URL=r2dbc:postgresql://postgres/stellio_search @@ -48,6 +48,18 @@ services: - APPLICATION_TENANTS_1_NAME=openiot - APPLICATION_TENANTS_1_ISSUER=https://sso.stellio.io/auth/realms/openiot - APPLICATION_TENANTS_1_DBSCHEMA=openiot + - APPLICATION_TENANTS_2_NAME=farmer + - APPLICATION_TENANTS_2_ISSUER=https://sso.stellio.io/auth/realms/farmer + - APPLICATION_TENANTS_2_DBSCHEMA=farmer + - APPLICATION_TENANTS_3_NAME=vet + - APPLICATION_TENANTS_3_ISSUER=https://sso.stellio.io/auth/realms/vet + - APPLICATION_TENANTS_3_DBSCHEMA=vet + - APPLICATION_TENANTS_4_NAME=contractor + - APPLICATION_TENANTS_4_ISSUER=https://sso.stellio.io/auth/realms/contractor + - APPLICATION_TENANTS_4_DBSCHEMA=contractor + - APPLICATION_TENANTS_5_NAME=weather + - APPLICATION_TENANTS_5_ISSUER=https://sso.stellio.io/auth/realms/weather + - APPLICATION_TENANTS_5_DBSCHEMA=weather - APPLICATION_PAGINATION_LIMIT-DEFAULT=30 - APPLICATION_PAGINATION_LIMIT-MAX=1000 ports: @@ -65,7 +77,7 @@ services: container_name: stellio-subscription-service labels: org.fiware: 'tutorial' - image: stellio/stellio-subscription-service:${STELLIO_DOCKER_TAG} + image: quay.io/fiware/stellio-subscription-service:${STELLIO_DOCKER_TAG} environment: - SPRING_PROFILES_ACTIVE=docker - SPRING_R2DBC_URL=r2dbc:postgresql://postgres/stellio_subscription @@ -79,6 +91,18 @@ services: - APPLICATION_TENANTS_1_NAME=openiot - APPLICATION_TENANTS_1_ISSUER=https://sso.stellio.io/auth/realms/openiot - APPLICATION_TENANTS_1_DBSCHEMA=openiot + - APPLICATION_TENANTS_2_NAME=farmer + - APPLICATION_TENANTS_2_ISSUER=https://sso.stellio.io/auth/realms/farmer + - APPLICATION_TENANTS_2_DBSCHEMA=farmer + - APPLICATION_TENANTS_3_NAME=vet + - APPLICATION_TENANTS_3_ISSUER=https://sso.stellio.io/auth/realms/vet + - APPLICATION_TENANTS_3_DBSCHEMA=vet + - APPLICATION_TENANTS_4_NAME=contractor + - APPLICATION_TENANTS_4_ISSUER=https://sso.stellio.io/auth/realms/contractor + - APPLICATION_TENANTS_4_DBSCHEMA=contractor + - APPLICATION_TENANTS_5_NAME=weather + - APPLICATION_TENANTS_5_ISSUER=https://sso.stellio.io/auth/realms/weather + - APPLICATION_TENANTS_5_DBSCHEMA=weather - APPLICATION_PAGINATION_LIMIT-DEFAULT=30 - APPLICATION_PAGINATION_LIMIT-MAX=1000 ports: @@ -95,7 +119,7 @@ services: kafka: labels: org.fiware: 'tutorial' - image: confluentinc/cp-kafka:7.3.1 + image: confluentinc/cp-kafka:7.6.0 container_name: kafka ports: - 29092:29092 @@ -114,9 +138,7 @@ services: KAFKA_INTER_BROKER_LISTENER_NAME: 'PLAINTEXT' KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER' KAFKA_LOG4J_ROOT_LOGLEVEL: INFO - volumes: - - ./stellio/kafka/update_run.sh:/tmp/update_run.sh - command: "bash -c 'if [ ! -f /tmp/update_run.sh ]; then echo \"ERROR: Did you forget the update_run.sh file that came with this docker-compose.yml file?\" && exit 1 ; else /tmp/update_run.sh && /etc/confluent/docker/run ; fi'" + CLUSTER_ID: NjExODZhMWVjMzllMTFlZm # Databases @@ -124,7 +146,6 @@ services: labels: org.fiware: 'tutorial' image: stellio/stellio-timescale-postgis:${STELLIO_TIMESCALE_POSTGIS} - hostname: postgres container_name: db-postgres environment: @@ -145,8 +166,7 @@ services: timeout: 5s retries: 20 start_period: 10s - - + # Tutorial acts as a series of dummy IoT Sensors over HTTP and connects to the Stellio Broker tutorial: environment: diff --git a/docker-compose/stellio/kafka/update_run.sh b/docker-compose/stellio/kafka/update_run.sh deleted file mode 100755 index 18de790..0000000 --- a/docker-compose/stellio/kafka/update_run.sh +++ /dev/null @@ -1,11 +0,0 @@ -#!/bin/sh - -# Docker workaround: Remove check for KAFKA_ZOOKEEPER_CONNECT parameter -sed -i '/KAFKA_ZOOKEEPER_CONNECT/d' /etc/confluent/docker/configure - -# Docker workaround: Ignore cub zk-ready -sed -i 's/cub zk-ready/echo ignore zk-ready/' /etc/confluent/docker/ensure - -# KRaft required step: Format the storage directory with a new cluster ID -echo "kafka-storage format --ignore-formatted -t $(kafka-storage random-uuid) -c /etc/kafka/kafka.properties" >> /etc/confluent/docker/ensure - diff --git a/import-data b/import-data index d74bc6e..0b7e006 100755 --- a/import-data +++ b/import-data @@ -21,7 +21,7 @@ curl -s -o /dev/null -X POST 'http://'"${CONTEXT_BROKER}"'/ngsi-ld/v1/entityOper { "id": "urn:ngsi-ld:Building:farm001", "type": "Building", - "category": {"type": "Property", "value": ["farm"]}, + "category": {"type": "VocabProperty", "vocab": ["farm"]}, "address": { "type": "Property", "value": { "streetAddress": "Großer Stern 1", "addressRegion": "Berlin", "addressLocality": "Tiergarten", "postalCode": "10557"}, @@ -37,7 +37,7 @@ curl -s -o /dev/null -X POST 'http://'"${CONTEXT_BROKER}"'/ngsi-ld/v1/entityOper { "id": "urn:ngsi-ld:Building:barn002", "type": "Building", - "category": {"type": "Property", "value": ["barn"]}, + "category": {"type": "VocabProperty", "vocab": ["barn"]}, "address": { "type": "Property", "value": { "streetAddress": "Straße des 17. Juni", "addressRegion": "Berlin", "addressLocality": "Tiergarten", "postalCode": "10557"}, @@ -53,7 +53,7 @@ curl -s -o /dev/null -X POST 'http://'"${CONTEXT_BROKER}"'/ngsi-ld/v1/entityOper { "id": "urn:ngsi-ld:Building:tower003", "type": "Building", - "category": {"type": "Property", "value": ["water_tower"]}, + "category": {"type": "VocabProperty", "vocab": ["water_tower"]}, "address": { "type": "Property", "value": { "streetAddress": "John-Foster-Dulles-Allee", "addressRegion": "Berlin", "addressLocality": "Tiergarten", "postalCode": "10557"}, @@ -69,7 +69,7 @@ curl -s -o /dev/null -X POST 'http://'"${CONTEXT_BROKER}"'/ngsi-ld/v1/entityOper { "id": "urn:ngsi-ld:Building:farm002", "type": "Building", - "category": {"type": "Property", "value": ["farm"]}, + "category": {"type": "VocabProperty", "vocab": ["farm"]}, "address": { "type": "Property", "value": { "streetAddress": "Hardenbergplatz 8", "addressRegion": "Berlin", "addressLocality": "Tiergarten", "postalCode": "10787"}, diff --git a/import-ngsi-v2 b/import-ngsi-v2 index 0277ee6..dda37ff 100755 --- a/import-ngsi-v2 +++ b/import-ngsi-v2 @@ -19,7 +19,7 @@ curl -o /dev/null -X POST \ "entities": [ { "id":"urn:ngsi-ld:Store:001","type":"Store", - "category": {"type": "VocabularyProperty", "value": "supermarket"}, + "category": {"type": "Property", "value": "supermarket"}, "address":{"type":"PostalAddress","value":{"streetAddress":"Bornholmer Straße 65","addressRegion":"Berlin","addressLocality":"Prenzlauer Berg","postalCode":"10439"}}, "location":{"type":"geo:json","value":{"type":"Point","coordinates":[13.3986,52.5547]}}, "name":{"type":"Text","value":"Bösebrücke Einkauf"}, @@ -27,27 +27,27 @@ curl -o /dev/null -X POST \ "type":"Relationship", "value":"urn:ngsi-ld:Person:001", "metadata": { "objectType": { - "type":"VocabularyProperty", "value": "Person" + "type":"Property", "value": "Person" } } } }, { "id":"urn:ngsi-ld:Store:002","type":"Store", - "category": {"type": "VocabularyProperty", "value": "supermarket"}, + "category": {"type": "Property", "value": "supermarket"}, "address":{"type":"PostalAddress","value":{"streetAddress":"Friedrichstraße 44","addressRegion":"Berlin","addressLocality":"Kreuzberg","postalCode":"10969"}}, "location":{"type":"geo:json","value":{"type":"Point","coordinates":[13.3903,52.5075]}}, "name":{"type":"Text","value":"Checkpoint Markt"}}, { "id":"urn:ngsi-ld:Store:003","type":"Store", - "category": {"type": "VocabularyProperty", "value": "supermarket"}, + "category": {"type": "Property", "value": "supermarket"}, "address":{"type":"PostalAddress","value":{"streetAddress":"Mühlenstrasse 10","addressRegion":"Berlin","addressLocality":"Friedrichshain","postalCode":"10243"}}, "location":{"type":"geo:json","value":{"type":"Point","coordinates":[13.4447,52.5031]}}, "name":{"type":"Text","value":"East Side Galleria"} }, { "id":"urn:ngsi-ld:Store:004","type":"Store", - "category": {"type": "VocabularyProperty", "value": "supermarket"}, + "category": {"type": "Property", "value": "supermarket"}, "address":{"type":"PostalAddress","value":{"streetAddress":"Panoramastraße 1A","addressRegion":"Berlin","addressLocality":"Mitte","postalCode":"10178"}}, "location":{"type":"geo:json","value":{"type":"Point","coordinates":[13.4094,52.5208]}}, "name":{"type":"Text","value":"Tower Trödelmarkt"} diff --git a/services b/services index 6ca7c79..bd6ea8e 100755 --- a/services +++ b/services @@ -207,7 +207,7 @@ case "${command}" in export $(cat .env | grep "#" -v) stoppingContainers waitForCoreContext - echo -e "Starting containers: \033[1;34mStellio (NGSI-LD)\033[0m, \033[1;34mOrion (NGSI-v2)\033[0m, \033[1;34mLepus\033[0m, \033[1mTutorial\033[0m, \033[1mKafka\033[0m, \033[1mZookeeper\033[0m, \033[1mNeo4j\033[0m, \033[1mPostgreSQL/TimescaleDB\033[0m, and a linked data \033[1mContext\033[0m." + echo -e "Starting containers: \033[1;34mStellio (NGSI-LD)\033[0m, \033[1;34mOrion (NGSI-v2)\033[0m, \033[1;34mLepus\033[0m, \033[1mTutorial\033[0m, \033[1mKafka\033[0m \033[1mPostgreSQL/TimescaleDB\033[0m, and a linked data \033[1mContext\033[0m." echo -e "- \033[1;34mStellio\033[0m is the NGSI-LD context broker" echo -e "- \033[1;34mOrion\033[0m is an NGSI-v2 context source" echo -e "- Data models \033[1m@context\033[0m (Smart Supermarket) is supplied externally"