Merge branch 'master' into

512-improve-configuration-of-the-knowledge-engine

README.md

@@ -182,7 +182,7 @@ This section gives more detailed information about the project's structure, and
 
 The Knowledge Engine project consists of the following Maven modules:
 - `smart-connector`
-  - This is the implementation of the smart connector, with the Java developer API. For instructions on how to use it, refer to [the documentation](./docs/docs/java_developer_api.md).
+  - This is the implementation of the smart connector, with the Java developer API. For instructions on how to use it, refer to [the documentation](./docs/docs/getting_started.md).
 - `smart-connector-api`
 	- This module contains interfaces for the smart connector and other classes. It is made as a separate module so that it is easy to use different implementations of the interfaces.
 - `smart-connector-rest-server`

admin-ui/pom.xml

@@ -15,10 +15,10 @@
 	<description>A user interface for managing a Knowledge Network.</description>
 
 	<properties>
-		<swagger-core-version>2.2.22</swagger-core-version>
-		<jetty-version>11.0.15</jetty-version>
+		<swagger-core-version>2.2.26</swagger-core-version>
+		<jetty-version>11.0.24</jetty-version>
 		<jersey3-version>3.1.9</jersey3-version>
-		<jackson-version>2.18.1</jackson-version>
+		<jackson-version>2.18.2</jackson-version>
 		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
 	</properties>
 
@@ -52,7 +52,7 @@
 		<dependency>
 			<groupId>org.slf4j</groupId>
 			<artifactId>slf4j-simple</artifactId>
-			<version>2.0.13</version>
+			<version>2.0.16</version>
 		</dependency>
 
 

docs/docs/common_error_messages.md

@@ -0,0 +1,41 @@
+---
+  sidebar_position: 9
+---
+# Common Error Messages
+
+On this page you can find a list of common error messages and what they mean.
+
+### `x` is not an unprefixed URI or literal
+Whenever you are specifying variable bindings, for example in a binding set when executing an ask, you may encounter this error.
+It occurs because all variable bindings need to be either RDF Literals or IRIs.
+If you want to identify a specific object or device `x`, you can use something like `<http://www.example.org/x>`.
+Note that if the variable binding is for a subject in a triple (?subject ?predicate ?object), then an IRI is always required.
+For more information on bindings and binding sets, see: [Bindings](https://docs.knowledge-engine.eu/java_developer_api#bindings).
+
+### There are lots of 'HTTP/1.1 header parser received no bytes' errors in the logs. How do I prevent these?
+This can be prevented by using a shorter timeout in the Java HTTP Client.
+Try using the following Java option: `-Djdk.httpclient.keepalive.timeout=3`.
+This can also be set in the docker configuration of a Knowledge Engine Runtime docker container by setting the JAVA_TOOL_OPTIONS as follows: `JAVA_TOOL_OPTIONS: "-Djdk.httpclient.keepalive.timeout=3"`.
+
+### java.lang.IllegalArgumentException: KB gave outgoing binding Binding [...], but this doesn't have a matching incoming binding!
+When using a REACT Knowledge Interaction that contains both an argument and result graph pattern which share a variable name, the Knowledge Engine expects all values for this variable in the result binding set to also occur as a value of the variable in the argument binding set.
+This has to do with how these graph patterns are internally used to form if … then … rules.
+If this is not the case, it gives the above error message.
+If this shared variable in the argument and result graph patterns is intended to be the same, make sure you align the values of this variable in the result binding set with the values of this variable in the argument binding set.
+If the variable is not intended to be the same thing, you can rename one of them to prevent the knowledge engine to expect them to share values.
+
+### The JSON Object should contain both a recipientSelector and a bindingSet key
+When executing an ASK or POST you have to provide either (a) a binding set, or (b) a recipient selector *and* binding set.
+The binding set specifies values that you are interested in.
+The recipient selector can be used to select a single Knowledge Base which should be contacted.
+In case you do not want to select a single Knowledge Base, you can pass an empty list to contact *all* relevant Knowledge Bases:
+```json
+{
+  "recipientSelector": {
+    "knowledgeBases": []
+  },
+  "bindingSet": [
+    {}
+  ]
+}
+```

docs/docs/get-started/_category_.json

@@ -1,8 +1,8 @@
 {
-  "label": "Tutorial",
+  "label": "Guides",
   "position": 6,
   "link": {
     "type": "generated-index",
-    "description": "In-depth details about how to use the various elements of the Knowledge Engine"
+    "description": "In-depth details about how to use the various elements of the Knowledge Engine."
   }
 }

docs/docs/get-started/graph-patterns.md

@@ -0,0 +1,136 @@
+---
+sidebar_position: 8
+---
+# Writing Graph Patterns
+This page discusses some good practices for writing graph patterns.
+
+## How to handle redundant variables if they are required in a graph pattern
+:::tip
+This is a common pitfall, so we recommend having a look at this section!
+:::
+
+Let's take the following graph pattern as an example, which defines a device which measures a property, and the type of that property.
+
+```sparql
+?device <https://saref.etsi.org/core/measuresProperty> ?property .
+?property a ?propertyType .
+```
+
+This graph pattern contains three variables: `device`, `property` and `propertyType`.
+What should you do when you are only interested in `device` and `propertyType`, but not in `property`?
+
+This case often occurs and a common pitfall is that people replace the `property` variable by a placeholder individual, 
+e.g. : `http://example.org/property#property`.
+The resulting pattern would then look like this:
+
+```sparql
+?device <https://saref.etsi.org/core/measuresProperty> <http://example.org/property#property> .
+<http://example.org/property#property> a ?propertyType .
+```
+
+However, this causes a problem. 
+Let's distinguish between syntax and semantics here.
+The graph pattern above is syntactically correct, and is accepted by the Knowledge Engine.
+However, there is some difficulty with the semantics that it expresses.
+
+Imagine the following binding set:
+
+```json
+{
+  "device": "<sensor1>",
+  "propertyType": "saref:Motion"
+},
+{
+  "device": "<sensor1>",
+  "propertyType": "saref:Smoke"
+},
+{
+  "device": "<sensor2>",
+  "propertyType": "saref:Energy"
+}
+```
+
+If we combine the above Graph Pattern with the above BindingSet, we get the following RDF triples:
+
+1. `<sensor1> <https://saref.etsi.org/core/measuresProperty> <http://example.org/property#property> .`
+2.   `<http://example.org/property#property> a saref:Motion .`
+3. `<sensor1> <https://saref.etsi.org/core/measuresProperty> <http://example.org/property#property> .`
+4. `<http://example.org/property#property> a saref:Smoke .`
+5. `<sensor2> <https://saref.etsi.org/core/measuresProperty> <http://example.org/property#property> .`
+6. `<http://example.org/property#property> a saref:Energy .`
+
+Now, if we draw these triples as a graph, we get something like:
+
+![Illustration of aforementioned RDF triples. It contains 2 nodes on the left, both connected to a central node, which is then connected to three nodes on the right.](./../../static/img/graph-pattern-pitfall.png)
+
+While we had 6 RDF triples, we only see 5 edges in this figure!
+This is caused by the fact that the 1st and 3rd triple of the graph pattern above are exactly the same and triples can only occur once in a graph.
+So, if you write them twice, they will still only occur once.
+So, by using a fixed individual for `property` in the graph pattern, it becomes semantically impossible to determine that `sensor1` is measuring the property Motion and Smoke, while sensor2 is measuring Energy!
+
+Therefore, while using a fixed property is syntactically correct, it is semantically incorrect.
+Using a fixed property in the current version of the KE (with a matcher instead of a reasoner) will probably work fine and the data is exchanged as expected.
+This is, however, probably not the case with the future version of the KE with reasoner, because the reasoner will actually semantically interpret the graph pattern and binding set and when you ask something like:
+
+  ```sparql
+  ?device <https://saref.etsi.org/core/measuresProperty> <http://interconnectproject.eu/pilots/greek/property#property> .
+  <http://interconnectproject.eu/pilots/greek/property#property> a saref:Energy .
+  ```
+  This graph pattern represents the query: “give me all devices that measure energy” and it will answer with the following bindingset:
+
+```json
+{
+  "device": "<sensor1>"
+},
+{
+  "device": "<sensor2>"
+}
+```
+
+Which is not correct and is caused by the fixed property.
+So, there are two solutions here.
+1. The practical one, which I think happens quite a lot, is for the ontology to provide an individual per property.
+So, you would have a `ex:motionIndividual`, `ex:energyIndividual` and `ex:smokeIndividual`.
+These can be used instead of the ex:property and will make sure that it remains semantically correct.
+2. A less practical (and philosophically debatable) one is to have a unique property individual for every property a device measures.
+So, you would get something like `<sensor1/individual/motion>`, `<sensor1/individual/smoke>` and `<sensor2/individual/energy>` and even more for all other devices.
+
+## Can you use volatile IRIs or should you use static or stored reference IRIs?
+Let's say we have some graph pattern like:
+
+```sparql
+?timeseries rdf:type ex:Timeseries .
+?timeseries ex:hasMeasurement ?measurement .
+?measurement rdf:type saref:Measurement .
+?measurement saref:hasFeatureOfInterest ?room .
+?room rdf:type saref:Room .
+?measurement saref:observedProperty saref:Temperature .
+?measurement saref:hasSimpleResult ?temperature .
+?measurement ex:hasTimestamp ?ts .
+```
+
+And the timeseries returns an array of temperature values and timestamp for each value.
+The binding set will be something like:
+```json
+[
+	{
+		"timeseries": "<https://www.example.org/timeseries-sensora-b-23>",
+		"measurement": "<https://www.example.org/measurement-42>",
+		"room": "<https://www.example.org/kitchen>",
+		"temperature": "\"21.2\"^^<http://www.w3.org/2001/XMLSchema#float>",
+		"ts": "\"2020-10-16T22:00Z\"^^some_timestamp_type"
+	},
+	{
+		"timeseries": "<https://www.example.org/timeseries-sensora-b-23>",
+		"measurement": "<https://www.example.org/measurement-43>",
+		"room": "<https://www.example.org/kitchen>",
+		"temperature": "\"21.4\"^^<http://www.w3.org/2001/XMLSchema#float>",
+		"ts": "\"2020-10-16T23:00Z\"^^some_timestamp_type"
+	}
+]
+```
+
+If you use a temporal or volatile IRI for the measurement, that would mean that you cannot retrieve more details for the object later (assuming an appropriate interaction exists).
+It is therefore recommended to use a stored or static IRI.
+In the case of measurements or observations, you could also choose an IRI like so: `<https://www.example.org/sensor/[sensor]/ts/[timestamp]>`, where you add the timestamp (e.g. in unix format) and a reference to the sensor doing the measurement.
+This makes it easy to later reconstruct the IRI when you want to request more info.

docs/docs/get-started/knowledge-base.md

@@ -0,0 +1,22 @@
+---
+sidebar_position: 99
+---
+# Implementing a Knowledge Base
+This page describes how to implement your own Knowledge Base given a data source.
+
+There are three approaches to implement your own Knowledge Base:
+1. Java
+2. REST Developer API
+3. Knowledge Mapper (based on a Python client)
+4. JavaScript client
+
+The Knowledge Mapper is a tool we have built to easily connect to several common data sources (SQL, RDF, APIs).
+If you're interested in using the Knowledge Mapper or JavaScript client, please reach out to us as they are not yet open source.
+
+## Implementing your own Knowledge Interaction
+When you receive a request for data via an ANSWER or REACT Knowledge Interaction, you should return the expected results, e.g. by retrieving data from an API.
+
+If you do not have a response, then you should send an empty binding set.
+Also, when your REACT Knowledge Interaction has no result graph pattern, you should always return an empty binding set to the Knowledge Engine.
+
+If an error occurs while responding, you can either return an empty BindingSet (although this does not give any information about the error occurring) or call the (in case you are using the asynchronous handler methods of the Java Developer API) `future.completeExceptionally(...)` method.

docs/docs/get-started/knowledge-directory.md

@@ -0,0 +1,13 @@
+---
+sidebar_position: 4
+---
+# Starting a Knowledge Directory
+This page describes how to setup a Knowledge Directory.
+
+You can start the Knowledge Directory on ports 8080 with the available JAR:
+```bash
+cd knowledge-directory/target/
+
+java -Dorg.slf4j.simpleLogger.logFile=kd.log -cp "knowledge-directory-1.2.5.jar:dependency/*" eu.knowledge.engine.knowledgedirectory.Main 8080
+```
+You can of course run the Knowledge Directory on another port by replacing 8080 by your preferred port number.

docs/docs/get-started/knowledge-interactions.md

@@ -1,32 +1,82 @@
 ---
+sidebar_position: 7
 ---
 import Tabs from '@theme/Tabs';
 import TabItem from '@theme/TabItem';
 
-# Knowledge Interactions
+# Using Knowledge Interactions
+This page describes how to register and execute Knowledge Interactions.
 
 ## How to instantiate a Knowledge Interaction?
+> You only need to register your Knowledge Interactions once. 
+> They are, however, dynamic and can be added and removed if needed.
+
 <Tabs groupId="tke-usage">
 <TabItem value="java" label="Java">
 
 ```java
 // ASK:
-AskKnowledgeInteraction askInteraction = new AskKnowledgeInteraction(graphPattern);
+AskKnowledgeInteraction askInteraction = new AskKnowledgeInteraction(communicativeAct, graphPattern);
 smartConnector.register(askInteraction);
 
 // ANSWER:
-AnswerKnowledgeInteraction answerInteraction = new AnswerKnowledgeInteraction(graphPattern);
+AnswerKnowledgeInteraction answerInteraction = new AnswerKnowledgeInteraction(communicativeAct, graphPattern);
 smartConnector.register(answerInteraction);
 
 // POST:
-PostKnowledgeInteraction postInteraction = new PostKnowledgeInteraction(graphPattern);
+PostKnowledgeInteraction postInteraction = new PostKnowledgeInteraction(communicativeAct, argumentGraphPattern, resultGraphPattern);
 smartConnector.register(postInteraction);
 
 // REACT:
-ReactKnowledgeInteraction reactInteraction = new ReactKnowledgeInteraction(graphPattern);
+ReactKnowledgeInteraction reactInteraction = new ReactKnowledgeInteraction(communicativeAct, argumentGraphPattern, resultGraphPattern);
 smartConnector.register(reactInteraction);
 ```
 
+You can also provide a name for your interaction, for example:
+```java
+AskKnowledgeInteraction askInteraction = new AskKnowledgeInteraction(communicativeAct, graphPattern, name); 
+```
+
+If you want to use prefixes in your graph pattern, these can be defined in the `graphPattern`:
+```java
+GraphPattern graphPattern = new GraphPattern(prefixes, pattern);
+```
+
+</TabItem>
+<TabItem value="JSON" label="Rest API">
+To instantiate a Knowledge Interaction, you need to send a POST to `/sc/ki` with a body that contains the type of knowledge interaction that you want to make, and any required graph patterns.
+ASK and ANSWER require one graph pattern, like so:
+
+```json
+{
+  "knowledgeInteractionType": "AskKnowledgeInteraction",
+  "graphPattern": "?s ?p ?o"
+}
+```
+
+POST and REACT require an `argumentGraphPattern`, and optionally use a `resultGraphPattern`. For example:
+
+```json
+{
+  "knowledgeInteractionType": "PostKnowledgeInteraction",
+  "argumentGraphPattern": "?s ?p ?o",
+  "resultGraphPattern": "?x ?y ?z"
+}
+```
+
+You can also provide a name for your interaction and define prefixes for your graph patterns:
+
+```json
+{
+  "knowledgeInteractionType": "AskKnowledgeInteraction",
+  "knowledgeInteractionName": "my-ask",
+  "graphPattern": "?a rdf:type ex:Book",
+  "prefixes": {
+    "rdf": "https://www.w3.org/1999/02/22-rdf-syntax-ns/"
+  }
+}
+```
+
 </TabItem>
 </Tabs>
 
@@ -50,6 +100,30 @@ AskKnowledgeInteraction askInteraction = new AskKnowledgeInteraction(graphPatter
 AskResult interactionResult = sc.ask(askInteraction, queryBindings).get();
 ```
 
+</TabItem>
+<TabItem value="JSON" label="Rest API">
+Send a POST to `/sc/ask` to execute an Ask Knowledge Interaction.
+To execute a Post Knowledge Interaction, send a POST to `/sc/post`.
+
+Triggering an interaction requires you to provide two parameters:
+* `Knowledge-Base-Id`: specifies the Knowledge Base Id for which to execute the ask
+* `Knowledge-Interaction-Id`: specifies the Ask Knowledge Interaction that should be executed
+
+In the body you can also specify a binding set, or a recipient selector *and* binding set.
+The recipient selector can be used to select a single Knowledge Base Id which should be contacted.
+The binding set specifies values that you are interested in. These must correspond to the variables in the graph pattern of the knowledge interaction.
+```json
+{
+  "recipientSelector": {
+    "knowledgeBases": []
+  },
+  "bindingSet": [
+    {}
+  ]
+}
+```
+If you leave the array for `knowledgeBases` empty, then it will simply ask all relevant KBs.
+
 </TabItem>
 </Tabs>