diff --git a/README.md b/README.md index f8ca11fe..e805d25c 100644 --- a/README.md +++ b/README.md @@ -182,7 +182,7 @@ This section gives more detailed information about the project's structure, and The Knowledge Engine project consists of the following Maven modules: - `smart-connector` - - This is the implementation of the smart connector, with the Java developer API. For instructions on how to use it, refer to [the documentation](./docs/docs/java_developer_api.md). + - This is the implementation of the smart connector, with the Java developer API. For instructions on how to use it, refer to [the documentation](./docs/docs/getting_started.md). - `smart-connector-api` - This module contains interfaces for the smart connector and other classes. It is made as a separate module so that it is easy to use different implementations of the interfaces. - `smart-connector-rest-server` diff --git a/admin-ui/pom.xml b/admin-ui/pom.xml index dc36f745..89336c8b 100644 --- a/admin-ui/pom.xml +++ b/admin-ui/pom.xml @@ -15,10 +15,10 @@ A user interface for managing a Knowledge Network. - 2.2.22 - 11.0.15 + 2.2.26 + 11.0.24 3.1.9 - 2.18.1 + 2.18.2 UTF-8 @@ -52,7 +52,7 @@ org.slf4j slf4j-simple - 2.0.13 + 2.0.16 diff --git a/docs/docs/common_error_messages.md b/docs/docs/common_error_messages.md new file mode 100644 index 00000000..31d90a73 --- /dev/null +++ b/docs/docs/common_error_messages.md @@ -0,0 +1,41 @@ +--- + sidebar_position: 9 +--- +# Common Error Messages + +On this page you can find a list of common error messages and what they mean. + +### `x` is not an unprefixed URI or literal +Whenever you are specifying variable bindings, for example in a binding set when executing an ask, you may encounter this error. +It occurs because all variable bindings need to be either RDF Literals or IRIs. +If you want to identify a specific object or device `x`, you can use something like ``. +Note that if the variable binding is for a subject in a triple (?subject ?predicate ?object), then an IRI is always required. +For more information on bindings and binding sets, see: [Bindings](https://docs.knowledge-engine.eu/java_developer_api#bindings). + +### There are lots of 'HTTP/1.1 header parser received no bytes' errors in the logs. How do I prevent these? +This can be prevented by using a shorter timeout in the Java HTTP Client. +Try using the following Java option: `-Djdk.httpclient.keepalive.timeout=3`. +This can also be set in the docker configuration of a Knowledge Engine Runtime docker container by setting the JAVA_TOOL_OPTIONS as follows: `JAVA_TOOL_OPTIONS: "-Djdk.httpclient.keepalive.timeout=3"`. + +### java.lang.IllegalArgumentException: KB gave outgoing binding Binding [...], but this doesn't have a matching incoming binding! +When using a REACT Knowledge Interaction that contains both an argument and result graph pattern which share a variable name, the Knowledge Engine expects all values for this variable in the result binding set to also occur as a value of the variable in the argument binding set. +This has to do with how these graph patterns are internally used to form if … then … rules. +If this is not the case, it gives the above error message. +If this shared variable in the argument and result graph patterns is intended to be the same, make sure you align the values of this variable in the result binding set with the values of this variable in the argument binding set. +If the variable is not intended to be the same thing, you can rename one of them to prevent the knowledge engine to expect them to share values. + +### The JSON Object should contain both a recipientSelector and a bindingSet key +When executing an ASK or POST you have to provide either (a) a binding set, or (b) a recipient selector *and* binding set. +The binding set specifies values that you are interested in. +The recipient selector can be used to select a single Knowledge Base which should be contacted. +In case you do not want to select a single Knowledge Base, you can pass an empty list to contact *all* relevant Knowledge Bases: +```json +{ + "recipientSelector": { + "knowledgeBases": [] + }, + "bindingSet": [ + {} + ] +} +``` \ No newline at end of file diff --git a/docs/docs/faq.md b/docs/docs/faq.md index 909367ec..b786ca13 100644 --- a/docs/docs/faq.md +++ b/docs/docs/faq.md @@ -6,24 +6,52 @@ FAQ === -*Question*: may be a question of dummies, what is the time of validity of the information? is it specified in the graph pattern? -- *Answer*: The validity of the information is not available by default, but you can include it into the graph pattern if you need it for your use case. - -*Question*: There is a limited and defined number of KI. How do we proceed to "generate" all possible graph patterns and so associated KI ? -- *Answer*: The KIs used should match and the KIs and their Graph Patterns are the end result of the SAREFization process for services. Also, if you need pilot specific KIs, you can agree on them within your pilot using SAREF or a pilot specific ontology. - -*Question*: what about ACKs of the requests? -- *Answer*: There are no explicit ACKs. The Ask receives an answer of one or more KBs and the Post receives the BindingSet for the optional Result Graph Pattern. If you need an explicit ACK (for example with the turn on the light example in the presentation) you can use the result Graph Pattern to contain the ACK. - -*Question*: Is there any specific requirements such as RAM, CPU and disk space for deploying the KE ? Is there aditional components to take into account such as an external DB or it is all inclusive ? Is there OS specific configuration such as network or ports ? -- *Answer*: We do not have minimal requirements for the KE yet. Locally I am running multiple Smart Connectors just fine on my Intel Core i7-8650 CPU @ 1.9GHz with 16 Gb RAM, but it also depends on the amount of traffic of course. -By the way, the current version (0.1.6) of the KE is a centralized one where all the Smart Connectors (with their reasoner) run on a server hosted by INESC TEC and partners create and access their Smart Connector via the REST Developer API of the generic adapter. A future version of the generic adapter will contain an instance of the Smart Connector (plus reasoner). -Currently, an instance of the Smart Connector is self contained, so no external database is required. The future version will of course need to use the network (ports are not yet decided) to communicate with other Smart Connectors (and the Knowledge Directory). - -*Question*: A Knowledge Interaction (KI) is a basic graph pattern. I suppose that this means just a sequence of triple patterns. Does this mean that for example the ‘FILTER’ keyword can’t be used in the KI? -- *Answer*: Indeed, the FILTER keyword (of the SPARQL language) is not available. Although this is a very useful keyword and we would love to support something like that, there needs to be an equivalent of filtering in the reasoner and most of the time this is not there. We do keep this in mind when looking/making for a new reasoner, but I do not expect this to be available anytime soon (there is still research required, I think). Note that a Knowledge Interaction is more than a single Basic Graph Pattern (although it is the most important part of it). It also has a type (Ask/Answer or Post/React) and a Communicative Act (to convey the 'reason' for the interaction). Also, the Post/React KI have two graph patterns attached to them; the argument and the result graph pattern. - -*Question*: It means also that a SparQL query like you see below is a SparQL query and is not something that can be used at the KE REST API interface. Only the part within the brackets is a basic graph pattern. Is that right? +### What is the time of validity of the exchanged information? +The validity of the exchanged information is not available by default, but you can include it into the graph pattern if you need it for your use case. + +### Are there any technical requirements, e.g. RAM, CPU and disk space, for deploying the Knowledge Engine? +We do not have minimal requirements for the Knowledge Engine yet. +Locally we can run multiple Smart Connectors on an Intel Core i7-8650 CPU @ 1.9GHz with 16 Gb RAM, but the requirements also depend on the amount of data that is exchanged. +An instance of the Smart Connector is self-contained and no external database or storage is required. + +For setting up your own Knowledge Network, including a Knowledge Directory, typically the following steps are required: +1. Have a machine or virtual machine ready: + * Ideally configure the (virtual) machine to be in a DMZ network, separate from other critical/sensitive resources (good security practice) +2. Deploy the Knowledge Engine and Knowledge Directory on the machine, i.e. deploy two Java servers. +3. Configure the firewall to allow external communication to that (virtual) machine. + * Depending on the local infrastructure, configure a proxy (if it exists) to forward the requests to that (virtual) machine. + +A medium range (virtual) machine with the following requirements should be sufficient to set up your own Knowledge Network: +* (Ideally) Linux-based OS +* Latest Java SE installed +* Outside world (inbound) internet access +* Low/medium CPU (2 cores at least) +* 16 GB RAM (nowadays a good minimum for a server, more is better) + +It is recommended that someone can access the (virtual) machine to collect any logs and troubleshoot when necessary. + +### How does the Knowledge Engine deal with privacy-sensitive information? +The Knowledge Engine (and the Smart Connector) functions as a serving hatch and does not store any data that is being exchanged. +All this data is stored in the Knowledge Bases which are responsible for protecting privacy-sensitive data. +Within the Knowledge Engine we distinguish between *graph patterns* and *binding sets*. +The graph patterns should not contain any privacy-sensitive data since they are part of the meta-data that is being stored as capability descriptions. +This information is needed to orchestrate the data exchange. +These graph patterns might also show up in the logs of the smart connector (at all log levels). +The binding sets, on the other hand, _can_ contain privacy-sensitive data which will not be stored. +Binding Sets are also not stored in the log files of Smart Connectors if these have a log level of INFO or higher. +Keep in mind, though, that the Knowledge Engine functions as an intelligent broker between consumers and producers of knowledge. +This might cause the Knowledge Engine to exchange your data with unexpected parties within the Knowledge Network. +So make sure your knowledge network only contains trusted KBs. + +### Can we use SPARQL keywords such as FILTER in Knowledge Interactions? +No, SPARQL keywords are not available. +We do not use SPARQL, because SPARQL is only usable for a question/answer interactions, while we also support publish/subscribe and function call interactions. +Although keywords such as FILTER are very useful keywords, and we would love to support something like that, there need to be equivalent options in the reasoner and most of the time this is not there. +We do keep this in mind when looking for/making a new reasoner, but do not expect this to be available anytime soon (there is still research required). +Note that a Knowledge Interaction is more than a single Basic Graph Pattern (although it is the most important part of it). +It also has a type (Ask/Answer or Post/React) and a Communicative Act (to convey the 'reason' for the interaction). + +Take for example the following SPARQL query: ```sparql SELECT ?sensor WHERE { ?building a saref4bldg:Building. @@ -33,162 +61,127 @@ SELECT ?sensor WHERE { ?vibrationSensor saref:hasState ?state . } ``` -- *Answer*: Exactly, the WHERE part contains the Basic Graph Patterns and those are used to create the Knowledge Interactions. We do not use SPARQL, because SPARQL is only usable for a question/answer interactions, while the Interoperability layer should also support publish/subscribe and function call interactions. +We can use the Basic Graph Pattern from the WHERE-clause to create a Knowledge Interaction. +We will then also need to specify the type of interaction, e.g. ASK, and the Communicative Act. -*Question*: In POST /sc/ki one registers a Knowledge Interaction along with the knowledge interaction type. In POST /sc/ask one queries for some results by referring to a KI and providing an incomplete binding set. The result will be a complete binding set. In your presentation KE for dummies slide 12, you mentioned that one could restrict the question (at the react side). I didn’t find in the rest of the slides on how one can do that, except by having a literal in the registered KI. In your example a person has a name and a email address, but the logic only allows to ask for the email address associated with a person with a certain name, but it does not allow to get the name associated with a specific email address. How do we impose such a restriction, or we can’t do this at this stage? -- *Answer*: If the logic does not allow the inverse, then you should not use an Ask/Answer Knowledge Interactions with a graph pattern like: - ```sparql - ?person :hasUsername ?userName . - ?person :hasEmailaddress ?emailAddress . - ``` +### Why do our two Knowledge Bases not exchange data even though they have matching graph patterns? +In this case, typically the error is in the Knowledge Interactions that you expect to match. +Two Knowledge Interactions match when: +* The types match +* The graph patterns match +* The communicative acts match - In that case you want to use the Post/React Knowledge Interactions. These have two Graph Patterns and the *argument* graph pattern would look something like: +In the table below 'yes' means those two types of Knowledge Interactions match, while 'no' means those two types of Knowledge Interactions do not match. - ```sparql - ?person :hasUsername ?userName . - ``` +| | | POST | | ASK | +|---------------|-----------------------------------------|-------------------------|-----------------------------------------|------------| +| | | *only argument GP* | *both argument and result GP* | | +| REACT | *only argument* GP | yes | no | n/a | +| | *both argument and result GP* | no | yes | n/a | +| ANSWER | | n/a | n/a | yes | - and the *result* graph pattern would look something like: +When your two Knowledge Interaction types have a 'yes', then you can take a look at whether the graph patterns match. - ```sparql - ?person :hasEmailaddress ?emailAddress . - ``` +Two graph patterns match when every triple of the first graph pattern is also in the second graph pattern and vice versa. +The ordering and names of variables like `?s` are ignored. +Note that in case of POST and REACT Knowledge Interactions, both the argument graph pattern and the result graph pattern must match. - This tells the Knowledge Engine that you cannot send it an email address and receive the username. +If you are sure that the graph patterns match (be careful of typos!), check which communicative acts they use. +The CommunicativeAct is meant to indicate the 'reason' for the interaction and in most cases the “InformPurpose” is sufficient, and therefore it is the default communicative act of every registered Knowledge Interaction. +Whenever the Knowledge Engine wants to exchange data it compares the sender Knowledge Interaction’s communicative act with the recipient Knowledge Interaction’s communicative act and if they ‘match’ the data will be exchanged. +If both Knowledge Bases use the REST API to register Knowledge Interactions and do not specify the communicative act, they will be able to exchange data. +However, when they _do_ specify the communicative act when registering a Knowledge Interaction, they should be compatible. -*Question*: Can you explain how to register the argument pattern and the result graph pattern? In the KE API I saw only one graph pattern in the register of a Knowledge interaction, and no parameter to indicate if it is an argument pattern or a result graph pattern. -- *Answer*: In the Java Developer API the constructors of the [PostKnowledgeInteraction](https://github.com/TNO/knowledge-engine/blob/master/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/PostKnowledgeInteraction.java) and [ReactKnowledgeInteraction](https://github.com/TNO/knowledge-engine/blob/master/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/ReactKnowledgeInteraction.java) objects require both an argument and a result graph pattern. - In the JSON body of the [REST Developer API ](https://github.com/TNO/knowledge-engine/blob/master/smart-connector-rest-server/src/main/resources/openapi-sc.yaml) `POST /sc/ki` operation, you specific the type of the Knowledge Interaction. If you choose the PostKnowledgeInteraction or ReactKnowledgeInteraction `knowledgeInteractionType`, the argument and result graph patterns are also expected (see also the schema of the request body): +### How do we get all possible graph patterns and their associated Knowledge Interactions that we need to register? +Within a specific use case or setting, the Knowledge Interactions and their graph patterns are the end result of the ontology engineering process. +In this process you need to decide what data you want to exchange, and which ontologies you want to use for this. +You can also build your own ontology, though we recommend reusing ontologies where possible as it's beneficial for the interoperability with other systems. - ```json - { - "knowledgeInteractionType": "PostKnowledgeInteraction", - "argumentGraphPattern": "?s ?p ?o", - "resultGraphPattern": "?x ?y ?z" - } - ``` - Note that the result graph pattern is optional. +### Do you send ACKs for all requests? +There are no explicit ACKs. +The Ask receives an answer of one or more KBs and the Post receives the BindingSet for the optional Result Graph Pattern. +If you need an explicit ACK, you can put it in the result graph pattern of a REACT Knowledge Interaction. -*Question*: In the context of graph pattern matching, can you explain the subset/superset condition: "when the Graph Pattern" of the sender is a superset of the Graph Pattern of the receiver' ? Is it at definition level or at data level? I found (at ontology level) the following explanation: "Ontology O1 is a subset of ontology O2 if all definitions in O1 are contained in O2 (O2 is the superset of O1)". -1) More concrete: is 'a b c .' graph pattern a subset or a superset of 'a b c . d e f.' ? -2) In case of Post/React knowledge interactions, both argument graph patterns must match and also both result graph patterns must match? -3) In case of Post/React knowledge interactions graph pattern matching, is the POST side regarded as the sender for the argument graph pattern and the REACT side as the sender for the result graph pattern? -4) Let's assume the REACT side result pattern is like 'a b c . d e f.' and the POST side result pattern is 'a b c .'. Is this allowed? So it is similar to a 'SELECT' in SQL? The result binding set at the POST side is then also reduced, I assume (not in number of _records_, but _fields_). -- *Answer*: We do not have defined the subset/superset terms within the context of the Knowledge Engine and Graph Patterns, but it would indeed be helpful to do so. Since the idea of the Knowledge Engine is that the *data* is always kept at the source and is only retrieved when necessary for an interaction, it is more suitable to talk about subset/superset at the definition level and not at the data level. This is because all data is simply not available in a single location. The definition level is also the level where currently the *matching* between graph patterns happens. The *matcher* (as opposed to the *reasoner*) does not support subset/superset matching. - 1) I would say graph pattern `a b c` is a subset of the graph pattern `a b c . d e f`. Note that graph pattern typically contain variables like `?a`. Note that graph pattern matching ignores variable names and triple order. - 2) Yes, the argument graph pattern and result graph pattern should both match if two Post/React Knowledge Interactions want to exchange data. Note that this probably changes when the Knowledge Engine uses a reasoner instead of a matcher. - 3) Yes, the PostKnowledgeInteraction sends the argument and the ReactKnowledgeInteraction sends the (optional) result. - 4) Currently, this will not work, because we are using a graph pattern *matcher* instead of a *reasoner*. I expect the reasoner to indeed allow them to interact if the POST side result pattern is a subset of the REACT side result pattern. In that case the result binding set at the POST side should also be a subset (in fields) of the binding set given from the REACT side. So, the results are always given to a Knowledge Base in its own terminology, this already happens by translating the variable names, but should also happen in the way you describe once the reasoner is active. +### Can we restrict the results of a Knowledge Interaction? +The first way to restrict a result is to limit which Knowledge Bases are contacted. +When executing an interaction, you can specify a single knowledge base. +In this case, it will only contact that specific knowledge base to try and answer your query. -*Question*: I successfully created smart connector (https://cybergrid.com/kb1) and the knowledge Interaction. When I wanted to execute the ask command with the following body: -```json -[ - { - "deviceName": "device1" - } -] -``` -I received the following expectation from the knowladge-engine: ```400 Bad Request: ['device1' is not an unprefixed URI or literal.]``` -- *Answer*: The reason your request fails is because variable bindings need to be either RDF Literals or IRIs. See also our [documentation](java_developer_api.md#bindings). - If you change your example value from `device1` to something like ``, this particular error should be resolved. +The second way to restrict a Knowledge Interaction is to use literals. +This can be done either in the graph pattern of the registered Knowledge Interaction _or_ in the binding set when executing an interaction. +This way you will only receive data related to that literal. -*Question*: Do we need a long polling connection for every Knowledge Interaction? Doesn't that get very complicated? -- *Answer*: No, per Smart Connector (or Knowledge Base) you need a single long polling connection to receive all interactions from the Knowledge Engine. Do remember that this long polling connection is returned with status code 202 every 29 seconds and also needs to be reestablished after you receive data via it. +But what if you want to make this more general? +Take for example a person with a name and an email address. +You may want to be able to request the email address for a person with a certain name, but do not want to provide the name for a specific email address. -*Question*: I’m trying to understand how timeseries as part of a larger graph pattern are expressed in a binding set. +In this case, if logic does not allow the inverse, then you should not use an Ask/Answer Knowledge Interactions with a graph pattern like: +```sparql +?person :hasUsername ?userName . +?person :hasEmailaddress ?emailAddress . +``` -For instance: -Let's say we have some graph pattern like: +Instead, you want to use the Post/React Knowledge Interactions. +These have two Graph Patterns and the *argument* graph pattern would look something like: ```sparql -?timeseries rdf:type ex:Timeseries . -?timeseries ex:hasMeasurement ?measurement . -?measurement rdf:type saref:Measurement . -?measurement saref:hasFeatureOfInterest ?room . -?room rdf:type saref:Room . -?measurement saref:observedProperty saref:Temperature . -?measurement saref:hasSimpleResult ?temperature . -?measurement ex:hasTimestamp ?ts . +?person :hasUsername ?userName . ``` -And the timeseries returns an array of temperature values and timestamp for each value. +and the *result* graph pattern would look something like: -In the ANWSER Knowledge interaction will the binding set be something like: -```json -[ - { - "timeseries": "", - "measurement": "", - "room": "", - "temperature": "\"21.2\"^^", - "ts": "\"2020-10-16T22:00Z\"^^some_timestamp_type" - }, - { - "timeseries": "", - "measurement": "", - "room": "", - "temperature": "\"21.4\"^^", - "ts": "\"2020-10-16T23:00Z\"^^some_timestamp_type" - }, - { - "timeseries": "", - "measurement": "", - "room": "", - "temperature": "\"21.6\"^^", - "ts": "\"2020-10-16T24:00Z\"^^some_timestamp_type" - } -] +```sparql +?person :hasEmailaddress ?emailAddress . ``` +This way the Knowledge Engine can send a username and receive the email address, but it cannot do the inverse. -Is the following statement right: the IRI is filled in by the service specific adapter? -Can this IRI then be used to for example ask more info about `` ? That would mean that the service specific adapter should use an ID that is stored, and not some temporal/volatile ID for the IRI. Because that means that you can give a reference to an object in the answers. Of course you have to provide then a graph pattern to allow the retrieval of this object resulting in another binding set. -- *Answer*: You are correct with respect to ``. Ideally you should be able to retrieve more information about it and the service should not randomly generate it, but use a stored id. +### How do the reactive Knowledge Interactions compare to a publish-subscribe broker? +The Knowledge Engine functions similarly to a publish-subscribe broker. +The React Knowledge Interaction can be seen as a *subscribe* and the Post Knowledge Interaction can be seen as a *publish*. +All *matching* React Knowledge Interactions will receive the Post Knowledge Interaction's data. -*Question*: I'm wondering if the reactive Knowledge Interactions function the same way that a publish/subscribe broker would work. Is it possible for one service to POST something to the KE, while multiple other services listen to the interaction. This way they all receive the data they need whenever it is available. +Aside from this interaction pattern, POST/REACT can also be used for an interaction pattern similar to function calls. +The POST/REACT Knowledge Interactions support an (optional) result graph pattern that describes what data is sent back (the results) after receiving data (the arguments). +The results from all *matching* REACTs are aggregated before being returned to the POST side. -If I understand correctly, a REACT KI can also send data back upon receiving it. If publish/subscribe is possible, how would this communication work? Would the Knowledge Base on the POST side receive all responses? -- *Answer*: Yes, it functions similarly to a publish/subscribe broker. The React Knowledge Interaction can be seen as a *subscribe* and the Post Knowledge Interaction can be seen as a *publish*. All *matching* React Knowledge Interactions will receive the Post Knowledge Interaction's data. - For a more functional interaction pattern, the Post/React Knowledge Interaction also support a (optional) result graph pattern that describes the data that will be sent back (the results) after receiving data (the arguments). The result will indeed be aggregated and returned to POST side. +### Inconsistent or missing results with POST/REACT +Users are sometimes surprised that their POST Knowledge Interaction is not returning with results. +This situation typically occurs when there are multiple Knowledge Bases available that can answer or want to react to a Knowledge Interaction. +The Knowledge Engine will aggregate all results before returning and thus delays in answers are possible. +If your Knowledge Base has not received a request for data, it may be waiting until others have answered or reacted. +If your Knowledge Base has received a request but the results are not returned, it is likely because the Knowledge Engine is waiting for the results of other matching Knowledge Bases. +We have an [issue](https://github.com/TNO/knowledge-engine/issues/109) which would allow you to instruct the Knowledge Engine to not wait indefinitely for an answer, but this is still on our todo list. +Until then, we recommend using more specific graph patterns. -*Question*: It's not clear for me how the interaction between the service store and the KE should be seen. In our case we have a system acting as a knowledgebase requesting information for other partners. This information will then be combined and using some specific application logic new information will be exposed. Both will be implemented through the knowledge engine. By looking at the examples I managed to implement a small application which is working fine but still I have some questions: +We have also seen the following situation: +* POST requests were sent +* The corresponding Knowledge Base with a corresponding REACT did not always receive this +* When receiving a POST, the REACT side sent a confirmation properly +* The Knowledge Base with the POST rarely received the confirmation from the REACT +* There were 2 REACT Knowledge Interactions with the same argument graph pattern but different result graph patterns. -The registration of the KB and KIs should only be done once I suppose? -Let's say we have a ASK-ANSWER (our KB will create answer) and a POST-REACT KI (our KB will react). Will we need to restart the connection with the KE? I thought something was mentioned during the sync call that an interaction is only active for 30 mins. -Is there a complete example available where the KE and the service store is combined? -- *Answer*: In general, the exact link between service store and knowledge engine is still evolving and under discussion. Currently, as far as I know, this link is that the service store keeps a list of services with metadata that have a generic adapters that also provide access to the Interoperability layer (i.e. Knowledge Engine). +In this setting, there were 2 REACT Knowledge Interactions with the same argument graph pattern but different result graph patterns. +One result graph pattern matched the POST, but the other did not. +This prevented the REACT interaction to react to the POST and thus the POST never got a response. +So if you have the same argument graph pattern for several interactions, be careful that *all* REACTs and *all* POSTs use the same result graph pattern - - The registration of the KB and KIs should only be done once I suppose? - - Typically, the Generic Adapter (and Smart Connector) should be available as long as your system (Knowledge Base) is available and in that case you only need to register your KIs once. KIs are, however, dynamic and can be added and removed if this is useful for the use case. - - Let's say we have a ASK-ANSWER (our KB will create answer) and a POST-REACT KI (our KB will react). Will we need to restart the connection with the KE? I thought something was mentioned during the sync call that an interaction is only active for 30 mins. - - The Knowledge Engine REST Developer API uses long-polling to notify you when your KB needs to react. This long-polling connection will automatically return every *29 seconds* with status code 202 to prevent certain proxies from blocking it. So, you need to reestablish this long-polling connection when you receive a 202. This does not affect the Knowledge Base and Knowledge Interactions. - - Is there a complete example available where the KE and the service store is combined? - - I think @aleksandar.tomcic.vizlore.com is working on examples that use the generic-adapter which maintains the link between the Service Store and the Knowledge Engine. For the Knowledge Engine only, we do have an very simple Python example available here: https://gitlab.inesctec.pt/interconnect/ke-python-examples -*Question*: I use a very generic graph pattern like `?s ?p ?o` for my PostKnowledgeInteraction, but my other KB does not get a request. Or, you do get a request, but your post is not returning with the results. -- *Answer*: We noticed multiple knowledge bases that register graph patterns like "?s ?p ?o" (i.e. from the examples we provided). If this is the case, it might occur that you ask a question or post some data and there are multiple KBs available that can answer or want to react to that type of data (i.e. they use a matching graph pattern). This means that you may not receive a request for data on your KB until one of the others has answered or reacted, or you might get a request, but you do not see the expected reaction in your other KB, because the Interoperability layer is waiting for the other matching KBs to answer/react. -We have an issue #95 which would allow you to instruct the Knowledge Engine to not wait indefinitely for an answer, but this is still on our todo list. Until then, we recommend using more specific graph patterns for testing. For example: -```json -{ - "knowledgeInteractionType": "ReactKnowledgeInteraction", - "argumentGraphPattern": "?s ?o ." -} -``` +### We see a spike in memory usage whenever an ASK or POST is executed. What is happening? +Most likely you have enabled the reasoner when you created a Smart Connector for your Knowledge Base. +When an ASK or POST is executed, the Knowledge Engine will use the reasoner to infer new data and orchestrate the data exchange (for more details see [Reasoning](./reasoning.md)). +When you have large graph patterns and/or many bindings, the reasoner's time and memory consumption can be quite large. +If you have no need for this reasoning capability, you can limit its resource usage by disabling the reasoner. +When using the REST Developer API, you can disable the reasoner by setting the JSON property `reasonerEnabled` to `false` or leave the property out altogether because by default the reasoner is disabled. -*Question*: In a Ask/Answer interaction the bindingSet is a list of items. The items should stay in the order there are inserted on the Answer side. Currently it is not the case. -- *Answer*: The Knowledge Engine cannot guarantee the ordering of the Bindings in a BindingSet due to several reasons. From a Semantic Technology perspective, the bindings are stored in a binding set in which the ordering is not fixed. Also, due to the matchers (and future reasoners) that mediate between different smart connectors and the fact that the response can be a union from BindingSets received from different knowledge bases, it is difficult to guarantee any ordering. Ideally, the ordering of the bindings can be derived from data, for example an ordering value or timestamps. This would allow the data to be sorted after receiving it from the interoperability layer. -*Question*: Some partners use a query like API request whereby they can specify via an input field what kind of results they expect back, together with the device id (temperature sensor, smart plug, meter,...) -For instance one can select the list of parameters (status, temperature, power, voltage, door_state, humidity, luminosity etc.) that should be returned. -In the result below is only the power_total returned, but additional fields could be selected/set in the input field (depending on the device type it can return more than one value) -This is a quite generic approach. So for this one API call there will be a lot of KIs, correct or can this realized with one KI? -Is the best approach to create a KI graph pattern per device type that returns all parameters? -What if I'm only interested in one parameter (not possible now because an exact match is required in this version, but possible in a next version)? -Result: +### How to deal with a query-like Knowledge Base where you can specify what properties you want to be returned? +Some users use a data source where you can specify what kind of results you expect back via an input field. +A common example we have seen is a query-like API request where you can specify what properties should be returned, e.g. for a device: ```json { @@ -206,10 +199,6 @@ Result: [ "2021-05-12T08:53:30.052924161Z", 241.63 - ], - [ - "2021-05-12T09:03:00.050337173Z", - 119.67 ] ] } @@ -218,188 +207,65 @@ Result: ] } ``` +Naturally, in this case you want to put other data in a Knowledge Interaction depending on which fields are selected. +The difficulty is, however, that a Knowledge Interaction is predefined and not dynamic. -- *Answer*: This is indeed quite a generic approach that, unfortunately, cannot be done with the current version of the KE (as you already correctly mention: because of exact matching). You could in theory register a lot of Knowledge Interactions, although I am not sure that is the best approach. If there is a limited set of fields that are always available, I would recommend providing a single large knowledge interaction. This would, however, mean that the asker registers this large knowledge interaction as well. - An alternative approach, which maybe mimics the generic behaviour of the API, could be to provide a measurement graph pattern like: - - ```sparql - ?deviceId . - ?deviceId ?m . - ?m ?p . - ?p ?fieldType . - ?m ?ts . - ?m ?val . - ``` - - This would allow the asking side to provide a binding set with a particular deviceId and 'fieldTypes': - - ```json - [ - { - "deviceId": "", - "fieldType": "" - }, - { - "deviceId": "", - "fieldType": "" - } - ] - ``` - - The answer side would need to parse this correctly (which is not trivial) and fill the bindingset correctly. - When we use a reasoner instead of a matcher, the ask side would not need to use the large graph pattern, but only those fieldTypes that it is interested in. The reasoner would still call the answer side, but limit the results to what the ask side needs. - So, there are several ways to handle this and each approach has advantages and disadvantages. Unfortunately, there is not a single best practice to solve this. - -*Question*: Have there been any discussions about availability and scalability of knowledge engine? Seeing as a knowledge base can only have a single polling connection, it does not seem possible to spin up multiple instances of a reactive adapter to the same knowledge base. This would limit the scalability potential by quite a lot. Are there any workarounds around this perhaps? - - *Answer*: Although scalability and availability have been mentioned several times now, there has not been any thorough discussions about them. One reason for this is that other things have been our priority in the last couple of months. This also means that the Knowledge Engine has not been designed to handle enormous amounts of data (although it is event-based and multi-threaded), but we expect it to be good enough for most use cases. The exact limitations with respect to throughput and latency will probably become clearer in the comings months and since we have not been really concerned with performance I expect there is also still room for improvement. - - Regarding multiple instances of a reactive adapter, we advice a single smart connector per knowledge base and each knowledge base is indeed limited to a single long polling connection. There are several ways to circumvent this limitation: - 1) consider using the _Java_ Developer API (not sure if the generic adapter will provide this in a future version). It uses handlers and is multi threaded, so it is much better scalable than the _REST_ Developer API. - 2) divide Knowledge Interactions over multiple Smart Connectors. This would allow you to have a single long polling connection per Knowledge Interaction. - - B.T.W.: We did not choose HTTP as the protocol in order to support web-server like scalability, but because most partners were familiar with it and the tooling and specification is very accessible. - -*Question*: I can hardly figure how data are exchanged between services in the Interworking layer architecure. When a service A ask for data form another service (B) through the KE (for example give me the temperature setpoint for this household) it is not clear if the final data (ie the temprature setpoint) is sent directly from Servica A Endpoint to Service B Endpoint or if it transit through the Interworking layer infrastructure. - - - *Answer*: If two services A and B want to exchange data in an interoperable manner, they both should use interoperability layer. They first register their capabilities (using the Ask and Answer Knowledge Interactions) and then the actual data exchange can happen. This data exchange is orchestrated by the interoperability layer and, so, the data transits through the interoperability layer. Service A asks the temperature set point to its Smart Connector and the interoperability layer will contact the Smart Connector of Service B to retrieve the answer from its Service B and sends the result back. - - For more information about the Knowledge Engine and Knowledge Interactions, see the recorded workshop on our shared drive: https://drive.inesctec.pt/f/16182787 - -*Quesiton*: We noticed that we cannot get consistent results when testing a POST/REACT exchange, both locally and with a partner. So to summarize: -* we can send a POST request -* the REACT sides don't receive 100% of the time (more like 25%), whether it is locally running or from our partner's platform -* When receiving the POST, the REACT side sends the confirmation properly -* The POST side rarely receives the confirmation from REACT (around 5% of the time) - -- *Answer*: We were doing tests with 2 REACT KB that had different result graph patterns. One matched our post but not the other. This prevented the react process to react to the post and so the post never got a response. So to remember to work properly: For the same argument graph pattern, ALL reacts and ALL posts need the same answer graph pattern. Also: the post will receive the answers from the react once they have all answered. The answers will be aggregated into one. - -*Question*: I have a question about how the compliance checker and reasoner are supposed to work. **If this is the wrong place for asking this, please direct me to the right people!** - -An example: the following graph pattern defines a **device** and the **type** of it’s **property**. - +There are several ways to tackle this. +The first option is to register a lot of Knowledge Interactions that cover all the possibilities. +The second option is to register one single graph pattern that covers all properties and to enable the reasoner. +Alternatively, you can instantiate an ASK with a generalized graph pattern like: ```sparql -?device ?property . -?property a ?propertyType . +?deviceId . +?deviceId ?m . +?m ?p . +?p ?fieldType . +?m ?ts . +?m ?val . ``` - -The only parameters we need here are **device** and **propertyType**. The **property** variable is redundant, but it still needs to be in the graph pattern. What we decided to do, is replace this variable by a placeholder individual : `http://interconnectproject.eu/pilots/greek/property#property`. The resulting pattern would then look like this: - -```sparql -?device . - a ?propertyType . +This allows the asking side to provide a binding set that specifies which properties should be returned (in this case 'fieldType' for a specific 'device'). +```json +[ + { + "deviceId": "", + "fieldType": "" + }, + { + "deviceId": "", + "fieldType": "" + } +] ``` - -I’m wondering if referring to individuals that are not in any ontology, in **subjects** and **objects**, will cause problems with compliance or reasoning? - -I guess my question is more specifically: “Will the compliance checker look at **subjects** and **objects** in graph patterns, or only at predicates? And will the reasoner be able to handle these kinds of structures?” - -- *Answer*: I think it helps if we distinguish between syntax and semantics here. Using a pattern like: - - ```sparql - ?device . - a ?propertyType . - ``` - - is syntactically correct and it should be accepted by both the validator and reasoner. The difficulty here is with the semantics that it expresses. Imagine the following bindingset: - - | ?device | ?propertyType | - |-----------|---------------| - | \ | saref:Motion | - | \ | saref:Smoke | - | \ | saref:Energy | - - If we combine the above Graph Pattern with the above BindingSet, we get the following RDF triples (I am a bit inconsistent with the prefixes): - - 1. ` .` - 2. ` a saref:Motion .` - - 3. ` .` - 4. ` a saref:Smoke .` - - 5. ` .` - 6. ` a saref:Energy .` - - Now, if we draw these triples as a graph figure, we get something like the following: - [Image no longer available] - - Note: - - that there are 6 triples in the above graph pattern, but only 5 edges in this figure. This is caused by the fact that the 1st and 3rd triple of the graph pattern above are exactly the same and triples can only occur once in a graph. So, if you write them twice, they will still only occur once. - - the effect that using a fixed individual greek:property in the graph pattern is causing; it makes it semantically impossible to determine that sensor1 is measuring property Motion and Smoke, while sensor2 is measuring Energy! - - Conclusion: while the fixed property is syntactically correct, it is semantically incorrect. So, using a fixed property in the current version of the KE (with a matcher instead of a reasoner) will probably work fine and the data is exchanged as expected. This is, however, probably not the case with the future version of the KE with reasoner, because the reasoner will actually semantically interpret the graph pattern and bindingset and when you ask something like: - - ```sparql - ?device . - a saref:Energy . - ``` - - (note that ?propertyType has been substituted with saref:Energy) - - This graph pattern represents the query: “give me all devices that measure energy” and it will answer with the following bindingset: - - | ?device | - |-----------| - | \ | - | \ | - - Which is not correct and is caused by the fixed property. So, there are two solutions here. The practical one, which I think happens quite a lot, is for the ontology to provide an individual per property. So, you would have a `interconnect:motionIndividual`, `interconnect:energyIndividual` and `interconnect:smokeIndividual`. These can be used instead of the greek:property and will make sure that it remains semantically correct. A less practical (and philosophically debatable) one is to have a unique property individual for every property a device measures. So, you would get something like ``, `` and `` and even more for all other devices. - - And last but not least, a short reaction to Georg’s remark: “I think it makes sense to think about the GP as the body of a query”. It certainly does, although within the context of the Knowledge Engine graph patterns are also used for non-query like interactions. - -*Question*: Which are the technical requirements needed to host the KE at pilot level for the next 22 months? Do you have an estimation about hosting/processing needs? And from the technical support, any idea about the effort that might require? -- *Answer*: From a technical perspective what needs to be done is: - 1. Have a machine or virtual machine ready (spec discussion separately); - 1.1. Ideally configure the machine/virtual machine to be in a DMZ network, separate from other critical/sensitive resources (for the sake of prevention) - 2. Deploy the KE and Knowledge Directory in the machine i.e., deploy two java servers. - 3. Configure firewall to allow external communication to that machine/virtual machine - 3.1 Depending on the local infrastructure, configure a proxy (if it exists) to forward the requests to that machine/virtual machine. - 4. Communicate me the public IP address where the KE is now available (so that we can use the project domain to identify that IP). - 5. If the proxy used uses a wildcard TLS certificate it can protect the KE instance (that is what we are doing in the cloud instance). If the wildcard certificate is not available, we need to acquire one and deploy it together with the KE. - - These are essentially the steps. These steps are standard, so any technical colleague should be able to do it. - - As for the technical requirements of the machine, we will collect that info and provide it to you. But a medium range machine/virtual machine should do it. - Requirements: - - (ideally) Linux based OS , e.g., latest ubuntu - - Latest Java SE installed - - Outside world (inbound) internet access - - Low/medium CPU (2 core at least) - - 16 GB RAM (nowadays a good minimum for a server. if if has more, better). - - Finally someone should be able to access that machine to collect any logs and trouble shoot whenever necessary. -*Question*: How does the Knowledge Engine deal with privacy sensitive information? -- *Answer*: The Knowledge Engine (and the Smart Connector) functions as a serving hatch and does not store any data that is being exchanged. All this data is stored in the Knowledge Bases and the KB are responsible for protecting privacy sensitive data. Within the Knowledge Engine we distinguish between *graph patterns* and *binding sets*. The graph patterns should not contain any privacy sensitive data since they are part of the meta data that is being stored as capability descriptions. This information is needed to orchestrate the data exchange. These graph patterns might also show up in the logs of the smart connector (at all log levels). The binding sets, on the other hand, _can_ contain privacy sensitive data which will not be stored. Binding Sets are also not stored in the log files of Smart Connectors if these have a log level of INFO or higher. Keep in mind, though, that the Knowledge Engine functions as an intelligent broker between consumers and producers of knowledge. This might cause the Knowledge Engine to exchange your data with unexpected parties within the Knowledge Network. So make sure your knowledge network only contains trusted KBs. - -*Question*: Why do our two KBs not exchange data, while they should? -- *Answer*: Assuming we are talking about the Matcher (default) instead of the Reasoner. The first thing to check are the Knowledge Interactions (KIs) of the two KBs that you expect to match. For two KIs to match, both their type needs to match and their graph patterns. In the table below 'yes' means those two types of KIs match, while 'no' means those two types of KIs do not match. When your two KI types have a 'yes', you can take a look at whether the graph patterns match. Two graph patterns match when every triple of the first graph pattern is also in the second graph pattern and vice versa. The ordering and names of variables like `?s` are ignored. Note that in case of POST and REACT KIs, both the argument graph pattern and the result graph pattern must match. - -| | | POST | | ASK | -|---------------|-----------------------------------------|-------------------------|-----------------------------------------|------------| -| | | *only argument GP* | *both argument and result GP* | | -| REACT | *only argument* GP | yes | no | n/a | -| | *both argument and result GP* | no | yes | n/a | -| ANSWER | | n/a | n/a | yes | - -*Question*: When making the ASK request, the binding set is a list of datapoints where the timestamps are increasing from the first to the last element. When the Answer service receives a response, the order of the datapoints inside the binding set is completely changed. -- *Answer*: You are correct to observe that the ordering of bindings in a binding set is not guaranteed by the Knowledge Engine. The reason why we call it a binding set is, because the elements in a set are unordered. Since JSON does not support sets and we are using JSON for our REST API, we put the bindings in a JSON list ([ … ]). I can imagine this is a bit misleading and you would expect the result to keep the ordering. However, due to the nature of RDF, the ordering of the bindings in a binding set cannot be used to encode any information when exchanging data. The ordering should, thus, be encoded explicitly using numbers or (as in your case) timestamps and the receiving end (ANSWER) should use this information to put the information in the correct order. - -*Question*: My KB has an ASK knowledge interaction with the same graph pattern as the ANSWER knowledge interaction of the Whirlpool KB with whom I want to exchange data, but the data exchange is not happing. What is going wrong? -- *Answer*: Double check the communicative acts that both knowledge interactions use. The CommunicativeAct is meant to indicate the 'reason' for the interaction and in most cases the “InformPurpose” is sufficient and therefore it is the default communicative act of every registered Knowledge Interaction. Whenever the KE wants to exchange data it compares the sender KI’s communicative act with the recipient KI’s communicative act and if they ‘match’ the data will be exchanged. If both KB use the REST API to register KIs and do not specify the communicative act, they will be able to exchange data. However, when they _do_ specify the communicative act when registering a KI, they should be compatible. In the case of Whirlpool the problem might be that Whirlpool explicitly configures the old default https://www.tno.nl/energy/ontology/interconnect#InformPurpose URI. This default was changed to https://w3id.org/knowledge-engine/InformPurpose in KE version `1.1.2` and sometimes prevents two knowledge bases from exchanging data. - -*Question*: What should our KB do When we receive a request for data (either via an ANSWER or REACT Knowledge Interaction), but we do not have a response? -- *Answer*: You should send an empty binding set when you do not have a response. Also when your REACT Knowledge Interaction has no result graph pattern, you should always return an empty binding set to the Knowledge Engine. If an error occurs while responding, you can either return an empty BindingSet (although this does not give any information about the error occurring) or call the (in case you are using the asynchronous handler methods of the Java Developer API) `future.completeExceptionally(...)` method. - -*Question*: There are lots of 'HTTP/1.1 header parser received no bytes' errors in the logs. How do I prevent these? -- *Answer*: This can be prevented by using a shorter timeout in the Java HTTP Client. Try using the following Java option: `-Djdk.httpclient.keepalive.timeout=3`. This can also be set in the docker configuration of a KER docker container by setting the JAVA_TOOL_OPTIONS as follows: `JAVA_TOOL_OPTIONS: "-Djdk.httpclient.keepalive.timeout=3"`. - -*Question*: Whenever I do a post or ask, the memory usage of the Knowledge Engine Runtime skyrockets and it fails with a error (`HTTP 500`) after a minute or two. -- *Answer*: Double check whether you are enabling the reasoner when you create a Smart Connector for your Knowledge Base. When using the REST Developer API, you can disable the reasoner by setting the JSON property `reasonerEnabled` to `false` or leave the property out altogether because by default the reasoner is disabled. Currently, the reasoner is not usable for scenario's where graph patterns are more than about 5 or 6 triple patterns, because the algorithm for graph pattern matching uses too much memory. We are working on improving this algorithm and hopefully allow more use cases to enable the reasoner and benefit the increased interoperability. - -*Question*: The result of my POST contains the following failed message in its ExchangeInfo: "java.lang.IllegalArgumentException: KB gave outgoing binding Binding [...], but this doesn't have a matching incoming binding!". What does this mean? -- *Answer*: The reason this error is given, is the following: REACT KIs that contain both an argument and result graph pattern and these two graph patterns share a variable name, the knowledge engine expects all values for this variable in the result binding set to also occur as a value of the variable in the argument binding set. This has to do with how these graph patterns are internally used to form if … then … rules. If this is not the case, it gives the above failedMessage. If this shared variable in the argument and result graph patterns is intended to be the same, make sure you align the values of this variable in the result binding set with the values of this variable in the argument binding set. If the variable is not intended to be the same thing, you can rename one of them to prevent the knowledge engine to expect them to share values. - -*Question*: There is an existing Knowledge Network with a Knowledge Directory (KD) and multiple Knowledge Engine Runtimes (KERs). How do I setup my own KER and configure it such that it participates in the existing Knowledge Network? -- *Answer*: Every Knowledge Engine Runtime (KER) in distributed mode consists of two APIs: [Knowledge Engine Developer REST API](https://github.com/TNO/knowledge-engine/blob/1.2.5/smart-connector-rest-server/src/main/resources/openapi-sc.yaml) and the [Inter-Knowledge Engine Runtime API](https://github.com/TNO/knowledge-engine/blob/1.2.5/smart-connector/src/main/resources/openapi-inter-ker.yaml). The former is started on port `8280` by default and you use this API to register your Knowledge Base and Knowledge Interactions. The latter API is meant for internal communication between KERs and you do not need to use it yourself. However, you do need to make sure this API is reachable for other KERs in the Knowledge Network. By default this API is available on port 8081, but sometimes you need to change this port using the `KE_RUNTIME_PORT` environment variable. Make sure the latter API of your KER is accessible from the internet and configure its URL when starting the KER with the `KE_RUNTIME_EXPOSED_URL` environment variable. To set this up correctly, you typically install a reverse proxy like NGNIX and open the correct ports in the firewall of the server. For this you need to contact the administrator of the server you are using. A KER starts in distributed mode when it detects the `KD_URL` environment variable. This variable points to the Knowledge Directory of the Knowledge Network. You can configure it using environment variables `KD_URL=`. If the Knowledge Directory is protected using Basic Authentication, you can add the credentials to the KD_URL as described [here](https://stackoverflow.com/a/50528734). - -*Question*: There is an existing Knowledge Engine Runtime (KER) that I want to use to develop a knowledge base. How do I connect to this KER? -- *Answer*: To connect to the existing KER you need to know and have access to the KER's [Knowledge Engine REST Developer API](https://github.com/TNO/knowledge-engine/blob/1.2.5/smart-connector-rest-server/src/main/resources/openapi-sc.yaml). If you have the URL of the KER you want to use, you can test it by activating its `GET /sc` operation via the browser with an URL that looks like this: `/sc` (if the KER is protected with Basic Authentication your browser might ask you for credentials, you can also put the credentials directly into the URL as user info). This operation returns JSON with all the Knowledge Bases (KBs) that are registered with that KER (if there are none, it returns an empty JSON array `[]`). If you run a KER on your own computer (for example using the provided docker image), the `` would typically be `http://localhost:8280`. Now that you tested the KER, you can use the `` to register your KB and Knowledge Interactions (KIs) by activating the different operations that are described in the Open API specification above. +The ANSWER side will need to parse this correctly (which is not trivial) and fill the binding set correctly. + +### Any thoughts on the scalability of the single long polling connection used by each Smart Connector? +Currently, each Smart Connector uses a single long polling connection to receive all interactions from the Knowledge Engine. +The Knowledge Engine is event-based and multithreaded, and while not designed to handle enormous amounts of data, this has not been a limiting factor in our use cases so far. + +If the current setup, a Smart Connector with a single long polling connection, is limiting for you, there are several ways to circumvent this: +1) Use the _Java_ Developer API. + It uses handlers and is multithreaded, so it is scales better than the _REST_ Developer API. +2) Divide Knowledge Interactions over multiple Smart Connectors. + This allows you to have a single long polling connection per Knowledge Interaction. + +### How does the Knowledge Engine deal with subsets/supersets in graph patterns? +We have not defined the subset/superset terms within the context of the Knowledge Engine but this would indeed be helpful. +For ontologies, the subset/superset definition is sometimes explained as follows: +"Ontology O1 is a subset of ontology O2 if all definitions in O1 are contained in O2 (O2 is the superset of O1)". + +Since the idea of the Knowledge Engine is that the *data* is always kept at the source and is only retrieved when necessary for an interaction, it is more suitable to talk about subset/superset at the definition level and not at the data level. +This is because all data is simply not available in a single location. +The definition level is also the level where currently the *matching* between graph patterns happens. +The *matcher* (as opposed to the *reasoner*) does not support subset/superset matching. + +We can consider the following concrete questions when dealing with subset/superset: +1. Is `a b c.` a subset or superset of `a b c . d e f .`? + - I would say graph pattern `a b c` is a subset of the graph pattern `a b c . d e f`. + Note that graph pattern typically contain variables like `?a`. + Graph pattern _matching_ ignores variable names and triple order. +2. When using POST/REACT, do argument and result graph patterns *both* need to match or can they be a subset/superset? + - Yes, the argument graph pattern and result graph pattern should both match if two POST/REACT interactions want to exchange data. + This may change when you use the reasoner instead of the matcher. +3. Would the following interactions match? A REACT with result graph pattern `a b c. d e f.` and a POST with result graph pattern `a b c.` + - This will not match if you are using the graph pattern _matcher_ instead of a _reasoner_. + The reasoner would allow them to interact if the POST result pattern is a subset of the REACT result pattern. + In that case, the result binding set at the POST side should also be a subset (in fields) of the binding set given by the REACT side. diff --git a/docs/docs/get-started/_category_.json b/docs/docs/get-started/_category_.json index 7e664cc1..a49f28a7 100644 --- a/docs/docs/get-started/_category_.json +++ b/docs/docs/get-started/_category_.json @@ -1,8 +1,8 @@ { - "label": "Tutorial", + "label": "Guides", "position": 6, "link": { "type": "generated-index", - "description": "In-depth details about how to use the various elements of the Knowledge Engine" + "description": "In-depth details about how to use the various elements of the Knowledge Engine." } } diff --git a/docs/docs/get-started/graph-patterns.md b/docs/docs/get-started/graph-patterns.md new file mode 100644 index 00000000..b40c06d0 --- /dev/null +++ b/docs/docs/get-started/graph-patterns.md @@ -0,0 +1,136 @@ +--- +sidebar_position: 8 +--- +# Writing Graph Patterns +This page discusses some good practices for writing graph patterns. + +## How to handle redundant variables if they are required in a graph pattern +:::tip +This is a common pitfall, so we recommend having a look at this section! +::: + +Let's take the following graph pattern as an example, which defines a device which measures a property, and the type of that property. + +```sparql +?device ?property . +?property a ?propertyType . +``` + +This graph pattern contains three variables: `device`, `property` and `propertyType`. +What should you do when you are only interested in `device` and `propertyType`, but not in `property`? + +This case often occurs and a common pitfall is that people replace the `property` variable by a placeholder individual, +e.g. : `http://example.org/property#property`. +The resulting pattern would then look like this: + +```sparql +?device . + a ?propertyType . +``` + +However, this causes a problem. +Let's distinguish between syntax and semantics here. +The graph pattern above is syntactically correct, and is accepted by the Knowledge Engine. +However, there is some difficulty with the semantics that it expresses. + +Imagine the following binding set: + +```json +{ + "device": "", + "propertyType": "saref:Motion" +}, +{ + "device": "", + "propertyType": "saref:Smoke" +}, +{ + "device": "", + "propertyType": "saref:Energy" +} +``` + +If we combine the above Graph Pattern with the above BindingSet, we get the following RDF triples: + +1. ` .` +2. ` a saref:Motion .` +3. ` .` +4. ` a saref:Smoke .` +5. ` .` +6. ` a saref:Energy .` + +Now, if we draw these triples as a graph, we get something like: + +![Illustration of aforementioned RDF triples. It contains 2 nodes on the left, both connected to a central node, which is then connected to three nodes on the right.](./../../static/img/graph-pattern-pitfall.png) + +While we had 6 RDF triples, we only see 5 edges in this figure! +This is caused by the fact that the 1st and 3rd triple of the graph pattern above are exactly the same and triples can only occur once in a graph. +So, if you write them twice, they will still only occur once. +So, by using a fixed individual for `property` in the graph pattern, it becomes semantically impossible to determine that `sensor1` is measuring the property Motion and Smoke, while sensor2 is measuring Energy! + +Therefore, while using a fixed property is syntactically correct, it is semantically incorrect. +Using a fixed property in the current version of the KE (with a matcher instead of a reasoner) will probably work fine and the data is exchanged as expected. +This is, however, probably not the case with the future version of the KE with reasoner, because the reasoner will actually semantically interpret the graph pattern and binding set and when you ask something like: + + ```sparql + ?device . + a saref:Energy . + ``` + This graph pattern represents the query: “give me all devices that measure energy” and it will answer with the following bindingset: + +```json +{ + "device": "" +}, +{ + "device": "" +} +``` + +Which is not correct and is caused by the fixed property. +So, there are two solutions here. +1. The practical one, which I think happens quite a lot, is for the ontology to provide an individual per property. +So, you would have a `ex:motionIndividual`, `ex:energyIndividual` and `ex:smokeIndividual`. +These can be used instead of the ex:property and will make sure that it remains semantically correct. +2. A less practical (and philosophically debatable) one is to have a unique property individual for every property a device measures. +So, you would get something like ``, `` and `` and even more for all other devices. + +## Can you use volatile IRIs or should you use static or stored reference IRIs? +Let's say we have some graph pattern like: + +```sparql +?timeseries rdf:type ex:Timeseries . +?timeseries ex:hasMeasurement ?measurement . +?measurement rdf:type saref:Measurement . +?measurement saref:hasFeatureOfInterest ?room . +?room rdf:type saref:Room . +?measurement saref:observedProperty saref:Temperature . +?measurement saref:hasSimpleResult ?temperature . +?measurement ex:hasTimestamp ?ts . +``` + +And the timeseries returns an array of temperature values and timestamp for each value. +The binding set will be something like: +```json +[ + { + "timeseries": "", + "measurement": "", + "room": "", + "temperature": "\"21.2\"^^", + "ts": "\"2020-10-16T22:00Z\"^^some_timestamp_type" + }, + { + "timeseries": "", + "measurement": "", + "room": "", + "temperature": "\"21.4\"^^", + "ts": "\"2020-10-16T23:00Z\"^^some_timestamp_type" + } +] +``` + +If you use a temporal or volatile IRI for the measurement, that would mean that you cannot retrieve more details for the object later (assuming an appropriate interaction exists). +It is therefore recommended to use a stored or static IRI. +In the case of measurements or observations, you could also choose an IRI like so: ``, where you add the timestamp (e.g. in unix format) and a reference to the sensor doing the measurement. +This makes it easy to later reconstruct the IRI when you want to request more info. diff --git a/docs/docs/get-started/knowledge-base.md b/docs/docs/get-started/knowledge-base.md new file mode 100644 index 00000000..1215d060 --- /dev/null +++ b/docs/docs/get-started/knowledge-base.md @@ -0,0 +1,22 @@ +--- +sidebar_position: 99 +--- +# Implementing a Knowledge Base +This page describes how to implement your own Knowledge Base given a data source. + +There are three approaches to implement your own Knowledge Base: +1. Java +2. REST Developer API +3. Knowledge Mapper (based on a Python client) +4. JavaScript client + +The Knowledge Mapper is a tool we have built to easily connect to several common data sources (SQL, RDF, APIs). +If you're interested in using the Knowledge Mapper or JavaScript client, please reach out to us as they are not yet open source. + +## Implementing your own Knowledge Interaction +When you receive a request for data via an ANSWER or REACT Knowledge Interaction, you should return the expected results, e.g. by retrieving data from an API. + +If you do not have a response, then you should send an empty binding set. +Also, when your REACT Knowledge Interaction has no result graph pattern, you should always return an empty binding set to the Knowledge Engine. + +If an error occurs while responding, you can either return an empty BindingSet (although this does not give any information about the error occurring) or call the (in case you are using the asynchronous handler methods of the Java Developer API) `future.completeExceptionally(...)` method. diff --git a/docs/docs/get-started/knowledge-directory.md b/docs/docs/get-started/knowledge-directory.md new file mode 100644 index 00000000..cd969d5f --- /dev/null +++ b/docs/docs/get-started/knowledge-directory.md @@ -0,0 +1,13 @@ +--- +sidebar_position: 4 +--- +# Starting a Knowledge Directory +This page describes how to setup a Knowledge Directory. + +You can start the Knowledge Directory on ports 8080 with the available JAR: +```bash +cd knowledge-directory/target/ + +java -Dorg.slf4j.simpleLogger.logFile=kd.log -cp "knowledge-directory-1.2.5.jar:dependency/*" eu.knowledge.engine.knowledgedirectory.Main 8080 +``` +You can of course run the Knowledge Directory on another port by replacing 8080 by your preferred port number. diff --git a/docs/docs/get-started/knowledge-interactions.md b/docs/docs/get-started/knowledge-interactions.md index 3f373365..f130241f 100644 --- a/docs/docs/get-started/knowledge-interactions.md +++ b/docs/docs/get-started/knowledge-interactions.md @@ -1,32 +1,82 @@ --- +sidebar_position: 7 --- import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -# Knowledge Interactions +# Using Knowledge Interactions +This page describes how to register and execute Knowledge Interactions. ## How to instantiate a Knowledge Interaction? +> You only need to register your Knowledge Interactions once. +> They are, however, dynamic and can be added and removed if needed. + ```java // ASK: -AskKnowledgeInteraction askInteraction = new AskKnowledgeInteraction(graphPattern); +AskKnowledgeInteraction askInteraction = new AskKnowledgeInteraction(communicativeAct, graphPattern); smartConnector.register(askInteraction); // ANSWER: -AnswerKnowledgeInteraction answerInteraction = new AnswerKnowledgeInteraction(graphPattern); +AnswerKnowledgeInteraction answerInteraction = new AnswerKnowledgeInteraction(communicativeAct, graphPattern); smartConnector.register(answerInteraction); // POST: -PostKnowledgeInteraction postInteraction = new PostKnowledgeInteraction(graphPattern); +PostKnowledgeInteraction postInteraction = new PostKnowledgeInteraction(communicativeAct, argumentGraphPattern, resultGraphPattern); smartConnector.register(postInteraction); // REACT: -ReactKnowledgeInteraction reactInteraction = new ReactKnowledgeInteraction(graphPattern); +ReactKnowledgeInteraction reactInteraction = new ReactKnowledgeInteraction(communicativeAct, argumentGraphPattern, resultGraphPattern); smartConnector.register(reactInteraction); ``` +You can also provide a name for your interaction, for example: +```java +AskKnowledgeInteraction askInteraction = new AskKnowledgeInteraction(communicativeAct, graphPattern, name); +``` + +If you want to use prefixes in your graph pattern, these can be defined in the `graphPattern`: +```java +GraphPattern graphPattern = new GraphPattern(prefixes, pattern); +``` + + + +To instantiate a Knowledge Interaction, you need to send a POST to `/sc/ki` with a body that contains the type of knowledge interaction that you want to make, and any required graph patterns. +ASK and ANSWER require one graph pattern, like so: + +```json +{ + "knowledgeInteractionType": "AskKnowledgeInteraction", + "graphPattern": "?s ?p ?o" +} +``` + +POST and REACT require an `argumentGraphPattern`, and optionally use a `resultGraphPattern`. For example: + +```json +{ + "knowledgeInteractionType": "PostKnowledgeInteraction", + "argumentGraphPattern": "?s ?p ?o", + "resultGraphPattern": "?x ?y ?z" +} +``` + +You can also provide a name for your interaction and define prefixes for your graph patterns: + +```json +{ + "knowledgeInteractionType": "AskKnowledgeInteraction", + "knowledgeInteractionName": "my-ask", + "graphPattern": "?a rdf:type ex:Book", + "prefixes": { + "rdf": "https://www.w3.org/1999/02/22-rdf-syntax-ns/" + } +} +``` + @@ -50,6 +100,30 @@ AskKnowledgeInteraction askInteraction = new AskKnowledgeInteraction(graphPatter AskResult interactionResult = sc.ask(askInteraction, queryBindings).get(); ``` + + +Send a POST to `/sc/ask` to execute an Ask Knowledge Interaction. +To execute a Post Knowledge Interaction, send a POST to `/sc/post`. + +Triggering an interaction requires you to provide two parameters: +* `Knowledge-Base-Id`: specifies the Knowledge Base Id for which to execute the ask +* `Knowledge-Interaction-Id`: specifies the Ask Knowledge Interaction that should be executed + +In the body you can also specify a binding set, or a recipient selector *and* binding set. +The recipient selector can be used to select a single Knowledge Base Id which should be contacted. +The binding set specifies values that you are interested in. These must correspond to the variables in the graph pattern of the knowledge interaction. +```json +{ + "recipientSelector": { + "knowledgeBases": [] + }, + "bindingSet": [ + {} + ] +} +``` +If you leave the array for `knowledgeBases` empty, then it will simply ask all relevant KBs. + diff --git a/docs/docs/get-started/smart-connector.md b/docs/docs/get-started/smart-connector.md index 04ffb75a..25a7edf1 100644 --- a/docs/docs/get-started/smart-connector.md +++ b/docs/docs/get-started/smart-connector.md @@ -5,12 +5,49 @@ sidebar_position: 6 import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -# Smart Connectors +# Connecting to a Knowledge Network +This page describes how to connect to an (existing) Knowledge Network using a Smart Connector. + +To connect to a Knowledge Network, you need a Knowledge Engine Runtime (KER). +Every KER in distributed mode consists of two APIs: [Knowledge Engine Developer REST API](https://github.com/TNO/knowledge-engine/blob/1.2.5/smart-connector-rest-server/src/main/resources/openapi-sc.yaml) and the [Inter-Knowledge Engine Runtime API](https://github.com/TNO/knowledge-engine/blob/1.2.5/smart-connector/src/main/resources/openapi-inter-ker.yaml). +The former is started on port `8280` by default, and you use this API to register your Knowledge Base and Knowledge Interactions. +The latter API is meant for internal communication between KERs and you do not need to use it yourself. +However, you do need to make sure this API is reachable for other KERs in the Knowledge Network. +By default, this API is available on port 8081, but sometimes you need to change this port using the `KE_RUNTIME_PORT` environment variable. +Make sure the latter API of your KER is accessible from the internet and configure its URL when starting the KER with the `KE_RUNTIME_EXPOSED_URL` environment variable. +To set this up correctly, you typically install a reverse proxy like NGINX and open the correct ports in the firewall of the server. +For this you need to contact the administrator of the server you are using. +A KER starts in distributed mode when it detects the `KD_URL` environment variable. +This variable points to the Knowledge Directory of the Knowledge Network. +You can configure it using environment variables `KD_URL=`. +If the Knowledge Directory is protected using Basic Authentication, you can add the credentials to the KD_URL as described [here](https://stackoverflow.com/a/50528734). + + + +To connect to a network, the following steps are required: +* Get access to the Knowledge Engine Runtime (KER) you want to connect to +* Start a Knowledge Engine Runtime (KER) on your computer +* Register your Knowledge Base via the REST Developer API +* [Register your Knowledge Interactions via the REST Developer API](./knowledge-interactions.md) + +## Getting access to the Knowledge Engine Runtime +To get access to the Knowledge Engine Runtime you want to connect to, you will need its URL. +You can test whether you have access to its REST Developer API by activating its `GET /sc` operation via the browser with a URL like: `/sc` +If the KER is protected with Basic Authentication, your browser might ask you for credentials. +This operation returns JSON with all the Knowledge Bases that are registered with that Knowledge Engine Runtime. +An empty list indicates that no Knowledge Bases are registered with this Knowledge Engine Runtime. +If you run a Knowledge Engine Runtime on your own computer, then the URL is typically `http://localhost:8280`. + +## How to start a Smart Connector? +> *Before starting a Smart Connector, please ensure that there is a Knowledge Directory available to connect to.* + +> Typically, you start a single Smart Connector which should be available as long as your system (Knowledge Base) is available. -## How to instantiate a Smart Connector? +Assuming `this` is your knowledge base, you can make a `SmartConnector` as follows: + ```java SmartConnector sc = SmartConnectorBuilder.newSmartConnector(this).create(); ``` @@ -31,9 +68,12 @@ java -Dorg.slf4j.simpleLogger.logFile=ke.log -cp "smart-connector-rest-dist-1.2. - -## How to add a Smart Connector? - ## How to remove a Smart Connector? -## How to renew the lease of a Smart Connector? \ No newline at end of file +## How to renew the lease of a Smart Connector? + +## How to deal with the long polling connection of a Smart Connector? +Each Smart Connector uses a single long polling connection to receive all interactions from the Knowledge Engine. +The Knowledge Engine REST Developer API uses long-polling to notify you when your KB needs to react. +This long-polling connection will automatically return every *29 seconds* with status code 202 to prevent certain proxies from blocking it. +You will need to reestablish this long polling connection when you receive a 202 and after you receive data via it. diff --git a/docs/docs/java_developer_api.md b/docs/docs/getting_started.md similarity index 63% rename from docs/docs/java_developer_api.md rename to docs/docs/getting_started.md index a8928b5e..eab82efb 100644 --- a/docs/docs/java_developer_api.md +++ b/docs/docs/getting_started.md @@ -2,17 +2,16 @@ sidebar_position: 5 --- -# Java Developer API +# Getting Started As a developer, you might be thinking "this all seems awfully complex and painful". If that sounds like you, we have good news: you don't have to bother with most of it! -This section explains the Java Developer API of the Knowledge Engine. - -Note that there is also a [REST Developer API](https://github.com/TNO/knowledge-engine/blob/master/smart-connector-rest-server/src/main/resources/openapi-sc.yaml) available. +This section will take you through the main steps that are needed to get started. +We will show this using the Java Developer API, but it can also be done using the [REST Developer API](https://github.com/TNO/knowledge-engine/blob/master/smart-connector-rest-server/src/main/resources/openapi-sc.yaml). We provide an implementation of a `SmartConnector` that you can instantiate as a Java object. -You are responsible for: +When you want to exchange information within a Knowledge Network, you are responsible for: - Registering the knowledge that your knowledge base requests or provides. - Implementing handlers that are called when certain knowledge is requested or provided. @@ -20,9 +19,9 @@ You are responsible for: The SmartConnector uses this to register itself in the knowledge network. -The following subsections further explain how a SmartConnector can be instantiated and, how the different kinds of knowledge can be registered. +The following sections further explain how a SmartConnector can be instantiated and, how you can register the knowledge that your knowledge base provides. -## Instantiating and configuring a SmartConnector +## Instantiating and Configuring a SmartConnector @@ -32,8 +31,8 @@ Assuming `this` is your knowledge base, you can make a `SmartConnector` as follo SmartConnector sc = SmartConnectorBuilder.newSmartConnector(this).create(); ``` -## Registering and using Knowledge Interactions -Currently, a SmartConnector is required to register the patterns of knowledge that it will request from the network. (See #67) +## Registering and Using Knowledge Interactions +Currently, a SmartConnector is required to register the patterns of knowledge that it will request from the network. A knowledge request can be registered as follows: ```java @@ -44,10 +43,9 @@ sc.register( ``` where `graphPattern` is a string describing an RDF graph pattern where variables are prefixed with a `?`. -Graph patterns consist of one or more triples separated by a dot (.) and each triple consists of a subject, predicate and object node. Each node can be either a variable (using a question mark `?var` prefix), a URI (using the `` or a literal (using quotes `"hello"`). -More information on graph patterns can be found in: - -- Graph Pattern syntax is based on W3C's [SPARQL 1.1 graph pattern](https://www.w3.org/TR/rdf-sparql-query/#BasicGraphPatterns). +Graph patterns consist of one or more triples separated by a dot (.) and each triple consists of a subject, predicate and object node. +Each node can be either a variable (using a question mark `?var` prefix), a URI (using the `` or a literal (using quotes `"hello"`). +Graph Pattern syntax is based on W3C's [SPARQL 1.1 basic graph patterns](https://www.w3.org/TR/rdf-sparql-query/#BasicGraphPatterns). As an example, assume `graphPattern` is the following graph pattern: ```sparql @@ -64,7 +62,7 @@ where the variables are represented by circles and the fixed URIs are represente The graph pattern above matches on temperature measurements in rooms. -### Querying the network +### Querying the Network When querying the network for the pattern, the variables (`?measurement`, `?room`, and `?temperature`) can be bound to known values, to limit the possible matches. @@ -82,9 +80,12 @@ BindingSet resultBindings = askResult.getBindings(); The results from the knowledge network are in the set of bindings. The `AskResult` contains other useful information, such as `AskExchangeInfo`, which gives information about the data's origins. -## Registering and using other kinds of Knowledge Interactions +### Other Types of Knowledge Interactions + +Aside from `ASK` knowledge interactions, there are also `ANSWER`, `REACT`, and `POST` interactions. +`ASK` is used to request information, `ANSWER` is used to reply to a request, `POST` is used to publish information, and `REACT` can be used to subscribe to information. +They can be registered and executed in a similar way to the `ASK` explained above (see [here](./get-started/knowledge-interactions.md)). -Aside from `ASK` knowledge interactions, there are also `ANSWER`, `REACT`, and `POST` interactions, which will be explained here in the future. ## Bindings The data that is shared is inside the `Binding` objects. @@ -102,16 +103,26 @@ As you can see, a `Binding` object is essentially a map from variable names to t Two important things should be noted: -1. The keys of the bindings MUST correspond to the variable names in the graph pattern, and they must be complete (all variables must have a value bound to them). (This last restriction does not apply to the bindings given with ASK requests; they can be partial of even empty.) +1. The keys of the bindings MUST correspond to the variable names in the graph pattern, and they must be complete (all variables must have a value bound to them). + (This last restriction does not apply to the bindings given with ASK requests; they can be partial of even empty.) 2. The values of the bindings MUST be valid IRIs (https://www.w3.org/TR/turtle/#sec-iri) (for now without prefixes, so full IRIs) or valid literals (https://www.w3.org/TR/turtle/#literals). -### Binding sets +### Binding Sets A result of a knowledge interaction can have more than 1 match. These matches are collected in a `BindingSet`, which is simply a set of bindings. -## Expressibility +The Knowledge Engine does not guarantee the ordering of bindings in a binding set. +The reason why we call it a binding set is because the elements in a set are unordered. +Due to the nature of RDF, the ordering of the bindings in a binding set cannot be used to encode any information when exchanging data. +Thus, if you need ordering, this should be encoded explicitly, e.g. by using numbers or timestamps, and the receiving end should use this information to put the information in the correct order. + +## Expressiveness -The Graph Pattern syntax has a limited expressibility. This means there are certain things that you might want to express with them, but are unable to. Sometimes this means it limits the actual data exchange, but sometimes there are work-arounds. One of the limitations is related to one-to-many relations. Take the following RDF about parents and children in which a parent has a one-to-many relation with its children: +The Graph Pattern syntax has a limited expressivity. +This means there are certain things that you might want to express with them, but are unable to. +Sometimes this means it limits the actual data exchange, but sometimes there are workarounds. +One of the limitations is related to one-to-many relations. +Take the following RDF about parents and children in which a parent has a one-to-many relation with its children: ```sparql ex:parent1 rdf:type ex:Parent . @@ -126,7 +137,11 @@ ex:parent1 ex:hasChild ex:child3 . ex:child3 rdf:type ex:Child . ``` -As you can see, RDF is perfectly capable of expressing one-to-many relations. Now imagine that you want to use the Interoperability layer to exchange information about parents and their children. For this you need to let the Interoperability layer know that you can participate in data exchanges about parents and their children. You do this using a graph pattern (and for example an AnswerKnowledgeInteraction). Let's take the following graph pattern: +As you can see, RDF is perfectly capable of expressing one-to-many relations. +Now imagine that you want to use the Knowledge Engine to exchange information about parents and their children. +For this you need to let the Knowledge Engine know that you can participate in data exchanges about parents and their children. +You do this using a graph pattern (and for example an AnswerKnowledgeInteraction). +Let's take the following graph pattern: ```sparql ?parent rdf:type ex:Parent . @@ -134,9 +149,14 @@ As you can see, RDF is perfectly capable of expressing one-to-many relations. No ?someChild rdf:type ex:Child . ``` -Since the syntax of graph patterns is limited, you cannot express that the `ex:hasChild` relation between a parent and a child is a one to many relation. So, how does the interoperability layer exchange the RDF data about parents and their children above? This is where bindingsets come in. +Since the syntax of graph patterns is limited, you cannot express that the `ex:hasChild` relation between a parent and a child is a one-to-many relation. +So, how does the Knowledge Engine exchange the RDF data about parents and their children above? +This is where binding sets come in. -A bindingset provides a collection of 'solutions' to the graph pattern, i.e. combinations of values for the available variables (those start with a question mark `?`) in the graph pattern. So, in the graph pattern above there are two variables (note that both variables occur twice). A binding set consists of zero or more bindings and each binding represents a single mapping of some or all of the variables to a value. For example, when the graph pattern above is applied to the RDF data above, this results in the following bindingset (note that we use JSON here to express the bindingset): +A binding set provides a collection of 'solutions' to the graph pattern, i.e. combinations of values for the available variables (those start with a question mark `?`) in the graph pattern. +So, in the graph pattern above there are two variables (note that both variables occur twice). +A binding set consists of zero or more bindings and each binding represents a single mapping of some or all of the variables to a value. +For example, when the graph pattern above is applied to the RDF data above, this results in the following binding set (note that we use JSON here to express the binding set): ```json [ @@ -155,11 +175,12 @@ A bindingset provides a collection of 'solutions' to the graph pattern, i.e. com ] ``` -As you can see, the one-to-many relations in the RDF data is represented in the bindingset by having 3 bindings. Note that all bindings have the same value for the `parent` variable. +As you can see, the one-to-many relations in the RDF data is represented in the binding set by having 3 bindings. +Note that all bindings have the same value for the `parent` variable. -### Hierarchy example +### Hierarchy Example -Imagine your graph pattern looks something like this (note that we use a non existing ontology): +Imagine your graph pattern looks something like this (note that we use a non-existing ontology): ```sparql ?ts rdf:type ex:TimeSeries . @@ -187,7 +208,7 @@ Imagine you have JSON data of the form: } ``` -How would the bindingset look like that represents the JSON corresponding to the graph pattern? +How would the binding set look like that represents the JSON corresponding to the graph pattern? ```json [ diff --git a/docs/docs/quickstart.md b/docs/docs/quickstart.md deleted file mode 100644 index 3c4248b7..00000000 --- a/docs/docs/quickstart.md +++ /dev/null @@ -1,33 +0,0 @@ ---- - sidebar_position: 3 ---- - -# Quickstart - -Setting up a Knowledge Network requires 3 steps: -1. Start a Knowledge Directory -2. Start one (or more) Smart Connectors. -3. Register Knowledge Interactions - -## Starting the Knowledge Directory -Start the Knowledge Directory on ports 8080: -```bash -cd knowledge-directory/target/ - -java -Dorg.slf4j.simpleLogger.logFile=kd.log -cp "knowledge-directory-1.2.5.jar:dependency/*" eu.knowledge.engine.knowledgedirectory.Main 8080 -``` -You can of course run the Knowledge Directory on another port by replacing 8080 by your preferred port number. - -## Starting a Smart Connector -After starting a Knowledge Directory, one can start smart connectors to join the network via: -```bash -cd smart-connector-rest-dist/target - -export KD_URL=http://localhost:8080 -export KE_RUNTIME_EXPOSED_URL=http://localhost:8081 -export KE_RUNTIME_PORT=8081 - -java -Dorg.slf4j.simpleLogger.logFile=ke.log -cp "smart-connector-rest-dist-1.2.5.jar:dependency/*" eu.knowledge.engine.rest.Main 8280 -``` - -## Registering Knowledge Interactions \ No newline at end of file diff --git a/docs/static/img/graph-pattern-pitfall.drawio b/docs/static/img/graph-pattern-pitfall.drawio new file mode 100644 index 00000000..6490fbba --- /dev/null +++ b/docs/static/img/graph-pattern-pitfall.drawio @@ -0,0 +1,83 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/static/img/graph-pattern-pitfall.png b/docs/static/img/graph-pattern-pitfall.png new file mode 100644 index 00000000..71d79a66 Binary files /dev/null and b/docs/static/img/graph-pattern-pitfall.png differ diff --git a/examples/java-api/pom.xml b/examples/java-api/pom.xml index e1a8588f..d3c048da 100644 --- a/examples/java-api/pom.xml +++ b/examples/java-api/pom.xml @@ -43,7 +43,7 @@ org.slf4j slf4j-simple - 2.0.13 + 2.0.16 diff --git a/examples/rest-api/docker-compose.yml b/examples/rest-api/docker-compose.yml index 20783f8f..a21d49db 100644 --- a/examples/rest-api/docker-compose.yml +++ b/examples/rest-api/docker-compose.yml @@ -2,7 +2,7 @@ services: knowledge-engine: image: ghcr.io/tno/knowledge-engine/smart-connector:1.2.5 healthcheck: - test: curl -f http://localhost:8280/rest/sc + test: wget http://localhost:8280/rest/sc -O /dev/null interval: 1s sensor: build: diff --git a/knowledge-directory/pom.xml b/knowledge-directory/pom.xml index 4f72a41c..ca319ffc 100644 --- a/knowledge-directory/pom.xml +++ b/knowledge-directory/pom.xml @@ -14,13 +14,12 @@ - 2.2.22 + 2.2.26 3.1.0 - 11.0.15 + 11.0.24 3.1.9 - 2.18.1 + 2.18.2 4.13.1 - 1.4.14 UTF-8 @@ -36,12 +35,6 @@ swagger-jaxrs2-servlet-initializer-v2-jakarta ${swagger-core-version} - - ch.qos.logback - logback-core - ${logback-version} - compile - org.junit.jupiter junit-jupiter-api @@ -124,7 +117,7 @@ org.slf4j slf4j-simple - 2.0.13 + 2.0.16 diff --git a/reasoner/pom.xml b/reasoner/pom.xml index 355254c5..bf282b0d 100644 --- a/reasoner/pom.xml +++ b/reasoner/pom.xml @@ -39,7 +39,7 @@ org.slf4j slf4j-simple - 2.0.13 + 2.0.16 diff --git a/reasoner/src/main/java/eu/knowledge/engine/reasoner/api/TriplePattern.java b/reasoner/src/main/java/eu/knowledge/engine/reasoner/api/TriplePattern.java index dd67c766..ee66c117 100644 --- a/reasoner/src/main/java/eu/knowledge/engine/reasoner/api/TriplePattern.java +++ b/reasoner/src/main/java/eu/knowledge/engine/reasoner/api/TriplePattern.java @@ -182,4 +182,8 @@ public boolean equals(Object obj) { return true; } + public Triple asTriple() { + return Triple.create(this.getSubject(), this.getPredicate(), this.getObject()); + } + } diff --git a/smart-connector-api/pom.xml b/smart-connector-api/pom.xml index 545faa38..b7f5c165 100644 --- a/smart-connector-api/pom.xml +++ b/smart-connector-api/pom.xml @@ -15,7 +15,7 @@ org.slf4j slf4j-simple - 2.0.13 + 2.0.16 eu.knowledge.engine diff --git a/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/AnswerKnowledgeInteraction.java b/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/AnswerKnowledgeInteraction.java index 0992ae1d..a12653a3 100644 --- a/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/AnswerKnowledgeInteraction.java +++ b/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/AnswerKnowledgeInteraction.java @@ -42,7 +42,7 @@ public AnswerKnowledgeInteraction(CommunicativeAct anAct, GraphPattern aPattern, */ public AnswerKnowledgeInteraction(CommunicativeAct anAct, GraphPattern aPattern, String aName, boolean anIsMeta, boolean anIncludeMetaKIs, MatchStrategy aMatchStrategy) { - super(anAct, aName, anIsMeta, anIncludeMetaKIs, aMatchStrategy); + super(anAct, aName, anIsMeta, anIncludeMetaKIs, false, aMatchStrategy); this.pattern = aPattern; } diff --git a/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/AskKnowledgeInteraction.java b/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/AskKnowledgeInteraction.java index 8fc7e2f2..3457bf6e 100644 --- a/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/AskKnowledgeInteraction.java +++ b/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/AskKnowledgeInteraction.java @@ -17,20 +17,21 @@ public final class AskKnowledgeInteraction extends KnowledgeInteraction { private final GraphPattern pattern; public AskKnowledgeInteraction(CommunicativeAct act, GraphPattern pattern) { - this(act, pattern, null, false, false, null); + this(act, pattern, null, false, false, false, null); } - public AskKnowledgeInteraction(CommunicativeAct act, GraphPattern pattern, String name) { - this(act, pattern, name, false, false, null); + public AskKnowledgeInteraction(CommunicativeAct act, GraphPattern pattern, String name, + boolean aKnowledgeGapsEnabled) { + this(act, pattern, name, false, false, aKnowledgeGapsEnabled, null); } public AskKnowledgeInteraction(CommunicativeAct act, GraphPattern pattern, boolean anIncludeMetaKIs) { - this(act, pattern, null, false, anIncludeMetaKIs, null); + this(act, pattern, null, false, anIncludeMetaKIs, false, null); } public AskKnowledgeInteraction(CommunicativeAct act, GraphPattern pattern, boolean anisMeta, boolean anIncludeMetaKIs) { - this(act, pattern, null, anisMeta, anIncludeMetaKIs, null); + this(act, pattern, null, anisMeta, anIncludeMetaKIs, false, null); } /** @@ -41,8 +42,8 @@ public AskKnowledgeInteraction(CommunicativeAct act, GraphPattern pattern, boole * that this {@link KnowledgeInteraction} asks for. */ public AskKnowledgeInteraction(CommunicativeAct act, GraphPattern pattern, String name, boolean anisMeta, - boolean anIncludeMetaKIs, MatchStrategy aMatchStrategy) { - super(act, name, anisMeta, anIncludeMetaKIs, aMatchStrategy); + boolean anIncludeMetaKIs, boolean aKnowledgeGapsEnabled, MatchStrategy aMatchStrategy) { + super(act, name, anisMeta, anIncludeMetaKIs, aKnowledgeGapsEnabled, aMatchStrategy); this.pattern = pattern; } diff --git a/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/AskResult.java b/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/AskResult.java index c90dfd8a..28e4d196 100644 --- a/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/AskResult.java +++ b/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/AskResult.java @@ -9,6 +9,7 @@ import org.slf4j.LoggerFactory; import eu.knowledge.engine.reasoner.ReasonerPlan; +import eu.knowledge.engine.reasoner.api.TriplePattern; /** * An {@link AskResult} contains the result of the @@ -26,6 +27,8 @@ public class AskResult { * Can be null, if the matcher is used instead of the reasoner. */ private ReasonerPlan reasonerPlan; + + private Set knowledgeGaps; private final Set exchangeInfos; @@ -38,14 +41,15 @@ public class AskResult { * value for every available variable in the * {@link GraphPattern}. */ - public AskResult(BindingSet someBindings, Set askExchangeInfos, ReasonerPlan aRootNode) { + public AskResult(BindingSet someBindings, Set askExchangeInfos, ReasonerPlan aRootNode, Set kGaps) { this.bindings = someBindings; this.exchangeInfos = askExchangeInfos; this.reasonerPlan = aRootNode; + this.knowledgeGaps = kGaps; } public AskResult(BindingSet someBindings, Set askExchangeInfos) { - this(someBindings, askExchangeInfos, null); + this(someBindings, askExchangeInfos, null, null); } /** @@ -99,8 +103,12 @@ public ReasonerPlan getReasonerPlan() { return this.reasonerPlan; } + public Set getKnowledgeGaps() { + return this.knowledgeGaps; + } + @Override public String toString() { - return "AskResult [bindings=" + bindings + ", exchangeInfoPerKnowledgeBase=" + exchangeInfos + "]"; + return "AskResult [bindings=" + bindings + ", exchangeInfoPerKnowledgeBase=" + exchangeInfos + ", knowledgeGaps=" + knowledgeGaps + "]"; } } diff --git a/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/KnowledgeGap.java b/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/KnowledgeGap.java new file mode 100644 index 00000000..0b498b62 --- /dev/null +++ b/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/KnowledgeGap.java @@ -0,0 +1,31 @@ +package eu.knowledge.engine.smartconnector.api; + +import java.util.HashSet; +import java.util.Set; + +import eu.knowledge.engine.reasoner.api.TriplePattern; + +/** + * A knowledge gap consists of a set of {@link TriplePattern}s that are missing and need ALL + * to be present in order to satisfy an ask/post interaction. + * + * Note that there can be multiple knowledge gaps in an ask or post interaction. + * They will be combined in a Sets that contains one or more + * {@link KnowledgeGap}s in an OR fashion. + * + * */ +public class KnowledgeGap extends HashSet { + + private static final long serialVersionUID = 1L; + + public KnowledgeGap() { + } + + /** + * Construct a KnowledgeGap from a set of TriplePatterns tps + */ + public KnowledgeGap(Set tps) { + this.addAll(tps); + } + +} diff --git a/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/KnowledgeInteraction.java b/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/KnowledgeInteraction.java index 8ddcf4c2..9f3b72f6 100644 --- a/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/KnowledgeInteraction.java +++ b/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/KnowledgeInteraction.java @@ -1,6 +1,5 @@ package eu.knowledge.engine.smartconnector.api; - /** * A {@link KnowledgeInteraction} represents an agreement about the exchange of * knowledge between the {@link SmartConnectorImpl} and the @@ -25,6 +24,8 @@ public abstract class KnowledgeInteraction { */ private final boolean includeMetaKIs; + private final boolean knowledgeGapsEnabled; + /** * {@code true} if this Knowledge Interaction is used for internal knowledge * engine communication. @@ -52,15 +53,15 @@ public abstract class KnowledgeInteraction { * whether it has side-effects or not. */ public KnowledgeInteraction(CommunicativeAct act) { - this(act, null, false, false, null); + this(act, null, false, false, false, null); } public KnowledgeInteraction(CommunicativeAct act, boolean isMeta) { - this(act, null, isMeta, false, null); + this(act, null, isMeta, false, false, null); } public KnowledgeInteraction(CommunicativeAct act, boolean isMeta, boolean anIncludeMetaKIs) { - this(act, null, isMeta, anIncludeMetaKIs, null); + this(act, null, isMeta, anIncludeMetaKIs, false, null); } /** @@ -84,13 +85,14 @@ public KnowledgeInteraction(CommunicativeAct act, boolean isMeta, boolean anIncl * used. */ public KnowledgeInteraction(CommunicativeAct act, String name, boolean isMeta, boolean anIncludeMetaKIs, - MatchStrategy aMatchConfig) { + boolean aKnowledgeGapsEnabled, MatchStrategy aMatchConfig) { this.validateName(name); this.anAct = act; this.name = name; this.isMeta = isMeta; this.includeMetaKIs = anIncludeMetaKIs; this.matchStrategy = aMatchConfig; + this.knowledgeGapsEnabled = aKnowledgeGapsEnabled; } public String getName() { @@ -116,6 +118,10 @@ public MatchStrategy getMatchStrategy() { return this.matchStrategy; } + public boolean knowledgeGapsEnabled() { + return this.knowledgeGapsEnabled; + } + /** * Throws an exception if {@code name} does not conform to the requirements of * knowledge interaction names. diff --git a/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/PostKnowledgeInteraction.java b/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/PostKnowledgeInteraction.java index 17625d06..0add9e7c 100644 --- a/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/PostKnowledgeInteraction.java +++ b/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/PostKnowledgeInteraction.java @@ -57,7 +57,7 @@ public PostKnowledgeInteraction(CommunicativeAct act, GraphPattern argument, Gra */ public PostKnowledgeInteraction(CommunicativeAct act, GraphPattern argument, GraphPattern result, String name, boolean anIsMeta, boolean anIncludeMetaKIs, MatchStrategy aMatchStrategy) { - super(act, name, anIsMeta, anIncludeMetaKIs, aMatchStrategy); + super(act, name, anIsMeta, anIncludeMetaKIs, false, aMatchStrategy); this.argument = argument; this.result = result; } diff --git a/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/ReactKnowledgeInteraction.java b/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/ReactKnowledgeInteraction.java index aa9951ba..312bb412 100644 --- a/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/ReactKnowledgeInteraction.java +++ b/smart-connector-api/src/main/java/eu/knowledge/engine/smartconnector/api/ReactKnowledgeInteraction.java @@ -19,6 +19,19 @@ public final class ReactKnowledgeInteraction extends KnowledgeInteraction { */ private final GraphPattern result; + /** + * Creates a {@link ReactKnowledgeInteraction}. See + * {@link KnowledgeInteraction#KnowledgeInteraction(CommunicativeAct, String, boolean, boolean, MatchStrategy)} + * + * @param argument The {@code argument} of this {@link KnowledgeInteraction}. It + * can be seen as the argument of a function call. + * @param result The {@code result} of this {@link PostKnowledgeInteraction}. + * It can be seen as the result of a function call. Can be + * {@code null} if this interaction does not expect any result. + * @apiNote TODO Can {@code argument} also be {@code null}? No, only the + * {@code result} graph pattern can be {@code null}. Note that not both + * {@code argument} and {@code result} can be {@code null}. + */ public ReactKnowledgeInteraction(CommunicativeAct act, GraphPattern argument, GraphPattern result) { this(act, argument, result, null, false, false, null); } @@ -28,31 +41,18 @@ public ReactKnowledgeInteraction(CommunicativeAct act, GraphPattern argument, Gr } public ReactKnowledgeInteraction(CommunicativeAct act, GraphPattern argument, GraphPattern result, - boolean anIsFullMatch) { - this(act, argument, result, null, false, anIsFullMatch, null); + boolean anIncludeMetaKIs) { + this(act, argument, result, null, false, anIncludeMetaKIs, null); } public ReactKnowledgeInteraction(CommunicativeAct act, GraphPattern argument, GraphPattern result, boolean anIsMeta, - boolean anIsFullMatch) { - this(act, argument, result, null, anIsMeta, anIsFullMatch, null); + boolean anIncludeMetaKIs) { + this(act, argument, result, null, anIsMeta, anIncludeMetaKIs, null); } - /** - * Creates a {@link ReactKnowledgeInteraction}. See - * {@link KnowledgeInteraction#KnowledgeInteraction(CommunicativeAct, String, boolean, boolean, MatchStrategy)} - * - * @param argument The {@code argument} of this {@link KnowledgeInteraction}. It - * can be seen as the argument of a function call. - * @param result The {@code result} of this {@link PostKnowledgeInteraction}. - * It can be seen as the result of a function call. Can be - * {@code null} if this interaction does not expect any result. - * @apiNote TODO Can {@code argument} also be {@code null}? No, only the - * {@code result} graph pattern can be {@code null}. Note that not both - * {@code argument} and {@code result} can be {@code null}. - */ public ReactKnowledgeInteraction(CommunicativeAct act, GraphPattern argument, GraphPattern result, String name, - boolean anIsMeta, boolean anIsFullMatch, MatchStrategy aMatchStrategy) { - super(act, name, anIsMeta, anIsFullMatch, aMatchStrategy); + boolean anIsMeta, boolean anIncludeMetaKIs, MatchStrategy aMatchStrategy) { + super(act, name, anIsMeta, anIncludeMetaKIs, false, aMatchStrategy); this.argument = argument; this.result = result; } diff --git a/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestAskAnswerReactWithGapsEnabled.java b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestAskAnswerReactWithGapsEnabled.java new file mode 100644 index 00000000..ef77a5b8 --- /dev/null +++ b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestAskAnswerReactWithGapsEnabled.java @@ -0,0 +1,240 @@ +package eu.knowledge.engine.rest.api; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.fail; + +import java.io.IOException; +import java.io.StringReader; +import java.net.MalformedURLException; +import java.net.URL; +import java.util.HashSet; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.CountDownLatch; + +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestInstance; + +import eu.knowledge.engine.rest.RestServerHelper; +import eu.knowledge.engine.test_utils.AsyncTester; +import eu.knowledge.engine.test_utils.HttpTester; +import jakarta.json.Json; +import jakarta.json.JsonArray; +import jakarta.json.JsonObject; +import jakarta.json.JsonObjectBuilder; +import jakarta.json.JsonReader; +import jakarta.json.JsonString; +import jakarta.json.JsonValue; + +@TestInstance(TestInstance.Lifecycle.PER_CLASS) +public class TestAskAnswerReactWithGapsEnabled { + private final RestServerHelper rsh = new RestServerHelper(); + private static int PORT = 8280; + + @BeforeAll + public void setUpServer() { + rsh.start(PORT); + } + + @Test + public void testAskAnswerReactWithGaps() throws IOException, InterruptedException { + + // In this test there will be an Ask KB with an AskKI with 2 triplepatterns, + // an AnswerKB with a single AnswerKI that answers only the first triplepattern + // of the Ask pattern, and + // a ReactKB that can answer the other triplepattern of the Ask, but needs + // another pattern to be satisfied. + // The test will execute the AskKI with knowledge gaps enabled. + // As a result, the set of knowledge gaps should contain a single gap. + + URL url = new URL("http://localhost:" + PORT + "/rest"); + + // sync between threads to make sure the ask is not activated before the others + // are ready. + CountDownLatch KBReady = new CountDownLatch(2); + + // activate the answer SC, KB, KI in a separate thread + var answeringSc = new AsyncTester(new Runnable() { + @Override + public void run() { + String answerKBId = "https://www.tno.nl/example/relationProvider"; + String answerKIId = answerKBId + "/interaction/answerRelations"; + try { + // register the AnswerKB + HttpTester registerAnswerKb = new HttpTester(new URL(url + "/sc"), "POST", + "{\"knowledgeBaseId\": \"https://www.tno.nl/example/relationProvider\", \"knowledgeBaseName\": \"RelationProvider\", \"knowledgeBaseDescription\": \"A KB that provides relations between people\", \"reasonerEnabled\" : true}", + Map.of("Content-Type", "application/json", "Accept", "*/*")); + registerAnswerKb.expectStatus(200); + + // register the AnswerKI + HttpTester registerAnswerKi = new HttpTester(new URL(url + "/sc/ki"), "POST", """ + { + "knowledgeInteractionType": "AnswerKnowledgeInteraction", + "knowledgeInteractionName": "answerRelations", + "graphPattern": "?a ?b ." + } + """, Map.of("Knowledge-Base-Id", "https://www.tno.nl/example/relationProvider", + "Content-Type", "application/json", "Accept", "*/*")); + registerAnswerKi.expectStatus(200); + + KBReady.countDown(); + + // get the handle for the answerKB to see if there are requests to be handled + var test = new HttpTester(new URL(url.toString() + "/sc/handle"), "GET", null, Map + .of("Knowledge-Base-Id", answerKBId, "Content-Type", "application/json", "Accept", "*/*")); + test.expectStatus(200); + + // build the body to answer the request: add handle request ID and dummy data + // bindingset + JsonObjectBuilder builder = Json.createObjectBuilder(); + JsonReader jp = Json.createReader(new StringReader(test.getBody())); + JsonObject jo = jp.readObject(); + int handleRequestId = jo.getInt("handleRequestId"); + builder.add("handleRequestId", handleRequestId); + JsonReader jr = Json.createReader(new StringReader( + "[{\"a\": \"\",\"b\": \"\"}]")); + JsonArray bs = jr.readArray(); + builder.add("bindingSet", bs); + JsonObject jo2 = builder.build(); + String body = jo2.toString(); + System.out.println("Handle an answer to a request with body: " + body); + + // fire the POST handle to execute the answer + var test2 = new HttpTester(new URL(url.toString() + "/sc/handle"), "POST", body, + Map.of("Knowledge-Base-Id", answerKBId, "Knowledge-Interaction-Id", answerKIId, + "Content-Type", "application/json", "Accept", "*/*")); + test2.expectStatus(200); + + } catch (MalformedURLException e) { + fail(); + } + } + }); + answeringSc.start(); + + // activate the answer SC, KB, KI in a separate thread + var reactingSc = new AsyncTester(new Runnable() { + @Override + public void run() { + String reactKBId = "https://www.tno.nl/example/relationReactor"; + String reactKIId = reactKBId + "/interaction/reactRelations"; + try { + // register the ReactKB + HttpTester registerReactKb = new HttpTester(new URL(url + "/sc"), "POST", + "{\"knowledgeBaseId\": \"https://www.tno.nl/example/relationReactor\", \"knowledgeBaseName\": \"RelationReactor\", \"knowledgeBaseDescription\": \"A KB that reacts to supply related people\", \"reasonerEnabled\" : true}", + Map.of("Content-Type", "application/json", "Accept", "*/*")); + registerReactKb.expectStatus(200); + + // register the ReactKI + HttpTester registerReactKi = new HttpTester(new URL(url + "/sc/ki"), "POST", """ + { + "knowledgeInteractionType": "ReactKnowledgeInteraction", + "knowledgeInteractionName": "reactRelations", + "argumentGraphPattern": "?a ?b .", + "resultGraphPattern": "?a ?b ." + } + """, Map.of("Knowledge-Base-Id", "https://www.tno.nl/example/relationReactor", + "Content-Type", "application/json", "Accept", "*/*")); + registerReactKi.expectStatus(200); + + KBReady.countDown(); + + System.out.println("Getting the handle for the reactKBId"); + // get the handle for the reactKB to see if there are requests to be handled => + // NOTE: it should never exit/return this handle for this test + var test = new HttpTester(new URL(url.toString() + "/sc/handle"), "GET", null, Map + .of("Knowledge-Base-Id", reactKBId, "Content-Type", "application/json", "Accept", "*/*")); + test.expectStatus(200); + + // build the body to react to the request: add handle request ID and dummy data + // bindingset + JsonObjectBuilder builder = Json.createObjectBuilder(); + JsonReader jp = Json.createReader(new StringReader(test.getBody())); + JsonObject jo = jp.readObject(); + int handleRequestId = jo.getInt("handleRequestId"); + builder.add("handleRequestId", handleRequestId); + // for now simply add an empty bindingset as a result + builder.add("bindingSet", JsonObject.EMPTY_JSON_ARRAY); + JsonObject jo2 = builder.build(); + String body = jo2.toString(); + System.out.println("Handle a react to a request with body: " + body); + + // fire the POST handle to execute the react + var test2 = new HttpTester(new URL(url.toString() + "/sc/handle"), "POST", body, + Map.of("Knowledge-Base-Id", reactKBId, "Knowledge-Interaction-Id", reactKIId, + "Content-Type", "application/json", "Accept", "*/*")); + test2.expectStatus(200); + + } catch (MalformedURLException e) { + fail(); + } + } + }); + reactingSc.start(); + + KBReady.await(); + + // register the AskKB + HttpTester registerKb = new HttpTester(new URL(url + "/sc"), "POST", + "{\"knowledgeBaseId\": \"https://www.tno.nl/example/relationAsker\", \"knowledgeBaseName\": \"RelationAsker\", \"knowledgeBaseDescription\": \"A KB that asks for relations between people\", \"reasonerEnabled\" : true}", + Map.of("Content-Type", "application/json", "Accept", "*/*")); + registerKb.expectStatus(200); + + // register the AskKI + HttpTester registerKiWithoutGapsEnabled = new HttpTester(new URL(url + "/sc/ki"), "POST", """ + { + "knowledgeInteractionType": "AskKnowledgeInteraction", + "knowledgeInteractionName": "askRelations", + "graphPattern": "?a ?b . ?a ?c .", + "knowledgeGapsEnabled": true + } + """, Map.of("Knowledge-Base-Id", "https://www.tno.nl/example/relationAsker", "Content-Type", + "application/json", "Accept", "*/*")); + registerKiWithoutGapsEnabled.expectStatus(200); + + // fire the ask KI + HttpTester askKiWithoutGapsEnabled = new HttpTester(new URL(url + "/sc/ask"), "POST", + "{\"recipientSelector\": {\"knowledgeBases\": []}, \"bindingSet\": []}", + Map.of("Knowledge-Base-Id", "https://www.tno.nl/example/relationAsker", "Knowledge-Interaction-Id", + "https://www.tno.nl/example/relationAsker/interaction/askRelations", "Content-Type", + "application/json", "Accept", "*/*")); + var result = askKiWithoutGapsEnabled.getBody(); + System.out.println("Result is:" + result); + + JsonObject jsonObject = Json.createReader(new StringReader(result)).readObject(); + JsonArray knowledgeGapsArray = jsonObject.getJsonArray("knowledgeGaps"); + + // tried to convert it using streams, but failed. So, just using for old + // fashioned loops :( + Set> actualKnowledgeGaps = new HashSet>(); + Set set; + for (JsonValue jv : knowledgeGapsArray) { + set = new HashSet(); + for (JsonString js : jv.asJsonArray().getValuesAs(JsonString.class)) { + set.add(js.getString()); + } + actualKnowledgeGaps.add(set); + } + + var expectedKnowledgeGaps = Set + .of(Set.of("?a ?c", "?a ?b")); + assertEquals(expectedKnowledgeGaps, actualKnowledgeGaps); + + JsonArray bindingArray = jsonObject.getJsonArray("bindingSet"); + var actualBindingSet = Set.of(bindingArray.toArray()); + var expectedBindingSet = Set.>of( /* empty */ ); + assertEquals(expectedBindingSet, actualBindingSet); + + } + + @AfterAll + public void cleanUp() throws IOException { + + TestUtil.unregisterAllKBs("http://localhost:" + PORT + "/rest"); + + rsh.cleanUp(); + } + +} diff --git a/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestAskAnswerWithGapsEnabled.java b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestAskAnswerWithGapsEnabled.java new file mode 100644 index 00000000..16541abc --- /dev/null +++ b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestAskAnswerWithGapsEnabled.java @@ -0,0 +1,137 @@ +package eu.knowledge.engine.rest.api; + +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.fail; + +import java.io.IOException; +import java.io.StringReader; +import java.net.MalformedURLException; +import java.net.URL; +import java.util.Map; +import java.util.concurrent.CountDownLatch; + +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestInstance; + +import eu.knowledge.engine.rest.RestServerHelper; +import eu.knowledge.engine.test_utils.AsyncTester; +import eu.knowledge.engine.test_utils.HttpTester; +import jakarta.json.Json; +import jakarta.json.JsonArray; +import jakarta.json.JsonObject; +import jakarta.json.JsonObjectBuilder; +import jakarta.json.JsonReader; + +@TestInstance(TestInstance.Lifecycle.PER_CLASS) +public class TestAskAnswerWithGapsEnabled { + private final RestServerHelper rsh = new RestServerHelper(); + private static int PORT = 8280; + + @BeforeAll + public void setUpServer() { + rsh.start(PORT); + } + + @Test + public void testAskAnswerWithGaps() throws IOException, InterruptedException { + + // In this test there will be an Ask KB with a single AskKI and + // an AnswerKB with a single AnswerKI that answers only part of the Ask pattern. + // The test will execute the AskKI with knowledge gaps enabled. + // As a result, the set of knowledge gaps should contain a single gap. + + CountDownLatch KBReady = new CountDownLatch(1); + + URL url = new URL("http://localhost:" + PORT + "/rest"); + + // activate the answer SC, KB, KI in a separate thread + var answeringSc = new AsyncTester(new Runnable() { + @Override + public void run() { + String answerKBId = "https://www.tno.nl/example/relationProvider"; + String answerKIId = answerKBId + "/interaction/answerRelations"; + try { + // register the AnswerKB + HttpTester registerAnswerKb = new HttpTester(new URL(url + "/sc"), "POST", + "{\"knowledgeBaseId\": \"https://www.tno.nl/example/relationProvider\", \"knowledgeBaseName\": \"RelationProvider\", \"knowledgeBaseDescription\": \"A KB that provides relations between people\", \"reasonerEnabled\" : true}", + Map.of("Content-Type", "application/json", "Accept", "*/*")); + registerAnswerKb.expectStatus(200); + + // register the AnswerKI + HttpTester registerAnswerKi = new HttpTester(new URL(url + "/sc/ki"), "POST", + "{\"knowledgeInteractionType\": \"AnswerKnowledgeInteraction\", \"knowledgeInteractionName\": \"answerRelations\", \"graphPattern\": \"?a ?b .\"}", + Map.of("Knowledge-Base-Id", "https://www.tno.nl/example/relationProvider", "Content-Type", + "application/json", "Accept", "*/*")); + registerAnswerKi.expectStatus(200); + + KBReady.countDown(); + // get the handle for the answerKB to see if there are requests to be handled + var test = new HttpTester(new URL(url.toString() + "/sc/handle"), "GET", null, Map + .of("Knowledge-Base-Id", answerKBId, "Content-Type", "application/json", "Accept", "*/*")); + test.expectStatus(200); + + // build the body to answer the request: add handle request ID and dummy data + // bindingset + JsonObjectBuilder builder = Json.createObjectBuilder(); + JsonReader jp = Json.createReader(new StringReader(test.getBody())); + JsonObject jo = jp.readObject(); + int handleRequestId = jo.getInt("handleRequestId"); + builder.add("handleRequestId", handleRequestId); + JsonReader jr = Json.createReader(new StringReader( + "[{\"a\": \"\",\"b\": \"\"}]")); + JsonArray bs = jr.readArray(); + builder.add("bindingSet", bs); + JsonObject jo2 = builder.build(); + String body = jo2.toString(); + System.out.println("Handle an answer to a request with body: " + body); + + // fire the POST handle to execute the answer + var test2 = new HttpTester(new URL(url.toString() + "/sc/handle"), "POST", body, + Map.of("Knowledge-Base-Id", answerKBId, "Knowledge-Interaction-Id", answerKIId, + "Content-Type", "application/json", "Accept", "*/*")); + test2.expectStatus(200); + + } catch (MalformedURLException e) { + fail(); + } + } + }); + answeringSc.start(); + + KBReady.await(); + // register the AskKB + HttpTester registerKb = new HttpTester(new URL(url + "/sc"), "POST", + "{\"knowledgeBaseId\": \"https://www.tno.nl/example/relationAsker\", \"knowledgeBaseName\": \"RelationAsker\", \"knowledgeBaseDescription\": \"A KB that asks for relations between people\", \"reasonerEnabled\" : true}", + Map.of("Content-Type", "application/json", "Accept", "*/*")); + registerKb.expectStatus(200); + + + // register the AskKI + HttpTester registerKiWithoutGapsEnabled = new HttpTester(new URL(url + "/sc/ki"), "POST", + "{\"knowledgeInteractionType\": \"AskKnowledgeInteraction\", \"knowledgeInteractionName\": \"askRelations\", \"graphPattern\": \"?a ?b . ?a ?c .\", \"knowledgeGapsEnabled\": true}", + Map.of("Knowledge-Base-Id", "https://www.tno.nl/example/relationAsker", "Content-Type", + "application/json", "Accept", "*/*")); + registerKiWithoutGapsEnabled.expectStatus(200); + + // fire the ask KI + HttpTester askKiWithoutGapsEnabled = new HttpTester(new URL(url + "/sc/ask"), "POST", + "{\"recipientSelector\": {\"knowledgeBases\": []}, \"bindingSet\": []}", + Map.of("Knowledge-Base-Id", "https://www.tno.nl/example/relationAsker", "Knowledge-Interaction-Id", + "https://www.tno.nl/example/relationAsker/interaction/askRelations", "Content-Type", + "application/json", "Accept", "*/*")); + var result = askKiWithoutGapsEnabled.getBody(); + System.out.println("Result is:" + result); + assertTrue(result.contains("\"knowledgeGaps\":[[\"?a ?c\"]]")); + + } + + @AfterAll + public void cleanUp() throws MalformedURLException { + + TestUtil.unregisterAllKBs("http://localhost:" + PORT + "/rest"); + rsh.cleanUp(); + } + +} diff --git a/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestAskAnswerWithGapsEnabledNoGaps.java b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestAskAnswerWithGapsEnabledNoGaps.java new file mode 100644 index 00000000..8ef721fe --- /dev/null +++ b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestAskAnswerWithGapsEnabledNoGaps.java @@ -0,0 +1,137 @@ +package eu.knowledge.engine.rest.api; + +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.fail; + +import java.io.IOException; +import java.io.StringReader; +import java.net.MalformedURLException; +import java.net.URL; +import java.util.Map; +import java.util.concurrent.CountDownLatch; + +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestInstance; + +import eu.knowledge.engine.rest.RestServerHelper; +import eu.knowledge.engine.test_utils.AsyncTester; +import eu.knowledge.engine.test_utils.HttpTester; +import jakarta.json.Json; +import jakarta.json.JsonArray; +import jakarta.json.JsonObject; +import jakarta.json.JsonObjectBuilder; +import jakarta.json.JsonReader; + +@TestInstance(TestInstance.Lifecycle.PER_CLASS) +public class TestAskAnswerWithGapsEnabledNoGaps { + private final RestServerHelper rsh = new RestServerHelper(); + private static int PORT = 8280; + + @BeforeAll + public void setUpServer() { + rsh.start(PORT); + } + + @Test + public void testAskAnswerWithGaps() throws IOException, InterruptedException { + + // In this test there will be an Ask KB with a single AskKI and + // an AnswerKB with a single AnswerKI that answers only part of the Ask pattern. + // The test will execute the AskKI with knowledge gaps enabled. + // As a result, there are no knowledge gaps, so the set should be empty. + + CountDownLatch KBReady = new CountDownLatch(1); + + URL url = new URL("http://localhost:" + PORT + "/rest"); + + // activate the answer SC, KB, KI in a separate thread + var answeringSc = new AsyncTester(new Runnable() { + @Override + public void run() { + String answerKBId = "https://www.tno.nl/example/relationProvider"; + String answerKIId = answerKBId + "/interaction/answerRelations"; + try { + // register the AnswerKB + HttpTester registerAnswerKb = new HttpTester(new URL(url + "/sc"), "POST", + "{\"knowledgeBaseId\": \"https://www.tno.nl/example/relationProvider\", \"knowledgeBaseName\": \"RelationProvider\", \"knowledgeBaseDescription\": \"A KB that provides relations between people\", \"reasonerEnabled\" : true}", + Map.of("Content-Type", "application/json", "Accept", "*/*")); + registerAnswerKb.expectStatus(200); + + // register the AnswerKI + HttpTester registerAnswerKi = new HttpTester(new URL(url + "/sc/ki"), "POST", + "{\"knowledgeInteractionType\": \"AnswerKnowledgeInteraction\", \"knowledgeInteractionName\": \"answerRelations\", \"graphPattern\": \"?a ?b .\"}", + Map.of("Knowledge-Base-Id", "https://www.tno.nl/example/relationProvider", "Content-Type", + "application/json", "Accept", "*/*")); + registerAnswerKi.expectStatus(200); + + KBReady.countDown(); + // get the handle for the answerKB to see if there are requests to be handled + var test = new HttpTester(new URL(url.toString() + "/sc/handle"), "GET", null, Map + .of("Knowledge-Base-Id", answerKBId, "Content-Type", "application/json", "Accept", "*/*")); + test.expectStatus(200); + + // build the body to answer the request: add handle request ID and dummy data + // bindingset + JsonObjectBuilder builder = Json.createObjectBuilder(); + JsonReader jp = Json.createReader(new StringReader(test.getBody())); + JsonObject jo = jp.readObject(); + int handleRequestId = jo.getInt("handleRequestId"); + builder.add("handleRequestId", handleRequestId); + JsonReader jr = Json.createReader(new StringReader( + "[{\"a\": \"\",\"b\": \"\"}]")); + JsonArray bs = jr.readArray(); + builder.add("bindingSet", bs); + JsonObject jo2 = builder.build(); + String body = jo2.toString(); + System.out.println("Handle an answer to a request with body: " + body); + + // fire the POST handle to execute the answer + var test2 = new HttpTester(new URL(url.toString() + "/sc/handle"), "POST", body, + Map.of("Knowledge-Base-Id", answerKBId, "Knowledge-Interaction-Id", answerKIId, + "Content-Type", "application/json", "Accept", "*/*")); + test2.expectStatus(200); + + } catch (MalformedURLException e) { + fail(); + } + } + }); + answeringSc.start(); + + KBReady.await(); + // register the AskKB + HttpTester registerKb = new HttpTester(new URL(url + "/sc"), "POST", + "{\"knowledgeBaseId\": \"https://www.tno.nl/example/relationAsker\", \"knowledgeBaseName\": \"RelationAsker\", \"knowledgeBaseDescription\": \"A KB that asks for relations between people\", \"reasonerEnabled\" : true}", + Map.of("Content-Type", "application/json", "Accept", "*/*")); + registerKb.expectStatus(200); + + + // register the AskKI + HttpTester registerKiWithoutGapsEnabled = new HttpTester(new URL(url + "/sc/ki"), "POST", + "{\"knowledgeInteractionType\": \"AskKnowledgeInteraction\", \"knowledgeInteractionName\": \"askRelations\", \"graphPattern\": \"?a ?b . \", \"knowledgeGapsEnabled\": true}", + Map.of("Knowledge-Base-Id", "https://www.tno.nl/example/relationAsker", "Content-Type", + "application/json", "Accept", "*/*")); + registerKiWithoutGapsEnabled.expectStatus(200); + + // fire the ask KI + HttpTester askKiWithoutGapsEnabled = new HttpTester(new URL(url + "/sc/ask"), "POST", + "{\"recipientSelector\": {\"knowledgeBases\": []}, \"bindingSet\": []}", + Map.of("Knowledge-Base-Id", "https://www.tno.nl/example/relationAsker", "Knowledge-Interaction-Id", + "https://www.tno.nl/example/relationAsker/interaction/askRelations", "Content-Type", + "application/json", "Accept", "*/*")); + var result = askKiWithoutGapsEnabled.getBody(); + System.out.println("Result is:" + result); + assertTrue(result.contains("\"knowledgeGaps\":[]")); + + } + + @AfterAll + public void cleanUp() throws MalformedURLException { + + TestUtil.unregisterAllKBs("http://localhost:" + PORT + "/rest"); + rsh.cleanUp(); + } + +} diff --git a/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestAskWithGapsEnabled.java b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestAskWithGapsEnabled.java new file mode 100644 index 00000000..ddf1b83c --- /dev/null +++ b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestAskWithGapsEnabled.java @@ -0,0 +1,74 @@ +package eu.knowledge.engine.rest.api; + +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.fail; + +import java.io.IOException; +import java.net.MalformedURLException; +import java.net.URL; +import java.util.Map; + +import org.apache.jena.atlas.logging.Log; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Disabled; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestInstance; + +import eu.knowledge.engine.rest.RestServerHelper; +import eu.knowledge.engine.test_utils.HttpTester; + +@TestInstance(TestInstance.Lifecycle.PER_CLASS) +public class TestAskWithGapsEnabled { + private final RestServerHelper rsh = new RestServerHelper(); + private static int PORT = 8280; + + @BeforeAll + public void setUpServer() { + rsh.start(PORT); + } + + @Test + public void testAskWithGaps() throws IOException { + + // In this test there will be only 1 KB with a single AskKI. + // The test will execute the AskKI with knowledge gaps enabled. + // As a result, the set of knowledge gaps should contain a single gap. + + URL url = new URL("http://localhost:" + PORT + "/rest"); + + HttpTester registerKb = new HttpTester(new URL(url + "/sc"), "POST", + "{\"knowledgeBaseId\": \"https://www.tno.nl/example/relationAsker\", \"knowledgeBaseName\": \"RelationAsker\", \"knowledgeBaseDescription\": \"A KB that asks for relations between people\", \"reasonerEnabled\" : true}", + Map.of("Content-Type", "application/json", "Accept", "*/*")); + registerKb.expectStatus(200); + + HttpTester registerKiWithoutGapsEnabled = new HttpTester(new URL(url + "/sc/ki"), "POST", + "{\"knowledgeInteractionType\": \"AskKnowledgeInteraction\", \"knowledgeInteractionName\": \"askRelations\", \"graphPattern\": \"?a ?b .\", \"knowledgeGapsEnabled\": true}", + Map.of("Knowledge-Base-Id", "https://www.tno.nl/example/relationAsker", "Content-Type", + "application/json", "Accept", "*/*")); + registerKiWithoutGapsEnabled.expectStatus(200); + + HttpTester getKiWithoutGapsEnabled = new HttpTester(new URL(url + "/sc/ki"), "GET", null, + Map.of("Knowledge-Base-Id", "https://www.tno.nl/example/relationAsker", "Content-Type", + "application/json", "Accept", "*/*")); + var body = getKiWithoutGapsEnabled.getBody(); + assertTrue(body.contains("\"https://www.tno.nl/example/relationAsker/interaction/askRelations\"")); + + HttpTester askKiWithoutGapsEnabled = new HttpTester(new URL(url + "/sc/ask"), "POST", + "{\"recipientSelector\": {\"knowledgeBases\": []}, \"bindingSet\": []}", + Map.of("Knowledge-Base-Id", "https://www.tno.nl/example/relationAsker", "Knowledge-Interaction-Id", + "https://www.tno.nl/example/relationAsker/interaction/askRelations", "Content-Type", + "application/json", "Accept", "*/*")); + var result = askKiWithoutGapsEnabled.getBody(); + System.out.println("Result is:" + result); + assertTrue(result.contains("\"knowledgeGaps\":[[\"?a ?b\"]]")); + } + + @AfterAll + public void cleanUp() { + TestUtil.unregisterAllKBs("http://localhost:" + PORT + "/rest"); + rsh.cleanUp(); + } + +} diff --git a/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestAskWithGapsNotEnabled.java b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestAskWithGapsNotEnabled.java new file mode 100644 index 00000000..8f4a29c0 --- /dev/null +++ b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestAskWithGapsNotEnabled.java @@ -0,0 +1,74 @@ +package eu.knowledge.engine.rest.api; + +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.fail; + +import java.io.IOException; +import java.net.MalformedURLException; +import java.net.URL; +import java.util.Map; + +import org.apache.jena.atlas.logging.Log; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Disabled; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestInstance; + +import eu.knowledge.engine.rest.RestServerHelper; +import eu.knowledge.engine.test_utils.HttpTester; + +@TestInstance(TestInstance.Lifecycle.PER_CLASS) +public class TestAskWithGapsNotEnabled { + private final RestServerHelper rsh = new RestServerHelper(); + private static int PORT = 8280; + + @BeforeAll + public void setUpServer() { + rsh.start(PORT); + } + + @Test + public void testAskWithoutGaps() throws IOException { + + // In this test there will be only 1 KB with a single AskKI. + // The test will execute the AskKI without knowledge gaps enabled. + // As a result, the set of knowledge gaps should be null. + + URL url = new URL("http://localhost:" + PORT + "/rest"); + + HttpTester registerKb = new HttpTester(new URL(url + "/sc"), "POST", + "{\"knowledgeBaseId\": \"https://www.tno.nl/example/relationAsker\", \"knowledgeBaseName\": \"RelationAsker\", \"knowledgeBaseDescription\": \"A KB that asks for relations between people\", \"reasonerEnabled\" : true}", + Map.of("Content-Type", "application/json", "Accept", "*/*")); + registerKb.expectStatus(200); + + HttpTester registerKiWithoutGapsEnabled = new HttpTester(new URL(url + "/sc/ki"), "POST", + "{\"knowledgeInteractionType\": \"AskKnowledgeInteraction\", \"knowledgeInteractionName\": \"askRelations\", \"graphPattern\": \"?a ?b .\"}", + Map.of("Knowledge-Base-Id", "https://www.tno.nl/example/relationAsker", "Content-Type", + "application/json", "Accept", "*/*")); + registerKiWithoutGapsEnabled.expectStatus(200); + + HttpTester getKiWithoutGapsEnabled = new HttpTester(new URL(url + "/sc/ki"), "GET", null, + Map.of("Knowledge-Base-Id", "https://www.tno.nl/example/relationAsker", "Content-Type", + "application/json", "Accept", "*/*")); + var body = getKiWithoutGapsEnabled.getBody(); + assertTrue(body.contains("\"https://www.tno.nl/example/relationAsker/interaction/askRelations\"")); + + HttpTester askKiWithoutGapsEnabled = new HttpTester(new URL(url + "/sc/ask"), "POST", + "{\"recipientSelector\": {\"knowledgeBases\": []}, \"bindingSet\": []}", + Map.of("Knowledge-Base-Id", "https://www.tno.nl/example/relationAsker", "Knowledge-Interaction-Id", + "https://www.tno.nl/example/relationAsker/interaction/askRelations", "Content-Type", + "application/json", "Accept", "*/*")); + var result = askKiWithoutGapsEnabled.getBody(); + System.out.println("Result is:" + result); + assertFalse(result.contains("\"knowledgeGaps\":")); + } + + @AfterAll + public void cleanUp() { + TestUtil.unregisterAllKBs("http://localhost:" + PORT + "/rest"); + + rsh.cleanUp(); + } +} diff --git a/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestPostMemoryLeak.java b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestPostMemoryLeak.java index b3e88b25..65e25d2a 100644 --- a/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestPostMemoryLeak.java +++ b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestPostMemoryLeak.java @@ -177,6 +177,7 @@ public void run() { @AfterAll public void cleanUp() { + TestUtil.unregisterAllKBs("http://localhost:" + PORT + "/rest"); rsh.cleanUp(); } } diff --git a/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestQuickScPosts.java b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestQuickScPosts.java index d3add0cb..797c0544 100644 --- a/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestQuickScPosts.java +++ b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestQuickScPosts.java @@ -32,7 +32,7 @@ public void testPostScsSimultaneously() throws IOException { // You can make the test pass consistently by uncommenting the following: // new HttpTester(url, "GET", null, Map.of( - // "Accept", "*/*" + // "Accept", "*/*" // )).expectStatus(200); // This latch is used to make the threads synchronize before they will @@ -58,12 +58,9 @@ public void run() { } new HttpTester(url, "POST", - "{\"knowledgeBaseId\": \""+ kb1Id + "\", \"knowledgeBaseName\": \"KB1\", \"knowledgeBaseDescription\": \"KB1\"}", - Map.of( - "Content-Type", "application/json", - "Accept", "*/*" - ) - ).expectStatus(200); + "{\"knowledgeBaseId\": \"" + kb1Id + + "\", \"knowledgeBaseName\": \"KB1\", \"knowledgeBaseDescription\": \"KB1\"}", + Map.of("Content-Type", "application/json", "Accept", "*/*")).expectStatus(200); afterRegister.countDown(); try { @@ -72,15 +69,9 @@ public void run() { fail(); } - new HttpTester(url, "GET", null, Map.of( - "Accept", "*/*", - "Knowledge-Base-Id", kb1Id - )).expectStatus(200); + new HttpTester(url, "GET", null, Map.of("Accept", "*/*", "Knowledge-Base-Id", kb1Id)).expectStatus(200); - new HttpTester(url, "GET", null, Map.of( - "Accept", "*/*", - "Knowledge-Base-Id", kb2Id - )).expectStatus(200); + new HttpTester(url, "GET", null, Map.of("Accept", "*/*", "Knowledge-Base-Id", kb2Id)).expectStatus(200); } }); @@ -95,13 +86,10 @@ public void run() { } new HttpTester(url, "POST", - "{\"knowledgeBaseId\": \""+ kb2Id + "\", \"knowledgeBaseName\": \"KB1\", \"knowledgeBaseDescription\": \"KB1\"}", - Map.of( - "Content-Type", "application/json", - "Accept", "*/*" - ) - ).expectStatus(200); - + "{\"knowledgeBaseId\": \"" + kb2Id + + "\", \"knowledgeBaseName\": \"KB1\", \"knowledgeBaseDescription\": \"KB1\"}", + Map.of("Content-Type", "application/json", "Accept", "*/*")).expectStatus(200); + afterRegister.countDown(); try { afterRegister.await(); @@ -109,15 +97,9 @@ public void run() { fail(); } - new HttpTester(url, "GET", null, Map.of( - "Accept", "*/*", - "Knowledge-Base-Id", kb1Id - )).expectStatus(200); + new HttpTester(url, "GET", null, Map.of("Accept", "*/*", "Knowledge-Base-Id", kb1Id)).expectStatus(200); - new HttpTester(url, "GET", null, Map.of( - "Accept", "*/*", - "Knowledge-Base-Id", kb2Id - )).expectStatus(200); + new HttpTester(url, "GET", null, Map.of("Accept", "*/*", "Knowledge-Base-Id", kb2Id)).expectStatus(200); } }); @@ -131,6 +113,7 @@ public void run() { @AfterAll public void cleanUp() { + TestUtil.unregisterAllKBs("http://localhost:" + PORT + "/rest"); rsh.cleanUp(); } } diff --git a/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestRegisterKnowledgeInteraction.java b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestRegisterKnowledgeInteraction.java index 2d2b5c22..ea135efc 100644 --- a/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestRegisterKnowledgeInteraction.java +++ b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestRegisterKnowledgeInteraction.java @@ -50,6 +50,7 @@ public void testRegisterKi() throws IOException { @AfterAll public void cleanUp() { + TestUtil.unregisterAllKBs("http://localhost:" + PORT + "/rest"); rsh.cleanUp(); } } diff --git a/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestScLifeCycle.java b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestScLifeCycle.java index 1bfc69d7..99db4193 100644 --- a/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestScLifeCycle.java +++ b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestScLifeCycle.java @@ -26,10 +26,8 @@ public void setUpServer() { public void testInvalidJson() throws IOException { URL url = new URL("http://localhost:" + PORT + "/rest/sc"); - HttpTester httpTest = new HttpTester(url, "POST", "{\"bla\"{}", Map.of( - "Content-Type", "application/json", - "Accept", "*/*" - )); + HttpTester httpTest = new HttpTester(url, "POST", "{\"bla\"{}", + Map.of("Content-Type", "application/json", "Accept", "*/*")); httpTest.expectStatus(400); } @@ -37,15 +35,15 @@ public void testInvalidJson() throws IOException { public void testValidJson() throws IOException { URL url = new URL("http://localhost:" + PORT + "/rest/sc"); - HttpTester httpTest = new HttpTester(url, "POST", "{\"knowledgeBaseId\": \"http://example.com/kb\", \"knowledgeBaseName\": \"KB\", \"knowledgeBaseDescription\": \"KB\"}", Map.of( - "Content-Type", "application/json", - "Accept", "*/*" - )); + HttpTester httpTest = new HttpTester(url, "POST", + "{\"knowledgeBaseId\": \"http://example.com/kb\", \"knowledgeBaseName\": \"KB\", \"knowledgeBaseDescription\": \"KB\"}", + Map.of("Content-Type", "application/json", "Accept", "*/*")); httpTest.expectStatus(200); } @AfterAll public void cleanUp() { + TestUtil.unregisterAllKBs("http://localhost:" + PORT + "/rest"); rsh.cleanUp(); } } diff --git a/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestSuspendedKnowledgeBase.java b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestSuspendedKnowledgeBase.java index 7b14f0ff..d739de31 100644 --- a/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestSuspendedKnowledgeBase.java +++ b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestSuspendedKnowledgeBase.java @@ -13,166 +13,125 @@ @Disabled @TestInstance(TestInstance.Lifecycle.PER_CLASS) public class TestSuspendedKnowledgeBase { - int NUM_UNHANDLED_REQUESTS = 5; - CountDownLatch latch = new CountDownLatch(NUM_UNHANDLED_REQUESTS); - private final RestServerHelper rsh = new RestServerHelper(); - private static final int PORT = 8280; - - @BeforeAll - public void setUpServer() { - rsh.start(PORT); - } - - @Test - public void testSuspendingKnowledgeBase() throws MalformedURLException { - URL url = new URL("http://localhost:" + PORT + "/rest"); - HttpTester registerKb = new HttpTester(new URL(url + "/sc"), - "POST", - "{\"knowledgeBaseId\": \"http://example.com/kb1\", " + - "\"knowledgeBaseName\": \"KB1\", " + - "\"knowledgeBaseDescription\": \"KB1\"}", Map.of( - "Content-Type", "application/json", - "Accept", "*/*" - )); - registerKb.expectStatus(200); - HttpTester registerKb2 = new HttpTester(new URL(url + "/sc"), - "POST", - "{\"knowledgeBaseId\": \"http://example.com/kb2\", " + - "\"knowledgeBaseName\": \"KB2\", " + - "\"knowledgeBaseDescription\": \"KB2\"}", Map.of( - "Content-Type", "application/json", - "Accept", "*/*" - )); - registerKb2.expectStatus(200); - - - var sc1 = new AsyncTester(() -> { - try { - HttpTester registerAsk = new HttpTester(new URL(url + "/sc/ki"), - "POST", - "{\"knowledgeInteractionType\": \"AskKnowledgeInteraction\", " + - "\"knowledgeInteractionName\": \"ask\"," + - "\"graphPattern\": \"?a ?b ?c.\"}", - Map.of( - "Knowledge-Base-Id", "http://example.com/kb1", - "Content-Type", "application/json", - "Accept", "*/*" - ) - ); - registerAsk.expectStatus(200); - System.out.println("Registered Ask"); - HttpTester executeAsk = new HttpTester(new URL(url + "/sc/ask"), - "POST", "[{}]", - Map.of( - "Knowledge-Base-Id", "http://example.com/kb1", - "Knowledge-Interaction-Id", "http://example.com/kb1/interaction/ask", - "Content-Type", "application/json", - "Accept", "*/*" - ) - ); - System.out.println("Executing Ask"); - var body = executeAsk.getBody(); - System.out.println(body); - - for (int i = 0; i < NUM_UNHANDLED_REQUESTS; i++) { - System.out.println("Making another ask"); - new AsyncTester(() -> { - HttpTester executeAsk1; - try { - executeAsk1 = new HttpTester(new URL(url + "/sc/ask"), - "POST", "[{}]", - Map.of( - "Knowledge-Base-Id", "http://example.com/kb1", - "Knowledge-Interaction-Id", "http://example.com/kb1/interaction/ask", - "Content-Type", "application/json", - "Accept", "*/*" - ) - ); - } catch (MalformedURLException e) { - throw new RuntimeException(e); - } - System.out.println("Executing Ask"); - latch.countDown(); - var body1 = executeAsk1.getBody(); - System.out.println(body1); - }).start(); - } - } catch (MalformedURLException e) { - System.err.println(e.getMessage()); - } - }); - - var sc2 = new AsyncTester(() -> { - try { - HttpTester registerAnswer = new HttpTester(new URL(url + "/sc/ki"), - "POST", - "{\"knowledgeInteractionType\": \"AnswerKnowledgeInteraction\", " + - "\"knowledgeInteractionName\": \"answer\"," + - "\"graphPattern\": \"?a ?b ?c.\"}", - Map.of( - "Knowledge-Base-Id", "http://example.com/kb2", - "Content-Type", "application/json", - "Accept", "*/*" - ) - ); - registerAnswer.expectStatus(200); - System.out.println("Registered Answer"); - - HttpTester waitForRequest = new HttpTester(new URL(url + "/sc/handle"), - "GET", null, - Map.of( - "Knowledge-Base-Id", "http://example.com/kb2", - "Content-Type", "application/json", - "Accept", "*/*" - ) - ); - var body = waitForRequest.getBody(); - System.out.println(body); - - System.out.println("Waiting for requests"); - var postAnswer = new HttpTester(new URL(url + "/sc/handle"), "POST", """ - { - "handleRequestId": 1, - "bindingSet": [{ - "a": "", - "b": "", - "c": "" - }] - } - """, - Map.of("Content-Type", "application/json", - "Accept", "*/*", - "Knowledge-Base-Id", "http://example.com/kb2", - "Knowledge-Interaction-Id", "http://example.com/kb2/interaction/answer")); - postAnswer.getBody(); - System.out.println("Posted answer"); - - latch.await(); - - HttpTester deleteSC = new HttpTester(new URL(url + "/sc/"), - "DELETE", "", - Map.of( - "Knowledge-Base-Id", "http://example.com/kb2", - "Content-Type", "application/json", - "Accept", "*/*" - )); - deleteSC.expectStatus(200); - } catch (MalformedURLException e) { - System.err.println(e.getMessage()); - } catch (InterruptedException e) { - throw new RuntimeException(e); - } - }); - - sc1.start(); - sc2.start(); - sc1.joinAndRethrow(); - sc2.joinAndRethrow(); - - } - - @AfterAll - public void cleanUp() { - rsh.cleanUp(); - } + int NUM_UNHANDLED_REQUESTS = 5; + CountDownLatch latch = new CountDownLatch(NUM_UNHANDLED_REQUESTS); + private final RestServerHelper rsh = new RestServerHelper(); + private static final int PORT = 8280; + + @BeforeAll + public void setUpServer() { + rsh.start(PORT); + } + + @Test + public void testSuspendingKnowledgeBase() throws MalformedURLException { + URL url = new URL("http://localhost:" + PORT + "/rest"); + HttpTester registerKb = new HttpTester(new URL(url + "/sc"), "POST", + "{\"knowledgeBaseId\": \"http://example.com/kb1\", " + "\"knowledgeBaseName\": \"KB1\", " + + "\"knowledgeBaseDescription\": \"KB1\"}", + Map.of("Content-Type", "application/json", "Accept", "*/*")); + registerKb.expectStatus(200); + HttpTester registerKb2 = new HttpTester(new URL(url + "/sc"), "POST", + "{\"knowledgeBaseId\": \"http://example.com/kb2\", " + "\"knowledgeBaseName\": \"KB2\", " + + "\"knowledgeBaseDescription\": \"KB2\"}", + Map.of("Content-Type", "application/json", "Accept", "*/*")); + registerKb2.expectStatus(200); + + var sc1 = new AsyncTester(() -> { + try { + HttpTester registerAsk = new HttpTester(new URL(url + "/sc/ki"), "POST", + "{\"knowledgeInteractionType\": \"AskKnowledgeInteraction\", " + + "\"knowledgeInteractionName\": \"ask\"," + "\"graphPattern\": \"?a ?b ?c.\"}", + Map.of("Knowledge-Base-Id", "http://example.com/kb1", "Content-Type", "application/json", + "Accept", "*/*")); + registerAsk.expectStatus(200); + System.out.println("Registered Ask"); + HttpTester executeAsk = new HttpTester(new URL(url + "/sc/ask"), "POST", "[{}]", + Map.of("Knowledge-Base-Id", "http://example.com/kb1", "Knowledge-Interaction-Id", + "http://example.com/kb1/interaction/ask", "Content-Type", "application/json", "Accept", + "*/*")); + System.out.println("Executing Ask"); + var body = executeAsk.getBody(); + System.out.println(body); + + for (int i = 0; i < NUM_UNHANDLED_REQUESTS; i++) { + System.out.println("Making another ask"); + new AsyncTester(() -> { + HttpTester executeAsk1; + try { + executeAsk1 = new HttpTester(new URL(url + "/sc/ask"), "POST", "[{}]", + Map.of("Knowledge-Base-Id", "http://example.com/kb1", "Knowledge-Interaction-Id", + "http://example.com/kb1/interaction/ask", "Content-Type", + "application/json", "Accept", "*/*")); + } catch (MalformedURLException e) { + throw new RuntimeException(e); + } + System.out.println("Executing Ask"); + latch.countDown(); + var body1 = executeAsk1.getBody(); + System.out.println(body1); + }).start(); + } + } catch (MalformedURLException e) { + System.err.println(e.getMessage()); + } + }); + + var sc2 = new AsyncTester(() -> { + try { + HttpTester registerAnswer = new HttpTester(new URL(url + "/sc/ki"), "POST", + "{\"knowledgeInteractionType\": \"AnswerKnowledgeInteraction\", " + + "\"knowledgeInteractionName\": \"answer\"," + "\"graphPattern\": \"?a ?b ?c.\"}", + Map.of("Knowledge-Base-Id", "http://example.com/kb2", "Content-Type", "application/json", + "Accept", "*/*")); + registerAnswer.expectStatus(200); + System.out.println("Registered Answer"); + + HttpTester waitForRequest = new HttpTester(new URL(url + "/sc/handle"), "GET", null, + Map.of("Knowledge-Base-Id", "http://example.com/kb2", "Content-Type", "application/json", + "Accept", "*/*")); + var body = waitForRequest.getBody(); + System.out.println(body); + + System.out.println("Waiting for requests"); + var postAnswer = new HttpTester(new URL(url + "/sc/handle"), "POST", """ + { + "handleRequestId": 1, + "bindingSet": [{ + "a": "", + "b": "", + "c": "" + }] + } + """, + Map.of("Content-Type", "application/json", "Accept", "*/*", "Knowledge-Base-Id", + "http://example.com/kb2", "Knowledge-Interaction-Id", + "http://example.com/kb2/interaction/answer")); + postAnswer.getBody(); + System.out.println("Posted answer"); + + latch.await(); + + HttpTester deleteSC = new HttpTester(new URL(url + "/sc/"), "DELETE", "", Map.of("Knowledge-Base-Id", + "http://example.com/kb2", "Content-Type", "application/json", "Accept", "*/*")); + deleteSC.expectStatus(200); + } catch (MalformedURLException e) { + System.err.println(e.getMessage()); + } catch (InterruptedException e) { + throw new RuntimeException(e); + } + }); + + sc1.start(); + sc2.start(); + sc1.joinAndRethrow(); + sc2.joinAndRethrow(); + + } + + @AfterAll + public void cleanUp() { + TestUtil.unregisterAllKBs("http://localhost:" + PORT + "/rest"); + rsh.cleanUp(); + } } diff --git a/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestUtil.java b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestUtil.java new file mode 100644 index 00000000..bcbcb6f8 --- /dev/null +++ b/smart-connector-rest-dist/src/test/java/eu/knowledge/engine/rest/api/TestUtil.java @@ -0,0 +1,52 @@ +package eu.knowledge.engine.rest.api; + +import static org.junit.jupiter.api.Assertions.fail; + +import java.io.StringReader; +import java.net.MalformedURLException; +import java.net.URL; +import java.util.Map; + +import eu.knowledge.engine.test_utils.HttpTester; +import jakarta.json.Json; +import jakarta.json.JsonArray; +import jakarta.json.JsonReader; +import jakarta.json.JsonValue; + +public class TestUtil { + + public static void unregisterAllKBs(String url) { + + HttpTester listKbs; + try { + listKbs = new HttpTester(new URL(url + "/sc"), "GET", null, null); + listKbs.expectStatus(200); + String body = listKbs.getBody(); + + JsonReader jsonReader = Json.createReader(new StringReader(body)); + JsonArray KBs = jsonReader.readArray(); + + for (JsonValue jo : KBs) { + unregisterKb(url, jo.asJsonObject().getString("knowledgeBaseId")); + } + + } catch (MalformedURLException e) { + fail(); + e.printStackTrace(); + } + + } + + public static void unregisterKb(String url, String id) { + + try { + HttpTester deleteKb = new HttpTester(new URL(url + "/sc"), "DELETE", null, Map.of("Knowledge-Base-Id", id)); + deleteKb.expectStatus(200); + } catch (MalformedURLException e) { + e.printStackTrace(); + fail(); + + } + } + +} diff --git a/smart-connector-rest-server/pom.xml b/smart-connector-rest-server/pom.xml index 567c84d1..57012e4f 100644 --- a/smart-connector-rest-server/pom.xml +++ b/smart-connector-rest-server/pom.xml @@ -11,10 +11,10 @@ - 2.2.22 - 11.0.15 + 2.2.26 + 11.0.24 3.1.9 - 2.18.1 + 2.18.2 UTF-8 @@ -41,7 +41,7 @@ org.slf4j slf4j-simple - 2.0.13 + 2.0.16 @@ -140,7 +140,7 @@ org.apache.maven maven-model - 3.9.4 + 3.9.9 diff --git a/smart-connector-rest-server/src/main/java/eu/knowledge/engine/rest/api/impl/ProactiveApiServiceImpl.java b/smart-connector-rest-server/src/main/java/eu/knowledge/engine/rest/api/impl/ProactiveApiServiceImpl.java index 374f8a0b..0d00c2ad 100644 --- a/smart-connector-rest-server/src/main/java/eu/knowledge/engine/rest/api/impl/ProactiveApiServiceImpl.java +++ b/smart-connector-rest-server/src/main/java/eu/knowledge/engine/rest/api/impl/ProactiveApiServiceImpl.java @@ -7,9 +7,12 @@ import java.util.HashMap; import java.util.List; import java.util.Map; +import java.util.Set; import java.util.concurrent.ExecutionException; import java.util.stream.Collectors; +import org.apache.jena.sparql.graph.PrefixMappingZero; +import org.apache.jena.sparql.util.FmtUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -24,6 +27,7 @@ import eu.knowledge.engine.rest.model.ResponseMessage; import eu.knowledge.engine.smartconnector.api.BindingSet; import eu.knowledge.engine.smartconnector.api.ExchangeInfo.Initiator; +import eu.knowledge.engine.smartconnector.api.KnowledgeGap; import io.swagger.v3.oas.annotations.Parameter; import jakarta.validation.Valid; import jakarta.validation.constraints.NotNull; @@ -145,10 +149,16 @@ public void scAskPost( .failedMessage(aei.getFailedMessage())) .collect(Collectors.toList()); - AskResult ar = new AskResult().bindingSet(this.bindingSetToList(askResult.getBindings())) - .exchangeInfo(infos); - - asyncResponse.resume(Response.status(Status.OK).entity(ar).build()); + LOG.debug("Bindings in result is {}", askResult.getBindings()); + LOG.debug("KnowledgeGapsEnabled is {}", ki.getKnowledgeGapsEnabled()); + + AskResult ar = new AskResult().bindingSet(this.bindingSetToList(askResult.getBindings())).exchangeInfo(infos); + // distinguish between knowledge gaps enabled or not to produce an AskResult or an AskResultWithGaps + if (ki.getKnowledgeGapsEnabled()) { + LOG.info("Knowledge gaps in result is {}", askResult.getKnowledgeGaps()); + ar.knowledgeGaps(this.knowledgeGapsToList(askResult.getKnowledgeGaps())); + } + asyncResponse.resume(Response.status(Status.OK).entity(ar).build()); }); } catch (URISyntaxException | InterruptedException | ExecutionException e) { @@ -178,6 +188,19 @@ private List> bindingSetToList(BindingSet bindings) { return listBindings; } + private List> knowledgeGapsToList(Set knowledgeGaps) { + List> listKnowledgeGaps = new ArrayList>(knowledgeGaps.size()); + knowledgeGaps.forEach((kg) -> { + List listKnowledgeGap = new ArrayList(); + kg.forEach((tp) -> { + String tpString = FmtUtils.stringForTriple(tp.asTriple(), new PrefixMappingZero()); + listKnowledgeGap.add(tpString); + }); + listKnowledgeGaps.add(listKnowledgeGap); + }); + return listKnowledgeGaps; + } + private AskExchangeInfo.InitiatorEnum toInitiatorEnumAsk(Initiator initiator) { switch (initiator) { case KNOWLEDGEBASE: diff --git a/smart-connector-rest-server/src/main/java/eu/knowledge/engine/rest/api/impl/RestKnowledgeBase.java b/smart-connector-rest-server/src/main/java/eu/knowledge/engine/rest/api/impl/RestKnowledgeBase.java index 5a64e547..be97ce11 100644 --- a/smart-connector-rest-server/src/main/java/eu/knowledge/engine/rest/api/impl/RestKnowledgeBase.java +++ b/smart-connector-rest-server/src/main/java/eu/knowledge/engine/rest/api/impl/RestKnowledgeBase.java @@ -47,6 +47,7 @@ import eu.knowledge.engine.smartconnector.api.KnowledgeBase; import eu.knowledge.engine.smartconnector.api.KnowledgeEngineException; import eu.knowledge.engine.smartconnector.api.KnowledgeInteraction; +import eu.knowledge.engine.smartconnector.api.MatchStrategy; import eu.knowledge.engine.smartconnector.api.PostKnowledgeInteraction; import eu.knowledge.engine.smartconnector.api.ReactExchangeInfo; import eu.knowledge.engine.smartconnector.api.ReactHandler; @@ -394,6 +395,12 @@ public String register(KnowledgeInteractionBase ki) { prefixMapping = new PrefixMappingZero(); } + boolean knowledgeGapsEnabled = ki.getKnowledgeGapsEnabled() == null ? false : ki.getKnowledgeGapsEnabled(); + if (knowledgeGapsEnabled && !this.sc.isReasonerEnabled()) { + throw new IllegalArgumentException( + "You can only set knowledgeGapsEnabled when the Knowledge Base is reasonerEnabled."); + } + String type = ki.getKnowledgeInteractionType(); URI kiId; if (type.equals("AskKnowledgeInteraction")) { @@ -403,8 +410,15 @@ public String register(KnowledgeInteractionBase ki) { if (aki.getGraphPattern() == null) { throw new IllegalArgumentException("graphPattern must be given for ASK knowledge interactions."); } + + MatchStrategy strategy = null; + if (aki.getKnowledgeGapsEnabled() != null && aki.getKnowledgeGapsEnabled()) { + strategy = MatchStrategy.SUPREME_LEVEL; + } + var askKI = new AskKnowledgeInteraction(ca, new GraphPattern(prefixMapping, aki.getGraphPattern()), - ki.getKnowledgeInteractionName()); + ki.getKnowledgeInteractionName(), false, false, knowledgeGapsEnabled, strategy); + kiId = this.sc.register(askKI); this.knowledgeInteractions.put(kiId, askKI); } else if (type.equals("AnswerKnowledgeInteraction")) { @@ -515,7 +529,7 @@ private KnowledgeInteractionWithId kiToModelKiWithId(URI kiId, KnowledgeInteract var requirements = act.getRequirementPurposes().stream().map(r -> r.toString()).collect(Collectors.toList()); var satisfactions = act.getSatisfactionPurposes().stream().map(r -> r.toString()).collect(Collectors.toList()); var kiwid = new KnowledgeInteractionWithId().knowledgeInteractionId(kiId.toString()) - .knowledgeInteractionName(ki.getName()) + .knowledgeInteractionName(ki.getName()).knowledgeGapsEnabled(ki.knowledgeGapsEnabled()) .communicativeAct(new eu.knowledge.engine.rest.model.CommunicativeAct().requiredPurposes(requirements) .satisfiedPurposes(satisfactions)); diff --git a/smart-connector-rest-server/src/main/java/eu/knowledge/engine/rest/api/impl/SmartConnectorLifeCycleApiServiceImpl.java b/smart-connector-rest-server/src/main/java/eu/knowledge/engine/rest/api/impl/SmartConnectorLifeCycleApiServiceImpl.java index 29d73bcc..8fcb8f3a 100644 --- a/smart-connector-rest-server/src/main/java/eu/knowledge/engine/rest/api/impl/SmartConnectorLifeCycleApiServiceImpl.java +++ b/smart-connector-rest-server/src/main/java/eu/knowledge/engine/rest/api/impl/SmartConnectorLifeCycleApiServiceImpl.java @@ -145,8 +145,8 @@ public void scPost(@Parameter(description = "", required = true) @NotNull @Valid final boolean reasonerEnabled = smartConnector.getReasonerEnabled() == null ? false : smartConnector.getReasonerEnabled(); - - LOG.info("Creating smart connector with ID {}.", kbId); + + LOG.info("Creating smart connector with ID {} and reasoner enabled '{}'.", kbId, reasonerEnabled); // Tell the manager to create a KB, store it, and have it set up a SC etc. this.manager.createKB(new SmartConnector().knowledgeBaseId(kbId.toString()).knowledgeBaseName(kbName) diff --git a/smart-connector-rest-server/src/main/resources/openapi-sc.yaml b/smart-connector-rest-server/src/main/resources/openapi-sc.yaml index d93b0153..60663eb9 100644 --- a/smart-connector-rest-server/src/main/resources/openapi-sc.yaml +++ b/smart-connector-rest-server/src/main/resources/openapi-sc.yaml @@ -180,6 +180,13 @@ paths: type: string requestBody: required: true + description: + The body of the ASK can contain the option "knowledgeGapEnabled" + (See the example "ASK with knowledge gaps enabled"). + This option is disabled by default. If enabled, each activation of the knowledge interaction will + provide a set of knowledge gaps as part of the result. If this set is empty, the binding set of the + result contains the answer. If the set of knowledge gaps is not empty, this contains one or more + knowledge gaps that need to be fixed for the knowledge interaction to produce an answer. content: application/json; charset=UTF-8: schema: @@ -216,6 +223,11 @@ paths: prefixes: rdf: "http://www.w3.org/1999/02/22-rdf-syntax-ns#" ex: "http://example.org/" + ASK with knowledge gaps enabled: + value: + knowledgeInteractionType: AskKnowledgeInteraction + graphPattern: "?a ?b ." + knowledgeGapsEnabled: "true" responses: '200': description: If the Knowledge Interaction is successfully registered, it returns the KnowledgeInteractionId object. @@ -321,10 +333,22 @@ paths: bindings together with additional exchange info. The returned bindings form a set and there is no quaranteed ordering. The exchange info shows the other Knowledge Bases that contributed to the answer with timing, initiator information. + If KnowledgeGaps are enabled, a list of knowledge gaps that are found will be returned + as well. See, the different examples below. content: application/json; charset=UTF-8: schema: $ref: '#/components/schemas/AskResult' + examples: + knowledge gaps NOT enabled: + value: + bindingSet: [{"a": "","b": ""}] + exchangeInfo: [] + knowledge gaps enabled: + value: + bindingSet: [{"a": "","b": ""}] + exchangeInfo: [] + knowledgeGaps: [["?a ?c", "?a ?b"]] '400': description: If the ask failed. content: @@ -608,6 +632,8 @@ components: knowledgeInteractionName: type: string pattern: '^[a-zA-Z0-9-]*$' + knowledgeGapsEnabled: + type: boolean communicativeAct: $ref: '#/components/schemas/CommunicativeAct' prefixes: @@ -686,6 +712,11 @@ components: type: array items: $ref: '#/components/schemas/AskExchangeInfo' + knowledgeGaps: + type: array + items: + $ref: '#/components/schemas/KnowledgeGap' + nullable: true PostResult: type: object required: [resultBindingSet, exchangeInfo] @@ -735,6 +766,11 @@ components: $ref: '#/components/schemas/BindingSet' resultBindingSet: $ref: '#/components/schemas/BindingSet' + KnowledgeGap: + type: array + items: + nullable: false # Enforced manually, but kept here nonetheless. + type: string HandleRequest: type: object properties: diff --git a/smart-connector/pom.xml b/smart-connector/pom.xml index 443dc12f..c48b3056 100644 --- a/smart-connector/pom.xml +++ b/smart-connector/pom.xml @@ -25,7 +25,7 @@ org.slf4j slf4j-simple - 2.0.13 + 2.0.16 @@ -221,10 +221,10 @@ - 2.2.22 + 2.2.26 11.0.24 3.1.9 - 2.18.1 + 2.18.2 6.1.0 3.1 diff --git a/smart-connector/src/main/java/eu/knowledge/engine/smartconnector/impl/MetaKnowledgeBaseImpl.java b/smart-connector/src/main/java/eu/knowledge/engine/smartconnector/impl/MetaKnowledgeBaseImpl.java index f2a73dc3..332cdd79 100644 --- a/smart-connector/src/main/java/eu/knowledge/engine/smartconnector/impl/MetaKnowledgeBaseImpl.java +++ b/smart-connector/src/main/java/eu/knowledge/engine/smartconnector/impl/MetaKnowledgeBaseImpl.java @@ -95,7 +95,7 @@ public MetaKnowledgeBaseImpl(LoggerProvider loggerProvider, MessageRouter aMessa (anAKI, anAnswerExchangeInfo) -> this.fillMetaBindings(anAnswerExchangeInfo.getIncomingBindings()), true); - this.metaAskKI = new AskKnowledgeInteraction(new CommunicativeAct(), this.metaGraphPattern, null, true, true, + this.metaAskKI = new AskKnowledgeInteraction(new CommunicativeAct(), this.metaGraphPattern, null, true, true, false, MatchStrategy.ENTRY_LEVEL); this.knowledgeBaseStore.register(this.metaAskKI, true); diff --git a/smart-connector/src/main/java/eu/knowledge/engine/smartconnector/impl/ReasonerProcessor.java b/smart-connector/src/main/java/eu/knowledge/engine/smartconnector/impl/ReasonerProcessor.java index 4f94d629..5640c5e5 100644 --- a/smart-connector/src/main/java/eu/knowledge/engine/smartconnector/impl/ReasonerProcessor.java +++ b/smart-connector/src/main/java/eu/knowledge/engine/smartconnector/impl/ReasonerProcessor.java @@ -6,6 +6,7 @@ import java.util.HashSet; import java.util.Iterator; import java.util.Map; +import java.util.Map.Entry; import java.util.Set; import java.util.concurrent.CompletableFuture; import java.util.concurrent.ConcurrentHashMap; @@ -21,7 +22,9 @@ import org.slf4j.LoggerFactory; import org.slf4j.helpers.MessageFormatter; +import eu.knowledge.engine.reasoner.AntSide; import eu.knowledge.engine.reasoner.BaseRule; +import eu.knowledge.engine.reasoner.BindingSetHandler; import eu.knowledge.engine.reasoner.ProactiveRule; import eu.knowledge.engine.reasoner.ReasonerPlan; import eu.knowledge.engine.reasoner.Rule; @@ -29,6 +32,7 @@ import eu.knowledge.engine.reasoner.TaskBoard; import eu.knowledge.engine.reasoner.TransformBindingSetHandler; import eu.knowledge.engine.reasoner.api.TriplePattern; +import eu.knowledge.engine.reasoner.rulenode.RuleNode; import eu.knowledge.engine.reasoner.rulestore.RuleStore; import eu.knowledge.engine.smartconnector.api.AnswerKnowledgeInteraction; import eu.knowledge.engine.smartconnector.api.AskExchangeInfo; @@ -39,6 +43,7 @@ import eu.knowledge.engine.smartconnector.api.ExchangeInfo.Initiator; import eu.knowledge.engine.smartconnector.api.ExchangeInfo.Status; import eu.knowledge.engine.smartconnector.api.GraphPattern; +import eu.knowledge.engine.smartconnector.api.KnowledgeGap; import eu.knowledge.engine.smartconnector.api.KnowledgeInteraction; import eu.knowledge.engine.smartconnector.api.MatchStrategy; import eu.knowledge.engine.smartconnector.api.PostExchangeInfo; @@ -72,6 +77,7 @@ public class ReasonerProcessor extends SingleInteractionProcessor { private final Set postExchangeInfos; private Set additionalDomainKnowledge; private ReasonerPlan reasonerPlan; + private Set knowledgeGaps; private MatchStrategy matchStrategy = MatchStrategy.NORMAL_LEVEL; @@ -164,6 +170,22 @@ public void planAskInteraction(MyKnowledgeInteractionInfo aAKI) { } } + /** + * In this ask operation that smart connector should do the following: 1: first, + * make a plan for executing the ask using the planAsk method in this smart + * connector this will return an object of type AskPlan. 2: second, execute the + * plan on the knowledge network to potentially get bindings for the pattern in + * the ask. 3: third, as part of the execution also find knowledge gaps in the + * reasoner plan when the resulting binding set is empty. This will result in an + * object of type AskResult that contains the resulting bindings, exchange info + * and optionally the reasoner plan plus knowledge gaps. + * + * This can lead to the following situations: 1: a plan, a non-empty binding set + * and no gaps => ask has a result 2: a plan, an empty binding set and no gaps + * => ask has an empty result 3: a plan, an empty binding set with gaps => ask + * has no result and gaps are found + * + */ @Override public CompletableFuture executeAskInteraction(BindingSet someBindings) { @@ -172,7 +194,12 @@ public CompletableFuture executeAskInteraction(BindingSet someBinding continueReasoningBackward(translateBindingSetTo(someBindings)); return this.finalBindingSetFuture.thenApply((bs) -> { - return new AskResult(translateBindingSetFrom(bs), this.askExchangeInfos, this.reasonerPlan); + if (myKnowledgeInteraction.getKnowledgeInteraction().knowledgeGapsEnabled()) { + this.knowledgeGaps = bs.isEmpty() + ? getKnowledgeGaps(this.reasonerPlan.getStartNode()) + : new HashSet(); + } + return new AskResult(translateBindingSetFrom(bs), this.askExchangeInfos, this.reasonerPlan, this.knowledgeGaps); }); } @@ -623,6 +650,118 @@ public ReasonerPlan getReasonerPlan() { return this.reasonerPlan; } + /** + * Returns the knowledge gaps of this reasoning node. A knowledge gap is a + * subset of this node's antecedent triple patterns that do not match any + * neighbor that has no knowledge gaps. + * + * @return returns all triples that have no matching nodes (and for which there + * are no alternatives). Note that it returns a set of sets. Where every + * set in this set represents a single way to resolve the knowledge gaps + * present in this reasoning graph. So, {@code [[A],[B]]} means either + * triple {@code A} OR triple {@code B} needs be added to + * solve the gap or both, while {@code [[A,B]]} means that both + * {@code A} AND {@code B} need to be added to solve the + * gap. + */ + public Set getKnowledgeGaps(RuleNode plan) { + + assert plan instanceof AntSide; + + Set existingOrGaps = new HashSet(); + + // TODO do we need to include the parent if we are not backward chaining? + Map> nodeCoverage = plan + .findAntecedentCoverage(((AntSide) plan).getAntecedentNeighbours()); + + // collect triple patterns that have an empty set + Set collectedOrGaps, someGaps = new HashSet(); + + for (Entry> entry : nodeCoverage.entrySet()) { + + LOG.debug("Entry key is {}", entry.getKey()); + LOG.debug("Entry value is {}", entry.getValue()); + + collectedOrGaps = new HashSet(); + boolean foundNeighborWithoutGap = false; + for (RuleNode neighbor : entry.getValue()) { + LOG.debug("Neighbor is {}", neighbor); + + if (!neighbor.getRule().getAntecedent().isEmpty()) { + // make sure neighbor has no knowledge gaps + LOG.debug("Neighbor has antecedents, so check if the neighbor has gaps"); + + // knowledge engine specific code. We ignore meta knowledge interactions when + // looking for knowledge gaps, because they are very generic and make finding + // knowledge gaps nearly impossible. + boolean isMeta = isMetaKI(neighbor); + + // TODO what if the graph contains loops? + if (!isMeta && (someGaps = getKnowledgeGaps(neighbor)).isEmpty()) { + // found neighbor without knowledge gaps for the current triple, so current + // triple is covered. + LOG.debug("Neighbor has no gaps"); + foundNeighborWithoutGap = true; + break; + } + LOG.debug("Neighbor has someGaps {}", someGaps); + collectedOrGaps.addAll(someGaps); + } else + foundNeighborWithoutGap = true; + } + LOG.debug("Found a neighbor without gaps is {}", foundNeighborWithoutGap); + + if (!foundNeighborWithoutGap) { + // there is a gap here, either in the current node or in a neighbor. + + if (collectedOrGaps.isEmpty()) { + KnowledgeGap kg = new KnowledgeGap(); + kg.add(entry.getKey()); + collectedOrGaps.add(kg); + } + LOG.debug("CollectedOrGaps is {}", collectedOrGaps); + + Set newExistingOrGaps = new HashSet(); + if (existingOrGaps.isEmpty()) { + existingOrGaps.addAll(collectedOrGaps); + LOG.debug("Added collectedOrGaps to existingOrGaps"); + } else { + KnowledgeGap newGap; + for (KnowledgeGap existingOrGap : existingOrGaps) { + for (KnowledgeGap collectedOrGap : collectedOrGaps) { + newGap = new KnowledgeGap(); + newGap.addAll(existingOrGap); + newGap.addAll(collectedOrGap); + LOG.debug("Found newGap {}", newGap); + newExistingOrGaps.add(newGap); + } + } + existingOrGaps = newExistingOrGaps; + } + + } + } + LOG.debug("Found existingOrGaps {}", existingOrGaps); + return existingOrGaps; + } + + private boolean isMetaKI(RuleNode neighbor) { + + assert neighbor.getRule() instanceof Rule; + + BindingSetHandler bsh = ((Rule) neighbor.getRule()).getBindingSetHandler(); + + if (bsh instanceof ReactBindingSetHandler) { + ReactBindingSetHandler rbsh = (ReactBindingSetHandler) bsh; + return rbsh.getKnowledgeInteractionInfo().isMeta(); + } else if (bsh instanceof AnswerBindingSetHandler) { + AnswerBindingSetHandler absh = (AnswerBindingSetHandler) bsh; + return absh.getKnowledgeInteractionInfo().isMeta(); + } + + return false; + } + public void setMatchStrategy(MatchStrategy aStrategy) { this.matchStrategy = aStrategy; } diff --git a/smart-connector/src/main/java/eu/knowledge/engine/smartconnector/impl/Util.java b/smart-connector/src/main/java/eu/knowledge/engine/smartconnector/impl/Util.java index 16d90eec..7dcdc7cc 100644 --- a/smart-connector/src/main/java/eu/knowledge/engine/smartconnector/impl/Util.java +++ b/smart-connector/src/main/java/eu/knowledge/engine/smartconnector/impl/Util.java @@ -26,16 +26,9 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import eu.knowledge.engine.reasoner.AntSide; -import eu.knowledge.engine.reasoner.BindingSetHandler; -import eu.knowledge.engine.reasoner.Rule; -import eu.knowledge.engine.reasoner.api.TriplePattern; -import eu.knowledge.engine.reasoner.rulenode.RuleNode; import eu.knowledge.engine.smartconnector.api.Binding; import eu.knowledge.engine.smartconnector.api.BindingSet; import eu.knowledge.engine.smartconnector.api.GraphPattern; -import eu.knowledge.engine.smartconnector.impl.ReasonerProcessor.AnswerBindingSetHandler; -import eu.knowledge.engine.smartconnector.impl.ReasonerProcessor.ReactBindingSetHandler; public class Util { private static final Logger LOG = LoggerFactory.getLogger(Util.class); @@ -106,110 +99,6 @@ public static Model generateModel(GraphPattern graphPattern, BindingSet variable return m; } - /** - * Returns the knowledge gap of this reasoning node. A knowledge gap is a subset - * of this node's antecedent triple patterns that do not match any neighbor that - * has no knowledge gaps. - * - * Currently, this method does not show how the knowledge gaps influence each - * other. Some knowledge gaps might have an {@code or}-relation (namely those - * that occur on the same triple) and some might have {@code and}-relations - * (i.e. those that do not occur on the same triple). This information is - * important if you want to know how to solve the gaps because 2 gaps related by - * {@code or} do not both need to be solved, but only one of them. While 2 gaps - * related by {@code and} both need to be solved to solve the gap. - * - * @return returns all triples that have no matching nodes (and for which there - * are no alternatives). Note that it returns a set of sets. Where every - * set in this set represents a single way to resolve the knowledge gaps - * present in this reasoning graph. So, {@code [[A],[B]]} means either - * triple {@code A} OR triple {@code B} needs be added to - * solve the gap or both, while {@code [[A,B]]} means that both - * {@code A} AND {@code B} need to be added to solve the - * gap. - */ - public static Set> getKnowledgeGaps(RuleNode plan) { - - assert plan instanceof AntSide; - - Set> existingOrGaps = new HashSet<>(); - - // TODO do we need to include the parent if we are not backward chaining? - Map> nodeCoverage = plan - .findAntecedentCoverage(((AntSide) plan).getAntecedentNeighbours()); - - // collect triple patterns that have an empty set - Set> collectedOrGaps, someGaps = new HashSet<>(); - for (Entry> entry : nodeCoverage.entrySet()) { - - collectedOrGaps = new HashSet<>(); - boolean foundNeighborWithoutGap = false; - for (RuleNode neighbor : entry.getValue()) { - if (!neighbor.getRule().getAntecedent().isEmpty()) { - // make sure neighbor has no knowledge gaps - - // knowledge engine specific code. We ignore meta knowledge interactions when - // looking for knowledge gaps, because they are very generic and make finding - // knowledge gaps nearly impossible. - boolean isMeta = isMetaKI(neighbor); - - //TODO what if the graph contains loops? - if (!isMeta && (someGaps = getKnowledgeGaps(neighbor)).isEmpty()) { - // found neighbor without knowledge gaps for the current triple, so current - // triple is covered. - foundNeighborWithoutGap = true; - break; - } - collectedOrGaps.addAll(someGaps); - } else - foundNeighborWithoutGap = true; - } - - if (!foundNeighborWithoutGap) { - // there is a gap here, either in the current node or in a neighbor. - - if (collectedOrGaps.isEmpty()) { - collectedOrGaps.add(new HashSet<>(Arrays.asList(entry.getKey()))); - } - - Set> newExistingOrGaps = new HashSet<>(); - if (existingOrGaps.isEmpty()) { - existingOrGaps.addAll(collectedOrGaps); - } else { - Set newGap; - for (Set existingOrGap : existingOrGaps) { - for (Set collectedOrGap : collectedOrGaps) { - newGap = new HashSet<>(); - newGap.addAll(existingOrGap); - newGap.addAll(collectedOrGap); - newExistingOrGaps.add(newGap); - } - } - existingOrGaps = newExistingOrGaps; - } - } - } - - return existingOrGaps; - } - - private static boolean isMetaKI(RuleNode neighbor) { - - assert neighbor.getRule() instanceof Rule; - - BindingSetHandler bsh = ((Rule) neighbor.getRule()).getBindingSetHandler(); - - if (bsh instanceof ReactBindingSetHandler) { - ReactBindingSetHandler rbsh = (ReactBindingSetHandler) bsh; - return rbsh.getKnowledgeInteractionInfo().isMeta(); - } else if (bsh instanceof AnswerBindingSetHandler) { - AnswerBindingSetHandler absh = (AnswerBindingSetHandler) bsh; - return absh.getKnowledgeInteractionInfo().isMeta(); - } - - return false; - } - public static void removeRedundantBindingsAnswer(BindingSet incoming, BindingSet outgoing) { if (incoming.isEmpty()) { // We should not remove any bindings in this case! diff --git a/smart-connector/src/main/java/eu/knowledge/engine/smartconnector/util/KnowledgeNetwork.java b/smart-connector/src/main/java/eu/knowledge/engine/smartconnector/util/KnowledgeNetwork.java index bf35d20d..810dd1bd 100644 --- a/smart-connector/src/main/java/eu/knowledge/engine/smartconnector/util/KnowledgeNetwork.java +++ b/smart-connector/src/main/java/eu/knowledge/engine/smartconnector/util/KnowledgeNetwork.java @@ -89,7 +89,7 @@ private void startAndWaitForReady() { for (MockedKnowledgeBase kb : justStartedKBs) { AskKnowledgeInteraction anAskKI = new AskKnowledgeInteraction(new CommunicativeAct(), gp, null, false, true, - MatchStrategy.ENTRY_LEVEL); + false, MatchStrategy.ENTRY_LEVEL); this.knowledgeInteractionMetadata.put(kb, anAskKI); kb.register(anAskKI); } diff --git a/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestAskAnswerWithGapsEnabled1.java b/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestAskAnswerWithGapsEnabled1.java new file mode 100644 index 00000000..4d850aca --- /dev/null +++ b/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestAskAnswerWithGapsEnabled1.java @@ -0,0 +1,107 @@ +package eu.knowledge.engine.smartconnector.api; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.fail; + +import java.net.URISyntaxException; +import java.util.Iterator; +import java.util.Set; +import java.util.concurrent.BrokenBarrierException; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.TimeoutException; + +import org.apache.jena.shared.PrefixMapping; +import org.apache.jena.sparql.graph.PrefixMappingMem; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import eu.knowledge.engine.smartconnector.util.KnowledgeNetwork; +import eu.knowledge.engine.smartconnector.util.MockedKnowledgeBase; + +public class TestAskAnswerWithGapsEnabled1 { + + private static final Logger LOG = LoggerFactory.getLogger(TestAskAnswerWithGapsEnabled1.class); + + private static MockedKnowledgeBase kbRelationAsker; + + private static KnowledgeNetwork kn; + + private static PrefixMappingMem prefixes; + + private AskKnowledgeInteraction askKIGaps; + + @BeforeAll + public static void setup() throws InterruptedException, BrokenBarrierException, TimeoutException { + + prefixes = new PrefixMappingMem(); + prefixes.setNsPrefixes(PrefixMapping.Standard); + prefixes.setNsPrefix("ex", "https://www.tno.nl/example/"); + + } + + @Test + public void testAskAnswerWithGapsEnabled() throws InterruptedException, URISyntaxException { + + // In this test there will be only 1 KB with a single AskKI. + // The test will execute the AskKI with knowledge gaps enabled. + // As a result, the set of knowledge gaps should contain a single gap. + + setupNetwork(); + + // Perform the ASK + try { + AskResult result = kbRelationAsker.ask(askKIGaps, new BindingSet()).get(); + // check whether set of knowledge gaps contains a single gap + Set gaps = result.getKnowledgeGaps(); + LOG.info("Found gaps: " + gaps); + assertFalse(gaps.isEmpty(),"The set of knowledge gaps should be empty"); + assertEquals(1, gaps.size(), "Number of gaps should be 1"); + Iterator iter = gaps.iterator(); + while (iter.hasNext()) { + KnowledgeGap gap = iter.next(); + assertEquals("[?a isRelatedTo ?b]", gap.toString(), "Gap should be [?a isRelatedTo ?b]"); + } + BindingSet bindings = result.getBindings(); + LOG.info("Resulting binding set is: " + bindings); + assertTrue(bindings.isEmpty(),"The resulting bindingset should be empty"); + } catch (InterruptedException | ExecutionException e) { + fail(); + } + + } + + private void setupNetwork() { + + instantiateAskRelationsKB(); + kn = new KnowledgeNetwork(); + kn.addKB(kbRelationAsker); + + long start = System.nanoTime(); + kn.sync(); + long end = System.nanoTime(); + LOG.info("Duration: {}", (((double) (end - start)) / 1_000_000)); + } + + public void instantiateAskRelationsKB() { + // start a knowledge base with the behavior "I am interested in related people" + kbRelationAsker = new MockedKnowledgeBase("RelationAsker"); + kbRelationAsker.setReasonerEnabled(true); + + // Register an Ask pattern for relations with knowledge gaps enabled + GraphPattern gp1 = new GraphPattern(prefixes, "?a ex:isRelatedTo ?b ."); + this.askKIGaps = new AskKnowledgeInteraction(new CommunicativeAct(), gp1, "askRelations", false, true, true, MatchStrategy.SUPREME_LEVEL); + kbRelationAsker.register(this.askKIGaps); + + } + + @AfterAll + public static void cleanup() throws InterruptedException, ExecutionException { + LOG.info("Clean up: {}", TestAskAnswerWithGapsEnabled1.class.getSimpleName()); + kn.stop().get(); + } +} diff --git a/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestAskAnswerWithGapsEnabled2.java b/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestAskAnswerWithGapsEnabled2.java new file mode 100644 index 00000000..9321ee1b --- /dev/null +++ b/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestAskAnswerWithGapsEnabled2.java @@ -0,0 +1,150 @@ +package eu.knowledge.engine.smartconnector.api; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.fail; + +import java.net.URISyntaxException; +import java.util.Iterator; +import java.util.Set; +import java.util.concurrent.BrokenBarrierException; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.TimeoutException; + +import org.apache.jena.shared.PrefixMapping; +import org.apache.jena.sparql.graph.PrefixMappingMem; +import org.apache.jena.sparql.graph.PrefixMappingZero; +import org.apache.jena.sparql.util.FmtUtils; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import eu.knowledge.engine.reasoner.api.TriplePattern; +import eu.knowledge.engine.smartconnector.util.KnowledgeNetwork; +import eu.knowledge.engine.smartconnector.util.MockedKnowledgeBase; + +public class TestAskAnswerWithGapsEnabled2 { + + private static final Logger LOG = LoggerFactory.getLogger(TestAskAnswerWithGapsEnabled2.class); + + private static MockedKnowledgeBase kbRelationAsker; + private static MockedKnowledgeBase kbRelationProvider; + + private static KnowledgeNetwork kn; + + private static PrefixMappingMem prefixes; + + private AskKnowledgeInteraction askKIGaps; + + @BeforeAll + public static void setup() throws InterruptedException, BrokenBarrierException, TimeoutException { + + prefixes = new PrefixMappingMem(); + prefixes.setNsPrefixes(PrefixMapping.Standard); + prefixes.setNsPrefix("ex", "https://www.tno.nl/example/"); + + } + + @Test + public void testAskAnswerWithGapsEnabled() throws InterruptedException, URISyntaxException { + + // In this test there will be an Ask KB with a single AskKI and + // an AnswerKB with a single AnswerKI that answers only part of the Ask pattern. + // The test will execute the AskKI with knowledge gaps enabled. + // As a result, the set of knowledge gaps should contain a single gap. + + setupNetwork(); + + // Perform the ASK + try { + AskResult result = kbRelationAsker.ask(askKIGaps, new BindingSet()).get(); + // check whether set of knowledge gaps contains a single gap + Set gaps = result.getKnowledgeGaps(); + + result.getReasonerPlan().getStore().printGraphVizCode(result.getReasonerPlan(), true); + + LOG.info("Found gaps: " + gaps); + assertFalse(gaps.isEmpty(), "The set of knowledge gaps should be empty"); + assertEquals(1, gaps.size(), "Number of gaps should be 1"); + Iterator iter = gaps.iterator(); + while (iter.hasNext()) { + KnowledgeGap gap = iter.next(); + Iterator gapiter = gap.iterator(); + while (gapiter.hasNext()) { + TriplePattern triple = gapiter.next(); + String tpString = FmtUtils.stringForTriple(triple.asTriple(), new PrefixMappingZero()); + LOG.info("Gap is " + tpString); + assertEquals("?a ?c", tpString, + "Gap should be ?a ?c"); + } + } + BindingSet bindings = result.getBindings(); + LOG.info("Resulting binding set is: " + bindings); + assertTrue(bindings.isEmpty(), "The resulting bindingset should be empty"); + } catch (InterruptedException | ExecutionException e) { + fail(); + } + + } + + private void setupNetwork() { + + instantiateAskRelationsKB(); + instantiateAnswerRelationsKB(); + kn = new KnowledgeNetwork(); + kn.addKB(kbRelationAsker); + kn.addKB(kbRelationProvider); + + long start = System.nanoTime(); + kn.sync(); + long end = System.nanoTime(); + LOG.info("Duration: {}", (((double) (end - start)) / 1_000_000)); + } + + public void instantiateAskRelationsKB() { + // start a knowledge base with the behavior "I am interested in related people" + kbRelationAsker = new MockedKnowledgeBase("RelationAsker"); + kbRelationAsker.setReasonerEnabled(true); + + // Register an Ask pattern for relations with knowledge gaps enabled + GraphPattern gp1 = new GraphPattern(prefixes, "?a ex:isRelatedTo ?b . ?a ex:isFatherOf ?c ."); + this.askKIGaps = new AskKnowledgeInteraction(new CommunicativeAct(), gp1, "askRelations", false, true, true, + MatchStrategy.SUPREME_LEVEL); + kbRelationAsker.register(this.askKIGaps); + + } + + public void instantiateAnswerRelationsKB() { + // start a knowledge base with the behavior "I can supply related people" + kbRelationProvider = new MockedKnowledgeBase("RelationProvider"); + kbRelationProvider.setReasonerEnabled(true); + + // Patterns for the RelationProvider: an Answer pattern for relations + GraphPattern gp1 = new GraphPattern(prefixes, "?a ex:isRelatedTo ?b ."); + AnswerKnowledgeInteraction aKI = new AnswerKnowledgeInteraction(new CommunicativeAct(), gp1, "answerRelations"); + kbRelationProvider.register(aKI, (AnswerHandler) (anAKI, anAnswerExchangeInfo) -> { + assertTrue( + anAnswerExchangeInfo.getIncomingBindings().isEmpty() + || anAnswerExchangeInfo.getIncomingBindings().iterator().next().getVariables().isEmpty(), + "Should not have bindings in this binding set."); + + // add 1 dummy binding to the answer + BindingSet bindingSet = new BindingSet(); + Binding binding1 = new Binding(); + binding1.put("a", ""); + binding1.put("b", ""); + bindingSet.add(binding1); + + return bindingSet; + }); + } + + @AfterAll + public static void cleanup() throws InterruptedException, ExecutionException { + LOG.info("Clean up: {}", TestAskAnswerWithGapsEnabled2.class.getSimpleName()); + kn.stop().get(); + } +} diff --git a/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestAskAnswerWithGapsEnabled3.java b/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestAskAnswerWithGapsEnabled3.java new file mode 100644 index 00000000..17bc5c98 --- /dev/null +++ b/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestAskAnswerWithGapsEnabled3.java @@ -0,0 +1,179 @@ +package eu.knowledge.engine.smartconnector.api; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.fail; + +import java.net.URISyntaxException; +import java.util.Iterator; +import java.util.Set; +import java.util.concurrent.BrokenBarrierException; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.TimeoutException; + +import org.apache.jena.shared.PrefixMapping; +import org.apache.jena.sparql.graph.PrefixMappingMem; +import org.apache.jena.sparql.graph.PrefixMappingZero; +import org.apache.jena.sparql.util.FmtUtils; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import eu.knowledge.engine.reasoner.api.TriplePattern; +import eu.knowledge.engine.smartconnector.util.KnowledgeNetwork; +import eu.knowledge.engine.smartconnector.util.MockedKnowledgeBase; + +public class TestAskAnswerWithGapsEnabled3 { + + private static final Logger LOG = LoggerFactory.getLogger(TestAskAnswerWithGapsEnabled3.class); + + private static MockedKnowledgeBase kbRelationAsker; + private static MockedKnowledgeBase kbRelationProvider; + private static MockedKnowledgeBase kbRelationReactor; + + private static KnowledgeNetwork kn; + + private static PrefixMappingMem prefixes; + + private AskKnowledgeInteraction askKIGaps; + + @BeforeAll + public static void setup() throws InterruptedException, BrokenBarrierException, TimeoutException { + + prefixes = new PrefixMappingMem(); + prefixes.setNsPrefixes(PrefixMapping.Standard); + prefixes.setNsPrefix("ex", "https://www.tno.nl/example/"); + + } + + @Test + public void testAskAnswerReactWithGapsEnabled() throws InterruptedException, URISyntaxException { + + // In this test there will be an Ask KB with an AskKI with 2 triplepatterns, + // an AnswerKB with a single AnswerKI that answers only the first triplepattern of the Ask pattern, and + // a ReactKB that can answer the , but needs another pattern to be satisfied. + // The test will execute the AskKI with knowledge gaps enabled. + // As a result, the set of knowledge gaps should contain a single gap. + + setupNetwork(); + + // Perform the ASK + try { + AskResult result = kbRelationAsker.ask(askKIGaps, new BindingSet()).get(); + // check whether set of knowledge gaps contains a single gap + Set gaps = result.getKnowledgeGaps(); + LOG.info("Found gaps: " + gaps); + assertFalse(gaps.isEmpty(),"The set of knowledge gaps should not be empty"); + assertEquals(1, gaps.size(), "Number of gaps should be 1"); + Iterator iter = gaps.iterator(); + while (iter.hasNext()) { + KnowledgeGap gap = iter.next(); + Iterator gapiter = gap.iterator(); + while (gapiter.hasNext()) { + TriplePattern triple = gapiter.next(); + String tpString = FmtUtils.stringForTriple(triple.asTriple(), new PrefixMappingZero()); + LOG.info("Gap is " + tpString); + } + } + BindingSet bindings = result.getBindings(); + LOG.info("Resulting binding set is: " + bindings); + assertTrue(bindings.isEmpty(),"The resulting bindingset should be empty"); + } catch (InterruptedException | ExecutionException e) { + fail(); + } + + } + + private void setupNetwork() { + + instantiateAskRelationsKB(); + instantiateAnswerRelationsKB(); + instantiateReactRelationsKB(); + kn = new KnowledgeNetwork(); + kn.addKB(kbRelationAsker); + kn.addKB(kbRelationProvider); + kn.addKB(kbRelationReactor); + + long start = System.nanoTime(); + kn.sync(); + long end = System.nanoTime(); + LOG.info("Duration: {}", (((double) (end - start)) / 1_000_000)); + } + + public void instantiateAskRelationsKB() { + // start a knowledge base with the behavior "I am interested in related people" + kbRelationAsker = new MockedKnowledgeBase("RelationAsker"); + kbRelationAsker.setReasonerEnabled(true); + + // Register an Ask pattern for relations with knowledge gaps enabled + GraphPattern gp1 = new GraphPattern(prefixes, "?a ex:isRelatedTo ?b . ?a ex:isFatherOf ?c ."); + this.askKIGaps = new AskKnowledgeInteraction(new CommunicativeAct(), gp1, "askRelations", false, true, true, MatchStrategy.SUPREME_LEVEL); + kbRelationAsker.register(this.askKIGaps); + + } + + public void instantiateAnswerRelationsKB() { + // start a knowledge base with the behavior "I can supply related people" + kbRelationProvider = new MockedKnowledgeBase("RelationProvider"); + kbRelationProvider.setReasonerEnabled(true); + + // Patterns for the RelationProvider: an Answer pattern for relations + GraphPattern gp1 = new GraphPattern(prefixes, "?a ex:isRelatedTo1 ?b ."); + AnswerKnowledgeInteraction aKI = new AnswerKnowledgeInteraction(new CommunicativeAct(), gp1, "answerRelations"); + kbRelationProvider.register(aKI, (AnswerHandler) (anAKI, anAnswerExchangeInfo) -> { + assertTrue( + anAnswerExchangeInfo.getIncomingBindings().isEmpty() + || anAnswerExchangeInfo.getIncomingBindings().iterator().next().getVariables().isEmpty(), + "Should not have bindings in this binding set."); + + // add 1 dummy binding to the answer + BindingSet bindingSet = new BindingSet(); + Binding binding1 = new Binding(); + binding1.put("a", ""); + binding1.put("b", ""); + bindingSet.add(binding1); + + return bindingSet; + }); + } + + public void instantiateReactRelationsKB() { + + // start a knowledge base with the behavior "I can react to supply related people" + // when I get couples of "people that live in the same house". + kbRelationReactor = new MockedKnowledgeBase("relationReactor"); + kbRelationReactor.setReasonerEnabled(true); + + // Patterns for the relationReactor: an React pattern to supply relations + GraphPattern gp1 = new GraphPattern(prefixes, "?a ex:liveInTheSameHouse ?b ."); + GraphPattern gp2 = new GraphPattern(prefixes, "?a ex:isRelatedTo ?b ."); + ReactKnowledgeInteraction reactKI = new ReactKnowledgeInteraction(new CommunicativeAct(), gp1, gp2); + kbRelationReactor.register(reactKI, (anRKI, aReactExchangeInfo) -> { + + LOG.info("RelationReactor reacting to incoming bindings..."); + BindingSet bindingSet = new BindingSet(); + var argument = aReactExchangeInfo.getArgumentBindings(); + Iterator iter = argument.iterator(); + while (iter.hasNext()) { + Binding b = iter.next(); + LOG.info("Incoming tuple of people living in the same house is {}", b); + Binding binding1 = new Binding(); + binding1.put("a", b.get("a")); + binding1.put("b", b.get("b")); + bindingSet.add(binding1); + } + + return bindingSet; + }); + + } + + @AfterAll + public static void cleanup() throws InterruptedException, ExecutionException { + LOG.info("Clean up: {}", TestAskAnswerWithGapsEnabled3.class.getSimpleName()); + kn.stop().get(); + } +} diff --git a/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestAskAnswerWithGapsNotEnabled.java b/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestAskAnswerWithGapsNotEnabled.java new file mode 100644 index 00000000..2e2a334f --- /dev/null +++ b/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestAskAnswerWithGapsNotEnabled.java @@ -0,0 +1,100 @@ +package eu.knowledge.engine.smartconnector.api; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.fail; + +import java.net.URISyntaxException; +import java.util.Set; +import java.util.concurrent.BrokenBarrierException; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.TimeoutException; + +import org.apache.jena.shared.PrefixMapping; +import org.apache.jena.sparql.graph.PrefixMappingMem; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import eu.knowledge.engine.smartconnector.util.KnowledgeNetwork; +import eu.knowledge.engine.smartconnector.util.MockedKnowledgeBase; + +public class TestAskAnswerWithGapsNotEnabled { + + private static final Logger LOG = LoggerFactory.getLogger(TestAskAnswerWithGapsNotEnabled.class); + + private static MockedKnowledgeBase kbRelationAsker; + + private static KnowledgeNetwork kn; + + private static PrefixMappingMem prefixes; + + private AskKnowledgeInteraction askKI; + + @BeforeAll + public static void setup() throws InterruptedException, BrokenBarrierException, TimeoutException { + + prefixes = new PrefixMappingMem(); + prefixes.setNsPrefixes(PrefixMapping.Standard); + prefixes.setNsPrefix("ex", "https://www.tno.nl/example/"); + + } + + @Test + public void testAskAnswerWithoutGapsEnabled() throws InterruptedException, URISyntaxException { + + // In this test there will be only 1 KB with a single AskKI. + // The test will execute the AskKI without knowledge gaps enabled. + // As a result, the set of knowledge gaps should be null. + + setupNetwork(); + + // Perform the ASK + try { + AskResult result = kbRelationAsker.ask(askKI, new BindingSet()).get(); + // check whether set of knowledge gaps is empty + Set gaps = result.getKnowledgeGaps(); + LOG.info("Found gaps: " + gaps); + assertEquals(null, gaps, "The set of knowledge gaps should be null"); + BindingSet bindings = result.getBindings(); + LOG.info("Resulting binding set is: " + bindings); + assertTrue(bindings.isEmpty(), "The resulting bindingset should be empty"); + } catch (InterruptedException | ExecutionException e) { + fail(); + } + + } + + private void setupNetwork() { + + instantiateAskRelationsKB(); + kn = new KnowledgeNetwork(); + kn.addKB(kbRelationAsker); + + long start = System.nanoTime(); + kn.sync(); + long end = System.nanoTime(); + LOG.info("Duration: {}", (((double) (end - start)) / 1_000_000)); + } + + public void instantiateAskRelationsKB() { + // start a knowledge base with the behavior "I am interested in related people" + kbRelationAsker = new MockedKnowledgeBase("RelationAsker"); + kbRelationAsker.setReasonerEnabled(true); + + // Register an Ask pattern for relations without knowledge gaps enabled + GraphPattern gp1 = new GraphPattern(prefixes, "?a ex:isRelatedTo ?b ."); + this.askKI = new AskKnowledgeInteraction(new CommunicativeAct(), gp1, "askRelations", false, true, false, + MatchStrategy.SUPREME_LEVEL); + kbRelationAsker.register(this.askKI); + + } + + @AfterAll + public static void cleanup() throws InterruptedException, ExecutionException { + LOG.info("Clean up: {}", TestAskAnswerWithGapsNotEnabled.class.getSimpleName()); + kn.stop().get(); + } +} diff --git a/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestComplexGraphPatternMatching.java b/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestComplexGraphPatternMatching.java index 7db735f5..0fc0bf69 100644 --- a/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestComplexGraphPatternMatching.java +++ b/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestComplexGraphPatternMatching.java @@ -72,19 +72,17 @@ public void testComplexGraphPattern() throws InterruptedException { try { LOG.trace("Before ask."); - AskPlan askPlan = dashboardKB.planAsk(askKI, new RecipientSelector()); + AskResult askResult = dashboardKB.ask(askKI, new RecipientSelector(), new BindingSet()).get(); - LOG.info("Gaps: {}", Util.getKnowledgeGaps(askPlan.getReasonerPlan().getStartNode())); + LOG.info("Gaps: {}", askResult.getKnowledgeGaps()); - askPlan.getReasonerPlan().getStore().printGraphVizCode(askPlan.getReasonerPlan()); + askResult.getReasonerPlan().getStore().printGraphVizCode(askResult.getReasonerPlan()); - result = askPlan.execute(new BindingSet()).get(); - - bindings = result.getBindings(); + bindings = askResult.getBindings(); LOG.trace("After ask."); - Set kbIds = result.getExchangeInfoPerKnowledgeBase().stream().map(AskExchangeInfo::getKnowledgeBaseId) - .collect(Collectors.toSet()); + Set kbIds = askResult.getExchangeInfoPerKnowledgeBase().stream() + .map(AskExchangeInfo::getKnowledgeBaseId).collect(Collectors.toSet()); assertEquals( new HashSet( @@ -263,7 +261,7 @@ public float convert(float fahrenheit) { GraphPattern gp2 = new GraphPattern(prefixes, TestUtils.convertGP(pattern)); AskKnowledgeInteraction askKI = new AskKnowledgeInteraction(new CommunicativeAct(), gp2, null, false, false, - MatchStrategy.NORMAL_LEVEL); + true, MatchStrategy.NORMAL_LEVEL); dashboardKB.setDomainKnowledge(new HashSet<>( Arrays.asList(locatedInRule, thermostatSensorRule, opencloseSensorRule, fahrenheitConverterRule))); dashboardKB.register(askKI); diff --git a/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestDynamicSemanticComposition.java b/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestDynamicSemanticComposition.java index 4b096693..40384a0b 100644 --- a/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestDynamicSemanticComposition.java +++ b/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestDynamicSemanticComposition.java @@ -30,10 +30,8 @@ import eu.knowledge.engine.reasoner.BaseRule.MatchFlag; import eu.knowledge.engine.reasoner.Match; -import eu.knowledge.engine.reasoner.ReasonerPlan; import eu.knowledge.engine.reasoner.Rule; import eu.knowledge.engine.reasoner.api.TriplePattern; -import eu.knowledge.engine.smartconnector.impl.Util; import eu.knowledge.engine.smartconnector.util.KnowledgeNetwork; import eu.knowledge.engine.smartconnector.util.MockedKnowledgeBase; @@ -104,20 +102,20 @@ public void testAskAnswer() throws InterruptedException, URISyntaxException { setupNetwork(); - // start planning ask for targets! + // perform an ASK that should produce gaps and if found update network to fix gaps + try { + AskResult resultWithGaps = kbHVTSearcher.ask(askKI, new BindingSet()).get(); + // check for knowledge gaps + Set gaps = resultWithGaps.getKnowledgeGaps(); + LOG.info("Found gaps: " + gaps); + // add KB that fills the knowledge gap + updateNetwork(gaps); + } catch (InterruptedException | ExecutionException e) { + fail(); + } + + // perform ask for targets and knowledge gaps should have been fixed BindingSet bindings = null; - AskPlan plan = kbHVTSearcher.planAsk(askKI, new RecipientSelector()); - ReasonerPlan rn = plan.getReasonerPlan(); - rn.getStore().printGraphVizCode(rn); - // check for knowledge gaps - Set> gaps = Util.getKnowledgeGaps(rn.getStartNode()); - LOG.info("Found gaps: " + gaps); - - // add KB that fills the knowledge gap - updateNetwork(gaps); - - // start testing ask for targets! - bindings = null; try { AskResult result = kbHVTSearcher.ask(askKI, new BindingSet()).get(); bindings = result.getBindings(); @@ -182,7 +180,7 @@ private void setupNetwork() { LOG.info("Duration: {}", (((double) (end - start)) / 1_000_000)); } - private void updateNetwork(Set> gaps) { + private void updateNetwork(Set gaps) { instantiateTargetCountrySupplierKB(); instantiateTargetLanguageSupplierKB(); @@ -252,7 +250,7 @@ public void instantiateHVTSearcherKB() { // Patterns for the HVTSearcher // a pattern to ask for High Value Target searches GraphPattern gp2 = new GraphPattern(prefixes, "?id rdf:type v1905:HighValueTarget . ?id v1905:hasName ?name ."); - this.askKI = new AskKnowledgeInteraction(new CommunicativeAct(), gp2, "askHVTargets"); + this.askKI = new AskKnowledgeInteraction(new CommunicativeAct(), gp2, "askHVTargets",true); kbHVTSearcher.register(this.askKI); // a pattern to react to incoming new High Value Targets ReactKnowledgeInteraction reactKIsearcher = new ReactKnowledgeInteraction(new CommunicativeAct(), gp2, null); diff --git a/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestRegisterKnowledgeInteraction.java b/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestRegisterKnowledgeInteraction.java index 3f7adf6c..baf67237 100644 --- a/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestRegisterKnowledgeInteraction.java +++ b/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/api/TestRegisterKnowledgeInteraction.java @@ -51,10 +51,12 @@ public void smartConnectorStopped(SmartConnector aSC) { String kiName = "some-name"; - sc1.register(new AskKnowledgeInteraction(new CommunicativeAct(), new GraphPattern("?a ?c"), kiName)); + sc1.register( + new AskKnowledgeInteraction(new CommunicativeAct(), new GraphPattern("?a ?c"), kiName, false)); assertThrows(IllegalArgumentException.class, () -> { - sc1.register(new AskKnowledgeInteraction(new CommunicativeAct(), new GraphPattern("?a ?c"), kiName)); + sc1.register(new AskKnowledgeInteraction(new CommunicativeAct(), new GraphPattern("?a ?c"), kiName, + false)); }); sc1.stop(); diff --git a/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/runtime/messaging/DistributedMessageDispatcherTest.java b/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/runtime/messaging/DistributedMessageDispatcherTest.java index d60ef235..3e7265aa 100644 --- a/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/runtime/messaging/DistributedMessageDispatcherTest.java +++ b/smart-connector/src/test/java/eu/knowledge/engine/smartconnector/runtime/messaging/DistributedMessageDispatcherTest.java @@ -25,49 +25,52 @@ void testLocalMessageExchange() throws Exception { MessageDispatcher md = new MessageDispatcher(8081, new URI("http://localhost:8081"), new URI("http://localhost:8080")); - kd.start(); + try { + kd.start(); - Thread.sleep(1000); + Thread.sleep(1000); - md.start(); + md.start(); - URI kb1Id = new URI("http://test.com/kb1"); - URI kb2Id = new URI("http://test.com/kb2"); - MockSmartConnector sc1 = new MockSmartConnector(kb1Id); - MockSmartConnector sc2 = new MockSmartConnector(kb2Id); + URI kb1Id = new URI("http://test.com/kb1"); + URI kb2Id = new URI("http://test.com/kb2"); + MockSmartConnector sc1 = new MockSmartConnector(kb1Id); + MockSmartConnector sc2 = new MockSmartConnector(kb2Id); - md.getLocalSmartConnectorConnectionManager().smartConnectorAdded(sc1); - md.getLocalSmartConnectorConnectionManager().smartConnectorAdded(sc2); + md.getLocalSmartConnectorConnectionManager().smartConnectorAdded(sc1); + md.getLocalSmartConnectorConnectionManager().smartConnectorAdded(sc2); - Thread.sleep(5000); + Thread.sleep(5000); - BindingSet bindingSet = new BindingSet(); - Binding binding = new Binding(); - binding.put("a", "b"); - binding.put("c", "d"); - bindingSet.add(binding); + BindingSet bindingSet = new BindingSet(); + Binding binding = new Binding(); + binding.put("a", "b"); + binding.put("c", "d"); + bindingSet.add(binding); - // -- Ask message --> - AskMessage message1 = new AskMessage(kb1Id, kb1Id, kb2Id, kb2Id, bindingSet); - sc1.send(message1); + // -- Ask message --> + AskMessage message1 = new AskMessage(kb1Id, kb1Id, kb2Id, kb2Id, bindingSet); + sc1.send(message1); - Thread.sleep(1000); + Thread.sleep(1000); - KnowledgeMessage message1Received = sc2.getLastMessage(); - assertEquals(message1.toString(), message1Received.toString()); + KnowledgeMessage message1Received = sc2.getLastMessage(); + assertEquals(message1.toString(), message1Received.toString()); - // <-- Answer message -- - AnswerMessage message2 = new AnswerMessage(kb2Id, kb2Id, kb1Id, kb1Id, message1.getMessageId(), bindingSet); - sc2.send(message2); + // <-- Answer message -- + AnswerMessage message2 = new AnswerMessage(kb2Id, kb2Id, kb1Id, kb1Id, message1.getMessageId(), bindingSet); + sc2.send(message2); - Thread.sleep(1000); + Thread.sleep(1000); - KnowledgeMessage message2Received = sc1.getLastMessage(); - assertEquals(message2.toString(), message2Received.toString()); + KnowledgeMessage message2Received = sc1.getLastMessage(); + assertEquals(message2.toString(), message2Received.toString()); - md.stop(); - kd.stop(); - assertTrue(NetUtils.portAvailable(8080)); + } finally { + md.stop(); + kd.stop(); + assertTrue(NetUtils.portAvailable(8080)); + } } @Test @@ -79,51 +82,54 @@ void testRemoteMessageExchange() throws Exception { MessageDispatcher md2 = new MessageDispatcher(8082, new URI("http://localhost:8082"), new URI("http://localhost:8080")); - kd.start(); + try { + kd.start(); - Thread.sleep(1000); + Thread.sleep(1000); - md1.start(); - md2.start(); + md1.start(); + md2.start(); - URI kb1Id = new URI("http://test.com/kb1"); - URI kb2Id = new URI("http://test.com/kb2"); - MockSmartConnector sc1 = new MockSmartConnector(kb1Id); - MockSmartConnector sc2 = new MockSmartConnector(kb2Id); + URI kb1Id = new URI("http://test.com/kb1"); + URI kb2Id = new URI("http://test.com/kb2"); + MockSmartConnector sc1 = new MockSmartConnector(kb1Id); + MockSmartConnector sc2 = new MockSmartConnector(kb2Id); - md1.getLocalSmartConnectorConnectionManager().smartConnectorAdded(sc1); - md2.getLocalSmartConnectorConnectionManager().smartConnectorAdded(sc2); + md1.getLocalSmartConnectorConnectionManager().smartConnectorAdded(sc1); + md2.getLocalSmartConnectorConnectionManager().smartConnectorAdded(sc2); - Thread.sleep(5000); + Thread.sleep(5000); - BindingSet bindingSet = new BindingSet(); - Binding binding = new Binding(); - binding.put("a", "b"); - binding.put("c", "d"); - bindingSet.add(binding); + BindingSet bindingSet = new BindingSet(); + Binding binding = new Binding(); + binding.put("a", "b"); + binding.put("c", "d"); + bindingSet.add(binding); - // -- Ask message --> - AskMessage message1 = new AskMessage(kb1Id, kb1Id, kb2Id, kb2Id, bindingSet); - sc1.send(message1); + // -- Ask message --> + AskMessage message1 = new AskMessage(kb1Id, kb1Id, kb2Id, kb2Id, bindingSet); + sc1.send(message1); - Thread.sleep(1000); + Thread.sleep(1000); - KnowledgeMessage message1Received = sc2.getLastMessage(); - assertEquals(message1.toString(), message1Received.toString()); + KnowledgeMessage message1Received = sc2.getLastMessage(); + assertEquals(message1.toString(), message1Received.toString()); - // <-- Answer message -- - AnswerMessage message2 = new AnswerMessage(kb2Id, kb2Id, kb1Id, kb1Id, message1.getMessageId(), bindingSet); - sc2.send(message2); + // <-- Answer message -- + AnswerMessage message2 = new AnswerMessage(kb2Id, kb2Id, kb1Id, kb1Id, message1.getMessageId(), bindingSet); + sc2.send(message2); - Thread.sleep(1000); + Thread.sleep(1000); - KnowledgeMessage message2Received = sc1.getLastMessage(); - assertEquals(message2.toString(), message2Received.toString()); + KnowledgeMessage message2Received = sc1.getLastMessage(); + assertEquals(message2.toString(), message2Received.toString()); - md1.stop(); - md2.stop(); + } finally { + md1.stop(); + md2.stop(); - kd.stop(); - assertTrue(NetUtils.portAvailable(8080)); + kd.stop(); + assertTrue(NetUtils.portAvailable(8080)); + } } }