Skip to content

Commit

Permalink
Merge pull request #59 from singnet/feature/issue-52-new-design-distr…
Browse files Browse the repository at this point in the history 
…ibutedatomspace

[# 52] Feature / New design DistributedAtomSpace
  • Loading branch information
@marcocapozzoli
marcocapozzoli authored Dec 12, 2023
2 parents 5539a99 + e9f9eb1 commit 3452333
Show file tree
Hide file tree
Showing 18 changed files with 1,148 additions and 2,266 deletions.
10 changes: 5 additions & 5 deletions Makefile
View file
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
ci-tests:
test-ci:
@py.test -sx -vv ./tests/unit --cov=./hyperon_das/ --cov-report=term-missing --cov-fail-under=70

integration-tests:
test-integration:
@py.test -sx -vv ./tests/integration

unit-tests:
test-unit:
@py.test -sx -vv ./tests/unit

isort:
@isort ./hyperon_das ./tests --multi-line=3 --trailing-comma --force-grid-wrap=0 --use-parentheses --line-width=79
@isort ./hyperon_das ./tests --multi-line=3 --trailing-comma --force-grid-wrap=0 --use-parentheses --line-width=100

black:
@black ./hyperon_das ./tests --line-length 79 -t py37 --skip-string-normalization
@black ./hyperon_das ./tests --line-length 100 -t py37 --skip-string-normalization

flake8:
@flake8 --show-source ./hyperon_das ./test
Expand Down
201 changes: 41 additions & 160 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ Hi! This package is a query engine API for Distributed AtomSpace (DAS). When is
- [Using-pip](#using-pip)
- [Using-Poetry](#using-poetry)
- [Usage](#usage)
- [Redis and MongoDB](#redis-and-mongodb)
- [Create a client API](#create-a-client-api)
- [In Memory](#in-memory)
- [Local](#local)
- [Remote](#remote)
- [Server](#server)
- [Tests](#tests)

## Installation
Expand Down Expand Up @@ -62,180 +62,61 @@ Now you can run the project within the Poetry virtual environment.

## Usage

So far we have two ways of making queries using the API. One that uses database persistence and another that doesn't. The way to create and execute the query is exactly the same, the only difference is when you need to instantiate the API class. Below you can see more details.
You can instantiate DAS in three different ways. see below:

### Redis and MongoDB
### Local
This is a local instance of DAS with default settings.

If you want to use data persistence, you must have Redis and MongoDB running in your environment and you must have the following variables configured with their respective values:
```python
from hyperon_das import DistributedAtomSpace
*Example*:
```scheme
DAS_MONGODB_HOSTNAME=172.17.0.2
DAS_MONGODB_PORT=27017
DAS_MONGODB_USERNAME=mongo
DAS_MONGODB_PASSWORD=mongo
DAS_REDIS_HOSTNAME=127.0.0.1
DAS_REDIS_PORT=6379
das = DistributedAtomSpace()
```

**TIP**: You can change the values in the *environment* file, which is in the root directory and run the command below:
### Remote

```bash
source environment
```
##### Create a client API
To create a remote DAS, you need to specify the 'query_engine' parameter as 'remote' and pass the machine, 'host' and 'port' parameters. See below how to do this:

```python
from hyperon_das import DistributedAtomSpace
api = DistributedAtomSpace('redis_mongo')
das = DistributedAtomSpace(query_engine='remote', host='0.0.0.0', port=1234)
```

### In Memory
This way you don't need anything just instantiate the class as shown below:


1. A simple query which is a `AND` operation on two links whose targets are variables.

```python
from hyperon_das import DistributedAtomSpace
from hyperon_das.pattern_matcher import And, Variable, Link
from hyperon_das.utils import QueryOutputFormat
api = DistributedAtomSpace('ram_only')
api.add_link({
'type': 'Evaluation',
'targets': [
{'type': 'Predicate', 'name': 'Predicate:has_name'},
{
'type': 'Evaluation',
'targets': [
{'type': 'Predicate', 'name': 'Predicate:has_name'},
{
'type': 'Set',
'targets': [
{'type': 'Reactome', 'name': 'Reactome:R-HSA-164843'},
{'type': 'Concept', 'name': 'Concept:2-LTR circle formation'},
]
},
],
},
],
})
expression = Link("Evaluation", ordered=True, targets=[Variable("V1"), Variable("V2")])
resp = api.pattern_matcher_query(expression, {'return_type': QueryOutputFormat.JSON, 'toplevel_only': True})
print(resp)
```

```bash
[
{
"V1": {
"type": "Predicate",
"name": "Predicate:has_name",
"is_link": false,
"is_node": true
},
"V2": {
"type": "Evaluation",
"targets": [
{
"type": "Predicate",
"name": "Predicate:has_name"
},
{
"type": "Set",
"targets": [
{
"type": "Reactome",
"name": "Reactome:R-HSA-164843"
},
{
"type": "Concept",
"name": "Concept:2-LTR circle formation"
}
]
}
],
"is_link": true,
"is_node": false
}
}
]
```

2. Add Node and And Link (It's possible only using [Ram Only](#in-memory))
```python
api.count_atoms() # (0, 0)
nodes = [
{
'type': 'Reactome',
'name': 'Reactome:R-HSA-164843',
},
{
'type': 'Concept',
'name': 'Concept:2-LTR circle formation',
}
]
for node in nodes:
api.add_node(node)
api.count_atoms() # (2, 0)
link = {
'type': 'Evaluation',
'targets': [
{
'type': 'Predicate',
'name': 'Predicate:has_name'
},
{
'type': 'Evaluation',
'targets': [
{
'type': 'Predicate',
'name': 'Predicate:has_name'
},
{
'type': 'Set',
'targets': [
{
'type': 'Reactome',
'name': 'Reactome:R-HSA-164843',
},
{
'type': 'Concept',
'name': 'Concept:2-LTR circle formation',
},
]
},
],
},
],
}
api.add_link(link)
api.count_atoms() # (3, 3)
```
### Server
To create a DAS server, you will need to specify the 'atomdb' parameter as 'redis_mongo' and pass the database parameters. The databases supported in this release are Redis and MongoDB. Therefore, the minimum expected parameters are:

- mongo_hostname
- mongo_port
- mongo_username
- mongo_password
- redis_hostname
- redis_port

**Note1:** in this example I add 2 nodes and 1 a link, but in the end I have 3 nodes and 3 links. Therefore, it is possible to add nested links and as links are composed of nodes, if the link node doesn't exist in the system it's added.
but it is possible to pass other configuration parameters:

**Note2:** For these methods to work well, both nodes and links must be a dict with the structure shown above, i.e, for **nodes** you need to send, at least, the parameters `type` and `name` and for **links** `type` and `targets`
- mongo_tls_ca_file
- redis_username
- redis_password
- redis_cluster
- redis_ssl

```python
from hyperon_das import DistributedAtomSpace
das = DistributedAtomSpace(
atomdb='redis_mongo',
mongo_hostname='127.0.0.2',
mongo_port=27017,
mongo_username='mongo',
mongo_password='mongo',
redis_hostname='127.0.0.1',
redis_port=6379
)
```
## Tests

You can run the command below to run the unit tests

```bash
make test-coverage
```
make test-unit
2 changes: 1 addition & 1 deletion hyperon_das/__init__.py
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
from hyperon_das.api import DistributedAtomSpace
from hyperon_das.das import DistributedAtomSpace

__all__ = ['DistributedAtomSpace']
Loading

0 comments on commit 3452333

Please sign in to comment.