Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mib version2.2 #27

Open
wants to merge 42 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
ee595b2
add validations of metadata form for strictly follows in insert and
milanjakhaniya Apr 24, 2024
790afaa
combine validations for filed label and num of fields
milanjakhaniya Apr 25, 2024
fda5964
Merge branch 'main' of https://github.com/LL08-MathematicalModelling-…
milanjakhaniya Apr 26, 2024
c2861ad
new changes added for validations
milanjakhaniya Apr 26, 2024
afd6614
implimetation for time and operation number for insert data api
milanjakhaniya May 2, 2024
0d161b4
according to new design is deleted, date time and operationsimplimented
milanjakhaniya May 3, 2024
b4559b6
add fieled UI created and handle it's validations
milanjakhaniya May 4, 2024
874a948
fixed fetch data for view is_deleted false data only
milanjakhaniya May 5, 2024
ca093dc
field wise date time implimentation for post and fetch api
milanjakhaniya May 9, 2024
9f21a5d
data_type field added for insert data api
milanjakhaniya May 9, 2024
9069488
Added date time and data_type in Update data API V2
milanjakhaniya May 10, 2024
d2e503e
add flag on deleted True for delete api
milanjakhaniya May 11, 2024
bf11598
data type field match is required
milanjakhaniya May 11, 2024
a05a551
not allow to add new fields during updates
milanjakhaniya May 13, 2024
28d1d86
fix bug and handle update with new field insertion
milanjakhaniya May 14, 2024
479178d
intigration of dowell clock api for datacube api
milanjakhaniya May 16, 2024
5c97d93
Merge pull request #18 from LL08-MathematicalModelling-dowell/mj-meta…
JacobDowell May 18, 2024
c6cb1cd
instead of username implimented userID in all functions
milanjakhaniya May 21, 2024
3ca178b
Merge pull request #19 from LL08-MathematicalModelling-dowell/mj-userID
JacobDowell May 21, 2024
5776dcf
new documentations created for datacube v2 apis
milanjakhaniya May 22, 2024
f5de064
fix small bug for data view load mongo collection
milanjakhaniya May 22, 2024
8be8ef5
Merge pull request #20 from LL08-MathematicalModelling-dowell/mj-doc-v2
JacobDowell May 23, 2024
03a8475
upload collection csv file and validations handling for frontend
milanjakhaniya Jun 5, 2024
1d025e2
fixed order of the collection list and handle untitle collections
milanjakhaniya Jun 6, 2024
312202f
fix bug for upload csv collection with rename db1 coll name
milanjakhaniya Jun 8, 2024
1d46ba3
created logic for search and page
milanjakhaniya Jun 9, 2024
64bcb2f
untitled filed default created 10000
milanjakhaniya Jun 9, 2024
e4c53ed
implimentation for field view page and collectin view page
milanjakhaniya Jun 13, 2024
9cff967
more APIs added 16_09_2024, MIB
Goldeno10 Sep 16, 2024
9ea91d2
Completed ADD CREATE DB/COLLECTIONS and LIST COLLECTIONS endpoints
Goldeno10 Sep 23, 2024
da7a949
Removed unnecessary Apps
Goldeno10 Sep 23, 2024
f66d2c4
re-added the api_key check
Goldeno10 Sep 23, 2024
2ecb69e
Improved and optimized the crud API View
Goldeno10 Sep 24, 2024
6753f79
Improved code quality and structure
Goldeno10 Sep 27, 2024
57f3675
Improved Api user and data validation
Goldeno10 Oct 1, 2024
d0a09dc
Improve API serializer structure
Goldeno10 Oct 8, 2024
3cb3e2b
updated git ignore
Goldeno10 Oct 8, 2024
0d5eca3
updated swagger documentation
Goldeno10 Oct 8, 2024
4e87096
comment out un-reqired serializer fields
Goldeno10 Oct 12, 2024
23a43b2
Optimized code base
Goldeno10 Oct 13, 2024
1ead89c
Fixed request body and REST API JSON response
Goldeno10 Oct 13, 2024
35df5b4
Updated the requirements.txt
Goldeno10 Oct 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
10 changes: 8 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,15 @@ coverage.xml
*.log
*.pot


config.json
config.json*
config*.json
config*.json*
config.json*
profile_images/
env/
venv/
.idea/
.vscode/
.vscode/
.env
.env*
160 changes: 59 additions & 101 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,38 +19,56 @@ There are two main steps to use DoWell DataCube


## Step 1: Add request for database
To submit a request for the database, please begin by logging into DataCube. You can access the login page by clicking on the following link: [DataCube Login](https://datacube.uxlivinglab.online/). <br>
To submit a request for the database, please begin by logging into DataCube. You can access the login page by clicking on the following link: [DataCube Login](https://uxlive.online/). <br>
Once you have successfully logged in, you will be redirected to the dashboard. From there, you can enter your request for the database.

![Dashboard](screenshots/dashboard.jpg)
<br>
<br>
After submitting your request for the database, you'll have the ability to view the created database and its associated collections. <br>
By clicking on "View Collection," you can access and inspect all the collections within the relevant database. <br>
Moreover, you also have the option to add additional collections to the database if needed
Moreover, you also have the option to add additional collections and fields to the database if needed

![Metadata Records](screenshots/metadata_record.jpg)

ADD NEW COLLECTION
![Add Collections](screenshots/add_collections_to_database.jpg)

ADD NEW FIELD
![Add fields](screenshots/add_fields_to_database.jpg)
<br>
<br>
You also have the option to import JSON or CSV files to add documents to a collection.
![Import File](screenshots/import_file.jpg)
* import JSON or CSV files feature under development**
<!-- You also have the option to import JSON or CSV files to add documents to a collection.
![Import File](screenshots/import_file.jpg) -->
<br>
<br>

You can also view the data in a data view by selecting the database and collection from the dropdown menu.
![Data View](screenshots/data_view.jpg)
<br>
<br>

After adding the database and collection, you can now utilize APIs to add, retrieve, and update data within the database.

## Step 2: Use Database using APIs
Click here to read the documentation in postman [Documentation](https://documenter.getpostman.com/view/29895764/2s9YRB1WyN)

# API Detailed Documentation
=============================================================================
## Allow Data Types
```
"data_type": "real_data"

"data_type": "testing_data"

"data_type": "learning_data"

"data_type": "deleted_data"
```

## Fetch Data Using the API

### URL: https://datacube.uxlivinglab.online/db_api/get_data/
### URL: https://uxlive.online/db_api/get_data/
### Method: POST

This API enables you to retrieve data from a specific collection in the Dowell database based on a query.
Expand All @@ -63,8 +81,9 @@ This API enables you to retrieve data from a specific collection in the Dowell d
"db_name": "dowell",
"coll_name": "test",
"operation": "fetch",
"data_type": "real_data",
"filters": {
"_id": "101001010101"
"_id": "664cf185ac91b6cab59e64f9"
},
"limit": 1,
"offset": 0
Expand All @@ -77,6 +96,7 @@ This API enables you to retrieve data from a specific collection in the Dowell d
- `db_name`: The name of the database.
- `coll_name`: The name of the collection to fetch data from.
- `operation`: The operation type, which is "fetch" for this request.
- `data_type`: The data type, which is "real_data" for this request.
- `filters`: A JSON object specifying the query criteria.
- `limit` (optional): The maximum number of records to retrieve per page.
- `offset` (optional): The page number for pagination.
Expand All @@ -87,14 +107,15 @@ This API enables you to retrieve data from a specific collection in the Dowell d
import requests
import json

url = "https://datacube.uxlivinglab.online/db_api/get_data/"
url = "https://uxlive.online/db_api/get_data/"

data = {
"api_key": "your-dowell-api-key",
"db_name": "dowell",
"coll_name": "test",
"operation": "fetch",
"filters": {"_id": "101001010101"},
"data_type": "real_data",
"filters": {"_id": "664cf185ac91b6cab59e64f9"},
"limit": 1,
"offset": 0
}
Expand All @@ -110,7 +131,7 @@ print(response.text)

## Insert Data

### URL: https://datacube.uxlivinglab.online/db_api/crud/
### URL: https://uxlive.online/db_api/crud/
### Method: POST

Use this API to add data to a specific collection within the Dowell database.
Expand All @@ -123,14 +144,11 @@ Use this API to add data to a specific collection within the Dowell database.
"db_name": "dowell",
"coll_name": "test",
"operation": "insert",
"data_type": "real_data",
"data": {
"id": "101001010101",
"info": {"name": "dowell"},
"records": [{
"record": "1",
"type": "overall"
}]
}
"first_name":"milan",
"last_name":"jakhaniya"
}
}
```

Expand All @@ -140,6 +158,7 @@ Use this API to add data to a specific collection within the Dowell database.
- `db_name`: The name of the database.
- `coll_name`: The name of the collection for data insertion.
- `operation`: The operation type, which is "insert" for this request.
- `data_type`: The data type, which is "real_data" for this request.
- `data`: The data to be inserted as a JSON object.

#### Example of Inserting Data in Python
Expand All @@ -148,21 +167,18 @@ Use this API to add data to a specific collection within the Dowell database.
import requests
import json

url = "https://datacube.uxlivinglab.online/db_api/crud/"
url = "https://uxlive.online/db_api/crud/"

data = {
"api_key": "your-dowell-api-key",
"db_name": "dowell",
"coll_name": "test5",
"operation": "insert",
"data_type": "real_data",
"data": {
"id": "101001010101",
"info": {'name': "dowell"},
"records": [{
"record": "1",
"type": "overall"
}]
}
"first_name":"milan",
"last_name":"jakhaniya"
}
}

response = requests.post(url, json=data)
Expand All @@ -176,11 +192,11 @@ print(response.text)

**Note: If a user tries to add more than 10,000 documents in the same collection, the following error will be received:**

For error: `{"message": "Sorry, You can not allow to add more than 10,000 documents."}`
For error: `{"message": "Sorry, 10,000 number of documents reached in 'your collection name'"}`

## Update Data

### URL: https://datacube.uxlivinglab.online/db_api/crud/
### URL: https://uxlive.online/db_api/crud/
### Method: PUT

Use this API to update data in a specific collection within the Dowell database.
Expand All @@ -193,14 +209,10 @@ Use this API to update data in a specific collection within the Dowell database.
"db_name": "dowell",
"coll_name": "test",
"operation": "update",
"query": {"_id": "64f6fac8ac03855a010559f2"},
"data_type": "real_data",
"query": {"_id": "664cf185ac91b6cab59e64f9"},
"update_data": {
"id": "101001010101",
"info": {"name": "dowell"},
"records": [{
"record": "1",
"type": "overall_updated"
}]
"age":"36"
}
}
```
Expand All @@ -211,6 +223,7 @@ Use this API to update data in a specific collection within the Dowell database.
- `db_name`: The name of the database.
- `coll_name`: The name of the collection to update data in.
- `operation`: The operation type, which is "update" for this request.
- `data_type`: The data type, which is "real_data" for this request.
- `query`: Query or filter the record you want to update.
- `update_data`: The data to be updated, is provided as a JSON object.

Expand All @@ -220,22 +233,18 @@ Use this API to update data in a specific collection within the Dowell database.
import requests
import json

url = "https://datacube.uxlivinglab.online/db_api/crud/"
url = "https://uxlive.online/db_api/crud/"

data = {
"api_key": "your-dowell-api-key",
"db_name": "dowell",
"coll_name": "test",
"operation": "update",
"query" : {"_id": "64f6fac8ac03855a010559f2"}
"data_type": "real_data",
"query" : {"_id": "664cf185ac91b6cab59e64f9"}
"update_data": {
"id": "101001010101",
"info": {'name': "dowell"},
"records": [{
"record": "1",
"type": "overall_updated"
}]
}
"age":"36"
}
}

response = requests.put(url, json=data)
Expand All @@ -250,7 +259,7 @@ print(response.text)

## Delete Data

### URL: https://datacube.uxlivinglab.online/db_api/crud/
### URL: https://uxlive.online/db_api/crud/
### Method: DELETE

Use this API to remove data from a specific collection within the Dowell database.
Expand All @@ -263,8 +272,9 @@ Use this API to remove data from a specific collection within the Dowell databas
"db_name": "dowell",
"coll_name": "test",
"operation": "delete",
"data_type": "real_data",
"query": {
"id": "101001010101"
"id": "664cf185ac91b6cab59e64f9"
}
}
```
Expand All @@ -275,22 +285,24 @@ Use this API to remove data from a specific collection within the Dowell databas
- `db_name`: The name of the database.
- `coll_name`: The name of the collection from which data will be deleted.
- `operation`: The operation type, which is "delete" for this request.
- `data_type`: The data type, which is "real_data" for this request.
- `query`: Query or filter the record you want to delete.

#### Example of Deleting Data in Python

```python
import requests
import json
url = "https://datacube.uxlivinglab.online/db_api/crud/"
url = "https://uxlive.online/db_api/crud/"

data = {
"api_key": "your-dowell-api-key",
"db_name": "dowell",
"coll_name": "test",
"operation": "delete",
"data_type": "real_data",
"query": {
"_id": "64f6fac8ac03855a010559f2",
"_id": "664cf185ac91b6cab59e64f9",
}

}
Expand All @@ -306,60 +318,6 @@ print(response.text)

```markdown

### Add Collection to Database

### URL: https://datacube.uxlivinglab.online/db_api/add_collection/
### Method: POST

The Metadata View API allows you to add collections to a specified database within the DoWell DataCube platform. This API provides endpoints for adding and managing collections, making it a powerful tool for data organization and management.

#### Request Data / API Payload

```json
{
"api_key": "your-dowell-api-key",
"db_name": "dowell",
"coll_names": "Collection_4",
"num_collections": 1
}
```
#### Parameters

- `api_key` (required): Your Dowell API key.
- `db_name`: The name of the database.
- `coll_names`: The name of the collection for add in existing database.
- `num_collections`: as per usecase give num of collections

#### Response:

```json
{
"success": true,
"message": "Collection added successfully!",
"data": []
}
```


## Add Collection to Database

### Example of Add Collection to Database in Python

```python
import requests

url = "https://datacube.uxlivinglab.online/db_api/add_collection/"

data_to_add = {
"api_key": "your-dowell-api-key",
"db_name": "doWell",
"coll_names": "Collection_4",
"num_collections": 1
}

response = requests.post(url, json=data_to_add)
print(response.text)
```

### Response Codes

Expand Down
File renamed without changes.
File renamed without changes.
24 changes: 23 additions & 1 deletion dbdetails/script.py → api/script.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,10 @@
from rest_framework.status import (HTTP_200_OK, HTTP_400_BAD_REQUEST)

from .helper import run_backup, run_restore

import traceback
import requests
from rest_framework.response import Response
from rest_framework import status

class MongoDatabases:
def __init__(self):
Expand Down Expand Up @@ -398,3 +401,22 @@ def insert_cronjob_in_db(self, cron_name, created_at, _time, interval):
result = collection.insert_one(document)
print("Inserted document ID:", result.inserted_id)

#### dowell_time_api for date & time
def dowell_time():
try:
url = "https://100009.pythonanywhere.com/dowellclock/"
payload = json.dumps({
"timezone": "Asia/Kolkata",
})
headers = {
'Content-Type': 'application/json'
}

response = requests.post(url, headers=headers, data=payload)
if response.status_code == 200:
return json.loads(response.text)
else:
return "Error: Unable to fetch time for the given timezone."
except Exception as e:
traceback.print_exc()
return Response({"success": False, "message": str(e), "data": []}, status=status.HTTP_400_BAD_REQUEST)
Loading