Skip to content

Commit

Permalink
chore: rename property endpoints
Browse files Browse the repository at this point in the history
  • Loading branch information
timurbazhirov committed Feb 2, 2024
1 parent 56f9806 commit 650c81c
Show file tree
Hide file tree
Showing 7 changed files with 119 additions and 119 deletions.
2 changes: 1 addition & 1 deletion examples/job/get-file-from-job.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@
"from exabyte_api_client.endpoints.projects import ProjectEndpoints\n",
"from exabyte_api_client.endpoints.materials import MaterialEndpoints\n",
"from exabyte_api_client.endpoints.bank_workflows import BankWorkflowEndpoints\n",
"from exabyte_api_client.endpoints.raw_properties import RawPropertiesEndpoints"
"from exabyte_api_client.endpoints.properties import PropertiesEndpoints"
]
},
{
Expand Down
50 changes: 25 additions & 25 deletions examples/job/get-file-from-job.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,43 +6,43 @@
# </a>

# # Get-File-From-Job
#
#
# This example demonstrates how to use Mat3ra RESTful API to check for and acquire files from jobs which have been run. This example assumes that the user is already familiar with the [creation and submission of jobs](create_and_submit_jobs.ipynb) using our API.
#
#
# > <span style="color: orange">**IMPORTANT NOTE**</span>: In order to run this example in full, an active Mat3ra.com account is required. Alternatively, Readers may substitute the workflow ID below with another one (an equivalent one for VASP, for example) and adjust extraction of the results ("Viewing job files" section). RESTful API credentials shall be updated in [settings](../../utils/settings.json).
#
#
#
#
# ## Steps
#
#
# After working through this notebook, you will be able to:
#
#
# 1. Import [the structure of Si](https://materialsproject.org/materials/mp-149/) from Materials Project
# 2. Set up and run a single-point calculation using Quantum Espresso.
# 3. List files currently in the job's directory
# 4. Check metadata for every file (modification date, size, etc)
# 5. Access file contents directly and print them to console
# 6. Download files to your local machine
#
#
# ## Pre-requisites
#
#
# The explanation below assumes that the reader is familiar with the concepts used in Mat3ra platform and RESTful API. We outline these below and direct the reader to the original sources of information:
#
#
# - [Generating RESTful API authentication parameters](../system/get_authentication_params.ipynb)
# - [Importing materials from materials project](../material/import_materials_from_materialsproject.ipynb)
# - [Creating and submitting jobs](../job/create_and_submit_job.ipynb)

# # Complete Authorization Form and Initialize Settings
#
#
# This will also determine environment and set all environment variables. We determine if we are using Jupyter Notebooks or Google Colab to run this tutorial.
#
#
# If you are running this notebook from Google Colab, Colab takes ~1 min to execute the following cell.
#
#
# ACCOUNT_ID and AUTH_TOKEN - Authentication parameters needed for when making requests to [Mat3ra.com's API Endpoints](https://docs.mat3ra.com/rest-api/endpoints/).
#
#
# MATERIALS_PROJECT_API_KEY - Authentication parameter needed for when making requests to [Material Project's API](https://materialsproject.org/open)
#
#
# ORGANIZATION_ID - Authentication parameter needed for when working with collaborative accounts https://docs.mat3ra.com/collaboration/organizations/overview/
#
#
# > <span style="color: orange">**NOTE**</span>: If you are running this notebook from Jupyter, the variables ACCOUNT_ID, AUTH_TOKEN, MATERIALS_PROJECT_API_KEY, and ORGANIZATION_ID should be set in the file [settings.json](../../utils/settings.json) if you need to use these variables. To obtain API token parameters, please see the following link to the documentation explaining how to get them: https://docs.mat3ra.com/accounts/ui/preferences/api/

# In[ ]:
Expand Down Expand Up @@ -88,19 +88,19 @@
from exabyte_api_client.endpoints.projects import ProjectEndpoints
from exabyte_api_client.endpoints.materials import MaterialEndpoints
from exabyte_api_client.endpoints.bank_workflows import BankWorkflowEndpoints
from exabyte_api_client.endpoints.raw_properties import RawPropertiesEndpoints
from exabyte_api_client.endpoints.properties import PropertiesEndpoints


# ### Create and submit the job
#
#
# For this job, we'll use the workflow located [here](https://platform.mat3ra.com/analytics/workflows/84DAjE9YyTFndx6z3).
#
#
# This workflow is a single-point total energy calculation using Density-Functional Energy as-implemented in Quantum Espresso version 5.4.0.
#
#
# The PBE functional is used in conjunction with an ultrasoft pseudopotential and a planewave basis set.
#
#
# The material we will investigate is elemental [Silicon](https://materialsproject.org/materials/mp-149/), as-is from Materials Project.
#
#
# > <span style="color: orange">Note</span>: This cell uses our API to copy the unit cell of silicon from Materials Project into your account. It then copies a workflow to get the total energy of a system using Quantum Espresso to your account. Finally, a job is created using the Quantum Espresso workflow for the silicon unit cell, and the job is submitted to the cluster. For more information, please refer to our [run-simulation-and-extract-properties](./run-simulations-and-extract-properties.ipynb) notebook, located in this directory.

# In[ ]:
Expand Down Expand Up @@ -137,7 +137,7 @@

# ## Viewing job files
# ### Retreive a list of job files
#
#
# Here, we'll get a list of all files that belong to the job.

# In[ ]:
Expand All @@ -152,7 +152,7 @@

# ### Get metadata for the Output File
# The .out file is where Quantum Espresso shows its work and prints its results, so you most likely will want to view this files. Let's print out some of its metadata.
#
#
# You'll find that we get a lot of data describing the file and its providence. Brief explanations of each entry are:
# - Key - Path to the file on the cluster
# - size - Size of the file, in bytes.
Expand All @@ -173,7 +173,7 @@


# ### Display file contents to console
#
#
# The signedUrl gives us a place to access the file and download it. Let's read it into memory, and print out the last few lines of our job.

# In[ ]:
Expand All @@ -195,7 +195,7 @@


# ### Save the input file and output file to disk.
#
#
# Now that we've verified the job is done, let's go ahead and save it and its input to disk.

# In[ ]:
Expand Down
8 changes: 4 additions & 4 deletions examples/job/ml-train-model-predict-properties.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@
"from exabyte_api_client.endpoints.materials import MaterialEndpoints\n",
"from exabyte_api_client.endpoints.workflows import WorkflowEndpoints\n",
"from exabyte_api_client.endpoints.bank_workflows import BankWorkflowEndpoints\n",
"from exabyte_api_client.endpoints.raw_properties import RawPropertiesEndpoints"
"from exabyte_api_client.endpoints.properties import PropertiesEndpoints"
]
},
{
Expand Down Expand Up @@ -253,7 +253,7 @@
"material_endpoints = MaterialEndpoints(*ENDPOINT_ARGS)\n",
"workflow_endpoints = WorkflowEndpoints(*ENDPOINT_ARGS)\n",
"bank_workflow_endpoints = BankWorkflowEndpoints(*ENDPOINT_ARGS)\n",
"raw_property_endpoints = RawPropertiesEndpoints(*ENDPOINT_ARGS)"
"property_endpoints = PropertiesEndpoints(*ENDPOINT_ARGS)"
]
},
{
Expand Down Expand Up @@ -497,7 +497,7 @@
"outputs": [],
"source": [
"ml_predict_workflow = get_property_by_subworkflow_and_unit_indicies(\n",
" raw_property_endpoints, \"workflow:ml_predict\", job, 0, 4\n",
" property_endpoints, \"workflow:ml_predict\", job, 0, 4\n",
")[\"data\"]\n",
"ml_predict_workflow_id = ml_predict_workflow[\"_id\"]"
]
Expand Down Expand Up @@ -620,7 +620,7 @@
"outputs": [],
"source": [
"predicted_properties = get_property_by_subworkflow_and_unit_indicies(\n",
" raw_property_endpoints, \"predicted_properties\", job, 0, 3\n",
" property_endpoints, \"predicted_properties\", job, 0, 3\n",
")[\"data\"][\"values\"]"
]
},
Expand Down
68 changes: 34 additions & 34 deletions examples/job/ml-train-model-predict-properties.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,43 +6,43 @@
# </a>

# # Overview
#
#
# This example demonstrates how to use Mat3ra RESTful API to build a machine learning (ML) model for a set of materials called "train materials" and use the model to predict properties of another set called "target materials". The general approach can work for multiple properties, we use the Electronic Band Gap in this example.
#
#
#
#
#
#
# ## Steps
#
#
# We follow the below steps:
#
#
# - Import materials from [materials project](https://materialsproject.org/)
# - Calculate band gap for the "train materials"
# - Build ML Train model based on the "train materials"
# - Create and submit a job to predict band gap for the "target materials"
# - Extract band gap for "target materials"
# - Output the results as Pandas dataFrame
#
#
# ## Pre-requisites
#
#
# The explanation below assumes that the reader is familiar with the concepts used in Mat3ra platform and RESTful API. We outline these below and direct the reader to the original sources of information:
#
#
# - [Generating RESTful API authentication parameters](../system/get_authentication_params.ipynb)
# - [Importing materials from materials project](../material/import_materials_from_materialsproject.ipynb)
# - [Creating and submitting jobs](./create_and_submit_job.ipynb)
# - [Running DFT calculations](./run-simulations-and-extract-properties.ipynb)

# # Complete Authorization Form and Initialize Settings
#
#
# This will also determine environment and set all environment variables. We determine if we are using Jupyter Notebooks or Google Colab to run this tutorial.
#
#
# If you are running this notebook from Google Colab, Colab takes ~1 min to execute the following cell.
#
#
# ACCOUNT_ID and AUTH_TOKEN - Authentication parameters needed for when making requests to [Mat3ra.com's API Endpoints](https://docs.mat3ra.com/rest-api/endpoints/).
#
#
# MATERIALS_PROJECT_API_KEY - Authentication parameter needed for when making requests to [Material Project's API](https://materialsproject.org/open)
#
#
# ORGANIZATION_ID - Authentication parameter needed for when working with collaborative accounts https://docs.mat3ra.com/collaboration/organizations/overview/
#
#
# > <span style="color: orange">**NOTE**</span>: If you are running this notebook from Jupyter, the variables ACCOUNT_ID, AUTH_TOKEN, MATERIALS_PROJECT_API_KEY, and ORGANIZATION_ID should be set in the file [settings.json](../../utils/settings.json) if you need to use these variables. To obtain API token parameters, please see the following link to the documentation explaining how to get them: https://docs.mat3ra.com/accounts/ui/preferences/api/

# In[ ]:
Expand Down Expand Up @@ -96,13 +96,13 @@
from exabyte_api_client.endpoints.materials import MaterialEndpoints
from exabyte_api_client.endpoints.workflows import WorkflowEndpoints
from exabyte_api_client.endpoints.bank_workflows import BankWorkflowEndpoints
from exabyte_api_client.endpoints.raw_properties import RawPropertiesEndpoints
from exabyte_api_client.endpoints.properties import PropertiesEndpoints


# #### Materials
#
#
# Set parameters for the materials to be imported:
#
#
# - **TRAIN_MATERIALS_PROJECT_IDS**: a list of material IDs to train ML model based on
# - **TARGET_MATERIALS_PROJECT_IDS**: a list of material IDs to predict the property for

Expand All @@ -114,9 +114,9 @@


# #### Jobs
#
#
# Set parameters for the jobs to be ran for the imported materials:
#
#
# - **JOB_NAME_PREFIX**: prefix to be used for the job name with "{JOB_NAME_PREFIX} {FORMULA}" convention (e.g. "Job Name Prefix - SiGe")

# In[ ]:
Expand All @@ -126,9 +126,9 @@


# #### Compute
#
#
# Setup compute parameters. See [this](https://docs.mat3ra.com/infrastructure/compute-settings/ui) for more information about compute parameters.
#
#
# - **NODES**: Number of nodes. Defaults to 1.
# - **PPN**: Number of MPI processes per each node, Defaults to 1.
# - **QUEUE**: The name of queue to submit the jobs into. Defaults to D.
Expand All @@ -155,7 +155,7 @@
material_endpoints = MaterialEndpoints(*ENDPOINT_ARGS)
workflow_endpoints = WorkflowEndpoints(*ENDPOINT_ARGS)
bank_workflow_endpoints = BankWorkflowEndpoints(*ENDPOINT_ARGS)
raw_property_endpoints = RawPropertiesEndpoints(*ENDPOINT_ARGS)
property_endpoints = PropertiesEndpoints(*ENDPOINT_ARGS)


# Retrieve the owner and project IDs as they are needed by the endpoints. The default material is used to extract the owner ID. One can extract the owner ID from any other account's [entities](https://docs.mat3ra.com/entities-general/overview/).
Expand All @@ -168,7 +168,7 @@


# ### Create workflows
#
#
# Copy "ML: Train Model" and "Band Gap" bank workflows to the account's workflows. We use exabyte bank workflows which are identified by "systemName" field. The below can be adjusted to get the bank workflows by ID.

# In[ ]:
Expand All @@ -179,7 +179,7 @@


# ### Import materials
#
#
# Import materials from materials project.

# In[ ]:
Expand All @@ -194,7 +194,7 @@


# ### Calculate Properties for "train materials"
#
#
# Create jobs for the "train materials".

# In[ ]:
Expand Down Expand Up @@ -225,7 +225,7 @@


# ### Build ML Train model
#
#
# Create ML Train job for the train materials.

# In[ ]:
Expand Down Expand Up @@ -254,14 +254,14 @@


# ### Extract ML model as workflow
#
#
# The resulting trained model is extracted from the last unit (train with index 4) of the first job's subworkflow (ML: Train Model with index 0) and is further referred to as "ML predict workflow".

# In[ ]:


ml_predict_workflow = get_property_by_subworkflow_and_unit_indicies(
raw_property_endpoints, "workflow:ml_predict", job, 0, 4
property_endpoints, "workflow:ml_predict", job, 0, 4
)["data"]
ml_predict_workflow_id = ml_predict_workflow["_id"]

Expand All @@ -275,7 +275,7 @@


# ### Predict property using the model
#
#
# Create ML Predict job for the predict materials.

# In[ ]:
Expand Down Expand Up @@ -304,19 +304,19 @@


# ### Extract predicted properties
#
#
# Predicted properties are extracted from the last unit (score with index 3) of the first job's subworkflow (ml_predict_subworkflow with index 0).

# In[ ]:


predicted_properties = get_property_by_subworkflow_and_unit_indicies(
raw_property_endpoints, "predicted_properties", job, 0, 3
property_endpoints, "predicted_properties", job, 0, 3
)["data"]["values"]


# ### Flatten results
#
#
# The below for-loop iterates over the results and flatten them to form the final Pandas dataFrame.

# In[ ]:
Expand All @@ -334,7 +334,7 @@


# ### Ouput results
#
#
# Create and print the final table as Pandas dataFrame.

# In[ ]:
Expand Down
Loading

0 comments on commit 650c81c

Please sign in to comment.