Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simulators API #2006

Open
wants to merge 128 commits into
base: master
Choose a base branch
from
Open

Simulators API #2006

wants to merge 128 commits into from

Conversation

lpereiracgn
Copy link
Contributor

@lpereiracgn lpereiracgn commented Nov 1, 2024

Description

Support for Simulators API - Read only.

Supported Resources:

  • Simulator resource
  • Simulator Integration
  • Model and model revisions.
  • Routine and routine revisions.
  • Simulation Runs

PR for write support #2041

@lpereiracgn lpereiracgn mentioned this pull request Nov 3, 2024
4 tasks
from cognite.client.data_classes._base import CogniteFilter


class SimulatorIntegrationFilter(CogniteFilter):
Copy link
Contributor

@evertoncolling evertoncolling Dec 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should these filters have some docstrings? @haakonvt

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally yes, but I wouldn't bother. Instead, add a usage example to the docstrings where applicable so that people see how to use them 🚀

@evertoncolling
Copy link
Contributor

I see we are missing Simulation Data and Simulator Logs. We can take it in a follow up PR, but they could fell into the umbrella of read endpoints.

lpereiracgn and others added 15 commits December 10, 2024 17:47
@polomani
Copy link
Contributor

polomani commented Dec 11, 2024

we are missing Simulation Data and Simulator Logs.

I agree, this can go as a follow-up after we merge this one

Copy link
Contributor

@haakonvt haakonvt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial round of review (I haven't looked at most of data_classes/simulators...).

First and foremost I'd like to say that there is some solid work here!! Really looking forward to when this is released 💯

Some general notes:

  • This PR is just way too large for the review process to be pleasant for any of us. I would greatly encourage you to split this into several pieces, then we can make one release at the end.
  • Please hold off with adding tests until you have all the endpoints you need
  • We typically add __call__ and list at the same time then showcase the iterative approach in the list-docstring.

Retrieves a list of simulation runs that match the given criteria

Args:
limit (int): The maximum number of simulation runs to return.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I almost agree with @evertoncolling comment above 😄

Suggested change
limit (int): The maximum number of simulation runs to return.
limit (int | None): Maximum number of assets to return. Defaults to 25. Set to -1, float("inf") or None to return all items.

Comment on lines +48 to +52
List simulation runs:

>>> from cognite.client import CogniteClient
>>> client = CogniteClient()
>>> res = client.simulators.runs.list()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need one example showing where to import SimulationRunsFilter and how to use it (can include the sort in the same example)

Comment on lines +57 to +60
if isinstance(sort, CreatedTimeSort):
sort_by = [CreatedTimeSort.load(sort).dump()]
elif isinstance(sort, SimulationTimeSort):
sort_by = [SimulationTimeSort.load(sort).dump()]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be enough? Also see my other comment regarding these sort-classes.

Suggested change
if isinstance(sort, CreatedTimeSort):
sort_by = [CreatedTimeSort.load(sort).dump()]
elif isinstance(sort, SimulationTimeSort):
sort_by = [SimulationTimeSort.load(sort).dump()]
if isinstance(sort, CreatedTimeSort | SimulationTimeSort):
sort_by = [sort.dump(camel_case=True)]

Comment on lines +1530 to +1552
class PropertySort(CogniteSort):
def dump(self, camel_case: bool = True) -> dict[str, Any]:
dumped = super().dump(camel_case=camel_case)
dumped["property"] = self.property
return dumped


class CreatedTimeSort(PropertySort):
def __init__(
self,
property: Literal["createdTime"] = "createdTime",
order: Literal["asc", "desc"] = "asc",
):
super().__init__(property, order)


class SimulationTimeSort(PropertySort):
def __init__(
self,
property: Literal["simulationTime"] = "simulationTime",
order: Literal["asc", "desc"] = "asc",
):
super().__init__(property, order)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge these three classes into one. May I suggest we do sort similar to existing classes? E.g. check out SortableAssetProperty and AssetSort

return self._list(
method="POST",
limit=limit,
url_path="/simulators/runs/list",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed as this follows the API design guidelines

Suggested change
url_path="/simulators/runs/list",

Comment on lines +30 to +36
def truncated_uuid4():
return str(uuid.uuid4())[:8]


model_unique_external_id = f"{resource_names['simulator_model_external_id']}_{truncated_uuid4()}"
model_revision_unique_external_id = f"{resource_names['simulator_model_revision_external_id']}_{truncated_uuid4()}"
simulator_routine_unique_external_id = f"{resource_names['simulator_routine_external_id']}_{truncated_uuid4()}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't work with how we run tests (xdist package). These need to be put behind a fixture so that dependent tests agree on what the random variable is.

Also, you can just import random_string from utils?

Comment on lines +46 to +48
# check if file already exists
file = cognite_client.files.retrieve(external_id=seed_resource_names["simulator_model_file_external_id"])
if (file is None) or (file is False):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# check if file already exists
file = cognite_client.files.retrieve(external_id=seed_resource_names["simulator_model_file_external_id"])
if (file is None) or (file is False):
file = cognite_client.files.retrieve(external_id=seed_resource_names["simulator_model_file_external_id"])
if not file:

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should add a comment about what seed is (people like me would interpret it as something to make RNGs deterministic)

Comment on lines +64 to +67
cognite_client.post(
f"/api/v1/projects/{cognite_client.config.project}/simulators",
json={"items": [simulator]},
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't add the test before you've added the create endpoint to save yourself some refactoring

Comment on lines +61 to +62
simulators = cognite_client.simulators.list()
simulator_exists = len(list(filter(lambda x: x.external_id == simulator_external_id, simulators))) > 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there no retrieve you can use? If you only have list you can do:

simulator_exists = simulators.get(external_id=simulator_external_id)

which is a O(1) lookup instead of O(N)

@haakonvt
Copy link
Contributor

One more note, please don't push each commit one by one; we have quite an expensive test suite and we shouldn't run it willy-nilly 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants