Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Upload dataset from il sdk #1116

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

MerlinKallenbornAA
Copy link
Collaborator

@MerlinKallenbornAA MerlinKallenbornAA commented Oct 30, 2024

Description

PHS-842

Before Merging

  • Review the code changes
    • Unused print / comments / TODOs
    • Missing docstrings for functions that should have them
    • Consistent variable names
    • ...
  • Update changelog.md if necessary
  • Commit messages should contain a semantic label and the ticket number
    • Consider squashing if this is not the case

@MerlinKallenbornAA MerlinKallenbornAA changed the title feat: 842 upload dataset from il sdk feat: Upload dataset from il sdk Oct 30, 2024
Dataset,
Example,
ExpectedOutput,
)


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a block comment somewhere that this is not exported to connectors.XXX. Maybe it would even be better to not export the StudioDataRepository to eval but rather export this to connectors

Comment on lines 218 to 219
dataset: :Dataset: to be uploaded
examples: :Examples: of Dataset
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where did you get this docstring? i feel like we always did this via ... instead of :...:

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair, had it mixed up. Corrected it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with backticks i mean

src/intelligence_layer/connectors/studio/studio.py Outdated Show resolved Hide resolved
src/intelligence_layer/connectors/studio/studio.py Outdated Show resolved Hide resolved
Comment on lines 266 to 271
_status_code_to_exception: ClassVar[dict[int, type[DataError]]] = {
HTTPStatus.SERVICE_UNAVAILABLE: DataExternalServiceUnavailable,
HTTPStatus.NOT_FOUND: DataResourceNotFound,
HTTPStatus.UNPROCESSABLE_ENTITY: DataInvalidInput,
HTTPStatus.FORBIDDEN: DataForbiddenError,
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite see the reason to make these our own errors

Comment on lines +63 to +68
@fixture
def examples() -> Sequence[Example[str, str]]:
return [
Example(input="input_str", expected_output="output_str"),
Example(input="input_str2", expected_output="output_str2"),
]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have you checked if we have this fixture already? otherwise not a big deal to redefine it if it does not fit

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but in the "wrong" conftest.

)
result = studio_client.submit_dataset(dataset=dataset, examples=examples)

assert result is not None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could check that the dataset we get back has the metadata / labels etc that we expect

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added an "has been called with" assertion type

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants