-
Notifications
You must be signed in to change notification settings - Fork 66
feat: add score document support in csv #696
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, I think the refactoring makes this much easier to read!
I think CSVHandler is a bit vague though, maybe CSVParser or CSVReader?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added some comments, if we really introduce the CSVContext class, we also need to update the documentation.
Ok probably, there is nothing in the documentation about building dataset from CSV, because we only apply it automatically if one provides a file path in the csv. |
documentation will be added in a seperate PR: #688 (comment) |
Co-authored-by: George Mastrapas <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@@ -70,7 +70,7 @@ def list_experiments(self, page: int = 1, size: int = 50) -> Dict[str, Any]: | |||
..note:: The maximum number for `size` per page is 100. | |||
""" | |||
params = {'page': page, 'size': size} | |||
url = self._construct_url(self._base_url, API_VERSION, EXPERIMENTS) | |||
url = self._construct_url(self._base_url, API_VERSION, EXPERIMENTS) + '/' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this be don in the construct_url
function?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
later we will investigate why this is happening
This PR allows user create a CSV file contains three columns, col1 and col2 are content, and col3 indicates the similarity between col1 and col2. Besides, I refactored the
build_finetuning_dataset
function.