Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simpler evaluations — part 1.1 #489

Merged
merged 1 commit into from
Oct 28, 2024
Merged

Simpler evaluations — part 1.1 #489

merged 1 commit into from
Oct 28, 2024

Conversation

csansoon
Copy link
Contributor

@csansoon csansoon commented Oct 22, 2024

Simpler evaluations schema — 🚂 lil train — 🚉 station #​1 part 1

Create the configuration column in evaluationMetadataLlmAsJudgeLegacy, and adapt the code to use configuration from either (evaluation.configuration ?? evaluation.metadata.configuration).

@csansoon csansoon added 🚧 wip Work in progress 🚂 lil train Implementation divided in several feature-specific PRs labels Oct 22, 2024
@csansoon csansoon force-pushed the easier-evaluations branch 9 times, most recently from e9184c9 to 2b14a32 Compare October 24, 2024 10:03
@csansoon csansoon marked this pull request as ready for review October 24, 2024 10:19
@csansoon csansoon force-pushed the easier-evaluations branch 15 times, most recently from 77720a5 to 747fc2f Compare October 28, 2024 09:16
@csansoon csansoon removed the 🚧 wip Work in progress label Oct 28, 2024
geclos
geclos previously approved these changes Oct 28, 2024
andresgutgon
andresgutgon previously approved these changes Oct 28, 2024
@csansoon csansoon force-pushed the easier-evaluations branch 4 times, most recently from af416e8 to 9efe044 Compare October 28, 2024 14:05
@csansoon csansoon changed the title Simpler evaluations Simpler evaluations — part 1.1 Oct 28, 2024
@csansoon csansoon force-pushed the easier-evaluations branch 4 times, most recently from 61d3897 to 4ba921f Compare October 28, 2024 14:19
@csansoon csansoon force-pushed the easier-evaluations branch 2 times, most recently from b611023 to e237713 Compare October 28, 2024 14:26
@csansoon csansoon merged commit 2e29dd7 into main Oct 28, 2024
3 checks passed
@csansoon csansoon deleted the easier-evaluations branch October 28, 2024 14:49
@github-actions github-actions bot locked and limited conversation to collaborators Oct 28, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
🚂 lil train Implementation divided in several feature-specific PRs
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Evaluations – Remove conflicts in evaluation objective between configuration and prompt
3 participants