Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix Code Mode] Directory Processing Stats Files #104

Open
Tracked by #103
Yiannis128 opened this issue Feb 3, 2024 · 0 comments
Open
Tracked by #103

[Fix Code Mode] Directory Processing Stats Files #104

Yiannis128 opened this issue Feb 3, 2024 · 0 comments
Labels
enhancement New feature or request
Milestone

Comments

@Yiannis128
Copy link
Collaborator

Yiannis128 commented Feb 3, 2024

When processing entire directories, each file should be accompanied by a file of the same name but with the added suffix esbmc-ai.stats.txt, this should be created optionally (from config) to display the following stats in JSON:

{
    "file_name": "file.c",
    "max_attempts": 5,
    "llm": {
        "temperature": 1.0,
        "name": "gpt-3.5-turbo",
        "max_tokens": 16000,
    },
    "attempts": [
        {
            "attempt": 0,
            "esbmc_time": 100.20,
            "llm_time": 5.3,
            "result": "VERIFICATION FAILED",
            "tokens": 3041
        },
        {
            "attempt": 1,
            "esbmc_time": 120.56,
            "llm_time": 8.87,
            "result": "VERIFICATION SUCCESSFUL"
            "tokens": 8203
        }
    ]
}

Requirements

@Yiannis128 Yiannis128 added the enhancement New feature or request label Feb 3, 2024
@Yiannis128 Yiannis128 changed the title [Fix Code Mode] [Fix Code Mode] Directory Processing Stats Files Feb 3, 2024
@Yiannis128 Yiannis128 added this to the Version 0.5.0 milestone Feb 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant