Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: add plugin file date/hash to plugin load messages & logs for auditing #988

Open
leondz opened this issue Nov 11, 2024 · 0 comments
Assignees
Labels
architecture Architectural upgrades

Comments

@leondz
Copy link
Collaborator

leondz commented Nov 11, 2024

goal

output run summaries, including scores, garak versioning info, dates, a supplied asset ID, and analysis, as a record for asset audit trails

In all cases, we want to get the answer quickly, with minimum forensic work load.

suggestion

  • at garak load,
    • log gh current hash if known
      • in garak.log
      • in report.jsonl
  • at plugin load,
    • log file date as info (use same date as the cache uses)

Scenario 1:

We discover a few months post-hoc that one of our dev runs was really interesting, and we want to know if it was done using using stock or adjusted probes.

Scenario 2:

We find an old .post1-versioned log and want to know which version of a detector was used.

Scenario 3:

An external collaborator reports a bug and includes the last screenful (say 80 rows) of the log. We want to work out if they're using a proper version of garak or their own tweaked one.

@leondz leondz added the architecture Architectural upgrades label Nov 11, 2024
@leondz leondz added this to the 24.12 milestone Nov 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
architecture Architectural upgrades
Projects
None yet
Development

No branches or pull requests

2 participants