Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tools: Block Generator - allow exporting to files #5714

Merged
merged 11 commits into from
Sep 21, 2023

Conversation

tzaffi
Copy link
Contributor

@tzaffi tzaffi commented Aug 30, 2023

Block Generator in File Exporting Mode + README + Makefile + Remove Python Runner

Functional changes

  • tools/block-generator/generator/server.go - adding some calling context in maybeWriteError()
  • allow exporting blocks into conduit's data directory. This is useful for generating Conduit fixtures. In particular:
    • tools/block-generator/runner/run.go - handle file exporting with a new conduit.yml template
    • tools/block-generator/runner/runner.go - don't enforce the need for a connection string in the case of a file exporter

Non functional changes:

  • New Makefile targets:
    • make pg-up - standalone docker run
    • make pg-query - query postgres (handy since the DB credentials are variables of the Makefile)
    • make run-file-exporter - runner command tailored for file system exporting
    • make benchmark-blocks-export - encapsulates the previous to create blocks with a top-level report directory named according to the scenario
  • Update the README with an explanation of the statistical distribution of block generator scenarios
  • Slight rewording of the runner and daemon's help outputs
  • Remove the python runner - all its previous functionality can be achieved with make targets

@tzaffi tzaffi self-assigned this Aug 30, 2023
@tzaffi tzaffi changed the title Tools blockgen file export tools: Block Generator - allow exporting to files Aug 30, 2023
@codecov
Copy link

codecov bot commented Aug 30, 2023

Codecov Report

Merging #5714 (316ff04) into master (02b99bb) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #5714   +/-   ##
=======================================
  Coverage   55.54%   55.54%           
=======================================
  Files         476      476           
  Lines       66853    66853           
=======================================
  Hits        37131    37131           
- Misses      27193    27201    +8     
+ Partials     2529     2521    -8     

see 12 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@tzaffi tzaffi marked this pull request as ready for review August 30, 2023 22:00
@tzaffi tzaffi requested review from a team, winder, Eric-Warehime, shiqizng and algochoi and removed request for a team August 30, 2023 22:01
tools/block-generator/README.md Outdated Show resolved Hide resolved
tools/block-generator/README.md Outdated Show resolved Hide resolved
tools/block-generator/README.md Outdated Show resolved Hide resolved
tools/block-generator/README.md Outdated Show resolved Hide resolved
tools/block-generator/README.md Show resolved Hide resolved
Copy link
Contributor

@winder winder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good overall.

tools/block-generator/runner/run.go Show resolved Hide resolved
tools/block-generator/runner/run.go Outdated Show resolved Hide resolved
tools/block-generator/runner/run.go Show resolved Hide resolved
tools/block-generator/runner/runner.go Show resolved Hide resolved
tools/block-generator/Makefile Outdated Show resolved Hide resolved
tools/block-generator/Makefile Outdated Show resolved Hide resolved
tools/block-generator/README.md Outdated Show resolved Hide resolved
tools/block-generator/README.md Show resolved Hide resolved
tools/block-generator/README.md Show resolved Hide resolved
tools/block-generator/runner/runner.go Show resolved Hide resolved
shiqizng
shiqizng previously approved these changes Sep 5, 2023
@winder winder merged commit 2f09710 into algorand:master Sep 21, 2023
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants