go-algorand 3.16.0-beta Release PR

[Algo-devops-service]

IMPORTANT This release requires a protocol upgrade. This release contains a consensus protocol upgrade, which implements the following spec: https://github.com/algorandfoundation/specs/tree/abd3d4823c6f77349fc04c3af7b1e99fe4df699f

Overview

This update includes a drop in round time of ~0.4 seconds, a robustness improvement to State Proofs, and several developer-oriented features in the AVM, simulate, and devmode.

⚠️ In order to avoid opcode ambiguity, access to assets or applications with an ID less than 256 within a contract will fail immediately. For new networks, asset and application IDs will now start at 1000 to avoid any potential issues due to this change.

What's New

• 🚀 Round times are reduced by 400 milliseconds. See here for details.
• 👥 Group resource sharing for app calls: access to resources (assets, apps, accounts, boxes) during smart contract evaluation is shared across the transaction group. See here for details.
• 🕦 Devmode timestamp control: when running a network in devmode, the block timestamp can now be manipulated with a new API. See here for details.
• 🔮 More logging and opcode budget in simulate: when calling the simulate endpoint, there are now options to let the algod evaluate the transaction group with much higher limits on logging and on opcode budget. See here for details.
• 📒 Transaction group updates API: a new algod API that returns the ledger updates caused by individual transaction groups. See here for details.
• ➕ Readiness Endpoint: algod has a new supplementary /ready endpoint to benefit any K8's deployment as a readiness probe. The endpoint will return a 200 response code once algod has caught up to the latest state.
• 📦 Algod Docker Container: algod container usage just got easier! The container now supports node profiles to simplify running common configurations in place of mounting a custom config.json file. There are also changes to make mounting a data directory more consistent. See DockerHub for more details.

Changelog

Protocol Upgrade

This release contains a double protocol upgrade (Consensus v37 and v38). No action is needed from node runners but is called out for transparency. Consensus v37 is a technical upgrade released in unison with Consensus v38. v37 is needed to allow nodes to build up a necessary state to support State Proof related options in consensus v38.

New Features

• AVM: Share resource arrays across transactions (#5035)
• Algod: Allow timestamp offsets in dev mode (#5296)
• Algod: Simulation run with extra budget per transaction group (#5354)
• Algod: Simulation run with increased limits on logs (#5247)
• API: Txn Group Delta Apis (#5350)
• Evaltracer: Txn group deltas Tracer (#5297)
• Algod: New health endpoint (k8s /ready endpoint) (#4844)
• Docker: add profile support and improved endpoint access. (#5323)
• Algod: support state proofs recoverability (#4803)

Enhancements

• API: Don't return a top level array from algod (#5404)
• API: Limit request body size to 10MB (#5246)
• AVM: Modify StackType to provide additional information (#5130)
• AVM: No low resources (#5328)
• AVM: Show opcode context for logicsigs (not just apps) (#5336)
• Algocfg(profile): Add file name to config file conflict message. (#5262)
• Algocfg(profile): improve invalid profile error message. (#5324)
• Algod: Add EvalTracer tests for StateDeltas (#5368)
• Algod: Better Box Reference Error Message (#5325)
• Algod: Minor refactoring REST client submitForm from go-sdk PR Add support for travis ARM64 integration tests #335 (#5253)
• Algod: Modify simulate endpoint request type (#5292)
• Algod: Move delta API tags (#5396)
• Algod: Remove unused 404 resp in simulate endpoint (#5211)
• Algod: Use future consensus version to test with no low resources (#5362)
• Algod: search phonebook in data directory in addition to bindir (#5235)
• Catchpoints: Small tweaks, mostly to comments (#5195)
• Chore: Focus CONTRIBUTING.md on code contributions. (#5294)
• Ci: set reviewdog golangci-lint go version (#5196)
• Consensus: Introduce versions v37 and v38 (#5397)
• Dev Tools: Integrate Logfile Navigator (lnav) (#5000)
• Devops: Add CODEOWNERS to restrict workflow edits (#5353)
• Devops: Update CODEOWNERS to only refer to the devops group (#5389)
• Docker: Updated user and data dir handling. (#5276)
• Docker: disable rewards in devmode template (#5270)
• Docker: option to override topology file and peer address. (#5209)
• Docs: Fix bug label (#5322)
• Docs: add titles of subsystems, minor md fixes (#5279)
• Docs: increase heading levels, thus only one h1 is used (#5291)
• Enhancement: Only look back to FirstValid for pending transactions that were found in the pool (#5295)
• Eval: Add block hooks to eval tracer (#5303)
• Fix: Tech Debt (wsNetwork "auto" and OnlineAccountsDelete comment) (#5207)
• Follow Mode: set sync round after fast catchup. (#5349)
• Follower: add round information for missing deltas request (#5277)
• Goal: Introduce new command for simulate (#5213)
• Goal: Make goal state schema optional (#5356)
• Goal: rename --allow-extra-opcode-budget to --allow-more-opcode-budget (#5407)
• Goal: user defined scheme to connect to remote host (#4922)
• Ledger: Check MaxAcctLookback in tracker. (#5300)
• Ledger: Exclude stake at R-320 that is expired by R (#5403)
• Ledger: convert FC unmarshalled nil value to empty byte slice on DB write KVs (#5225)
• Ledger: rename "internal" to "eval" (#5236)
• Lint: enable nilerr linter and fix errors (#5361)
• Netdeploy: Copy ledger directory for kv tracker database (#5392)
• Network: enforce maximum header size on outgoing ws conns (#5268)
• REST: add the round number to algod box endpoint response (#5340)
• Scripts: enhance upload_config.sh (#5260)
• Simulate: Add opcode costs and budgets into response (#5221)
• Simulate: Make optional signatures an opt-in feature (#5335)
• Tools: Cross Repo Types Comparison Tool (#5304)
• Tools: Don't use strings.Title in chopper (#5239)
• Tools: Run the Cross Repo Type Checker in C.I. (#5326)
• Tools: add block-generator into published build (#5351)
• Tools: allow dumpblocks work with wal-enabled dbs (#5380)
• Tools: block generator (#5245)
• Tools: block generator data reporting (#5339)
• Tools: new endpoints for block generator (#5257)
• Tools: run block generator using a preloaded db (#5384)
• Tools: search/replace indexer/conduit in block-generator (#5321)
• Tools: update block-generator to use conduit binary (#5306)

Bugfixes

• API: Fix vote status computation. (#5228)
• AVM: Avoid panics in disassembly when branch instructions are short (#5252)
• AVM: Avoid panics while type checking bad immediates (#5271)
• Assembler: Error if extra args are present in pragmas (#5400)
• Bug Fix: block generator test (#5410)
• Bugfix: reduce flakiness in follower node test (#5355)
• Chore: Rename ModStateProofNextRound to StateProofNext. (#5265)
• Dependency: Update codec version to 1.1.9 (#5395)
• Devops: fix codecov uploading (#5345)
• Docker: Fix ADMIN_TOKEN and add missing variables to echo statements. (#5357)
• Erl: Support zero-size reservations (#5192)
• Fix: Correct GeneratedAccountsMnemonics Check (#5274)
• Follower: Set initial sync round to latest tracker committed round. (#5251)
• Goal: Better formatting in goal clerk simulate (#5259)
• Ledger: fix commit tasks enqueueing (#5214)
• Ledger: fix error shadowing in accountsNewRound (#5266)
• Ledger: report catchpoint writing only when it actually started (#5413)
• Logging: fix logging to line. (#5359)
• Tests: Change dd argument to use a bytes value (#5263)
• Tests: Change truncate to dd for mac amd tests (#5256)
• Tests: Fix State-Proofs ledger tests (#5379)
• Tests: Fix devmode test (#5334)
• Tests: Fix error messages in ledger tests (#5363)
• Tests: Improve flaky test from non-deterministic order string (#5333)
• Tests: fix consensus version inconsistency in newTestLedger (#5342)
• Tests: remove follower node flaky TestSyncRoundWithRemake (#5415)
• Txhandler: make dedup working set independent from ERL (#5200)
• Txngroup-deltas: Fix pointer bug copying deltas for txngroups (#5375)

Additional Resources

• Algorand Forum
• Developer Documentation

[onetechnical]

v3.16.0-beta is not released/tagged yet, so this will be merged with the previous.

[onetechnical]

For reference, these are the additions:

058a9d58a Goal: rename `--allow-extra-opcode-budget` to `--allow-more-opcode-budget` (#5407)
3fb696411 ledger: report catchpoint writing only when it actually started (#5413)
9d2bfef24 docker: Fix ADMIN_TOKEN and add missing variables to echo statements. (#5357)
ce3802ecf test: remove follower node flaky TestSyncRoundWithRemake (#5415)
1abbd50ae bug fix: block generator test  (#5410)
a87307085 tools: run block generator using a preloaded db (#5384)

[codecov]

Codecov Report

Merging #5417 (cc2f223) into rel/beta (297a4a1) will decrease coverage by 4.37%.
The diff coverage is 44.00%.

@@             Coverage Diff              @@
##           rel/beta    #5417      +/-   ##
============================================
- Coverage     56.07%   51.70%   -4.37%     
============================================
  Files           456      452       -4     
  Lines         63877    63853      -24     
============================================
- Hits          35816    33015    -2801     
- Misses        25604    28291    +2687     
- Partials       2457     2547      +90     

Impacted Files	Coverage Δ
ledger/acctonline.go	78.62% <0.00%> (-0.85%)	⬇️
ledger/acctupdates.go	70.02% <0.00%> (-1.37%)	⬇️
ledger/bulletin.go	93.47% <0.00%> (-2.18%)	⬇️
ledger/ledger.go	68.51% <0.00%> (-2.52%)	⬇️
ledger/metrics.go	96.77% <0.00%> (-3.23%)	⬇️
ledger/notifier.go	86.27% <0.00%> (-1.97%)	⬇️
ledger/spverificationtracker.go	95.65% <0.00%> (-0.73%)	⬇️
ledger/txtail.go	74.30% <0.00%> (-3.92%)	⬇️
tools/block-generator/generator/server.go	26.74% <0.00%> (-1.31%)	⬇️
ledger/tracker.go	71.42% <25.00%> (-5.00%)	⬇️
... and 3 more

... and 144 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more