Releases: neondatabase/neon
Releases · neondatabase/neon
release-6933
What's Changed
- feat(pageserver): do space check before gc-compaction by @skyzh in #9250
- storage_controller: delete stale shards when deleting tenant by @erikgrinaker in #9333
- safekeeper: flush WAL on compute disconnect by @erikgrinaker in #9436
- Add logging of installed_extensions by @lubennikovaav in #9438
- refactor(pageserver): remove aux v1 code path by @skyzh in #9424
- pageserver: detect & warn on loading an old index which is probably the result of a bad generation by @jcsp in #9383
- Update Rust to 1.82.0 and mold to 2.34.0 by @arpad-m in #9445
- update rustls by @conradludgate in #9396
- 2024 oct new clippy lints by @conradludgate in #9448
- compute_ctl: Add endpoint that allows setting role grants by @devjv in #9395
- storcon: fix case where we might fail to send compute notifications after two opposite migrations by @jcsp in #9435
- Increase shared_buffers in test_subscriber_synchronous_commit. by @arssher in #9427
- Update ruff to much newer version by @cloneable in #9433
- compute_ctl: Add endpoint that allows extensions to be installed by @devjv in #9344
- walproposer: immediately exit if sync-safekeepers collected 0/0. by @arssher in #9442
- pageserver: disconnect when asking client to reconnect by @VladLazar in #9390
- [local_proxy]: install pg_session_jwt extension on demand by @conradludgate in #9370
- storcon: handle ongoing deletions gracefully by @VladLazar in #9449
- Take the collector name as argument when generating sql_exporter configs by @tristan957 in #9447
- Accept basebackup --gzip requests by @arpad-m in #9456
- bump pg-session-jwt version by @conradludgate in #9455
- Compute release 2024-10-20 by @ololobus in #9459
Full Changelog: release-6911...release-6933
release-6911
What's Changed
- proxy: split out the console-redirect backend flow by @conradludgate in #9270
- Ignore pg_dynshmem fiel when comparing directories by @knizhnik in #9374
- CI(build-build-tools): fix unexpected cancellations by @bayandin in #9357
- Add test for fixed storage broker issue by @arpad-m in #9311
- tests: make size comparisons more lenient by @VladLazar in #9388
- Also consider offloaded timelines for obtaining retain_lsn by @arpad-m in #9308
- Add more timing histogram and gauge metrics to the Neon extension by @MMeent in #9116
- Switch CI, Storage and Proxy to Debian 12 (Bookworm) by @bayandin in #9170
- pageserver: stabilize & refine controller scale test by @jcsp in #8971
- Shut down timelines during offload and add offload tests by @arpad-m in #9289
- [local_proxy] update api for pg_session_jwt by @conradludgate in #9359
- proxy: Migrate jwt module away from anyhow by @cloneable in #9361
- Remove redundant code by @knizhnik in #9373
- Generate sql_exporter config files with Jsonnet by @tristan957 in #9368
- fix(pageserver): use a buffer for basebackup; add aux basebackup metrics log by @skyzh in #9401
- Add newly added metrics from #9116 to exports by @MMeent in #9402
- Fix jsonnet_files wildcard by @tristan957 in #9403
- Add jsonnetfmt targets by @tristan957 in #9404
- don't trigger workflows in two repos by @fcdm in #9340
- pageserver: improve handling of archival_config calls during Timeline shutdown by @jcsp in #9415
- proxy: format imports with nightly rustfmt by @cloneable in #9414
- storcon: use the same trace fields for reconciler and results by @VladLazar in #9410
- tests: stabilize test_timelines_parallel_endpoints by @jcsp in #9413
- Remove allow(clippy::unknown_lints) by @arpad-m in #9416
- Add support of extensions for v17 (part 2) by @lubennikovaav in #9389
- Activate timelines during unoffload by @arpad-m in #9399
- refactor(pageserver): upgrade remote_storage to use hyper1 by @skyzh in #9405
- proxy: merge AuthError and AuthErrorImpl by @cloneable in #9418
- Fix beta clippy warnings by @arpad-m in #9419
- Fix typo in sql_exporter generator by @tristan957 in #9420
- Fix some sql_exporter metrics for PG 17 by @tristan957 in #9406
- readme: fix build instructions for debian 12 by @problame in #9371
- Check access_count in lfc_evict by @knizhnik in #9407
- storage_scrubber: fixes to garbage commands by @jcsp in #9409
- proxy: move the connection pools to separate file by @awarus in #9398
- Add config variable for timeline offloading by @arpad-m in #9421
- Add support of extensions for v17 (part 3) by @lubennikovaav in #9430
- test_runner: Update the README.md to build neon with 'testing' by @awarus in #9437
- Releases/2024 10 17 compute kq only by @lubennikovaav in #9451
Full Changelog: release-6872...release-6911
release-6872
What's Changed
- Fix update of statistic for LFC/prefetch by @knizhnik in #9272
- proxy: rename console -> control_plane, rename web -> console_redirect by @conradludgate in #9266
- proxy: Move module base files into module directory by @cloneable in #9297
- Make sure BufferTag padding bytes are cleared in hash keys by @hlinnaka in #9292
- feat(compute_tools): Create JWKS Postgres roles without attributes by @devjv in #9031
- Replace BUFFERTAGS_EQUAL compatibility macro with new-style function by @hlinnaka in #9294
- Improve logging on changes in a compute's status by @tristan957 in #9265
- storage_broker: update hyper and tonic again by @arpad-m in #9299
- vm-monitor: Ignore LFC in postgres cgroup memory threshold by @sharnoff in #8668
- pgxn: Fix compiler warnings, and enable them by @hlinnaka in #9290
- Remove non-existent entries from .dockerignore by @hlinnaka in #9209
- Remove get_self_dir() by @tristan957 in #9306
- Fix compiler warnings on macOS by @bayandin in #9319
- storcon: apply all node status changes before handling transitions by @VladLazar in #9281
- Enable all pyupgrade checks in ruff by @tristan957 in #9323
- Add timeline offload mechanism by @arpad-m in #8907
- tests: Remove redundant log lines when starting an endpoint by @hlinnaka in #9316
- remote_storage: add
DownloadOpts::byte_(start|end)
by @erikgrinaker in #9293 - tests: Reduce noise from logging renamed files by @hlinnaka in #9315
- proxy: Unalias hyper1 and replace one use of hyper0 in test by @cloneable in #9324
- storcon: make observed state updates more granular by @VladLazar in #9276
- test_runner: fix path to tpc-h queries by @bayandin in #9327
- safekeeper: add evicted_timelines gauge. by @arssher in #9318
- Add /installed_extensions endpoint to collect statistics about extension usage. by @lubennikovaav in #8917
- pageserver: add direct io config to virtual file by @yliang412 in #9214
- tests: Use ThreadedMotoServer python class to launch mock S3 server by @hlinnaka in #9313
- tests: Remove redundant log lines when stopping storage nodes by @hlinnaka in #9317
- added workflow Report Workflow Stats by @fedordikarev in #9330
- Add partial support of extensions for v17: by @lubennikovaav in #9322
- local_proxy: integrate with pg_session_jwt extension by @conradludgate in #9086
- Move PgBenchInitResult.EXTRACTORS to a private module constant by @tristan957 in #9334
- Improve some typing in test_runner by @tristan957 in #9303
- add local-proxy suffix to wake-compute requests, respect the returned port by @conradludgate in #9298
- storage controller: use 'infra' JWT scope for node registration by @jcsp in #9343
- CODEOWNERS: assign
storage_scrubber
to storage by @erikgrinaker in #9346 - storage_scrubber: fix faulty assertion when no timelines by @erikgrinaker in #9345
- Move path-related fixtures into their own module by @tristan957 in #9304
- tests: allow a log warning in test_cli_start_stop_multi by @jcsp in #9320
- Enable cargo caching in check-codestyle-rust by @tristan957 in #9307
- Add compute_config_dir fixture by @tristan957 in #9352
- Use valid type hints for Python 3.9 by @tristan957 in #9349
- Introduce "quota" ErrorKind by @awarus in #9300
- tests: stabilize test_storage_controller_heartbeats by @jcsp in #9347
- proxy: Demote some errors to warnings in logs by @cloneable in #9354
- ci: gh-workflow-stats-action v0.1.4: remove debug output and proper pagination by @fedordikarev in #9356
- Test versions mismatch by @a-masterov in #9167
- CI(run-python-test-set): allow to skip missing compatibility snapshot by @bayandin in #9365
- proxy: refactor auth backends by @conradludgate in #9271
- Storage & Compute release 2024-10-14 by @vipvap in #9372
New Contributors
- @devjv made their first contribution in #9031
- @fedordikarev made their first contribution in #9330
Full Changelog: release-6812...release-6872
release-6812
What's Changed
- Expose more granular wait event data to the user by @MMeent in #9163
- proxy: Add timeout to webauth confirmation wait by @cloneable in #9227
- pageserver: tweak logging of "became visible" for layers by @jcsp in #9224
- storcon: ignore deleted timelines on new location catch-up by @VladLazar in #9244
- storcon: add metric for long running reconciles by @VladLazar in #9207
- fix(test): only test num of L1 layers in compaction smoke test by @skyzh in #9186
- tests: sync_after_each_test -> sync_between_tests by @jcsp in #9239
- Fix metric name of the 'getpage_wait_seconds_bucket' metric by @hlinnaka in #9242
- Change parentheses to clarify conditions in walproposer by @dotdister in #9180
- test_runner: bump
requests
andpsycopg2-binary
by @bayandin in #9248 - Add compute_tools/ to the list of paths that trigger an E2E run on a PR by @hlinnaka in #9251
- Use hyper 1.0 and tonic 0.12 in storage broker by @arpad-m in #9234
- Suppress wal lag timeout warnings right after tenant attachment by @arpad-m in #9232
- Remove unnecessary dependencies from postgis-build image by @hlinnaka in #9211
- chore: smaller layer changes by @koivunej in #9247
- vm-monitor: Upgrade axum from 0.6 to 0.7 by @hlinnaka in #9257
- Rename hyper 1.0 to hyper and hyper 0.14 to hyper0 by @arpad-m in #9254
- pageserver: tweak oversized key read path warning by @VladLazar in #9221
- safekeeper: fix panic in debug_dump. by @arssher in #9097
- Revert hyper and tonic updates by @arpad-m in #9268
- chore: remove unnecessary comments in compute/Dockerfile.compute-node by @davidgomes in #9253
- chore: makes some onboarding document improvements by @davidgomes in #9216
- tests: Refactor NeonCli test class by @hlinnaka in #9195
- Cargo.toml: enable
sso
foraws-config
by @erikgrinaker in #9261 - pageserver: use conditional GET for secondary tenant heatmaps by @erikgrinaker in #9236
- storcon: do timeline creation on all attached location by @VladLazar in #9237
- remote_storage: add
head_object
integration test by @erikgrinaker in #9274 - neon walsender_hooks: take basebackup LSN directly. by @arssher in #9263
- proxy: add local-proxy to compute image by @conradludgate in #8823
- proxy: exclude triple logging of connect compute errors by @awarus in #9277
- safekeeper: remove local WAL files ignoring peer_horizon_lsn. by @arssher in #8900
- Storage & Compute release 2024-10-07 by @vipvap in #9291
New Contributors
Full Changelog: release-6777...release-6812
release-6777
What's Changed
- fix: ignore tonic to resolve advisories by @skyzh in #9230
- Update aws SDK crates by @arpad-m in #9233
- Fix compute metrics collector config by @hlinnaka in #9235
- Compute release 2024-10-02 (2) by @lubennikovaav in #9249
Full Changelog: release-6773...release-6777
release-6773
What's Changed
- Bump default neon extension version to 1.5 by @hlinnaka in #9188
- pageserver: refactor immediate_gc into TenantManager by @jcsp in #9183
- test_compatibility: fix the regexes detecting the version by @a-masterov in #9205
- tests: longer timeout in
test_delete_timeline_client_hangup
by @jcsp in #9161 - Misc python test cleanup by @hlinnaka in #9194
- add proxy-protocol header disable option by @conradludgate in #9203
- Bump vm-builder v0.29.3 -> v0.35.0 by @petuhovskiy in #9208
- proxy: auth broker by @conradludgate in #8855
- Remove unnecessary dev package from compute image by @hlinnaka in #9210
- storage controller: drop out of blocking compute notification loop if migration origin becomes unavailable by @jcsp in #9147
- Upgrade OpenTelemetry and other tracing crates by @cloneable in #9200
- tests: make test_multi_attach more stable by @jcsp in #9202
- feat(compute): adds pg_session_jwt extension to compute image by @davidgomes in #8888
- storage_scrubber: retry on index deletion failures by @jcsp in #9204
- proxy: ignore typ and use jwt.alg rather than jwk.alg by @conradludgate in #9215
- Use debian packages for cmake again by @hlinnaka in #9212
- Add new compute metrics to sql exporter by @hlinnaka in #9190
- docs: add aux file v2 RFC by @skyzh in #9115
- safekeeper: Fix a log message of HTTP worker by @shinyaaa in #9213
- Fix post_apply_config() by @lubennikovaav in #9220
- pageserver: remove
TenantConfOpt::TryFrom<toml_edit::Item>
by @erikgrinaker in #9219 - Fix small memory accounting bug in libpagestore by @MMeent in #9223
- tests: wait walreceiver on sks to be gone on 'immediate' ep restart. by @arssher in #9099
- Backpressure: reset ps display after it is done. by @arssher in #8980
- Fix the size of the perf counters shared memory array by @hlinnaka in #9226
- Compute release 2024-10-02 by @lubennikovaav in #9228
New Contributors
- @davidgomes made their first contribution in #8888
Full Changelog: release-6745...release-6773
release-6745
What's Changed
- CI: Leave out unnecessary build files from binary artifact by @hlinnaka in #9135
- tests: remove patching up of az id column by @VladLazar in #8968
- CI(promote-images): fix prod ECR auth (#9131) by @bayandin in #9146
- fix(pageserver): passthrough partition cancel error by @skyzh in #9154
- Fix compute_logical_snapshot_files for v14 by @tristan957 in #9155
- chore: Silence clippy warning with nightly by @hlinnaka in #9157
- Forbid creation of child timelines of archived timeline by @arpad-m in #9122
- CI: run compatibility tests on Postgres 17 by @bayandin in #9145
- fix(compute_ctl): race condition in configurator by @petuhovskiy in #9162
- pageserver: rename control plane client & chunk validation requests by @jcsp in #8997
- fix(pageserver): categorize client error in basebackup metrics by @skyzh in #9110
- Fix flakiness of test_timeline_copy. by @arssher in #9061
- proxy: make local-proxy config by signal with PID, refine JWKS apis with role caching by @conradludgate in #9164
- safekeeper: skip syncfs on start if --no-sync is specified. by @arssher in #9166
- libs: improve logging on PG connection errors by @VladLazar in #9130
- Dump backtrace if a core dump is called just "core" by @hlinnaka in #9125
- Use long options when calling initdb by @tristan957 in #9174
- fix(pageserver): LSN lease edge cases around restarts/migrations by @yliang412 in #9055
- proxy: add jwks endpoint to control plane and mock providers by @conradludgate in #9165
- pageserver: remove AdjacentVectoredReadBuilder and bump minmimum io_buffer_alignment to 512 by @yliang412 in #9175
- WalProposer: Read WAL directly from buffers by @MMeent in #9171
- tests: Reduce the # of iterations in safekeeper::test_random_schedules by @hlinnaka in #9182
- fix(pageserver): abort process if fsync fails by @skyzh in #9108
- neon_local: Use clap derive macros to parse the CLI args by @hlinnaka in #9103
- Set disk quotas on bind in compute_ctl by @petuhovskiy in #8936
- CI(benchmarking): fix Nightly Benchmarks by @bayandin in #9178
- CI(benchmarking): fix setting LD_LIBRARY_PATH by @bayandin in #9191
- Storage & Compute release 2024-09-30 by @vipvap in #9196
Full Changelog: release-6715...release-6745
release-6715
What's Changed
- chore(#9077): cleanups & code dedup by @problame in #9082
- Move the patch to compute by @a-masterov in #9120
- test: Make test_hot_standby_feedback more forgiving of slow initialization by @hlinnaka in #9113
- Little test startup speedups by @hlinnaka in #9094
- pageserver: handle decompression outside vectored
read_blobs
by @yliang412 in #8942 - Catch Cancelled and don't print a warning for it by @arpad-m in #9121
- Fix compiler warnings on macOS by @bayandin in #9128
- test: Make test_lfc_resize more robust by @hlinnaka in #9117
- Remove TenantState::Loading by @hlinnaka in #9118
- chore(docker-compose): fix typo in readme by @Damian972 in #9133
- fix(test): storage scrubber should only log to stdout with info by @skyzh in #9067
- storcon: add tags to scheduler logs by @VladLazar in #9127
- Re-export git-version from utils and remove as direct dep by @cloneable in #9138
- storcon: do az aware scheduling by @VladLazar in #9083
- storage controller: make proxying of GETs to pageservers more robust by @jcsp in #9065
- docs(rfc): Independent compute release flow by @ololobus in #8881
- Try building images using Debian 12 "Bookworm" for PG17 by @MMeent in #9132
- storcon: include timeline ID in LSN waiting logs by @VladLazar in #9141
- fix(pageserver): handle lsn lease requests for unnormalized lsns by @yliang412 in #9137
- Compute release 2024-09-25 by @lubennikovaav in #9151
New Contributors
- @Damian972 made their first contribution in #9133
Full Changelog: release-6694...release-6715
release-6694
What's Changed
- Bump vendor/postgres to include extenision path fix by @lubennikovaav in #9076
- Fix init of WAL page header at startup by @hlinnaka in #8914
- Move load_timeline_metadata into separate function by @arpad-m in #9080
- Check submodule forward progress by @hlinnaka in #8949
- Misc dependency cleanup by @hlinnaka in #9092
- storcon: update compute hook state on detach by @VladLazar in #9045
- walredo: add a ping method by @problame in #8952
- compact_level0_phase1: remove final traces of value access mode config by @problame in #8935
- change HTTP method to comply with spec by @nikitakalyanov in #9100
- feat(pageserver): filter keys with gc-compaction by @skyzh in #9004
- test: Downgrade info message on removing empty directories by @hlinnaka in #9093
- Add views for metrics about pageserver requests by @hlinnaka in #9008
- Maintain number of used pages for LFC by @knizhnik in #9088
- NOAI by @MMeent in #9084
- Fix compiler warnings with nightly rustc about elided lifetimes having names by @hlinnaka in #9105
- Move files related to building compute image into compute/ dir by @hlinnaka in #9090
- scrubber: retry when missing index key in the listing by @yliang412 in #8873
- Add regression tests for a cloud-based Neon instance by @a-masterov in #8681
- storcon: improve initial shard scheduling by @VladLazar in #9081
- proxy: check if IP is allowed during webauth flow by @cloneable in #9101
- Update PostgreSQL 17 from 17rc1 to 17.0 by @MMeent in #9119
- Storage & Compute release 2024-09-23 by @vipvap in #9095
- CI(promote-images): fix prod ECR auth by @bayandin in #9131
Full Changelog: release-6667...release-6694
release-6667
What's Changed
- Grant access to pg_show_replication_origin_status for neon_superuser by @tristan957 in #8978
- Fix PG17's extension modifications by @MMeent in #9010
- Remove misc unused code by @hlinnaka in #9014
- Generate pgbench data on the server by @tristan957 in #8977
- Little python test cleanup by @hlinnaka in #9018
- storcon: forward requests from stepped down instance to the current leader by @VladLazar in #8954
- CI: fix release pipeline by @bayandin in #9017
- Remove unnecessary dependencies by @hlinnaka in #9000
- Fix test code related to migrations by @tristan957 in #9019
- Rename some S3 usages to "remote storage" in exposed messages by @arpad-m in #8999
- CI: don't fetch the whole git history if it's not required by @bayandin in #9021
- refactor(walredo): process launch & kill-on-error machinery by @problame in #8951
- add
neon_local
subcommand for the broker & use that from regression tests by @problame in #8948 - Refactor ImageLayerWriter to avoid passing a Timeline to finish() by @hlinnaka in #9028
- proxy: make sql-over-http max request/response sizes configurable by @cloneable in #9029
- CI(deploy): fix deploy to staging and prod by @bayandin in #9030
- Add v17 to revisions.json by @tristan957 in #9027
- proxy: mock cplane usable without allowed-ips table by @cloneable in #9046
neon_local start
: parallel startup to break cyclic dependency by @problame in #8950- fix(storage-scrubber): log version after initialize the logger by @skyzh in #9049
- Remove misc unused code by @hlinnaka in #9054
- Remove leftover dead code and other cleanup in config.rs by @hlinnaka in #9053
- safekeeper: add wal_last_modified to debug_dump. by @arssher in #8994
- pageserver: simple fix for vectored read image layer skip by @VladLazar in #9026
- impr(pageserver): log image layer creation by @skyzh in #9050
- proxy: remove auth info from http conn info & fixup jwt api trait by @conradludgate in #9047
- pageserver: wait for lsn lease duration after transition into AttachedSingle by @yliang412 in #9024
- refactor(pageserver): check layer map valid in one place by @skyzh in #9051
- Bump vendor/postgres by @arssher in #8986
- safekeeper: fix OpenAPI spec by @bayandin in #9066
- pageserver: add counters for started smgr/getpage requests by @problame in #9069
- Skip running clippy in --release mode. by @hlinnaka in #9073
- fix(pageserver): receive body error now 500 by @skyzh in #9052
- utils: leaky bucket should only report throttled if the notify queue is blocked on sleep by @conradludgate in #9072
- pageserver: suppress noisy "layer became visible" logs by @jcsp in #9064
- pageserver: throttling: per-tenant metrics + more metrics to help understand throttle queue depth by @problame in #9077
- storage hotfix release 2024-09-20 by @problame in #9085
Full Changelog: release-6616...release-6667