v1.18.0-rc1

September 18, 2024

CHANGES:

• activity (enterprise): filter all fields in client count responses by the request namespace [GH-27790]
• activity (enterprise): remove deprecated fields distinct_entities and non_entity_tokens [GH-27830]
• activity log: Deprecated the field "default_report_months". Instead, the billing start time will be used to determine the start time
  when querying the activity log endpoints. [GH-27350]
• activity log: Deprecates the current_billing_period field for /sys/internal/counters/activity. The default start time
  will automatically be set the billing period start date. [GH-27426]
• activity: The activity export API now requires the sudo ACL capability. [GH-27846]
• activity: The activity export API now responds with a status of 204 instead 400 when no data exists within the time range specified by start_time and end_time. [GH-28064]
• activity: The startTime will be set to the start of the current billing period by default.
  The endTime will be set to the end of the current month. This applies to /sys/internal/counters/activity,
  /sys/internal/counters/activity/export, and the vault operator usage command that utilizes /sys/internal/counters/activity. [GH-27379]
• api: Update backoff/v3 to backoff/v4.3.0 [GH-26868]
• auth/alicloud: Update plugin to v0.19.0 [GH-28263]
• auth/azure: Update plugin to v0.19.0 [GH-28294]
• auth/cf: Update plugin to v0.18.0 [GH-27724]
• auth/cf: Update plugin to v0.19.0 [GH-28266]
• auth/gcp: Update plugin to v0.19.0 [GH-28366]
• auth/jwt: Update plugin to v0.21.0 [GH-27498]
• auth/jwt: Update plugin to v0.22.0 [GH-28349]
• auth/kerberos: Update plugin to v0.13.0 [GH-28264]
• auth/kubernetes: Update plugin to v0.20.0 [GH-28289]
• auth/oci: Update plugin to v0.17.0 [GH-28307]
• cli: The undocumented -dev-three-node and -dev-four-cluster CLI options have been removed. [GH-27578]
• consul-template: updated to version 0.39.1 [GH-27799]
• core(enterprise): Updated the following two control group related errors responses to respond with response code 400 instead of 500: control group: could not find token, and control group: token is not a valid control group token.
• core: Bump Go version to 1.22.7
• database/couchbase: Update plugin to v0.12.0 [GH-28327]
• database/elasticsearch: Update plugin to v0.16.0 [GH-28277]
• database/mongodbatlas: Update plugin to v0.13.0 [GH-28268]
• database/redis-elasticache: Update plugin to v0.5.0 [GH-28293]
• database/redis: Update plugin to v0.4.0 [GH-28404]
• database/snowflake: Update plugin to v0.12.0 [GH-28275]
• sdk: Upgrade to go-secure-stdlib/[email protected], which also bumps github.com/docker/docker to v26.1.5+incompatible [GH-28269]
• secrets/ad: Update plugin to v0.19.0 [GH-28361]
• secrets/alicloud: Update plugin to v0.18.0 [GH-28271]
• secrets/azure: Update plugin to v0.19.2 [GH-27652]
• secrets/azure: Update plugin to v0.20.0 [GH-28267]
• secrets/gcp: Update plugin to v0.20.0 [GH-28324]
• secrets/gcpkms: Update plugin to v0.18.0 [GH-28300]
• secrets/gcpkms: Update plugin to v0.19.0 [GH-28360]
• secrets/kubernetes: Update plugin to v0.9.0 [GH-28287]
• secrets/kv: Update plugin to v0.20.0 [GH-28334]
• secrets/mongodbatlas: Update plugin to v0.13.0 [GH-28348]
• secrets/openldap: Update plugin to v0.14.0 [GH-28325]
• secrets/terraform: Update plugin to v0.10.0 [GH-28312]
• secrets/terraform: Update plugin to v0.9.0 [GH-28016]
• ui: Uses the internal/counters/activity/export endpoint for client count export data. [GH-27455]

FEATURES:

• AWS secrets engine STS session tags support: Adds support for setting STS
  session tags when generating temporary credentials using the AWS secrets
  engine. [GH-27620]
• Adaptive Overload Protection (enterprise): Enables Adaptive Overload Protection
  for write requests as a GA feature (enabled by default) for Integrated Storage.
• Audit Entry Exclusion: Audit devices support excluding fields from entries being written to them, with expression-based rules (powered by go-bexpr) to determine when the specific fields are excluded.
• Workload Identity Federation UI for AWS (enterprise): Add WIF fields to AWS secrets engine. [GH-28148]
• KV v2 Patch/Subkey (enterprise): Adds GUI support to read the subkeys of a KV v2 secret and patch (partially update) secret data. [GH-28212]
• Self-Managed Static Roles: Self-Managed Static Roles are now supported for select SQL database engines (Postgres, Oracle). Requires Vault Enterprise. [GH-28199]
• Vault Minimal Version: Add the ability to build a minimal version of Vault
  with only core features using the BUILD_MINIMAL environment variable. [GH-27394]
• Vault PKI 3GPP CMPv2 Server (Enterprise): Support for the PKI 3GPP CMPv2 certificate management protocol has been added to the Vault PKI Plugin. This allows standard CMPv2 clients to request certificates from a Vault server with no knowledge of Vault APIs.

IMPROVEMENTS:

• activity log: Changes how new client counts in the current month are estimated, in order to return more
  visibly sensible totals. [GH-27547]
• activity: The activity export API can now be called in non-root namespaces. Resulting records will be filtered to include the requested namespace (via X-Vault-Namespace header or within the path) and all child namespaces. [GH-27846]
• activity: The activity export API now includes identity metadata about entity clients. [GH-28064]
• activity: /sys/internal/counters/activity will now include a warning if the specified usage period contains estimated client counts. [GH-28068]
• agent/sink: Allow configuration of the user and group ID of the file sink. [GH-27123]
• agent: Add metric (vault.agent.authenticated) that is set to 1 when vault agent has a valid token and zero if it does not. [GH-26570]
• agent: Add the ability to dump pprof to the filesystem using SIGUSR2 [GH-27510]
• audit: Adds TRACE logging to log request/response under certain circumstances, and further improvements to the audit subsystem. [GH-28056]
• audit: Ensure that any underyling errors from audit devices are logged even if we consider auditing to be a success. [GH-27809]
• audit: Internal implementation changes to the audit subsystem which improve performance. [GH-27952]
• audit: Internal implementation changes to the audit subsystem which improve relability. [GH-28286]
• audit: sinks (file, socket, syslog) will attempt to log errors to the server operational
  log before returning (if there are errors to log, and the context is done). [GH-27859]
• auth/cert: Cache full list of role trust information separately to avoid
  eviction, and avoid duplicate loading during multiple simultaneous logins on
  the same role. [GH-27902]
• cli: Add a --dev-no-kv flag to prevent auto mounting a key-value secret backend when running a dev server [GH-16974]
• cli: Allow vault CLI HTTP headers to be specified using the JSON-encoded VAULT_HEADERS environment variable [GH-21993]
• cli: vault operator usage will now include a warning if the specified usage period contains estimated client counts. [GH-28068]
• core/activity: Ensure client count queries that include the current month return consistent results by sorting the clients before performing estimation [GH-28062]
• core/cli: Example 'help' pages for vault read / write docs improved. [GH-19064]
• core/identity: allow identity backend to be tuned using standard secrets backend tuning parameters. [GH-14723]
• core/metrics: ensure core HA metrics are always output to Prometheus. [GH-27966]
• core: make authLock and mountsLock in Core configurable via the detect_deadlocks configuration parameter. [GH-27633]
• database/postgres: Add new fields to the plugin's config endpoint for client certificate authentication. [GH-28024]
• db/cassandra: Add disable_host_initial_lookup option to backend, allowing the disabling of initial host lookup. [GH-9733]
• identity: alias metadata is now returned when listing entity aliases [GH-26073]
• license utilization reporting (enterprise): Auto-roll billing start date. [GH-27656]
• proxy/sink: Allow configuration of the user and group ID of the file sink. [GH-27123]
• proxy: Add the ability to dump pprof to the filesystem using SIGUSR2 [GH-27510]
• raft-snapshot (enterprise): add support for managed identity credentials for azure snapshots
• raft/autopilot: Persist Raft server versions so autopilot always knows the versions of all servers in the cluster. Include server versions in the Raft bootstrap challenge answer so autopilot immediately knows the versions of new nodes. [GH-28186]
• sdk/helper: Allow setting environment variables when using NewTestDockerCluster [GH-27457]
• secrets-sync (enterprise): add support for specifying the replication regions for secret storage within GCP Secret Manager destinations
• secrets-sync (enterprise): add support for syncing secrets to github environments within repositories
• secrets-sync (enterprise): add support for syncing secrets to github organizations (beta)
• secrets/database/hana: Update HANA db client to v1.10.1 [GH-27950]
• secrets/database: Add support for GCP CloudSQL private IP's. [GH-26828]
• secrets/pki: Key Usage can now be set on intermediate and root CAs, and CSRs generated by the PKI secret's engine. [GH-28237]
• serviceregistration: Added support for Consul ServiceMeta tags from config file from the new service_meta config field. [GH-11084]
• storage/azure: Updated metadata endpoint to GetMSIEndpoint, which supports more than just the metadata service. [GH-10624]
• storage/dynamodb: Speed up list and delete of large directories by only requesting keys from DynamoDB [GH-21159]
• storage/etcd: Update etcd3 client to v3.5.13 to allow use of TLSv1.3. [GH-26660]
• storage/raft: Bump raft to v1.7.0 which includes pre-vote. This should make clusters more stable during network partitions. [GH-27605]
• storage/raft: Improve autopilot logging on startup to show config values clearly and avoid spurious logs [GH-27464]
• ui/secrets-sync: Hide Secrets Sync from the sidebar nav if user does not have access to the feature. [GH-27262]
• ui: AWS credentials form sets credential_type from backing role [GH-27405]
• ui: Creates separate section for updating sensitive creds for Secrets sync create/edit view. [GH-27538]
• ui: For AWS and SSH secret engines hide mount configuration details in toggle and display configuration details or cta. [GH-27831]
• ui: Mask obfuscated fields when creating/editing a Secrets sync destination. [GH-27348]
• ui: Move secret-engine configuration create/edit from routing vault/settings/secrets/configure/<backend> to vault/secrets/<backend>/configuration/edit [GH-27918]
• ui: Remove deprecated current_billing_period from dashboard activity log request [GH-27559]
• ui: Update the client count dashboard to use API namespace filtering and other UX improvements [GH-28036]
• ui: remove initial start/end parameters on the activity call for client counts dashboard. [GH-27816]
• ui: simplify the date range editing experience in the client counts dashboard. [GH-27796]
• website/docs: Added API documentation for Azure Secrets Engine delete role [GH-27883]
• website/docs: corrected invalid json in sample payload for azure secrets engine create/update role [GH-28076]

BUG FIXES:

• activity: The sys/internal/counters/activity endpoint will return current month data when the end_date parameter is set to a future date. [GH-28042]
• agent: Fixed an issue causing excessive CPU usage during normal operation [GH-27518]
• auth/appid, auth/cert, auth/github, auth/ldap, auth/okta, auth/radius, auth/userpass: fixed an issue with policy name normalization that would prevent a token associated with a policy containing an uppercase character to be renewed. [GH-16484]
• auth/aws: fixes an issue where not supplying an external id was interpreted as an empty external id [GH-27858]
• auth/cert: Merge error messages returned in login failures and include error when present [GH-27202]
• auth/cert: Use subject's serial number, not issuer's within error message text in OCSP request errors [GH-27696]
• auth/cert: ocsp_ca_certificates field was not honored when validating OCSP responses signed by a CA that did not issue the certificate. [GH-28309]
• auth/token: fixes an edge case bug that "identity_policies" is nil and causes cli vault login error [GH-17007]
• cli: Fixed an erroneous warning appearing about -address not being set when it is. [GH-27265]
• cli: Fixed issue with vault hcp connect where HCP resources with uppercase letters were inaccessible when entering the correct project name. [GH-27694]
• command: The vault secrets move and vault auth move command will no longer attempt to write to storage on performance standby nodes. [GH-28059]
• config: Vault TCP listener config now correctly supports the documented proxy_protocol_behavior
  setting of 'deny_unauthorized' [GH-27459]
• core (enterprise): Fix 500 errors that occurred querying sys/internal/ui/mounts for a mount prefixed by a namespace path when path filters are configured. [GH-27939]
• core (enterprise): Fix HTTP redirects in namespaces to use the correct path and (in the case of event subscriptions) the correct URI scheme. [GH-27660]
• core (enterprise): Fix deletion of MFA login-enforcement configurations on standby nodes
• core/audit: Audit logging a Vault request/response checks if the existing context
  is cancelled and will now use a new context with a 5 second timeout.
  If the existing context is cancelled a new context, will be used. [GH-27531]
• core/config: fix issue when using proxy_protocol_behavior with deny_unauthorized,
  which causes the Vault TCP listener to close after receiving an untrusted upstream proxy connection. [GH-27589]
• core/identity: Fixed an issue where deleted/reassigned entity-aliases were not removed from in-memory database. [GH-27750]
• core: Fixed an issue where maximum request duration timeout was not being added to all requests containing strings sys/monitor and sys/events. With this change, timeout is now added to all requests except monitor and events endpoint. [GH-28230]
• core: Fixed an issue with performance standbys not being able to handle rotate root requests. [GH-27631]
• helper/pkcs7: Fix parsing certain messages containing only certificates [GH-27435]
• identity/oidc: prevent JWKS from being generated by multiple concurrent requests [GH-27929]
• licensing (enterprise): fixed issue where billing start date might not be correctly updated on performance standbys
• proxy/cache (enterprise): Fixed an issue where Proxy with static secret caching enabled would not correctly handle requests to older secret versions for KVv2 secrets. Proxy's static secret cache now properly handles all requests relating to older versions for KVv2 secrets. [GH-28207]
• proxy/cache (enterprise): Fixed an issue where Proxy would not correctly update KV secrets when talking to a perf standby. Proxy will now attempt to forward requests to update secrets triggered by events to the active node. Note that this requires allow_forwarding_via_header to be configured on the cluster. [GH-27891]
• proxy/cache (enterprise): Fixed an issue where cached static secrets could fail to update if the secrets belonged to a non-root namespace. [GH-27730]
• proxy: Fixed an issue causing excessive CPU usage during normal operation [GH-27518]
• raft/autopilot: Fixed panic that may occur during shutdown [GH-27726]
• replication (enterprise): fix cache invalidation issue leading to namespace custom metadata not being shown correctly on performance secondaries
• secrets-sync (enterprise): Destination set/remove operations will no longer be blocked as "purge in progress" after a purge job ended in failure.
• secrets-sync (enterprise): Normalize custom_tag keys and values for recoverable invalid characters.
• secrets-sync (enterprise): Normalize secret key names before storing the external_name in a secret association.
• secrets-sync (enterprise): Patching github sync destination credentials will properly update and save the new credentials.
• secrets-sync (enterprise): Properly remove tags from secrets in AWS when they are removed from the source association
• secrets-sync (enterprise): Return an error immediately on destination creation when providing invalid custom_tags based on destination type.
• secrets-sync (enterprise): Return more accurate error code for invalid connection details
• secrets-sync (enterprise): Secondary nodes in a cluster now properly check activation-flags values.
• secrets-sync (enterprise): Skip invalid GitHub repository names when creating destinations
• secrets-sync (enterprise): Validate corresponding GitHub app parameters app_name and installation_id are set
• secrets/database: Skip connection verification on reading existing DB connection configuration [GH-28139]
• secrets/identity (enterprise): Fix a bug that can cause DR promotion to fail in rare cases where a PR secondary has inconsistent alias information in storage.
• secrets/pki: fix lack of serial number to a certificate read resulting in a server side error. [GH-27681]
• secrets/transit (enterprise): Fix an issue that caused input data be returned as part of generated CMAC values.
• storage/azure: Fix invalid account name initialization bug [GH-27563]
• storage/raft (enterprise): Fix issue with namespace cache not getting cleared on snapshot restore, resulting in namespaces not found in the snapshot being inaccurately represented by API responses. [GH-27474]
• storage/raft: Fix auto_join not working with mDNS provider. [GH-25080]
• sys: Fix a bug where mounts of external plugins that were registered before Vault v1.0.0 could not be tuned to
  use versioned plugins. [GH-27881]
• ui: Allow creation of session_token type roles for AWS secret backend [GH-27424]
• ui: Display an error and force a timeout when TOTP passcode is incorrect [GH-27574]
• ui: Ensure token expired banner displays when batch token expires [GH-27479]
• ui: Fix UI improperly checking capabilities for enabling performance and dr replication [GH-28371]
• ui: Fix cursor jump on KVv2 json editor that would occur after pressing ENTER. [GH-27569]
• ui: fix issue where enabling then disabling "Tidy ACME" in PKI results in failed API call. [GH-27742]
• ui: fix namespace picker not working when in small screen where the sidebar is collapsed by default. [GH-27728]
• ui: fixes renew-self being called right after login for non-renewable tokens [GH-28204]
• ui: fixes toast (flash) alert message saying "created" when deleting a kv v2 secret [GH-28093]