Allow legacy VaultKV configuration keys in kapitan.secrets.vaultkv (#…

…1249)

## Proposed Changes

* Add validation aliases which allow the old envvar style configuration
for the VaultKV references backend config through the inventory
* Update the vaultkv documentation to note that the envvar style configs
in the inventory are deprecated.

## TODO

* [x] Document limitation regarding duplicated configs through different
field aliases. Due to some implementation details in pydantic-settings,
having e.g. the Vault address set both through environment variable
`VAULT_ADDR` and initializer field `addr` will cause a validation error
unless `extra` is set to `ignore` or `allow` neither of which have the
desired effect of protecting users against config errors in the
inventory.

## Docs and Tests

* [x] Tests added
* [x] Updated documentation

docs/references.md

@@ -659,13 +659,19 @@ Leave `mount` empty to use the specified mount from vault params from the invent
 
 Parameters in the secret file are collected from the inventory of the target we gave from CLI `-t <target_name>`. If target isn't provided then kapitan will identify the variables from the environment when revealing secret.
 
-Environment variables that can be defined in kapitan inventory are `VAULT_ADDR`, `VAULT_NAMESPACE`, `VAULT_SKIP_VERIFY`, `VAULT_CLIENT_CERT`, `VAULT_CLIENT_KEY`, `VAULT_CAPATH` & `VAULT_CACERT`.
+The environment variables which can also be defined in kapitan inventory are `VAULT_ADDR`, `VAULT_NAMESPACE`, `VAULT_SKIP_VERIFY`, `VAULT_CLIENT_CERT`, `VAULT_CLIENT_KEY`, `VAULT_CAPATH` & `VAULT_CACERT`.
+Note that providing these variables through the inventory in envvar style is deprecated.
+Users should update their inventory to set these values in keys without the `VAULT_` prefix and in all lowercase.
+For example `VAULT_ADDR: https://127.0.0.1:8200` should be given as `addr: https://127.0.0.1:8200` in the inventory.
+Please note that configuring one of these values in both `kapitan.secrets.vaultkv` in the inventory and in the environment will cause a validation error.
+
 Extra parameters that can be defined in inventory are:
 
 - `auth`: specify which authentication method to use like `token`,`userpass`,`ldap`,`github` & `approle`
 - `mount`: specify the mount point of key's path. e.g if path=`alpha-secret/foo/bar` then `mount: alpha-secret` (default `secret`)
 - `engine`: secret engine used, either `kv-v2` or `kv` (default `kv-v2`)
-Environment variables cannot be defined in inventory are `VAULT_TOKEN`,`VAULT_USERNAME`,`VAULT_PASSWORD`,`VAULT_ROLE_ID`,`VAULT_SECRET_ID`.
+
+The environment variables which cannot be defined in inventory are `VAULT_TOKEN`,`VAULT_USERNAME`,`VAULT_PASSWORD`,`VAULT_ROLE_ID`,`VAULT_SECRET_ID`.
 
   ```yaml
   parameters:
@@ -675,11 +681,11 @@ Environment variables cannot be defined in inventory are `VAULT_TOKEN`,`VAULT_US
   auth: userpass
   engine: kv-v2
   mount: team-alpha-secret
-  VAULT_ADDR: http://127.0.0.1:8200
-  VAULT_NAMESPACE: CICD-alpha
-  VAULT_SKIP_VERIFY: false
-  VAULT_CLIENT_KEY: /path/to/key
-  VAULT_CLIENT_CERT: /path/to/cert
+  addr: http://127.0.0.1:8200
+  namespace: CICD-alpha
+  skip_verify: false
+  client_key: /path/to/key
+  client_cert: /path/to/cert
   ```
 
 ### `vaulttransit`

kapitan/inventory/model/references.py

@@ -1,7 +1,7 @@
 from typing import List, Literal, Optional, Union
 
-from pydantic import BaseModel, ConfigDict
-from pydantic_settings import BaseSettings, SettingsConfigDict
+from pydantic import AliasChoices, BaseModel, ConfigDict, Field
+from pydantic_settings import BaseSettings
 
 from kapitan.utils import StrEnum
 
@@ -14,17 +14,23 @@ class KapitanReferenceGPGConfig(KapitanReferenceBaseConfig):
  recipients: List[dict[str, str]] = []
 
 
+# Must be pydantic_settings.BaseSettings so that environment variables are actually used to
+# populate the object. Any of the validation alias choices can be used as environment variable
+# names.
+# Note that this will break if both alias choices are set for a field (either through envvar or
+# initializer).
 class KapitanReferenceVaultEnv(BaseSettings):
- model_config = SettingsConfigDict(
- env_prefix="VAULT_",
+ addr: Optional[str] = Field(None, validation_alias=AliasChoices("addr", "VAULT_ADDR"))
+ skip_verify: Optional[bool] = Field(
+ True, validation_alias=AliasChoices("skip_verify", "VAULT_SKIP_VERIFY")
  )
- addr: Optional[str] = None
- skip_verify: Optional[bool] = True
- client_key: Optional[str] = None
- client_cert: Optional[str] = None
- cacert: Optional[str] = None
- capath: Optional[str] = None
- namespace: Optional[str] = None
+ client_key: Optional[str] = Field(None, validation_alias=AliasChoices("client_key", "VAULT_CLIENT_KEY"))
+ client_cert: Optional[str] = Field(
+  None, validation_alias=AliasChoices("client_cert", "VAULT_CLIENT_CERT")
+ )
+ cacert: Optional[str] = Field(None, validation_alias=AliasChoices("cacert", "VAULT_CACERT"))
+ capath: Optional[str] = Field(None, validation_alias=AliasChoices("capath", "VAULT_CAPATH"))
+ namespace: Optional[str] = Field(None, validation_alias=AliasChoices("namespace", "VAULT_NAMESPACE"))
 
 
 class VaultEngineTypes(StrEnum):