Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
…195974) # Backport This will backport the following commits from `main` to `8.x`: - [[Core] log on external requests to internal routes (#195696)](#195696) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Christiane (Tina) Heiligers","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-10-11T18:02:31Z","message":"[Core] log on external requests to internal routes (#195696)\n\nfix [#194772](https://github.com/elastic/kibana/issues/194772)\r\n\r\nKibana logs a warning when detecting requests from integrations with\r\ninternal APIs when the restriction is not enforced or explicitly set to\r\n`false`.\r\nConsumers can use these logs to audit their integrations and make\r\nchanges before the restriction becomes enforced.\r\n\r\n### Note ###\r\nAfter 9.0, the restriction will be enforced. Explicitly disabling the\r\nrestriction effectively opts into using internal routes, which are\r\nsubject to change and not recommended for non-Elastic consumption.\r\n\r\nBypassing the restriction for specific routes by adding the necessary\r\nheader or query parameter that allows access is also not recommended for\r\nnon-Elastic consumption.\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n## How to test this ##\r\n\r\n#### Test as an external consumer making a request to an internal route\r\n####\r\n\r\n<details>\r\n<summary>1. Enforce restricted internal APIs & enable logging:</summary>\r\n\r\n```//kibana.yml\r\nserver.restrictInternalApis: true // or don't declare\r\n...\r\nlogging\r\n appenders:\r\n http-custom:\r\n type: console\r\n layout:\r\n type: pattern\r\n highlight: true\r\n pattern: \"HTTP_CUSTOM--[%date][%level][%logger]---%message\"\r\n root:\r\n appenders: [console]\r\n level: warn\r\n loggers\r\n - name: http.server.kbn-internal-api-restricted\r\n level: warn\r\n appenders: [http-custom]\r\n```\r\n</details>\r\n\r\n2. Start ES (any license)\r\n3. Start Kbn\r\n\r\n<details>\r\n<summary>4. GET global settings:</summary>\r\n\r\n```\r\ncurl --location 'localhost:5601/abc/internal/kibana/global_settings' \\\r\n--header 'Content-Type: application/json' \\\r\n--header 'Accept-Encoding: gzip, deflate, br' \\\r\n--header 'kbn-xsrf: kibana' \\\r\n--header 'Kbn-Version: 9.0.0' \\\r\n--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ=='\r\n```\r\n</details>\r\n\r\n5. The response from the curl request shoud be:\r\n```\r\n{\"statusCode\":400,\"error\":\"Bad Request\",\"message\":\"uri [/internal/kibana/global_settings] with method [get] exists but is not available with the current configuration\"}\r\n```\r\n6. You should see an error log from the\r\n`http.server.kbnInternalApiRestricted` logger:\r\n```\r\nHTTP_CUSTOM--[2024-10-10T13:04:51.287-07:00][ERROR][http.server.kbn-internal-api-restricted]---Access to uri [/internal/kibana/global_settings] with method [get] is not available with the current configuration\r\n```\r\n\r\n#### Bypass the restriction for the global settings route (opt in to use\r\nthe internal route)\r\nKeeping the same configuration and with ES and Kbn still running, add\r\nthe 'x-elastic-internal-origin' header to the curl request ####\r\n\r\n<details>\r\n<summary>1. Opt in to use the global settings route:</summary>\r\n\r\n```\r\ncurl --location 'localhost:5601/abc/internal/kibana/global_settings' \\\r\n--header 'Content-Type: application/json' \\\r\n--header 'Accept-Encoding: gzip, deflate, br' \\\r\n--header 'kbn-xsrf: kibana' \\\r\n--header 'Kbn-Version: 9.0.0' \\\r\n--header 'x-elastic-internal-origin: kibana' \\\r\n--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ=='\r\n```\r\n</details>\r\n\r\n2. The response from the curl request shoud be:\r\n```\r\n{\"settings\":{\"buildNum\":{\"userValue\":9007199254740991},\"isDefaultIndexMigrated\":{\"userValue\":true}}}\r\n```\r\n3. You shouldn't see any new logs from the\r\n`http.server.kbnInternalApiRestricted` logger\r\n\r\n#### Test as an external consumer explicitly disabling the restriction\r\ncompletely (not recommended and not guaranteed) ####\r\n\r\n<details>\r\n<summary>1. Disable restricted internal APIs & enable logging:</summary>\r\n\r\n```//kibana.yml\r\nserver.restrictInternalApis: false\r\n...\r\nlogging\r\n appenders:\r\n http-custom:\r\n type: console\r\n layout:\r\n type: pattern\r\n highlight: true\r\n pattern: \"HTTP_CUSTOM--[%date][%level][%logger]---%message\"\r\n root:\r\n appenders: [console]\r\n level: warn\r\n loggers\r\n - name: http.server.kbn-internal-api-restricted\r\n level: warn\r\n appenders: [http-custom]\r\n```\r\n</details>\r\n\r\n2. Start ES (any license)\r\n3. Start Kbn\r\n\r\n<details>\r\n<summary>4. GET global settings:</summary>\r\n\r\n```\r\ncurl --location 'localhost:5601/abc/internal/kibana/global_settings' \\\r\n--header 'Content-Type: application/json' \\\r\n--header 'Accept-Encoding: gzip, deflate, br' \\\r\n--header 'kbn-xsrf: kibana' \\\r\n--header 'Kbn-Version: 9.0.0' \\\r\n--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ=='\r\n```\r\n</details>\r\n\r\n5. The response from the curl request shoud be:\r\n```\r\n{\"settings\":{\"buildNum\":{\"userValue\":9007199254740991},\"isDefaultIndexMigrated\":{\"userValue\":true}}}\r\n```\r\n6. You should see a warning log from the\r\n`http.server.kbnInternalApiRestricted` logger:\r\n```\r\nHTTP_CUSTOM--[2024-10-10T13:31:48.729-07:00][WARN ][http.server.kbn-internal-api-restricted]---Access to uri [/internal/kibana/global_settings] with method [get] is deprecated\r\n```\r\n<details>\r\n<summary>7. Add the internal origin header to the request:</summary>\r\n\r\n```\r\ncurl --location 'localhost:5601/abc/internal/kibana/global_settings' \\\r\n--header 'Content-Type: application/json' \\\r\n--header 'Accept-Encoding: gzip, deflate, br' \\\r\n--header 'kbn-xsrf: kibana' \\\r\n--header 'Kbn-Version: 9.0.0' \\\r\n--header 'x-elastic-internal-origin: kibana' \\\r\n--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ=='\r\n```\r\n</details>\r\n\r\n8. The response from the curl request shoud be:\r\n```\r\n{\"settings\":{\"buildNum\":{\"userValue\":9007199254740991},\"isDefaultIndexMigrated\":{\"userValue\":true}}}\r\n```\r\n9. You shouldn't see any new logs from the\r\n`http.server.kbnInternalApiRestricted` logger.\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine <[email protected]>","sha":"d0bdbdddb200a2656567c9b9f05d1e934c5a4cea","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Core","release_note:skip","v9.0.0","backport:prev-minor"],"title":"[Core] log on external requests to internal routes","number":195696,"url":"https://github.com/elastic/kibana/pull/195696","mergeCommit":{"message":"[Core] log on external requests to internal routes (#195696)\n\nfix [#194772](https://github.com/elastic/kibana/issues/194772)\r\n\r\nKibana logs a warning when detecting requests from integrations with\r\ninternal APIs when the restriction is not enforced or explicitly set to\r\n`false`.\r\nConsumers can use these logs to audit their integrations and make\r\nchanges before the restriction becomes enforced.\r\n\r\n### Note ###\r\nAfter 9.0, the restriction will be enforced. Explicitly disabling the\r\nrestriction effectively opts into using internal routes, which are\r\nsubject to change and not recommended for non-Elastic consumption.\r\n\r\nBypassing the restriction for specific routes by adding the necessary\r\nheader or query parameter that allows access is also not recommended for\r\nnon-Elastic consumption.\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n## How to test this ##\r\n\r\n#### Test as an external consumer making a request to an internal route\r\n####\r\n\r\n<details>\r\n<summary>1. Enforce restricted internal APIs & enable logging:</summary>\r\n\r\n```//kibana.yml\r\nserver.restrictInternalApis: true // or don't declare\r\n...\r\nlogging\r\n appenders:\r\n http-custom:\r\n type: console\r\n layout:\r\n type: pattern\r\n highlight: true\r\n pattern: \"HTTP_CUSTOM--[%date][%level][%logger]---%message\"\r\n root:\r\n appenders: [console]\r\n level: warn\r\n loggers\r\n - name: http.server.kbn-internal-api-restricted\r\n level: warn\r\n appenders: [http-custom]\r\n```\r\n</details>\r\n\r\n2. Start ES (any license)\r\n3. Start Kbn\r\n\r\n<details>\r\n<summary>4. GET global settings:</summary>\r\n\r\n```\r\ncurl --location 'localhost:5601/abc/internal/kibana/global_settings' \\\r\n--header 'Content-Type: application/json' \\\r\n--header 'Accept-Encoding: gzip, deflate, br' \\\r\n--header 'kbn-xsrf: kibana' \\\r\n--header 'Kbn-Version: 9.0.0' \\\r\n--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ=='\r\n```\r\n</details>\r\n\r\n5. The response from the curl request shoud be:\r\n```\r\n{\"statusCode\":400,\"error\":\"Bad Request\",\"message\":\"uri [/internal/kibana/global_settings] with method [get] exists but is not available with the current configuration\"}\r\n```\r\n6. You should see an error log from the\r\n`http.server.kbnInternalApiRestricted` logger:\r\n```\r\nHTTP_CUSTOM--[2024-10-10T13:04:51.287-07:00][ERROR][http.server.kbn-internal-api-restricted]---Access to uri [/internal/kibana/global_settings] with method [get] is not available with the current configuration\r\n```\r\n\r\n#### Bypass the restriction for the global settings route (opt in to use\r\nthe internal route)\r\nKeeping the same configuration and with ES and Kbn still running, add\r\nthe 'x-elastic-internal-origin' header to the curl request ####\r\n\r\n<details>\r\n<summary>1. Opt in to use the global settings route:</summary>\r\n\r\n```\r\ncurl --location 'localhost:5601/abc/internal/kibana/global_settings' \\\r\n--header 'Content-Type: application/json' \\\r\n--header 'Accept-Encoding: gzip, deflate, br' \\\r\n--header 'kbn-xsrf: kibana' \\\r\n--header 'Kbn-Version: 9.0.0' \\\r\n--header 'x-elastic-internal-origin: kibana' \\\r\n--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ=='\r\n```\r\n</details>\r\n\r\n2. The response from the curl request shoud be:\r\n```\r\n{\"settings\":{\"buildNum\":{\"userValue\":9007199254740991},\"isDefaultIndexMigrated\":{\"userValue\":true}}}\r\n```\r\n3. You shouldn't see any new logs from the\r\n`http.server.kbnInternalApiRestricted` logger\r\n\r\n#### Test as an external consumer explicitly disabling the restriction\r\ncompletely (not recommended and not guaranteed) ####\r\n\r\n<details>\r\n<summary>1. Disable restricted internal APIs & enable logging:</summary>\r\n\r\n```//kibana.yml\r\nserver.restrictInternalApis: false\r\n...\r\nlogging\r\n appenders:\r\n http-custom:\r\n type: console\r\n layout:\r\n type: pattern\r\n highlight: true\r\n pattern: \"HTTP_CUSTOM--[%date][%level][%logger]---%message\"\r\n root:\r\n appenders: [console]\r\n level: warn\r\n loggers\r\n - name: http.server.kbn-internal-api-restricted\r\n level: warn\r\n appenders: [http-custom]\r\n```\r\n</details>\r\n\r\n2. Start ES (any license)\r\n3. Start Kbn\r\n\r\n<details>\r\n<summary>4. GET global settings:</summary>\r\n\r\n```\r\ncurl --location 'localhost:5601/abc/internal/kibana/global_settings' \\\r\n--header 'Content-Type: application/json' \\\r\n--header 'Accept-Encoding: gzip, deflate, br' \\\r\n--header 'kbn-xsrf: kibana' \\\r\n--header 'Kbn-Version: 9.0.0' \\\r\n--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ=='\r\n```\r\n</details>\r\n\r\n5. The response from the curl request shoud be:\r\n```\r\n{\"settings\":{\"buildNum\":{\"userValue\":9007199254740991},\"isDefaultIndexMigrated\":{\"userValue\":true}}}\r\n```\r\n6. You should see a warning log from the\r\n`http.server.kbnInternalApiRestricted` logger:\r\n```\r\nHTTP_CUSTOM--[2024-10-10T13:31:48.729-07:00][WARN ][http.server.kbn-internal-api-restricted]---Access to uri [/internal/kibana/global_settings] with method [get] is deprecated\r\n```\r\n<details>\r\n<summary>7. Add the internal origin header to the request:</summary>\r\n\r\n```\r\ncurl --location 'localhost:5601/abc/internal/kibana/global_settings' \\\r\n--header 'Content-Type: application/json' \\\r\n--header 'Accept-Encoding: gzip, deflate, br' \\\r\n--header 'kbn-xsrf: kibana' \\\r\n--header 'Kbn-Version: 9.0.0' \\\r\n--header 'x-elastic-internal-origin: kibana' \\\r\n--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ=='\r\n```\r\n</details>\r\n\r\n8. The response from the curl request shoud be:\r\n```\r\n{\"settings\":{\"buildNum\":{\"userValue\":9007199254740991},\"isDefaultIndexMigrated\":{\"userValue\":true}}}\r\n```\r\n9. You shouldn't see any new logs from the\r\n`http.server.kbnInternalApiRestricted` logger.\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine <[email protected]>","sha":"d0bdbdddb200a2656567c9b9f05d1e934c5a4cea"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/195696","number":195696,"mergeCommit":{"message":"[Core] log on external requests to internal routes (#195696)\n\nfix [#194772](https://github.com/elastic/kibana/issues/194772)\r\n\r\nKibana logs a warning when detecting requests from integrations with\r\ninternal APIs when the restriction is not enforced or explicitly set to\r\n`false`.\r\nConsumers can use these logs to audit their integrations and make\r\nchanges before the restriction becomes enforced.\r\n\r\n### Note ###\r\nAfter 9.0, the restriction will be enforced. Explicitly disabling the\r\nrestriction effectively opts into using internal routes, which are\r\nsubject to change and not recommended for non-Elastic consumption.\r\n\r\nBypassing the restriction for specific routes by adding the necessary\r\nheader or query parameter that allows access is also not recommended for\r\nnon-Elastic consumption.\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n## How to test this ##\r\n\r\n#### Test as an external consumer making a request to an internal route\r\n####\r\n\r\n<details>\r\n<summary>1. Enforce restricted internal APIs & enable logging:</summary>\r\n\r\n```//kibana.yml\r\nserver.restrictInternalApis: true // or don't declare\r\n...\r\nlogging\r\n appenders:\r\n http-custom:\r\n type: console\r\n layout:\r\n type: pattern\r\n highlight: true\r\n pattern: \"HTTP_CUSTOM--[%date][%level][%logger]---%message\"\r\n root:\r\n appenders: [console]\r\n level: warn\r\n loggers\r\n - name: http.server.kbn-internal-api-restricted\r\n level: warn\r\n appenders: [http-custom]\r\n```\r\n</details>\r\n\r\n2. Start ES (any license)\r\n3. Start Kbn\r\n\r\n<details>\r\n<summary>4. GET global settings:</summary>\r\n\r\n```\r\ncurl --location 'localhost:5601/abc/internal/kibana/global_settings' \\\r\n--header 'Content-Type: application/json' \\\r\n--header 'Accept-Encoding: gzip, deflate, br' \\\r\n--header 'kbn-xsrf: kibana' \\\r\n--header 'Kbn-Version: 9.0.0' \\\r\n--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ=='\r\n```\r\n</details>\r\n\r\n5. The response from the curl request shoud be:\r\n```\r\n{\"statusCode\":400,\"error\":\"Bad Request\",\"message\":\"uri [/internal/kibana/global_settings] with method [get] exists but is not available with the current configuration\"}\r\n```\r\n6. You should see an error log from the\r\n`http.server.kbnInternalApiRestricted` logger:\r\n```\r\nHTTP_CUSTOM--[2024-10-10T13:04:51.287-07:00][ERROR][http.server.kbn-internal-api-restricted]---Access to uri [/internal/kibana/global_settings] with method [get] is not available with the current configuration\r\n```\r\n\r\n#### Bypass the restriction for the global settings route (opt in to use\r\nthe internal route)\r\nKeeping the same configuration and with ES and Kbn still running, add\r\nthe 'x-elastic-internal-origin' header to the curl request ####\r\n\r\n<details>\r\n<summary>1. Opt in to use the global settings route:</summary>\r\n\r\n```\r\ncurl --location 'localhost:5601/abc/internal/kibana/global_settings' \\\r\n--header 'Content-Type: application/json' \\\r\n--header 'Accept-Encoding: gzip, deflate, br' \\\r\n--header 'kbn-xsrf: kibana' \\\r\n--header 'Kbn-Version: 9.0.0' \\\r\n--header 'x-elastic-internal-origin: kibana' \\\r\n--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ=='\r\n```\r\n</details>\r\n\r\n2. The response from the curl request shoud be:\r\n```\r\n{\"settings\":{\"buildNum\":{\"userValue\":9007199254740991},\"isDefaultIndexMigrated\":{\"userValue\":true}}}\r\n```\r\n3. You shouldn't see any new logs from the\r\n`http.server.kbnInternalApiRestricted` logger\r\n\r\n#### Test as an external consumer explicitly disabling the restriction\r\ncompletely (not recommended and not guaranteed) ####\r\n\r\n<details>\r\n<summary>1. Disable restricted internal APIs & enable logging:</summary>\r\n\r\n```//kibana.yml\r\nserver.restrictInternalApis: false\r\n...\r\nlogging\r\n appenders:\r\n http-custom:\r\n type: console\r\n layout:\r\n type: pattern\r\n highlight: true\r\n pattern: \"HTTP_CUSTOM--[%date][%level][%logger]---%message\"\r\n root:\r\n appenders: [console]\r\n level: warn\r\n loggers\r\n - name: http.server.kbn-internal-api-restricted\r\n level: warn\r\n appenders: [http-custom]\r\n```\r\n</details>\r\n\r\n2. Start ES (any license)\r\n3. Start Kbn\r\n\r\n<details>\r\n<summary>4. GET global settings:</summary>\r\n\r\n```\r\ncurl --location 'localhost:5601/abc/internal/kibana/global_settings' \\\r\n--header 'Content-Type: application/json' \\\r\n--header 'Accept-Encoding: gzip, deflate, br' \\\r\n--header 'kbn-xsrf: kibana' \\\r\n--header 'Kbn-Version: 9.0.0' \\\r\n--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ=='\r\n```\r\n</details>\r\n\r\n5. The response from the curl request shoud be:\r\n```\r\n{\"settings\":{\"buildNum\":{\"userValue\":9007199254740991},\"isDefaultIndexMigrated\":{\"userValue\":true}}}\r\n```\r\n6. You should see a warning log from the\r\n`http.server.kbnInternalApiRestricted` logger:\r\n```\r\nHTTP_CUSTOM--[2024-10-10T13:31:48.729-07:00][WARN ][http.server.kbn-internal-api-restricted]---Access to uri [/internal/kibana/global_settings] with method [get] is deprecated\r\n```\r\n<details>\r\n<summary>7. Add the internal origin header to the request:</summary>\r\n\r\n```\r\ncurl --location 'localhost:5601/abc/internal/kibana/global_settings' \\\r\n--header 'Content-Type: application/json' \\\r\n--header 'Accept-Encoding: gzip, deflate, br' \\\r\n--header 'kbn-xsrf: kibana' \\\r\n--header 'Kbn-Version: 9.0.0' \\\r\n--header 'x-elastic-internal-origin: kibana' \\\r\n--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ=='\r\n```\r\n</details>\r\n\r\n8. The response from the curl request shoud be:\r\n```\r\n{\"settings\":{\"buildNum\":{\"userValue\":9007199254740991},\"isDefaultIndexMigrated\":{\"userValue\":true}}}\r\n```\r\n9. You shouldn't see any new logs from the\r\n`http.server.kbnInternalApiRestricted` logger.\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine <[email protected]>","sha":"d0bdbdddb200a2656567c9b9f05d1e934c5a4cea"}}]}] BACKPORT--> Co-authored-by: Christiane (Tina) Heiligers <[email protected]>
- Loading branch information