Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing test: Jest Integration Tests.src/core/server/integration_tests/saved_objects/migrations/group2 - migration v2 migrates saved objects normally when multiple Kibana instances are started with an average interval #156117

Open
kibanamachine opened this issue Apr 27, 2023 · 34 comments · Fixed by #158182, #162648, #189579 or #190679
Assignees
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Apr 27, 2023

A test failed on a tracked branch

[Error: [{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard count of [ALL] (have 1, needed 2). Timeout: [1m], request: [BulkShardRequest [[.kibana_8.8.0_reindex_temp][0]] containing [100] requests]"},{"type":"unavailable_shards_exception","reason":"[.kibana_8.8.0_reindex_temp][0] Not enough active copies to meet shard cou
[report_failure] output truncated to 8192 characters

First failure: CI Build - 8.8

@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Apr 27, 2023
@botelastic botelastic bot added the needs-team Issues missing a team label label Apr 27, 2023
@kibanamachine kibanamachine added the Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc label Apr 27, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-core (Team:Core)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Apr 27, 2023
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - 8.8

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@mistic
Copy link
Member

mistic commented May 3, 2023

Skipped.

main: d0761f3
8.8: 3e9e167

@rudolf rudolf self-assigned this May 3, 2023
@rudolf
Copy link
Contributor

rudolf commented May 4, 2023

Excerpt of relevant ES logs

│ info [o.e.c.c.ClusterBootstrapService] [node-01] this node is locked into cluster UUID [yaK92qMFQo2lCIpIFrqQWA] but [cluster.initial_master_nodes] is set to [node-01]; remove this setting to avoid possible data loss caused by subsequent cluster bootstrap attempts; for further information see https://www.elastic.co/guide/en/elasticsearch/reference/master/important-settings.html#initial_master_nodes
 info [o.e.c.m.MetadataCreateIndexService] [node-01] [.kibana_8.8.0_reindex_temp] creating index, cause [api], templates [], shards [1]/[1]
 info [o.e.c.r.a.AllocationService] [node-01] updating number_of_replicas to [0] for indices [.kibana_8.8.0_reindex_temp]
 info [o.e.c.r.a.AllocationService] [node-01] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.kibana_8.8.0_reindex_temp][0]]])." previous.health="YELLOW" reason="shards started [[.kibana_8.8.0_reindex_temp][0]]"
info [o.e.c.m.MetadataDeleteIndexService] [node-01] [.kibana_8.8.0_reindex_temp/ZtDHgeT0SGONkOp_imLQhQ] deleting index

At the moment I have two theories:

  1. Maybe if we're busy bulk indexing and the index is deleted it could surface as an unavailable shards exception
  2. Maybe there's a time when auto-expand-replica's starts being set to 1 before updating number_of_replicas to [0] happens and we might be indexing in that small window

@gsoldevila
Copy link
Contributor

Re (1) we should not delete an index (the temporary index I presume) until all migrators have finished bulk indexing.
At least, that's the goal of the second synchronisation point.
In fact, the migrator responsible for the index will first clone into the target index, before proceeding to delete the temporary one.

@rudolf
Copy link
Contributor

rudolf commented May 9, 2023

That's true for a single Kibana node. But multiple Kibana nodes don't get synchronised.

@rudolf
Copy link
Contributor

rudolf commented May 22, 2023

I've so far been unable to reproduce (2).

But I can reproduce (1). If we're busy bulk creating into a temp index and then the temp index gets deleted, Elasticsearch will automatically create a new index, but the newly created index won't have "auto_expand_replicas": "0-1" so it expects a replica shard to be available which then fails.

We can confirm that this is indeed what happened due to "cause [auto(bulk api)]" in the logs:

info [o.e.c.m.MetadataCreateIndexService] [node-01] [.kibana_8.8.0_reindex_temp] creating index, cause [auto(bulk api)], templates [], shards [1]/[1]

rudolf added a commit that referenced this issue Jun 4, 2023
## Summary

Try to fix
#156117 (comment)

## Release notes
Fixes a race condition that could cause intermittent upgrade migration
failures when Kibana connects to a single node Elasticsearch cluster.

### Checklist

Delete any items that are not applicable to this PR.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)


### Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.

When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:

| Risk | Probability | Severity | Mitigation/Notes |

|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |


### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Jun 4, 2023
## Summary

Try to fix
elastic#156117 (comment)

## Release notes
Fixes a race condition that could cause intermittent upgrade migration
failures when Kibana connects to a single node Elasticsearch cluster.

### Checklist

Delete any items that are not applicable to this PR.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)

### Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.

When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:

| Risk | Probability | Severity | Mitigation/Notes |

|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

(cherry picked from commit 8e7e263)
kibanamachine added a commit that referenced this issue Jun 4, 2023
# Backport

This will backport the following commits from `main` to `8.8`:
- [Migrations: Don't auto-create temp index
(#158182)](#158182)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Rudolf
Meijering","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-06-04T22:34:08Z","message":"Migrations:
Don't auto-create temp index (#158182)\n\n## Summary\r\n\r\nTry to
fix\r\nhttps://github.com//issues/156117#issuecomment-1557029863\r\n\r\n##
Release notes\r\nFixes a race condition that could cause intermittent
upgrade migration\r\nfailures when Kibana connects to a single node
Elasticsearch cluster.\r\n\r\n### Checklist\r\n\r\nDelete any items that
are not applicable to this PR.\r\n\r\n- [ ] Any text added follows
[EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [ ] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] Any UI
touched in this PR is usable by keyboard only (learn more\r\nabout
[keyboard accessibility](https://webaim.org/techniques/keyboard/))\r\n-
[ ] Any UI touched in this PR does not create any new axe
failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
Risk Matrix\r\n\r\nDelete this section if it is not applicable to this
PR.\r\n\r\nBefore closing this PR, invite QA, stakeholders, and other
developers to\r\nidentify risks that should be tested prior to the
change/feature\r\nrelease.\r\n\r\nWhen forming the risk matrix, consider
some of the following examples\r\nand how they may potentially impact
the change:\r\n\r\n| Risk | Probability | Severity | Mitigation/Notes
|\r\n\r\n|---------------------------|-------------|----------|-------------------------|\r\n|
Multiple Spaces&mdash;unexpected behavior in non-default Kibana
Space.\r\n| Low | High | Integration tests will verify that all features
are still\r\nsupported in non-default Kibana Space and when user
switches between\r\nspaces. |\r\n| Multiple nodes&mdash;Elasticsearch
polling might have race conditions\r\nwhen multiple Kibana nodes are
polling for the same tasks. | High | Low\r\n| Tasks are idempotent, so
executing them multiple times will not result\r\nin logical error, but
will degrade performance. To test for this case we\r\nadd plenty of unit
tests around this logic and document manual testing\r\nprocedure. |\r\n|
Code should gracefully handle cases when feature X or plugin Y
are\r\ndisabled. | Medium | High | Unit tests will verify that any
feature flag\r\nor plugin combination still results in our service
operational. |\r\n| [See more potential
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
|\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for
breaking API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"8e7e2632bbb5f2dfffe8ab6c2563e176c5d7cf6b","branchLabelMapping":{"^v8.9.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","Team:Core","release_note:fix","Feature:Migrations","backport:prev-minor","v8.9.0"],"number":158182,"url":"https://github.com/elastic/kibana/pull/158182","mergeCommit":{"message":"Migrations:
Don't auto-create temp index (#158182)\n\n## Summary\r\n\r\nTry to
fix\r\nhttps://github.com//issues/156117#issuecomment-1557029863\r\n\r\n##
Release notes\r\nFixes a race condition that could cause intermittent
upgrade migration\r\nfailures when Kibana connects to a single node
Elasticsearch cluster.\r\n\r\n### Checklist\r\n\r\nDelete any items that
are not applicable to this PR.\r\n\r\n- [ ] Any text added follows
[EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [ ] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] Any UI
touched in this PR is usable by keyboard only (learn more\r\nabout
[keyboard accessibility](https://webaim.org/techniques/keyboard/))\r\n-
[ ] Any UI touched in this PR does not create any new axe
failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
Risk Matrix\r\n\r\nDelete this section if it is not applicable to this
PR.\r\n\r\nBefore closing this PR, invite QA, stakeholders, and other
developers to\r\nidentify risks that should be tested prior to the
change/feature\r\nrelease.\r\n\r\nWhen forming the risk matrix, consider
some of the following examples\r\nand how they may potentially impact
the change:\r\n\r\n| Risk | Probability | Severity | Mitigation/Notes
|\r\n\r\n|---------------------------|-------------|----------|-------------------------|\r\n|
Multiple Spaces&mdash;unexpected behavior in non-default Kibana
Space.\r\n| Low | High | Integration tests will verify that all features
are still\r\nsupported in non-default Kibana Space and when user
switches between\r\nspaces. |\r\n| Multiple nodes&mdash;Elasticsearch
polling might have race conditions\r\nwhen multiple Kibana nodes are
polling for the same tasks. | High | Low\r\n| Tasks are idempotent, so
executing them multiple times will not result\r\nin logical error, but
will degrade performance. To test for this case we\r\nadd plenty of unit
tests around this logic and document manual testing\r\nprocedure. |\r\n|
Code should gracefully handle cases when feature X or plugin Y
are\r\ndisabled. | Medium | High | Unit tests will verify that any
feature flag\r\nor plugin combination still results in our service
operational. |\r\n| [See more potential
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
|\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for
breaking API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"8e7e2632bbb5f2dfffe8ab6c2563e176c5d7cf6b"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.9.0","labelRegex":"^v8.9.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/158182","number":158182,"mergeCommit":{"message":"Migrations:
Don't auto-create temp index (#158182)\n\n## Summary\r\n\r\nTry to
fix\r\nhttps://github.com//issues/156117#issuecomment-1557029863\r\n\r\n##
Release notes\r\nFixes a race condition that could cause intermittent
upgrade migration\r\nfailures when Kibana connects to a single node
Elasticsearch cluster.\r\n\r\n### Checklist\r\n\r\nDelete any items that
are not applicable to this PR.\r\n\r\n- [ ] Any text added follows
[EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [ ] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] Any UI
touched in this PR is usable by keyboard only (learn more\r\nabout
[keyboard accessibility](https://webaim.org/techniques/keyboard/))\r\n-
[ ] Any UI touched in this PR does not create any new axe
failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
Risk Matrix\r\n\r\nDelete this section if it is not applicable to this
PR.\r\n\r\nBefore closing this PR, invite QA, stakeholders, and other
developers to\r\nidentify risks that should be tested prior to the
change/feature\r\nrelease.\r\n\r\nWhen forming the risk matrix, consider
some of the following examples\r\nand how they may potentially impact
the change:\r\n\r\n| Risk | Probability | Severity | Mitigation/Notes
|\r\n\r\n|---------------------------|-------------|----------|-------------------------|\r\n|
Multiple Spaces&mdash;unexpected behavior in non-default Kibana
Space.\r\n| Low | High | Integration tests will verify that all features
are still\r\nsupported in non-default Kibana Space and when user
switches between\r\nspaces. |\r\n| Multiple nodes&mdash;Elasticsearch
polling might have race conditions\r\nwhen multiple Kibana nodes are
polling for the same tasks. | High | Low\r\n| Tasks are idempotent, so
executing them multiple times will not result\r\nin logical error, but
will degrade performance. To test for this case we\r\nadd plenty of unit
tests around this logic and document manual testing\r\nprocedure. |\r\n|
Code should gracefully handle cases when feature X or plugin Y
are\r\ndisabled. | Medium | High | Unit tests will verify that any
feature flag\r\nor plugin combination still results in our service
operational. |\r\n| [See more potential
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
|\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for
breaking API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"8e7e2632bbb5f2dfffe8ab6c2563e176c5d7cf6b"}}]}]
BACKPORT-->

Co-authored-by: Rudolf Meijering <[email protected]>
@rudolf
Copy link
Contributor

rudolf commented Jun 9, 2023

#158182 Hopefully fixed the root cause, but I forgot to unskip the skipped tests

adelisle pushed a commit to Makila-AI/kibana that referenced this issue Aug 5, 2024
…stic#188646)

# Backport

This will backport the following commits from `main` to `7.17`:
- [Migrations: Don't auto-create temp index
(elastic#158182)](elastic#158182)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Rudolf
Meijering","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-06-04T22:34:08Z","message":"Migrations:
Don't auto-create temp index (elastic#158182)\n\n## Summary\r\n\r\nTry to
fix\r\nhttps://github.com/elastic/issues/156117#issuecomment-1557029863\r\n\r\n##
Release notes\r\nFixes a race condition that could cause intermittent
upgrade migration\r\nfailures when Kibana connects to a single node
Elasticsearch cluster.\r\n\r\n### Checklist\r\n\r\nDelete any items that
are not applicable to this PR.\r\n\r\n- [ ] Any text added follows
[EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [ ] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] Any UI
touched in this PR is usable by keyboard only (learn more\r\nabout
[keyboard accessibility](https://webaim.org/techniques/keyboard/))\r\n-
[ ] Any UI touched in this PR does not create any new axe
failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
Risk Matrix\r\n\r\nDelete this section if it is not applicable to this
PR.\r\n\r\nBefore closing this PR, invite QA, stakeholders, and other
developers to\r\nidentify risks that should be tested prior to the
change/feature\r\nrelease.\r\n\r\nWhen forming the risk matrix, consider
some of the following examples\r\nand how they may potentially impact
the change:\r\n\r\n| Risk | Probability | Severity | Mitigation/Notes
|\r\n\r\n|---------------------------|-------------|----------|-------------------------|\r\n|
Multiple Spaces&mdash;unexpected behavior in non-default Kibana
Space.\r\n| Low | High | Integration tests will verify that all features
are still\r\nsupported in non-default Kibana Space and when user
switches between\r\nspaces. |\r\n| Multiple nodes&mdash;Elasticsearch
polling might have race conditions\r\nwhen multiple Kibana nodes are
polling for the same tasks. | High | Low\r\n| Tasks are idempotent, so
executing them multiple times will not result\r\nin logical error, but
will degrade performance. To test for this case we\r\nadd plenty of unit
tests around this logic and document manual testing\r\nprocedure. |\r\n|
Code should gracefully handle cases when feature X or plugin Y
are\r\ndisabled. | Medium | High | Unit tests will verify that any
feature flag\r\nor plugin combination still results in our service
operational. |\r\n| [See more potential
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
|\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for
breaking API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"8e7e2632bbb5f2dfffe8ab6c2563e176c5d7cf6b","branchLabelMapping":{"^v8.9.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","Team:Core","release_note:fix","Feature:Migrations","backport:prev-minor","backport:prev-MAJOR","v8.9.0","v8.8.1","v8.16.0"],"number":158182,"url":"https://github.com/elastic/kibana/pull/158182","mergeCommit":{"message":"Migrations:
Don't auto-create temp index (elastic#158182)\n\n## Summary\r\n\r\nTry to
fix\r\nhttps://github.com/elastic/issues/156117#issuecomment-1557029863\r\n\r\n##
Release notes\r\nFixes a race condition that could cause intermittent
upgrade migration\r\nfailures when Kibana connects to a single node
Elasticsearch cluster.\r\n\r\n### Checklist\r\n\r\nDelete any items that
are not applicable to this PR.\r\n\r\n- [ ] Any text added follows
[EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [ ] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] Any UI
touched in this PR is usable by keyboard only (learn more\r\nabout
[keyboard accessibility](https://webaim.org/techniques/keyboard/))\r\n-
[ ] Any UI touched in this PR does not create any new axe
failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
Risk Matrix\r\n\r\nDelete this section if it is not applicable to this
PR.\r\n\r\nBefore closing this PR, invite QA, stakeholders, and other
developers to\r\nidentify risks that should be tested prior to the
change/feature\r\nrelease.\r\n\r\nWhen forming the risk matrix, consider
some of the following examples\r\nand how they may potentially impact
the change:\r\n\r\n| Risk | Probability | Severity | Mitigation/Notes
|\r\n\r\n|---------------------------|-------------|----------|-------------------------|\r\n|
Multiple Spaces&mdash;unexpected behavior in non-default Kibana
Space.\r\n| Low | High | Integration tests will verify that all features
are still\r\nsupported in non-default Kibana Space and when user
switches between\r\nspaces. |\r\n| Multiple nodes&mdash;Elasticsearch
polling might have race conditions\r\nwhen multiple Kibana nodes are
polling for the same tasks. | High | Low\r\n| Tasks are idempotent, so
executing them multiple times will not result\r\nin logical error, but
will degrade performance. To test for this case we\r\nadd plenty of unit
tests around this logic and document manual testing\r\nprocedure. |\r\n|
Code should gracefully handle cases when feature X or plugin Y
are\r\ndisabled. | Medium | High | Unit tests will verify that any
feature flag\r\nor plugin combination still results in our service
operational. |\r\n| [See more potential
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
|\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for
breaking API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"8e7e2632bbb5f2dfffe8ab6c2563e176c5d7cf6b"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.9.0","labelRegex":"^v8.9.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/158182","number":158182,"mergeCommit":{"message":"Migrations:
Don't auto-create temp index (elastic#158182)\n\n## Summary\r\n\r\nTry to
fix\r\nhttps://github.com/elastic/issues/156117#issuecomment-1557029863\r\n\r\n##
Release notes\r\nFixes a race condition that could cause intermittent
upgrade migration\r\nfailures when Kibana connects to a single node
Elasticsearch cluster.\r\n\r\n### Checklist\r\n\r\nDelete any items that
are not applicable to this PR.\r\n\r\n- [ ] Any text added follows
[EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [ ] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] Any UI
touched in this PR is usable by keyboard only (learn more\r\nabout
[keyboard accessibility](https://webaim.org/techniques/keyboard/))\r\n-
[ ] Any UI touched in this PR does not create any new axe
failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
Risk Matrix\r\n\r\nDelete this section if it is not applicable to this
PR.\r\n\r\nBefore closing this PR, invite QA, stakeholders, and other
developers to\r\nidentify risks that should be tested prior to the
change/feature\r\nrelease.\r\n\r\nWhen forming the risk matrix, consider
some of the following examples\r\nand how they may potentially impact
the change:\r\n\r\n| Risk | Probability | Severity | Mitigation/Notes
|\r\n\r\n|---------------------------|-------------|----------|-------------------------|\r\n|
Multiple Spaces&mdash;unexpected behavior in non-default Kibana
Space.\r\n| Low | High | Integration tests will verify that all features
are still\r\nsupported in non-default Kibana Space and when user
switches between\r\nspaces. |\r\n| Multiple nodes&mdash;Elasticsearch
polling might have race conditions\r\nwhen multiple Kibana nodes are
polling for the same tasks. | High | Low\r\n| Tasks are idempotent, so
executing them multiple times will not result\r\nin logical error, but
will degrade performance. To test for this case we\r\nadd plenty of unit
tests around this logic and document manual testing\r\nprocedure. |\r\n|
Code should gracefully handle cases when feature X or plugin Y
are\r\ndisabled. | Medium | High | Unit tests will verify that any
feature flag\r\nor plugin combination still results in our service
operational. |\r\n| [See more potential
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
|\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for
breaking API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"8e7e2632bbb5f2dfffe8ab6c2563e176c5d7cf6b"}},{"branch":"8.8","label":"v8.8.1","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/158996","number":158996,"state":"MERGED","mergeCommit":{"sha":"46217b9129c9de9d7be064b6ad6d6f51897981e0","message":"[8.8]
Migrations: Don't auto-create temp index (elastic#158182) (elastic#158996)\n\n#
Backport\n\nThis will backport the following commits from `main` to
`8.8`:\n- [Migrations: Don't auto-create temp
index\n(elastic#158182)](https://github.com/elastic/kibana/pull/158182)\n\n<!---
Backport version: 8.9.7 -->\n\n### Questions ?\nPlease refer to the
[Backport
tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT
[{\"author\":{\"name\":\"Rudolf\nMeijering\",\"email\":\"[email protected]\"},\"sourceCommit\":{\"committedDate\":\"2023-06-04T22:34:08Z\",\"message\":\"Migrations:\nDon't
auto-create temp index (elastic#158182)\\n\\n## Summary\\r\\n\\r\\nTry
to\nfix\\r\\nhttps://github.com/elastic/issues/156117#issuecomment-1557029863\\r\\n\\r\\n##\nRelease
notes\\r\\nFixes a race condition that could cause intermittent\nupgrade
migration\\r\\nfailures when Kibana connects to a single
node\nElasticsearch cluster.\\r\\n\\r\\n### Checklist\\r\\n\\r\\nDelete
any items that\nare not applicable to this PR.\\r\\n\\r\\n- [ ] Any text
added
follows\n[EUI's\nwriting\\r\\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),\nuses\\r\\nsentence
case text and
includes\n[i18n\\r\\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\\r\\n-\n[\n]\\r\\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\\r\\nwas\nadded
for features that require explanation or tutorials\\r\\n- [ ]
[Unit\nor\nfunctional\\r\\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\\r\\nwere\nupdated
or added to match the most common scenarios\\r\\n- [ ] Any UI\ntouched
in this PR is usable by keyboard only (learn more\\r\\nabout\n[keyboard
accessibility](https://webaim.org/techniques/keyboard/))\\r\\n-\n[ ] Any
UI touched in this PR does not create any new axe\nfailures\\r\\n(run
axe
in\nbrowser:\\r\\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\\r\\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\\r\\n-\n[
] If a plugin configuration key changed, check if it needs
to\nbe\\r\\nallowlisted in the cloud and added to
the\n[docker\\r\\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\\r\\n-\n[
] This renders correctly on smaller devices using
a\nresponsive\\r\\nlayout. (You can test this
[in\nyour\\r\\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\\r\\n-\n[
] This was checked
for\n[cross-browser\\r\\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\\r\\n\\r\\n\\r\\n###\nRisk
Matrix\\r\\n\\r\\nDelete this section if it is not applicable to
this\nPR.\\r\\n\\r\\nBefore closing this PR, invite QA, stakeholders,
and other\ndevelopers to\\r\\nidentify risks that should be tested prior
to the\nchange/feature\\r\\nrelease.\\r\\n\\r\\nWhen forming the risk
matrix, consider\nsome of the following examples\\r\\nand how they may
potentially impact\nthe change:\\r\\n\\r\\n| Risk | Probability |
Severity |
Mitigation/Notes\n|\\r\\n\\r\\n|---------------------------|-------------|----------|-------------------------|\\r\\n|\nMultiple
Spaces&mdash;unexpected behavior in non-default Kibana\nSpace.\\r\\n|
Low | High | Integration tests will verify that all features\nare
still\\r\\nsupported in non-default Kibana Space and when user\nswitches
between\\r\\nspaces. |\\r\\n| Multiple
nodes&mdash;Elasticsearch\npolling might have race conditions\\r\\nwhen
multiple Kibana nodes are\npolling for the same tasks. | High |
Low\\r\\n| Tasks are idempotent, so\nexecuting them multiple times will
not result\\r\\nin logical error, but\nwill degrade performance. To test
for this case we\\r\\nadd plenty of unit\ntests around this logic and
document manual testing\\r\\nprocedure. |\\r\\n|\nCode should gracefully
handle cases when feature X or plugin Y\nare\\r\\ndisabled. | Medium |
High | Unit tests will verify that any\nfeature flag\\r\\nor plugin
combination still results in our service\noperational. |\\r\\n| [See
more
potential\nrisk\\r\\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n|\\r\\n\\r\\n\\r\\n###
For maintainers\\r\\n\\r\\n- [ ] This was checked for\nbreaking API
changes and
was\n[labeled\\r\\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\",\"sha\":\"8e7e2632bbb5f2dfffe8ab6c2563e176c5d7cf6b\",\"branchLabelMapping\":{\"^v8.9.0$\":\"main\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"bug\",\"Team:Core\",\"release_note:fix\",\"Feature:Migrations\",\"backport:prev-minor\",\"v8.9.0\"],\"number\":158182,\"url\":\"https://github.com/elastic/kibana/pull/158182\",\"mergeCommit\":{\"message\":\"Migrations:\nDon't
auto-create temp index (elastic#158182)\\n\\n## Summary\\r\\n\\r\\nTry
to\nfix\\r\\nhttps://github.com/elastic/issues/156117#issuecomment-1557029863\\r\\n\\r\\n##\nRelease
notes\\r\\nFixes a race condition that could cause intermittent\nupgrade
migration\\r\\nfailures when Kibana connects to a single
node\nElasticsearch cluster.\\r\\n\\r\\n### Checklist\\r\\n\\r\\nDelete
any items that\nare not applicable to this PR.\\r\\n\\r\\n- [ ] Any text
added
follows\n[EUI's\nwriting\\r\\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),\nuses\\r\\nsentence
case text and
includes\n[i18n\\r\\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\\r\\n-\n[\n]\\r\\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\\r\\nwas\nadded
for features that require explanation or tutorials\\r\\n- [ ]
[Unit\nor\nfunctional\\r\\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\\r\\nwere\nupdated
or added to match the most common scenarios\\r\\n- [ ] Any UI\ntouched
in this PR is usable by keyboard only (learn more\\r\\nabout\n[keyboard
accessibility](https://webaim.org/techniques/keyboard/))\\r\\n-\n[ ] Any
UI touched in this PR does not create any new axe\nfailures\\r\\n(run
axe
in\nbrowser:\\r\\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\\r\\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\\r\\n-\n[
] If a plugin configuration key changed, check if it needs
to\nbe\\r\\nallowlisted in the cloud and added to
the\n[docker\\r\\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\\r\\n-\n[
] This renders correctly on smaller devices using
a\nresponsive\\r\\nlayout. (You can test this
[in\nyour\\r\\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\\r\\n-\n[
] This was checked
for\n[cross-browser\\r\\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\\r\\n\\r\\n\\r\\n###\nRisk
Matrix\\r\\n\\r\\nDelete this section if it is not applicable to
this\nPR.\\r\\n\\r\\nBefore closing this PR, invite QA, stakeholders,
and other\ndevelopers to\\r\\nidentify risks that should be tested prior
to the\nchange/feature\\r\\nrelease.\\r\\n\\r\\nWhen forming the risk
matrix, consider\nsome of the following examples\\r\\nand how they may
potentially impact\nthe change:\\r\\n\\r\\n| Risk | Probability |
Severity |
Mitigation/Notes\n|\\r\\n\\r\\n|---------------------------|-------------|----------|-------------------------|\\r\\n|\nMultiple
Spaces&mdash;unexpected behavior in non-default Kibana\nSpace.\\r\\n|
Low | High | Integration tests will verify that all features\nare
still\\r\\nsupported in non-default Kibana Space and when user\nswitches
between\\r\\nspaces. |\\r\\n| Multiple
nodes&mdash;Elasticsearch\npolling might have race conditions\\r\\nwhen
multiple Kibana nodes are\npolling for the same tasks. | High |
Low\\r\\n| Tasks are idempotent, so\nexecuting them multiple times will
not result\\r\\nin logical error, but\nwill degrade performance. To test
for this case we\\r\\nadd plenty of unit\ntests around this logic and
document manual testing\\r\\nprocedure. |\\r\\n|\nCode should gracefully
handle cases when feature X or plugin Y\nare\\r\\ndisabled. | Medium |
High | Unit tests will verify that any\nfeature flag\\r\\nor plugin
combination still results in our service\noperational. |\\r\\n| [See
more
potential\nrisk\\r\\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n|\\r\\n\\r\\n\\r\\n###
For maintainers\\r\\n\\r\\n- [ ] This was checked for\nbreaking API
changes and
was\n[labeled\\r\\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\",\"sha\":\"8e7e2632bbb5f2dfffe8ab6c2563e176c5d7cf6b\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[],\"targetPullRequestStates\":[{\"branch\":\"main\",\"label\":\"v8.9.0\",\"labelRegex\":\"^v8.9.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/158182\",\"number\":158182,\"mergeCommit\":{\"message\":\"Migrations:\nDon't
auto-create temp index (elastic#158182)\\n\\n## Summary\\r\\n\\r\\nTry
to\nfix\\r\\nhttps://github.com/elastic/issues/156117#issuecomment-1557029863\\r\\n\\r\\n##\nRelease
notes\\r\\nFixes a race condition that could cause intermittent\nupgrade
migration\\r\\nfailures when Kibana connects to a single
node\nElasticsearch cluster.\\r\\n\\r\\n### Checklist\\r\\n\\r\\nDelete
any items that\nare not applicable to this PR.\\r\\n\\r\\n- [ ] Any text
added
follows\n[EUI's\nwriting\\r\\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),\nuses\\r\\nsentence
case text and
includes\n[i18n\\r\\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\\r\\n-\n[\n]\\r\\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\\r\\nwas\nadded
for features that require explanation or tutorials\\r\\n- [ ]
[Unit\nor\nfunctional\\r\\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\\r\\nwere\nupdated
or added to match the most common scenarios\\r\\n- [ ] Any UI\ntouched
in this PR is usable by keyboard only (learn more\\r\\nabout\n[keyboard
accessibility](https://webaim.org/techniques/keyboard/))\\r\\n-\n[ ] Any
UI touched in this PR does not create any new axe\nfailures\\r\\n(run
axe
in\nbrowser:\\r\\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\\r\\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\\r\\n-\n[
] If a plugin configuration key changed, check if it needs
to\nbe\\r\\nallowlisted in the cloud and added to
the\n[docker\\r\\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\\r\\n-\n[
] This renders correctly on smaller devices using
a\nresponsive\\r\\nlayout. (You can test this
[in\nyour\\r\\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\\r\\n-\n[
] This was checked
for\n[cross-browser\\r\\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\\r\\n\\r\\n\\r\\n###\nRisk
Matrix\\r\\n\\r\\nDelete this section if it is not applicable to
this\nPR.\\r\\n\\r\\nBefore closing this PR, invite QA, stakeholders,
and other\ndevelopers to\\r\\nidentify risks that should be tested prior
to the\nchange/feature\\r\\nrelease.\\r\\n\\r\\nWhen forming the risk
matrix, consider\nsome of the following examples\\r\\nand how they may
potentially impact\nthe change:\\r\\n\\r\\n| Risk | Probability |
Severity |
Mitigation/Notes\n|\\r\\n\\r\\n|---------------------------|-------------|----------|-------------------------|\\r\\n|\nMultiple
Spaces&mdash;unexpected behavior in non-default Kibana\nSpace.\\r\\n|
Low | High | Integration tests will verify that all features\nare
still\\r\\nsupported in non-default Kibana Space and when user\nswitches
between\\r\\nspaces. |\\r\\n| Multiple
nodes&mdash;Elasticsearch\npolling might have race conditions\\r\\nwhen
multiple Kibana nodes are\npolling for the same tasks. | High |
Low\\r\\n| Tasks are idempotent, so\nexecuting them multiple times will
not result\\r\\nin logical error, but\nwill degrade performance. To test
for this case we\\r\\nadd plenty of unit\ntests around this logic and
document manual testing\\r\\nprocedure. |\\r\\n|\nCode should gracefully
handle cases when feature X or plugin Y\nare\\r\\ndisabled. | Medium |
High | Unit tests will verify that any\nfeature flag\\r\\nor plugin
combination still results in our service\noperational. |\\r\\n| [See
more
potential\nrisk\\r\\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)\n|\\r\\n\\r\\n\\r\\n###
For maintainers\\r\\n\\r\\n- [ ] This was checked for\nbreaking API
changes and
was\n[labeled\\r\\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\",\"sha\":\"8e7e2632bbb5f2dfffe8ab6c2563e176c5d7cf6b\"}}]}]\nBACKPORT-->\n\nCo-authored-by:
Rudolf Meijering
<[email protected]>"}},{"branch":"8.16","label":"v8.16.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
@kibanamachine kibanamachine reopened this Aug 8, 2024
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.15

@afharo
Copy link
Member

afharo commented Aug 8, 2024

The latest error is caused by Kibana taking too long start the 3 test nodes. I'll consider it a one-off for now.

@afharo afharo closed this as not planned Won't fix, can't repro, duplicate, stale Aug 8, 2024
@kibanamachine kibanamachine reopened this Aug 12, 2024
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@mistic
Copy link
Member

mistic commented Aug 29, 2024

Skipped.

main: ed2e81e

@gsoldevila
Copy link
Contributor

This test has been removed / replaced through #193699, #193328, #193856, #193696

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.15

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc
Projects
None yet
9 participants