Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fleet] The agent upgrade downloading state does not show download details or metadata #171943

Closed
cmacknz opened this issue Nov 24, 2023 · 7 comments · Fixed by #173614
Closed
Assignees
Labels
Team:Fleet Team label for Observability Data Collection Fleet team

Comments

@cmacknz
Copy link
Member

cmacknz commented Nov 24, 2023

The upgrade downloading state only shows that the agent is in the downloading state, even though the agent can and will report additional context such as the download rate and completion percent.

Here is an example of the full upgrade details reported by the agent:

{
  "log.level": "info",
  "@timestamp": "2023-11-24T16:18:26.540Z",
  "log.origin": {
    "file.name": "coordinator/coordinator.go",
    "file.line": 500
  },
  "message": "updated upgrade details",
  "log": {
    "source": "elastic-agent"
  },
  "upgrade_details": {
    "target_version": "8.11.1",
    "state": "UPG_DOWNLOADING",
    "action_id": "aa72eec8-90b0-4889-b329-8472f5ee9e1d",
    "metadata": {
      "download_percent": 1,
      "download_rate": "16.87223912MBps"
    }
  },
  "ecs.version": "1.6.0"
}

The download metadata does not appear in the agent document in Fleet, it is not clear if this is a problem in what the agent sends in the checkin, in what Fleet server writes to the .fleet-agents index, or in Fleet itself:

Screenshot 2023-11-24 at 2 56 48 PM

Fleet only shows the agent as downloading with no other context, this is a UI presentation issue

Screenshot 2023-11-24 at 2 31 50 PM

We should include the metadata so users can find it. A core use case for the downloading state is to enable users to detect when an agent is failing because of a slow connection and right now that is not easy to observe. Possibly we could include this information in the agent details view, see #171942

@cmacknz cmacknz added the Team:Fleet Team label for Observability Data Collection Fleet team label Nov 24, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@cmacknz cmacknz changed the title [Fleet] The agent upgrade downloading state does not show all available information [Fleet] The agent upgrade downloading state does not show download details or metadata Nov 24, 2023
@cmacknz
Copy link
Member Author

cmacknz commented Nov 24, 2023

The download metadata does not appear in the agent document in Fleet, it is not clear if this is a problem in what the agent sends in the checkin, in what Fleet server writes to the .fleet-agents index, or in Fleet itself:

This problem may be true for all upgrade states, not just downloading. We could be omitting some fields from all of the states. This could be from the agent and Fleet API definitions of upgrade details are drifting out of sync.

CC @ycombinator

@juliaElastic juliaElastic self-assigned this Dec 19, 2023
@juliaElastic
Copy link
Contributor

juliaElastic commented Dec 19, 2023

@cmacknz I couldn't reproduce this issue, is there a way to upgrade from 8.12-snapshot to 8.13-snapshot to trigger download? Is there a working agent binary download source with snapshot versions?

I created an agent document manually with metadata.download_percent and the percent shows up in the tooltip:
image

The download_rate is not used by the UI, do we want to display that in the tooltip?
It was not part of the original design, that's why it's missing.

The download_rate seems to be a float (bytes per second), that should be formatted on the UI.

Also, do we want to show 0% and 0 Bps if the values are set to 0?

juliaElastic added a commit that referenced this issue Dec 19, 2023
## Summary

Closes #171943

Showing download rate in the upgrade details tooltip.

Used fake data as I couldn't get an actual agent to be in downloading
state with download percent and rate.

<img width="659" alt="image"
src="https://github.com/elastic/kibana/assets/90178898/1488fa7f-b003-4d5a-a98a-27fcc8a4c70c">


Insert test ES data with curl and go to Agent list/details to see the
tooltip, replace `existing_agent_id` with an existing agent's id or
insert a full agent doc.
```
curl -sk -XPOST --user elastic:changeme -H 'content-type:application/json' \
http://localhost:9200/_security/role/fleet_superuser -d '
   {
      "indices": [
            {
               "names": [".fleet*",".kibana*"],
               "privileges": ["all"],
               "allow_restricted_indices": true
            }
      ]
   }'

curl -sk -XPOST --user elastic:changeme -H 'content-type:application/json' \
http://localhost:9200/_security/user/fleet_superuser -d '
   {
      "password": "password",
      "roles": ["superuser", "fleet_superuser"]
   }'

   curl -sk -XPOST --user fleet_superuser:password -H 'content-type:application/json' \
   -H'x-elastic-product-origin:fleet' \
   http://localhost:9200/.fleet-agents/_update_by_query -d '
   {
      "script": {
        "source": "ctx._source.upgrade_details.state = \"UPG_DOWNLOADING\"; ctx._source.upgrade_details.metadata.download_percent = 22; ctx._source.upgrade_details.metadata.download_rate = 1223912;",
        "lang": "painless"
      },
      "query": {
        "term": {
          "agent.id":"existing_agent_id"
        }
      }
    }'

```

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Dec 19, 2023
## Summary

Closes elastic#171943

Showing download rate in the upgrade details tooltip.

Used fake data as I couldn't get an actual agent to be in downloading
state with download percent and rate.

<img width="659" alt="image"
src="https://github.com/elastic/kibana/assets/90178898/1488fa7f-b003-4d5a-a98a-27fcc8a4c70c">

Insert test ES data with curl and go to Agent list/details to see the
tooltip, replace `existing_agent_id` with an existing agent's id or
insert a full agent doc.
```
curl -sk -XPOST --user elastic:changeme -H 'content-type:application/json' \
http://localhost:9200/_security/role/fleet_superuser -d '
   {
      "indices": [
            {
               "names": [".fleet*",".kibana*"],
               "privileges": ["all"],
               "allow_restricted_indices": true
            }
      ]
   }'

curl -sk -XPOST --user elastic:changeme -H 'content-type:application/json' \
http://localhost:9200/_security/user/fleet_superuser -d '
   {
      "password": "password",
      "roles": ["superuser", "fleet_superuser"]
   }'

   curl -sk -XPOST --user fleet_superuser:password -H 'content-type:application/json' \
   -H'x-elastic-product-origin:fleet' \
   http://localhost:9200/.fleet-agents/_update_by_query -d '
   {
      "script": {
        "source": "ctx._source.upgrade_details.state = \"UPG_DOWNLOADING\"; ctx._source.upgrade_details.metadata.download_percent = 22; ctx._source.upgrade_details.metadata.download_rate = 1223912;",
        "lang": "painless"
      },
      "query": {
        "term": {
          "agent.id":"existing_agent_id"
        }
      }
    }'

```

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

(cherry picked from commit b9d3c86)
@cmacknz
Copy link
Member Author

cmacknz commented Dec 19, 2023

Also, do we want to show 0% and 0 Bps if the values are set to 0?

Yes because it's possible the agent is trying to download but on a connection that can't make progress.

@cmacknz
Copy link
Member Author

cmacknz commented Dec 19, 2023

Download rate is useful to show, I added it as an addition in #173370 so it can be dealt with at the same time.

@juliaElastic
Copy link
Contributor

Also, do we want to show 0% and 0 Bps if the values are set to 0?

Yes because it's possible the agent is trying to download but on a connection that can't make progress.

Added the display of download_rate including 0 values to the UI in this pr: #173614

@cmacknz
Copy link
Member Author

cmacknz commented Dec 19, 2023

Thanks!

kibanamachine referenced this issue Dec 21, 2023
… (#173651)

# Backport

This will backport the following commits from `main` to `8.12`:
- [[Fleet] show download rate in upgrade details tooltlip
(#173614)](#173614)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Julia
Bardi","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-12-19T15:07:25Z","message":"[Fleet]
show download rate in upgrade details tooltlip (#173614)\n\n##
Summary\r\n\r\nCloses
https://github.com/elastic/kibana/issues/171943\r\n\r\nShowing download
rate in the upgrade details tooltip.\r\n\r\nUsed fake data as I couldn't
get an actual agent to be in downloading\r\nstate with download percent
and rate.\r\n\r\n<img width=\"659\"
alt=\"image\"\r\nsrc=\"https://github.com/elastic/kibana/assets/90178898/1488fa7f-b003-4d5a-a98a-27fcc8a4c70c\">\r\n\r\n\r\nInsert
test ES data with curl and go to Agent list/details to see
the\r\ntooltip, replace `existing_agent_id` with an existing agent's id
or\r\ninsert a full agent doc.\r\n```\r\ncurl -sk -XPOST --user
elastic:changeme -H 'content-type:application/json'
\\\r\nhttp://localhost:9200/_security/role/fleet_superuser -d '\r\n
{\r\n \"indices\": [\r\n {\r\n \"names\":
[\".fleet*\",\".kibana*\"],\r\n \"privileges\": [\"all\"],\r\n
\"allow_restricted_indices\": true\r\n }\r\n ]\r\n }'\r\n\r\ncurl -sk
-XPOST --user elastic:changeme -H 'content-type:application/json'
\\\r\nhttp://localhost:9200/_security/user/fleet_superuser -d '\r\n
{\r\n \"password\": \"password\",\r\n \"roles\": [\"superuser\",
\"fleet_superuser\"]\r\n }'\r\n\r\n curl -sk -XPOST --user
fleet_superuser:password -H 'content-type:application/json' \\\r\n
-H'x-elastic-product-origin:fleet' \\\r\n
http://localhost:9200/.fleet-agents/_update_by_query -d '\r\n {\r\n
\"script\": {\r\n \"source\": \"ctx._source.upgrade_details.state =
\\\"UPG_DOWNLOADING\\\";
ctx._source.upgrade_details.metadata.download_percent = 22;
ctx._source.upgrade_details.metadata.download_rate = 1223912;\",\r\n
\"lang\": \"painless\"\r\n },\r\n \"query\": {\r\n \"term\": {\r\n
\"agent.id\":\"existing_agent_id\"\r\n }\r\n }\r\n
}'\r\n\r\n```\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios","sha":"b9d3c8611876632c587ca854125315c6a5c303bb","branchLabelMapping":{"^v8.13.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Fleet","v8.12.0","v8.13.0"],"number":173614,"url":"https://github.com/elastic/kibana/pull/173614","mergeCommit":{"message":"[Fleet]
show download rate in upgrade details tooltlip (#173614)\n\n##
Summary\r\n\r\nCloses
https://github.com/elastic/kibana/issues/171943\r\n\r\nShowing download
rate in the upgrade details tooltip.\r\n\r\nUsed fake data as I couldn't
get an actual agent to be in downloading\r\nstate with download percent
and rate.\r\n\r\n<img width=\"659\"
alt=\"image\"\r\nsrc=\"https://github.com/elastic/kibana/assets/90178898/1488fa7f-b003-4d5a-a98a-27fcc8a4c70c\">\r\n\r\n\r\nInsert
test ES data with curl and go to Agent list/details to see
the\r\ntooltip, replace `existing_agent_id` with an existing agent's id
or\r\ninsert a full agent doc.\r\n```\r\ncurl -sk -XPOST --user
elastic:changeme -H 'content-type:application/json'
\\\r\nhttp://localhost:9200/_security/role/fleet_superuser -d '\r\n
{\r\n \"indices\": [\r\n {\r\n \"names\":
[\".fleet*\",\".kibana*\"],\r\n \"privileges\": [\"all\"],\r\n
\"allow_restricted_indices\": true\r\n }\r\n ]\r\n }'\r\n\r\ncurl -sk
-XPOST --user elastic:changeme -H 'content-type:application/json'
\\\r\nhttp://localhost:9200/_security/user/fleet_superuser -d '\r\n
{\r\n \"password\": \"password\",\r\n \"roles\": [\"superuser\",
\"fleet_superuser\"]\r\n }'\r\n\r\n curl -sk -XPOST --user
fleet_superuser:password -H 'content-type:application/json' \\\r\n
-H'x-elastic-product-origin:fleet' \\\r\n
http://localhost:9200/.fleet-agents/_update_by_query -d '\r\n {\r\n
\"script\": {\r\n \"source\": \"ctx._source.upgrade_details.state =
\\\"UPG_DOWNLOADING\\\";
ctx._source.upgrade_details.metadata.download_percent = 22;
ctx._source.upgrade_details.metadata.download_rate = 1223912;\",\r\n
\"lang\": \"painless\"\r\n },\r\n \"query\": {\r\n \"term\": {\r\n
\"agent.id\":\"existing_agent_id\"\r\n }\r\n }\r\n
}'\r\n\r\n```\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios","sha":"b9d3c8611876632c587ca854125315c6a5c303bb"}},"sourceBranch":"main","suggestedTargetBranches":["8.12"],"targetPullRequestStates":[{"branch":"8.12","label":"v8.12.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.13.0","labelRegex":"^v8.13.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/173614","number":173614,"mergeCommit":{"message":"[Fleet]
show download rate in upgrade details tooltlip (#173614)\n\n##
Summary\r\n\r\nCloses
https://github.com/elastic/kibana/issues/171943\r\n\r\nShowing download
rate in the upgrade details tooltip.\r\n\r\nUsed fake data as I couldn't
get an actual agent to be in downloading\r\nstate with download percent
and rate.\r\n\r\n<img width=\"659\"
alt=\"image\"\r\nsrc=\"https://github.com/elastic/kibana/assets/90178898/1488fa7f-b003-4d5a-a98a-27fcc8a4c70c\">\r\n\r\n\r\nInsert
test ES data with curl and go to Agent list/details to see
the\r\ntooltip, replace `existing_agent_id` with an existing agent's id
or\r\ninsert a full agent doc.\r\n```\r\ncurl -sk -XPOST --user
elastic:changeme -H 'content-type:application/json'
\\\r\nhttp://localhost:9200/_security/role/fleet_superuser -d '\r\n
{\r\n \"indices\": [\r\n {\r\n \"names\":
[\".fleet*\",\".kibana*\"],\r\n \"privileges\": [\"all\"],\r\n
\"allow_restricted_indices\": true\r\n }\r\n ]\r\n }'\r\n\r\ncurl -sk
-XPOST --user elastic:changeme -H 'content-type:application/json'
\\\r\nhttp://localhost:9200/_security/user/fleet_superuser -d '\r\n
{\r\n \"password\": \"password\",\r\n \"roles\": [\"superuser\",
\"fleet_superuser\"]\r\n }'\r\n\r\n curl -sk -XPOST --user
fleet_superuser:password -H 'content-type:application/json' \\\r\n
-H'x-elastic-product-origin:fleet' \\\r\n
http://localhost:9200/.fleet-agents/_update_by_query -d '\r\n {\r\n
\"script\": {\r\n \"source\": \"ctx._source.upgrade_details.state =
\\\"UPG_DOWNLOADING\\\";
ctx._source.upgrade_details.metadata.download_percent = 22;
ctx._source.upgrade_details.metadata.download_rate = 1223912;\",\r\n
\"lang\": \"painless\"\r\n },\r\n \"query\": {\r\n \"term\": {\r\n
\"agent.id\":\"existing_agent_id\"\r\n }\r\n }\r\n
}'\r\n\r\n```\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common
scenarios","sha":"b9d3c8611876632c587ca854125315c6a5c303bb"}}]}]
BACKPORT-->

Co-authored-by: Julia Bardi <[email protected]>
Co-authored-by: Julien Lind <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Fleet Team label for Observability Data Collection Fleet team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants