Skip to content

Commit

Permalink
[Fleet] Feature to support columns when exporting agents to CSV (elas…
Browse files Browse the repository at this point in the history
…tic#203103)

## Summary

Closes elastic/ingest-dev#4325

Added modal window with column selector table when agents are exported.

There are some differences compared to the design due to technical
limitations:
- `Filters applied` badge not included, it's not really possible to
accurately calculate the count of filters, since they are stored in a
single string. We could potentially count by splitting AND/OR
conditions, but it may not be accurate. Do we still want to include it?
- We don't have display names for these mappings coming from the agent
index, and currently I don't have a way to show display names on the
exported file. For this reason showing the original field names on the
modal too.
- Added a Description column that transforms the field name to a more
readable name, we can also hardcode the descriptions if this is not good
enough.
- After some consideration, I decided to hardcode the allowed field list
to export other than querying dynamically from the agent index mappings,
otherwise new sensitive field mappings would show up on the UI.
- Caveat: searching on columns in the modal removes the already selected
columns that do not match. This seems to be a default behaviour of the
EuiTable, I find it a little strange, didn't see a straightforward way
to change it. I can spend more time on it if needed.

To verify:
- Select a few agents, click on Export CSV action
- Verify that the modal window opens with the agent mappings visible, by
default the columns on the UI selected
- Select a few columns to export
- Submit the modal
- Wait for the report to be ready and download it
- Verify that the exported csv includes the columns selected

<img width="1301" alt="image"
src="https://github.com/user-attachments/assets/71b549a7-a316-45d4-b017-f5eedec6ea7e">


Figma design:

![image](https://github.com/user-attachments/assets/bcf347a7-a68e-4f83-8f6b-37bdf43c6b54)

<img width="2138" alt="image"
src="https://github.com/user-attachments/assets/77ffc5d7-343f-4d5c-8b65-251e1cac94d4">


### Checklist

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
  • Loading branch information
juliaElastic authored Dec 10, 2024
1 parent 7f3c642 commit 6e145f9
Show file tree
Hide file tree
Showing 6 changed files with 408 additions and 21 deletions.
2 changes: 1 addition & 1 deletion x-pack/plugins/fleet/common/experimental_features.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ const _allowedExperimentalValues = {
useSpaceAwareness: false,
enableReusableIntegrationPolicies: true,
asyncDeployPolicies: true,
enableExportCSV: false,
enableExportCSV: true,
};

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ import { AgentRequestDiagnosticsModal } from '../../components/agent_request_dia

import { useExportCSV } from '../hooks/export_csv';

import { AgentExportCSVModal } from '../../components/agent_export_csv_modal';

import type { SelectionMode } from './types';
import { TagsAddRemove } from './tags_add_remove';

Expand Down Expand Up @@ -82,6 +84,7 @@ export const AgentBulkActions: React.FunctionComponent<Props> = ({
const [isTagAddVisible, setIsTagAddVisible] = useState<boolean>(false);
const [isRequestDiagnosticsModalOpen, setIsRequestDiagnosticsModalOpen] =
useState<boolean>(false);
const [isExportCSVModalOpen, setIsExportCSVModalOpen] = useState<boolean>(false);

// update the query removing the "managed" agents in any state (unenrolled, offline, etc)
const selectionQuery = useMemo(() => {
Expand Down Expand Up @@ -241,10 +244,7 @@ export const AgentBulkActions: React.FunctionComponent<Props> = ({
icon: <EuiIcon type="exportAction" size="m" />,
onClick: () => {
closeMenu();
generateReportingJobCSV(agents, {
field: sortField,
direction: sortOrder,
});
setIsExportCSVModalOpen(true);
},
},
]
Expand Down Expand Up @@ -288,6 +288,23 @@ export const AgentBulkActions: React.FunctionComponent<Props> = ({
/>
</EuiPortal>
)}
{isExportCSVModalOpen && (
<EuiPortal>
<AgentExportCSVModal
onSubmit={(columns: Array<{ field: string }>) => {
generateReportingJobCSV(agents, columns, {
field: sortField,
direction: sortOrder,
});
setIsExportCSVModalOpen(false);
}}
onClose={() => {
setIsExportCSVModalOpen(false);
}}
agentCount={agentCount}
/>
</EuiPortal>
)}
{upgradeModalState.isOpen && (
<EuiPortal>
<AgentUpgradeAgentModal
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,12 +62,13 @@ describe('export_csv', () => {
field: 'agent.id',
direction: 'asc',
};
const columns = [{ field: 'agent.id' }];

act(() => {
result.result.current.generateReportingJobCSV(agents, sortOptions);
result.result.current.generateReportingJobCSV(agents, columns, sortOptions);
});

expect(mockGetDecoratedJobParams.mock.calls[0][0].columns.length).toEqual(6);
expect(mockGetDecoratedJobParams.mock.calls[0][0].columns.length).toEqual(1);
expect(mockGetDecoratedJobParams.mock.calls[0][0].searchSource).toEqual(
expect.objectContaining({
filter: expect.objectContaining({
Expand Down Expand Up @@ -127,12 +128,13 @@ describe('export_csv', () => {

it('should generate reporting job for export csv with agents query', () => {
const agents = 'policy_id:1 AND status:online';
const columns = [{ field: 'agent.id' }];

act(() => {
result.result.current.generateReportingJobCSV(agents, undefined);
result.result.current.generateReportingJobCSV(agents, columns, undefined);
});

expect(mockGetDecoratedJobParams.mock.calls[0][0].columns.length).toEqual(6);
expect(mockGetDecoratedJobParams.mock.calls[0][0].columns.length).toEqual(1);
expect(mockGetDecoratedJobParams.mock.calls[0][0].searchSource).toEqual(
expect.objectContaining({
filter: expect.objectContaining({
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ import {
import type { Agent } from '../../../../../../../common';
import { getSortConfig, removeSOAttributes } from '../../../../../../../common';

import type { ExportField } from '../../components/agent_export_csv_modal';

import { getSortFieldForAPI } from './use_fetch_agents_data';

export function useExportCSV(enableExportCSV?: boolean) {
Expand All @@ -36,19 +38,9 @@ export function useExportCSV(enableExportCSV?: boolean) {

const getJobParams = (
agents: Agent[] | string,
columns: Array<{ field: string }>,
sortOptions?: { field?: string; direction?: string }
) => {
// TODO pass columns from Agent list UI
// TODO set readable column names
const columns = [
{ field: 'agent.id' },
{ field: 'status' },
{ field: 'local_metadata.host.hostname' },
{ field: 'policy_id' }, // policy name would need to be enriched
{ field: 'last_checkin' },
{ field: 'local_metadata.elastic.agent.version' },
];

const index = new DataView({
spec: {
title: '.fleet-agents',
Expand Down Expand Up @@ -108,9 +100,12 @@ export function useExportCSV(enableExportCSV?: boolean) {
// copied and adapted logic from here: https://github.com/elastic/kibana/blob/2846a162de7e56d2107eeb2e33e006a3310a4ae1/packages/kbn-reporting/public/share/share_context_menu/register_csv_modal_reporting.tsx#L86
const generateReportingJobCSV = (
agents: Agent[] | string,
columns: ExportField[],
sortOptions?: { field?: string; direction?: string }
) => {
const decoratedJobParams = apiClient.getDecoratedJobParams(getJobParams(agents, sortOptions));
const decoratedJobParams = apiClient.getDecoratedJobParams(
getJobParams(agents, columns, sortOptions)
);
return apiClient
.createReportingShareJob('csv_searchsource', decoratedJobParams)
.then(() => {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,234 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { i18n } from '@kbn/i18n';

export const INITIAL_AGENT_FIELDS_TO_EXPORT = [
{
field: 'agent.id',
description: i18n.translate('xpack.fleet.exportCSV.agentIdField', {
defaultMessage: 'Agent ID',
}),
},
{
field: 'status',
description: i18n.translate('xpack.fleet.exportCSV.statusField', { defaultMessage: 'Status' }),
},
{
field: 'local_metadata.host.hostname',
description: i18n.translate('xpack.fleet.exportCSV.hostnameField', {
defaultMessage: 'Host Name',
}),
},
{
field: 'policy_id',
description: i18n.translate('xpack.fleet.exportCSV.policyIdField', {
defaultMessage: 'Policy ID',
}),
}, // policy name would need to be enriched
{
field: 'last_checkin',
description: i18n.translate('xpack.fleet.exportCSV.lastCheckinField', {
defaultMessage: 'Last Checkin Time',
}),
},
{
field: 'local_metadata.elastic.agent.version',
description: i18n.translate('xpack.fleet.exportCSV.agentVersionField', {
defaultMessage: 'Agent Version',
}),
},
];

export const AGENT_FIELDS_TO_EXPORT = [
{
field: 'active',
description: i18n.translate('xpack.fleet.exportCSV.activeField', { defaultMessage: 'Active' }),
},
{
field: 'audit_unenrolled_reason',
description: i18n.translate('xpack.fleet.exportCSV.auditUnenrolledReasonField', {
defaultMessage: 'Audit Unenrolled Reason',
}),
},
{
field: 'audit_unenrolled_time',
description: i18n.translate('xpack.fleet.exportCSV.auditUnenrolledTimeField', {
defaultMessage: 'Audit Unenrolled Time',
}),
},
{
field: 'enrolled_at',
description: i18n.translate('xpack.fleet.exportCSV.enrolledAtField', {
defaultMessage: 'Enrolled At',
}),
},
{
field: 'last_checkin_message',
description: i18n.translate('xpack.fleet.exportCSV.lastCheckinMessageField', {
defaultMessage: 'Last Checkin Message',
}),
},
{
field: 'last_checkin_status',
description: i18n.translate('xpack.fleet.exportCSV.lastCheckinStatusField', {
defaultMessage: 'Last Checkin Status',
}),
},
{
field: 'last_updated',
description: i18n.translate('xpack.fleet.exportCSV.lastUpdatedField', {
defaultMessage: 'Last Updated Time',
}),
},
{
field: 'local_metadata.elastic.agent.build.original',
description: i18n.translate('xpack.fleet.exportCSV.agentBuildOriginalField', {
defaultMessage: 'Agent Build Original',
}),
},
{
field: 'local_metadata.elastic.agent.log_level',
description: i18n.translate('xpack.fleet.exportCSV.logLevelField', {
defaultMessage: 'Agent Log Level',
}),
},
{
field: 'local_metadata.elastic.agent.snapshot',
description: i18n.translate('xpack.fleet.exportCSV.agentSnapshotField', {
defaultMessage: 'Agent Snapshot',
}),
},
{
field: 'local_metadata.elastic.agent.unprivileged',
description: i18n.translate('xpack.fleet.exportCSV.agentUnprivilegedField', {
defaultMessage: 'Agent Unprivileged',
}),
},
{
field: 'local_metadata.elastic.agent.upgradeable',
description: i18n.translate('xpack.fleet.exportCSV.agentUpgradeableField', {
defaultMessage: 'Agent Upgradeable',
}),
},
{
field: 'local_metadata.host.architecture',
description: i18n.translate('xpack.fleet.exportCSV.hostArchitectureField', {
defaultMessage: 'Host Architecture',
}),
},
{
field: 'local_metadata.host.id',
description: i18n.translate('xpack.fleet.exportCSV.hostIdField', { defaultMessage: 'Host ID' }),
},
{
field: 'local_metadata.host.ip',
description: i18n.translate('xpack.fleet.exportCSV.hostIpField', { defaultMessage: 'Host IP' }),
},
{
field: 'local_metadata.host.mac',
description: i18n.translate('xpack.fleet.exportCSV.hostMacField', {
defaultMessage: 'Host Mac',
}),
},
{
field: 'local_metadata.host.name',
description: i18n.translate('xpack.fleet.exportCSV.hostNameField', {
defaultMessage: 'Host Name',
}),
},
{
field: 'local_metadata.os.family',
description: i18n.translate('xpack.fleet.exportCSV.osFamilyField', {
defaultMessage: 'OS Family',
}),
},
{
field: 'local_metadata.os.full',
description: i18n.translate('xpack.fleet.exportCSV.osFullField', { defaultMessage: 'OS Full' }),
},
{
field: 'local_metadata.os.kernel',
description: i18n.translate('xpack.fleet.exportCSV.osKernelField', {
defaultMessage: 'OS Kernel',
}),
},
{
field: 'local_metadata.os.name',
description: i18n.translate('xpack.fleet.exportCSV.osNameField', { defaultMessage: 'OS Name' }),
},
{
field: 'local_metadata.os.platform',
description: i18n.translate('xpack.fleet.exportCSV.osPlatformField', {
defaultMessage: 'OS Platform',
}),
},
{
field: 'local_metadata.os.version',
description: i18n.translate('xpack.fleet.exportCSV.osVersionField', {
defaultMessage: 'OS Version',
}),
},
{
field: 'tags',
description: i18n.translate('xpack.fleet.exportCSV.tagsField', { defaultMessage: 'Tags' }),
},
{
field: 'unenrolled_at',
description: i18n.translate('xpack.fleet.exportCSV.unenrolledAtField', {
defaultMessage: 'Unenrolled At',
}),
},
{
field: 'unenrolled_reason',
description: i18n.translate('xpack.fleet.exportCSV.unenrolledReasonField', {
defaultMessage: 'Unenrolled Reason',
}),
},
{
field: 'unenrollment_started_at',
description: i18n.translate('xpack.fleet.exportCSV.unenrolledStartedAtField', {
defaultMessage: 'Unenrolled Started At',
}),
},
{
field: 'unhealthy_reason',
description: i18n.translate('xpack.fleet.exportCSV.unhealthyReasonField', {
defaultMessage: 'Unhealthy Reason',
}),
},
{
field: 'updated_at',
description: i18n.translate('xpack.fleet.exportCSV.updatedAtField', {
defaultMessage: 'Updated At',
}),
},
{
field: 'upgrade_started_at',
description: i18n.translate('xpack.fleet.exportCSV.upgradeStartedAtField', {
defaultMessage: 'Upgrade Started At',
}),
},
{
field: 'upgrade_status',
description: i18n.translate('xpack.fleet.exportCSV.upgradeStatusField', {
defaultMessage: 'Upgrade Status',
}),
},
{
field: 'upgraded_at',
description: i18n.translate('xpack.fleet.exportCSV.upgradedAtField', {
defaultMessage: 'Upgraded At',
}),
},
{
field: 'user_provided_metadata',
description: i18n.translate('xpack.fleet.exportCSV.userProvidedMetadataField', {
defaultMessage: 'User Provided Metadata',
}),
},
];
Loading

0 comments on commit 6e145f9

Please sign in to comment.