Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Osquerybeat: Fix data_stream configuration, enforce the default values used before 8.6.0 #34246

Merged
merged 4 commits into from
Jan 13, 2023
Merged

Osquerybeat: Fix data_stream configuration, enforce the default values used before 8.6.0 #34246

Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
ab9c374
Osquerybeat: Fix data_stream configuration, enforce the default value…
aleksmaus Jan 12, 2023
9b0d02d
Merge branch 'main' into fix/osquery_logstash
aleksmaus Jan 12, 2023
b8ef343
Added changelog entry
aleksmaus Jan 12, 2023
3ec07bb
Merge branch 'main' into fix/osquery_logstash
aleksmaus Jan 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion x-pack/osquerybeat/cmd/root.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import (

_ "github.com/elastic/beats/v7/x-pack/libbeat/include"
"github.com/elastic/beats/v7/x-pack/osquerybeat/beater"
"github.com/elastic/beats/v7/x-pack/osquerybeat/internal/config"
"github.com/elastic/beats/v7/x-pack/osquerybeat/internal/install"
)

Expand Down Expand Up @@ -74,14 +75,30 @@ func genVerifyCmd(_ instance.Settings) *cobra.Command {
func osquerybeatCfg(rawIn *proto.UnitExpectedConfig, agentInfo *client.AgentInfo) ([]*reload.ConfigWithMeta, error) {
// Convert to streams, osquerybeat doesn't use streams
streams := make([]*proto.Stream, 1)

// Enforce the datastream dataset and type because the libbeat call to CreateInputsFromStreams
// provides it's own defaults that are breaking the osquery with logstash
// The target datastream for the publisher is expected to be logs-osquery_manager.result-<namespace>
// while the libebeat management.CreateInputsFromStreams defaults to osquery-generic-default
var datastream *proto.DataStream
if rawIn.GetDataStream() != nil {
// Copy by value and modify dataset and type
ds := *rawIn.GetDataStream()
ds.Dataset = config.DefaultDataset
ds.Type = config.DefaultType
datastream = &ds
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works, but if the dataset or type ever becomes configurable it will break. That's probably unlikely though.

}

streams[0] = &proto.Stream{
Source: rawIn.GetSource(),
Id: rawIn.GetId(),
DataStream: rawIn.GetDataStream(),
DataStream: datastream,
}

rawIn.Streams = streams

procs := defaultProcessors()

Copy link
Member

@cmacknz cmacknz Jan 12, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't comment directly on this, but this block below can be removed:

beats/x-pack/osquerybeat/cmd/root.go

Lines 89 to 91 in b5246c4

for iter := range modules {
modules[iter]["type"] = "log"
}

That is trying to set an input type, which I don't think matters for osquerybeat. It is just adding an arbitrary type: log into the generated config that osquerybeat will be ignoring.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, don't remember the reason for this. can clean up later, separately from this more urgent fix

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is a copy/paste error from one of the other Beats, agree this can be removed separately.

modules, err := management.CreateInputsFromStreams(rawIn, "osquery", agentInfo, procs...)
if err != nil {
return nil, fmt.Errorf("error creating input list from raw expected config: %w", err)
Expand Down