Skip to content

Commit

Permalink
Suppress errors when connecting to Buildkite
Browse files Browse the repository at this point in the history
It's important the collector does not raise exceptions and cause test
suites to fail. Some customers have seen exceptions raised when
connecting to Buildkite (not a surprise as networks are complicated and
unreliable).

This PR modifies the connection process to catch all StandardErrors and
log them. Trace data will be lost; however, the customer's build will
not fail which is more important.

We also intend to address the underlying issues; however, this PR is an
important defensive measure to protect customer builds.
  • Loading branch information
swebb committed Jul 28, 2022
1 parent d8924b2 commit 0cdec78
Show file tree
Hide file tree
Showing 4 changed files with 31 additions and 2 deletions.
6 changes: 6 additions & 0 deletions lib/buildkite/test_collector.rb
Original file line number Diff line number Diff line change
Expand Up @@ -101,5 +101,11 @@ def self.enable_tracing!
Buildkite::TestCollector::Uploader.tracer&.backfill(:sql, finish - start, **{ query: payload[:sql] })
end
end

def self.safe(&block)
block.call
rescue StandardError => e
logger.error("Buildkite::TestCollector received exception: #{e}")
end
end
end
2 changes: 1 addition & 1 deletion lib/buildkite/test_collector/library_hooks/minitest.rb
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,4 @@ class Minitest::Test

Buildkite::TestCollector.enable_tracing!

Buildkite::TestCollector::Uploader.configure
Buildkite::TestCollector.safe { Buildkite::TestCollector::Uploader.configure }
2 changes: 1 addition & 1 deletion lib/buildkite/test_collector/library_hooks/rspec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
config.before(:suite) do
config.add_formatter Buildkite::TestCollector::RSpecPlugin::Reporter

Buildkite::TestCollector::Uploader.configure
Buildkite::TestCollector.safe { Buildkite::TestCollector::Uploader.configure }
end

config.around(:each) do |example|
Expand Down
23 changes: 23 additions & 0 deletions spec/test_collector_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,27 @@
expect(analytics.url).to eq "https://analytics-api.buildkite.com/v1/uploads"
end
end

describe ".safe" do
let(:logger) { TestLogger.new }

class TestLogger
attr_reader :errors

def initialize
@errors = []
end

def error(message)
@errors << message
end
end

before { Buildkite::TestCollector.logger = logger }

it "suppresses exceptions and logs them to logger.error" do
expect{ described_class.safe { raise "penguines dance" } }.to_not raise_error
expect(logger.errors).to eq(["Buildkite::TestCollector received exception: penguines dance"])
end
end
end

0 comments on commit 0cdec78

Please sign in to comment.