Option to enable/disable reporting/uploading on certain tests

[spickermann]

Hey there,

we are using the buildkite-test_collector Ruby gem in our application, and we are running our test in SemaphoreCI.

Because of how the test collector is initialized in the rails_helper.rb with a simple Buildkite::TestCollector.configure(hook: :rspec) call, it seems like there is no option to have more control over when to collect and report data to Buildkite and when not.

More specifically, we are looking for a way to only enable the test collector when running test on the main branch, but to disable the collector when running tests on feature branches. Especially when those features are not ready for review yet, reporting failing tests feels like messing up data and statistics on Buildkite.

I didn't find any simple way to enable or disable the test collector in the docs or the code. I am looking for example to set or not setting a specific ENV. Or by passing an additional conditional argument to the Buildkite::TestCollector.configure call.

Did I miss something? Is there already a way to disable the collector depending on certain conditions on CI?

[swebb]

Hi @spickermann. This isn't something that is explicitly supported; however, we are considering adding support for. As a workaround, the SemaphoreCI docs say the SEMAPHORE_GIT_BRANCH environment variable includes the branch name, so I think this would work:

if ENV["SEMAPHORE_GIT_BRANCH"] == "main"
  Buildkite::TestCollector.configure(hook: :rspec)
end

Does that solve your problem?