Use new API endpoint to fetch and create the test plan

[nprizal]

⚠️ BREAKING CHANGES

These changes introduced 3 required configs

• BUILDKITE_API_ACCESS_TOKEN. You now need Buildkite API access token that has read_suites, read_test_plan, and write_test_plan scopes.
• BUILDKITE_ORGANIZATION_SLUG. This is available in your Buildkite Pipeline environment, so you don't need to set it manually.
• BUILDKITE_SPLITTER_SUITE_SLUG. This replaces BUILDKITE_SPLITTER_SUITE_TOKEN and need to be set manually.

Description

As part of split by example implementation, we want to change how the client fetch and create test plan from the server. The new flow will look like this

// main.go
func fetchOrCreatePlan() {
  plan := api.FetchPlan()

  if plan != nil {
    return plan
  }

  params := buildParamsToCreatePlan()

  plan := api.CreatePlan(params)

  if plan != nil {
    return plan
  }
}

This PR also replaced the endpoint to the new endpoint.

Changes

• Add 3 new configs
• Add httpClient to api.client struct that has Transport to attach access token to each request
• Implement FetchTestPlan in the api.client struct
• Replace endpoint in CreateTestPlan to the new endpoint
• Removed BUILDKITE_SPLITTER_SUITE_TOKEN from config

Testing

As this is a breaking change, I suggest testing it manually against a local Buildkite server (or Prod). I tested it manually against my local Buildkite server.

[amybiyuliu]

Nice work! I just left a couple of questions.

Also wondering if we need the foo_spec.rb for this PR?

[wooly]

I don't think the authentication settings below this are valid any more, is that correct?

[nprizal]

Good catch! I have updated the readme, but don't want to change it too much for now. What do you think?

[nprizal]

@wooly @niceking I'm ignoring server error, except for 400 ~ 403, so the client can proceed and attempt to make POST request. Do you think this make sense? Or should we implement a retry mechanism for this as well?

The reason behind handling 400 ~ 403 is that we want to terminate the program if there is bad request (miss config), unauthorized, or not permitted (token doesn't have required scope)

[wooly]

I haven't fully thought through the implications of falling back to cache-miss yet, but it probably makes sense to document the individual status codes that we respond to in the splitter and what they mean. At least to have as a point of reference, even if the code groups them all together.

[nprizal]

Good suggestion. I have improve the readability and the test. Let me know what you think.

[amybiyuliu]

@nprizal and I reviewed the changes in this PR at today's team time. The changes look good to me

[wooly]

Removing the DO NOT MERGE tag from this as it's now suitable to be merged into the 0.6.0 branch.