-
Notifications
You must be signed in to change notification settings - Fork 125
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unit Test SB Release Pipeline #4408
Comments
Ideally we would minimize changes to the actual pipeline to do this testing. The intent is that the validation pipeline behaves the same as the official pipeline. So I'd like to avoid injecting steps into the existing stages that are used by the validation and official pipeline. One possibility is to introduce a new stage at the end that is only used by the validation pipeline. It would include steps which does this kind of validation, either by examining artifacts or querying AzDO to get log output of the current build. |
Agreed. I don't think that these tests should be a part of the actual pipeline - my thinking is that these tests would help validate changes to the release pipeline rather than validate runs of the actual pipeline, if that makes sense. In other words, these tests help us to ensure that the changes we're making don't cause unintended consequences. |
@oleksandr-didyk - Is this the type of validation you were intending in #4131 or orthogonal? |
Currently, we validate changes to the SB release pipeline by running PRs for changes against fake release runs. If the test pipeline run succeeds, then the changes are considered "successful". This type of validation falls into the category of functional testing where it's about validating that the software behaves as expected and works, but it doesn't check the actual content or internal structures.
While this type of testing is important, we should consider writing tests and validating changes against the actual content of the pipeline. For example, we should write tests that address the following questions:
The text was updated successfully, but these errors were encountered: