Secret PAT Token does not work on Azure DevOps OnPremise

[RealCoda]

Summary

PAT Token can not be set to secret to work with PR Metrics in AzureDevops on Premise

Reproduction Steps

Use PR Metrics as a (last) task in some pipeline on Azure DevOps Server On Premise.
Add a pipeline variable called "PR_METRICS_ACCESS_TOKEN" to the pipeline and activate "secret"
-> leads to error "Could not access the Personal Access Token (PAT). Add 'PR_Metrics_Access_Token' as a secret environment variable."
if you deactivate "secret" it works like a charm.

Troubleshooting Undertaken

• tried to make it case sensitive more correct (PR_Metrics_Access_Token instead of PR_METRICS_ACCESS_TOKEN) and vice versa
• tried to call it SECRET_PR_METRICS_ACCESS_TOKEN and put it to secret

Additional Information

Versions

Version Azure DevOps Server 2022.1 (AzureDevOpsServer_20231128.1)
PR Metrics v1.5.11

Yaml

yaml export file says: (so it was really added)
Variable 'PR_METRICS_ACCESS_TOKEN' was defined in the Variables tab

Debug Log

debug log of an unsuccessful run (name = pr_metrics_access_token and secret = on)
##[debug]Evaluating condition for step: 'PR Metrics'
##[debug]Evaluating: succeeded()
##[debug]Evaluating succeeded:
##[debug]=> True
##[debug]Result: True
Starting: PR Metrics
==============================================================================
Task         : PR Metrics v1.5.11
Description  : Augments pull request titles to let reviewers quickly determine PR size and test coverage.
Version      : 1.5.11
Author       : Microsoft Corporation
Help         : https://aka.ms/PRMetrics/README
==============================================================================
##[debug]Using node path: C:\agent\externals\node16\bin\node.exe
##[debug]agent.TempDirectory=C:\agent\_work\_temp
##[debug]loading inputs and endpoints
##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
##[debug]loading SECRET_PR_METRICS_ACCESS_TOKEN
##[debug]loaded 4
##[debug]Agent.ProxyUrl=undefined
##[debug]Agent.CAInfo=undefined
##[debug]Agent.ClientCert=undefined
##[debug]Agent.SkipCertValidation=undefined
##[debug]check path : C:\agent\_work\_tasks\PRMetrics_907d3b28-6b37-4ac7-ac75-9631ee53e512\1.5.11\task.json
##[debug]adding resource file: C:\agent\_work\_tasks\PRMetrics_907d3b28-6b37-4ac7-ac75-9631ee53e512\1.5.11\task.json
##[debug]system.culture=en-US
##[debug]* CodeMetricsCalculator.shouldSkip
##[debug]* PullRequest.isPullRequest
##[debug]* PullRequest.isSupportedProvider
##[debug]* CodeMetricsCalculator.shouldStop()
##[debug]* ReposInvoker.isAccessTokenAvailable
##[debug]* ReposInvoker.getReposInvoker()
##[debug]* AzureReposInvoker.isAccessTokenAvailable
##[debug]task result: Failed
##[error]Could not access the Personal Access Token (PAT). Add 'PR_Metrics_Access_Token' as a secret environment variable.
##[debug]Processed: ##vso[task.issue type=error;source=TaskInternal;]Could not access the Personal Access Token (PAT). Add 'PR_Metrics_Access_Token' as a secret environment variable.
##[debug]Processed: ##vso[task.complete result=Failed;]Could not access the Personal Access Token (PAT). Add 'PR_Metrics_Access_Token' as a secret environment variable.
Finishing: PR Metrics

[muiriswoulfe]

Hi @RealCoda, your Azure DevOps YAML definition should look like the following, so that the secret is explicitly picked up by Azure DevOps and sent to PR Metrics:

- task: ms-omex.prmetrics.prmetrics.PRMetrics@1
  displayName: 'PR Metrics'
  env:
    PR_METRICS_ACCESS_TOKEN: $(PR_Metrics_Access_Token)
  continueOnError: true

The key here is the env element, for setting the secret within the task.

You can find more information about this at https://github.com/microsoft/PR-Metrics/blob/main/docs/azure-pipelines-task.md.

Could you ensure this is present?

If it is, could you provide the YAML definition you're using so that I can debug further. Thanks!

[RealCoda]

thx for commenting!
I see that one main thing was missing from my description, we use still the classic pipeline mode. (not the pure yaml definition).
Interestingly enough, i was not aware that there is a possible difference....

As we can see in the debug log above, there is obviously a difference:
loading SECRET_PR_METRICS_ACCESS_TOKEN
Maybe the classic pipeline is treating things different.(e.g. prefixes automatically with SECRET_ )

I will test this further more and keep you updated.

[muiriswoulfe]

Hi @RealCoda, thanks for clarifying your scenario.

I believe the key is the "Environment variables" section of the task definition:

For this, the first column would list PR_METRICS_ACCESS_TOKEN and the second column would be something like $(PR_Metrics_Access_Token) as far as I recall.

Unfortunately, my test Azure DevOps instance doesn't allow for classic pipelines to be used, so I can't validate this. If this works for you, I'd appreciate if you could let me know so that I can document it at https://github.com/microsoft/PR-Metrics/blob/main/docs/azure-pipelines-task.md for future reference.

[RealCoda]

Thanks for your input! Well, you were fully right ;-)
In classic pipelines i did the following:

• Added the Token via "Variables" Tag to the pipeline (or linked some variables block from the library) with "secret" enabled.
• Added the environment variable as you suggested.

=> works perfectly. thx !

[muiriswoulfe]

Thanks @RealCoda for the info. I've now added this as documentation for future reference via #471.