[kube-prometheus-stack] Pagerduty secret from file

[JMG81]

Describe the bug a clear and concise description of what the bug is.

Greetings, I hope this is the proper place to report.

The following PR was merged to the main alertmanager branch on October 14th, it allows reading pagerduty secrets from a file by using routing_key_file as opposed to routing_key: prometheus/alertmanager#3107

However, i'm seeing the following error in Prometheus operator logs:

level=error ts=2022-11-14T13:06:01.759399749Z caller=klog.go:116 component=k8s_client_runtime func=ErrorDepth msg="Sync \"default/prometheus-kube-prometheus-alertmanager\" failed: provision alertmanager configuration: failed to initialize from secret: yaml: unmarshal errors:\n line 29: field routing_key_file not found in type config.plain"

Everything works if routing_key_file is replaced with routing_key.

What's your helm version?

version.BuildInfo{Version:"v3.9.2", GitCommit:"1addefbfe665c350f4daf868a9adc5600cc064fd", GitTreeState:"clean", GoVersion:"go1.18.4"}

What's your kubectl version?

Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.3", GitCommit:"aef86a93758dc3cb2c658dd9657ab4ad4afc21cb", GitTreeState:"clean", BuildDate:"2022-07-13T14:21:56Z", GoVersion:"go1.18.4", Compiler:"gc", Platform:"darwin/arm64"} Kustomize Version: v4.5.4 Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.8-gke.1900", GitCommit:"79209257257c051b27df67c567755783eda93353", GitTreeState:"clean", BuildDate:"2022-07-15T09:23:51Z", GoVersion:"go1.17.11b7", Compiler:"gc", Platform:"linux/amd64"}

Which chart?

kube-prometheus-stack

What's the chart version?

41.7.4

What happened?

Custom config fails to apply

What you expected to happen?

Custom config applies and my PagerDuty secret is read from a secrets file

How to reproduce it?

Install the latest helm chart (at time of writing, 41.7.4) with custom configs

Enter the changed values of values.yaml?

alertmanager:
  alertmanagerSpec:
    secrets:
      - pagerduty
  config:
    route:
      group_by: ['instance']
      group_wait: 30s
      group_interval: 5m
      repeat_interval: 12h
      routes:
      - receiver: 'pagerduty'
        matchers:  
          - alertname = "DownInstance"
    receivers:    
    - name: 'null'
    - name: 'pagerduty'
      pagerduty_configs:
      - routing_key_file: '/etc/alertmanager/secrets/pagerduty/key'
         description: '{{ .CommonAnnotations.summary }}'

Enter the command that you execute and failing/misfunctioning.

helm upgrade --install prometheus prometheus-community/kube-prometheus-stack --namespace monitoring --version 41.7.4 --values custom.yaml

Anything else we need to know?

Prometheus Operator version: quay.io/prometheus-operator/prometheus-operator:v0.60.1
Alertmanager version: quay.io/prometheus/alertmanager:v0.24.0

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.

[stale]

This issue is being automatically closed due to inactivity.

[sirjobzy]

I am also facing this issue, has anyone found a solution to this?
cc @JMG81

[amnk]

For everyone who will stumble on this: routing_key_file was introduced in alertmanager 0.25 (https://github.com/prometheus/alertmanager/releases/tag/v0.25.0).