Problem with trivy/trivyoperator

[itsmednkv]

Hello,

DmitriyLewen say on Mar 22, 2024
to me: "Can you please open new discusson in https://github.com/aquasecurity/trivy-operator/discussions? It seems like some kind of trivy-operator setting uses old Trivy version.`"

We are using the software in kubernetes.

helm.sh/chart: trivy-operator-0.21.2
app.kubernetes.io/name: trivy-operator
app.kubernetes.io/instance: trivy-operator
app.kubernetes.io/version: "0.19.2"
app.kubernetes.io/managed-by: Helm

Now the Problem is here the following:

scan error: scan error: scan failed: scan failed: failed to detect vulnerabilities via RPC: twirp error internal: failed scan, docker.io/bitnami/xxxxxx

but as i read this problem was removed this error in v0.42.1 (see aquasecurity/trivy#4577)?!

[chen-keinan]

@itsmednkv can you please add your config maps?

[itsmednkv]

trivy.yml
`---
Source: trivy-operator/templates/configmaps/trivy.yaml

apiVersion: v1
kind: ConfigMap
metadata:
name: trivy-operator-trivy-config
namespace: security
labels:
helm.sh/chart: trivy-operator-0.21.2
app.kubernetes.io/name: trivy-operator
app.kubernetes.io/instance: trivy-operator
app.kubernetes.io/version: "0.19.2"
app.kubernetes.io/managed-by: Helm
data:
trivy.repository: "ghcr.io/aquasecurity/trivy"
trivy.tag: "0.50.1"
trivy.imagePullPolicy: "IfNotPresent"
trivy.additionalVulnerabilityReportFields: ""
trivy.severity: "UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL"
trivy.slow: "true"
trivy.skipJavaDBUpdate: "false"
trivy.includeDevDeps: "false"
trivy.imageScanCacheDir: "/tmp/trivy/.cache"
trivy.filesystemScanCacheDir: "/var/trivyoperator/trivy-db"
trivy.dbRepository: "ghcr.io/aquasecurity/trivy-db"
trivy.javaDbRepository: "ghcr.io/aquasecurity/trivy-java-db"
trivy.command: "image"
trivy.sbomSources: ""
trivy.dbRepositoryInsecure: "false"
trivy.useBuiltinRegoPolicies: "true"
trivy.supportedConfigAuditKinds: "Workload,Service,Role,ClusterRole,NetworkPolicy,Ingress,LimitRange,ResourceQuota"
trivy.timeout: "5m0s"
trivy.mode: "ClientServer"
trivy.serverURL: "https://xxxx.xxxxx.xxxxxx/"
trivy.resources.requests.cpu: "100m"
trivy.resources.requests.memory: "100M"
trivy.resources.limits.cpu: "500m"
trivy.resources.limits.memory: "500M"`

trivy-operator-config.yml
`---
Source: trivy-operator/templates/configmaps/trivy-operator-config.yaml

kind: ConfigMap
apiVersion: v1
metadata:
name: trivy-operator-config
namespace: security
labels:
helm.sh/chart: trivy-operator-0.21.2
app.kubernetes.io/name: trivy-operator
app.kubernetes.io/instance: trivy-operator
app.kubernetes.io/version: "0.19.2"
app.kubernetes.io/managed-by: Helm
data:
OPERATOR_LOG_DEV_MODE: "false"
OPERATOR_SCAN_JOB_TTL: ""
OPERATOR_SCAN_JOB_TIMEOUT: "5m"
OPERATOR_CONCURRENT_SCAN_JOBS_LIMIT: "10"
OPERATOR_CONCURRENT_NODE_COLLECTOR_LIMIT: "1"
OPERATOR_SCAN_JOB_RETRY_AFTER: "30s"
OPERATOR_BATCH_DELETE_LIMIT: "10"
OPERATOR_BATCH_DELETE_DELAY: "10s"
OPERATOR_METRICS_BIND_ADDRESS: ":8080"
OPERATOR_METRICS_FINDINGS_ENABLED: "true"
OPERATOR_METRICS_VULN_ID_ENABLED: "true"
OPERATOR_HEALTH_PROBE_BIND_ADDRESS: ":9090"
OPERATOR_VULNERABILITY_SCANNER_ENABLED: "true"
OPERATOR_SBOM_GENERATION_ENABLED: "true"
OPERATOR_CLUSTER_SBOM_CACHE_ENABLED: "false"
OPERATOR_VULNERABILITY_SCANNER_SCAN_ONLY_CURRENT_REVISIONS: "true"
OPERATOR_SCANNER_REPORT_TTL: "24h"
OPERATOR_CACHE_REPORT_TTL: "120h"
CONTROLLER_CACHE_SYNC_TIMEOUT: "5m"
OPERATOR_CONFIG_AUDIT_SCANNER_ENABLED: "true"
OPERATOR_RBAC_ASSESSMENT_SCANNER_ENABLED: "true"
OPERATOR_INFRA_ASSESSMENT_SCANNER_ENABLED: "true"
OPERATOR_CONFIG_AUDIT_SCANNER_SCAN_ONLY_CURRENT_REVISIONS: "true"
OPERATOR_EXPOSED_SECRET_SCANNER_ENABLED: "true"
OPERATOR_METRICS_EXPOSED_SECRET_INFO_ENABLED: "false"
OPERATOR_METRICS_CONFIG_AUDIT_INFO_ENABLED: "false"
OPERATOR_METRICS_RBAC_ASSESSMENT_INFO_ENABLED: "false"
OPERATOR_METRICS_INFRA_ASSESSMENT_INFO_ENABLED: "false"
OPERATOR_METRICS_IMAGE_INFO_ENABLED: "true"
OPERATOR_METRICS_CLUSTER_COMPLIANCE_INFO_ENABLED: "false"
OPERATOR_WEBHOOK_BROADCAST_URL: ""
OPERATOR_WEBHOOK_BROADCAST_TIMEOUT: "30s"
OPERATOR_SEND_DELETED_REPORTS: "false"
OPERATOR_PRIVATE_REGISTRY_SCAN_SECRETS_NAMES: "{}"
OPERATOR_ACCESS_GLOBAL_SECRETS_SERVICE_ACCOUNTS: "true"
OPERATOR_BUILT_IN_TRIVY_SERVER: "false"
TRIVY_SERVER_HEALTH_CHECK_CACHE_EXPIRATION: "10h"
OPERATOR_MERGE_RBAC_FINDING_WITH_CONFIG_AUDIT: "false"
OPERATOR_CLUSTER_COMPLIANCE_ENABLED: "true"`

operator.yml
`---
Source: trivy-operator/templates/configmaps/operator.yaml

apiVersion: v1
kind: ConfigMap
metadata:
name: trivy-operator
namespace: security
labels:
helm.sh/chart: trivy-operator-0.21.2
app.kubernetes.io/name: trivy-operator
app.kubernetes.io/instance: trivy-operator
app.kubernetes.io/version: "0.19.2"
app.kubernetes.io/managed-by: Helm
data:
nodeCollector.volumes: "[{"hostPath":{"path":"/var/lib/etcd"},"name":"var-lib-etcd"},{"hostPath":{"path":"/var/lib/kubelet"},"name":"var-lib-kubelet"},{"hostPath":{"path":"/var/lib/kube-scheduler"},"name":"var-lib-kube-scheduler"},{"hostPath":{"path":"/var/lib/kube-controller-manager"},"name":"var-lib-kube-controller-manager"},{"hostPath":{"path":"/etc/systemd"},"name":"etc-systemd"},{"hostPath":{"path":"/lib/systemd"},"name":"lib-systemd"},{"hostPath":{"path":"/etc/kubernetes"},"name":"etc-kubernetes"},{"hostPath":{"path":"/etc/cni/net.d/"},"name":"etc-cni-netd"}]"
nodeCollector.volumeMounts: "[{"mountPath":"/var/lib/etcd","name":"var-lib-etcd","readOnly":true},{"mountPath":"/var/lib/kubelet","name":"var-lib-kubelet","readOnly":true},{"mountPath":"/var/lib/kube-scheduler","name":"var-lib-kube-scheduler","readOnly":true},{"mountPath":"/var/lib/kube-controller-manager","name":"var-lib-kube-controller-manager","readOnly":true},{"mountPath":"/etc/systemd","name":"etc-systemd","readOnly":true},{"mountPath":"/lib/systemd/","name":"lib-systemd","readOnly":true},{"mountPath":"/etc/kubernetes","name":"etc-kubernetes","readOnly":true},{"mountPath":"/etc/cni/net.d/","name":"etc-cni-netd","readOnly":true}]"
scanJob.podTemplateContainerSecurityContext: "{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"privileged":false,"readOnlyRootFilesystem":true}"
scanJob.compressLogs: "true"
vulnerabilityReports.scanner: "Trivy"
configAuditReports.scanner: "Trivy"
compliance.failEntriesLimit: "10"
report.recordFailedChecksOnly: "true"
node.collector.imageRef: "ghcr.io/aquasecurity/node-collector:0.1.2"
policies.bundle.oci.ref: "ghcr.io/aquasecurity/trivy-policies:0"
node.collector.nodeSelector: "true"
`

policies.yml
`---
Source: trivy-operator/templates/configmaps/policies.yaml

apiVersion: v1
kind: ConfigMap
metadata:
name: trivy-operator-policies-config
namespace: security
labels:
helm.sh/chart: trivy-operator-0.21.2
app.kubernetes.io/name: trivy-operator
app.kubernetes.io/instance: trivy-operator
app.kubernetes.io/version: "0.19.2"
app.kubernetes.io/managed-by: Helm
data:`

[chen-keinan]

@itsmednkv have you consider using the built-in trivy server it works well and might make deployment process easy

[itsmednkv]

first image that i have tested was tag: 0.18.9 i think

[chen-keinan]

my question is, did you ever got scanning to work (generating reports with vulns) for you with trivy-operator while using this central server approach, by the error log its look like communication (networking) issue between trivy cli and server

[itsmednkv]

of cause yes. it scan some images...
in my case it not work with openldap

Scan job container","job":"security/scan-vulnerabilityreport-65bfcbb547","container":"openldap","status.reason":"Error","status.message":"2024-04-04T03:10:01.060Z\t\u001b[31mFATAL\u001b[0m\timage scan error: scan error: scan failed: scan failed: failed to detect vulnerabilities via RPC: twirp error internal: failed scan, ..../docker-images/bitnami/openldap:2.6:

[chen-keinan]

Do you have a referance to Trivy issue/discussion? I would like to comment their as its look like trivy client/server issue.
the trivy-operator version you are using do include tha latest trivy , so it not related to an old trivy version

[itsmednkv]

you mean this one?
aquasecurity/trivy#6330