Skip to content

Commit 5ba9a83

Browse files
simar7DmitriyLewen
simar7
and
DmitriyLewen
authoredNov 18, 2024··
feat: Update registry fallbacks (#7679)
Co-authored-by: DmitriyLewen <dmitriy.lewen@smartforce.io>
·
v0.60.0v0.58.0
1 parent 07915da commit 5ba9a83

File tree

15 files changed

+62
-22
lines changed

15 files changed

+62
-22
lines changed
 

‎docs/docs/configuration/db.md

+17-1
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,19 @@ You can reference the OCI manifest of [trivy-db].
9292
`trivy-db-registry:latest` => `trivy-db-registry:latest`, but `trivy-db-registry` => `trivy-db-registry:2`.
9393

9494

95+
### Rate limits
96+
Trivy hosts its databases on public OCI registries that are subject to their respective rate limits. While we strive to make the databases available to every
97+
Trivy user, there are certain recommendations that one can make in order to ensure rate limits are not hit.
98+
99+
#### Authenticated use of Registries
100+
By authenticating with the registries that Trivy hosts its DBs on can significantly increase the limit for users. For Amazon ECR, the details for rate limits can be found [ecr-limits].
101+
102+
Please see more info on how to authenticate with ECR [auth-ecr].
103+
104+
#### Caching DBs
105+
Trivy DB and Trivy Java DB are published every 6 hours and 24 hours, respectively. If you are running Trivy scans more often than this, you can significantly benefit from caching the DBs on each run and updating them as needed.
106+
Once example of this can be seen in Trivy Action, where with caching multiple CI invocations can be performed with a single download of the DBs. More on info Trivy Action caching can be found [trivy-action-cache].
107+
95108
## Java Index Database
96109
The same options are also available for the Java index DB, which is used for scanning Java applications.
97110
Skipping an update can be done by using the `--skip-java-db-update` option, while `--download-java-db-only` can be used to only download the Java index DB.
@@ -123,4 +136,7 @@ $ trivy clean --vuln-db --java-db
123136
```
124137

125138
[trivy-db]: https://github.com/aquasecurity/trivy-db/pkgs/container/trivy-db
126-
[trivy-java-db]: https://github.com/aquasecurity/trivy-java-db/pkgs/container/trivy-java-db
139+
[trivy-java-db]: https://github.com/aquasecurity/trivy-java-db/pkgs/container/trivy-java-db
140+
[ecr-limits]: https://docs.aws.amazon.com/AmazonECR/latest/public/public-service-quotas.html
141+
[auth-ecr]: https://aws.amazon.com/blogs/compute/authenticating-amazon-ecr-repositories-for-docker-cli-with-credential-helper/
142+
[trivy-action-cache]: https://github.com/aquasecurity/trivy-action?tab=readme-ov-file#cache

‎docs/docs/references/configuration/cli/trivy_filesystem.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ trivy filesystem [flags] PATH
2929
--config-data strings specify paths from which data for the Rego checks will be recursively loaded
3030
--config-file-schemas strings specify paths to JSON configuration file schemas to determine that a file matches some configuration and pass the schema to Rego checks for type checking
3131
--custom-headers strings custom headers in client mode
32-
--db-repository strings OCI repository(ies) to retrieve trivy-db in order of priority (default [ghcr.io/aquasecurity/trivy-db:2])
32+
--db-repository strings OCI repository(ies) to retrieve trivy-db in order of priority (default [mirror.gcr.io/aquasec/trivy-db:2,ghcr.io/aquasecurity/trivy-db:2])
3333
--dependency-tree [EXPERIMENTAL] show dependency origin tree of vulnerable packages
3434
--detection-priority string specify the detection priority:
3535
- "precise": Prioritizes precise by minimizing false positives.
@@ -56,7 +56,7 @@ trivy filesystem [flags] PATH
5656
--include-deprecated-checks include deprecated checks
5757
--include-dev-deps include development dependencies in the report (supported: npm, yarn)
5858
--include-non-failures include successes, available with '--scanners misconfig'
59-
--java-db-repository strings OCI repository(ies) to retrieve trivy-java-db in order of priority (default [ghcr.io/aquasecurity/trivy-java-db:1])
59+
--java-db-repository strings OCI repository(ies) to retrieve trivy-java-db in order of priority (default [mirror.gcr.io/aquasec/trivy-java-db:1,ghcr.io/aquasecurity/trivy-java-db:1])
6060
--license-confidence-level float specify license classifier's confidence level (default 0.9)
6161
--license-full eagerly look for licenses in source code headers and license files
6262
--list-all-pkgs output all packages in the JSON report regardless of vulnerability

‎docs/docs/references/configuration/cli/trivy_image.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ trivy image [flags] IMAGE_NAME
4343
--config-data strings specify paths from which data for the Rego checks will be recursively loaded
4444
--config-file-schemas strings specify paths to JSON configuration file schemas to determine that a file matches some configuration and pass the schema to Rego checks for type checking
4545
--custom-headers strings custom headers in client mode
46-
--db-repository strings OCI repository(ies) to retrieve trivy-db in order of priority (default [ghcr.io/aquasecurity/trivy-db:2])
46+
--db-repository strings OCI repository(ies) to retrieve trivy-db in order of priority (default [mirror.gcr.io/aquasec/trivy-db:2,ghcr.io/aquasecurity/trivy-db:2])
4747
--dependency-tree [EXPERIMENTAL] show dependency origin tree of vulnerable packages
4848
--detection-priority string specify the detection priority:
4949
- "precise": Prioritizes precise by minimizing false positives.
@@ -74,7 +74,7 @@ trivy image [flags] IMAGE_NAME
7474
--include-deprecated-checks include deprecated checks
7575
--include-non-failures include successes, available with '--scanners misconfig'
7676
--input string input file path instead of image name
77-
--java-db-repository strings OCI repository(ies) to retrieve trivy-java-db in order of priority (default [ghcr.io/aquasecurity/trivy-java-db:1])
77+
--java-db-repository strings OCI repository(ies) to retrieve trivy-java-db in order of priority (default [mirror.gcr.io/aquasec/trivy-java-db:1,ghcr.io/aquasecurity/trivy-java-db:1])
7878
--license-confidence-level float specify license classifier's confidence level (default 0.9)
7979
--license-full eagerly look for licenses in source code headers and license files
8080
--list-all-pkgs output all packages in the JSON report regardless of vulnerability

‎docs/docs/references/configuration/cli/trivy_kubernetes.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ trivy kubernetes [flags] [CONTEXT]
3838
--config-check strings specify the paths to the Rego check files or to the directories containing them, applying config files
3939
--config-data strings specify paths from which data for the Rego checks will be recursively loaded
4040
--config-file-schemas strings specify paths to JSON configuration file schemas to determine that a file matches some configuration and pass the schema to Rego checks for type checking
41-
--db-repository strings OCI repository(ies) to retrieve trivy-db in order of priority (default [ghcr.io/aquasecurity/trivy-db:2])
41+
--db-repository strings OCI repository(ies) to retrieve trivy-db in order of priority (default [mirror.gcr.io/aquasec/trivy-db:2,ghcr.io/aquasecurity/trivy-db:2])
4242
--dependency-tree [EXPERIMENTAL] show dependency origin tree of vulnerable packages
4343
--detection-priority string specify the detection priority:
4444
- "precise": Prioritizes precise by minimizing false positives.
@@ -70,7 +70,7 @@ trivy kubernetes [flags] [CONTEXT]
7070
--include-kinds strings indicate the kinds included in scanning (example: node)
7171
--include-namespaces strings indicate the namespaces included in scanning (example: kube-system)
7272
--include-non-failures include successes, available with '--scanners misconfig'
73-
--java-db-repository strings OCI repository(ies) to retrieve trivy-java-db in order of priority (default [ghcr.io/aquasecurity/trivy-java-db:1])
73+
--java-db-repository strings OCI repository(ies) to retrieve trivy-java-db in order of priority (default [mirror.gcr.io/aquasec/trivy-java-db:1,ghcr.io/aquasecurity/trivy-java-db:1])
7474
--k8s-version string specify k8s version to validate outdated api by it (example: 1.21.0)
7575
--kubeconfig string specify the kubeconfig file path to use
7676
--list-all-pkgs output all packages in the JSON report regardless of vulnerability

‎docs/docs/references/configuration/cli/trivy_repository.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ trivy repository [flags] (REPO_PATH | REPO_URL)
2929
--config-data strings specify paths from which data for the Rego checks will be recursively loaded
3030
--config-file-schemas strings specify paths to JSON configuration file schemas to determine that a file matches some configuration and pass the schema to Rego checks for type checking
3131
--custom-headers strings custom headers in client mode
32-
--db-repository strings OCI repository(ies) to retrieve trivy-db in order of priority (default [ghcr.io/aquasecurity/trivy-db:2])
32+
--db-repository strings OCI repository(ies) to retrieve trivy-db in order of priority (default [mirror.gcr.io/aquasec/trivy-db:2,ghcr.io/aquasecurity/trivy-db:2])
3333
--dependency-tree [EXPERIMENTAL] show dependency origin tree of vulnerable packages
3434
--detection-priority string specify the detection priority:
3535
- "precise": Prioritizes precise by minimizing false positives.
@@ -56,7 +56,7 @@ trivy repository [flags] (REPO_PATH | REPO_URL)
5656
--include-deprecated-checks include deprecated checks
5757
--include-dev-deps include development dependencies in the report (supported: npm, yarn)
5858
--include-non-failures include successes, available with '--scanners misconfig'
59-
--java-db-repository strings OCI repository(ies) to retrieve trivy-java-db in order of priority (default [ghcr.io/aquasecurity/trivy-java-db:1])
59+
--java-db-repository strings OCI repository(ies) to retrieve trivy-java-db in order of priority (default [mirror.gcr.io/aquasec/trivy-java-db:1,ghcr.io/aquasecurity/trivy-java-db:1])
6060
--license-confidence-level float specify license classifier's confidence level (default 0.9)
6161
--license-full eagerly look for licenses in source code headers and license files
6262
--list-all-pkgs output all packages in the JSON report regardless of vulnerability

‎docs/docs/references/configuration/cli/trivy_rootfs.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ trivy rootfs [flags] ROOTDIR
3131
--config-data strings specify paths from which data for the Rego checks will be recursively loaded
3232
--config-file-schemas strings specify paths to JSON configuration file schemas to determine that a file matches some configuration and pass the schema to Rego checks for type checking
3333
--custom-headers strings custom headers in client mode
34-
--db-repository strings OCI repository(ies) to retrieve trivy-db in order of priority (default [ghcr.io/aquasecurity/trivy-db:2])
34+
--db-repository strings OCI repository(ies) to retrieve trivy-db in order of priority (default [mirror.gcr.io/aquasec/trivy-db:2,ghcr.io/aquasecurity/trivy-db:2])
3535
--dependency-tree [EXPERIMENTAL] show dependency origin tree of vulnerable packages
3636
--detection-priority string specify the detection priority:
3737
- "precise": Prioritizes precise by minimizing false positives.
@@ -58,7 +58,7 @@ trivy rootfs [flags] ROOTDIR
5858
--ignorefile string specify .trivyignore file (default ".trivyignore")
5959
--include-deprecated-checks include deprecated checks
6060
--include-non-failures include successes, available with '--scanners misconfig'
61-
--java-db-repository strings OCI repository(ies) to retrieve trivy-java-db in order of priority (default [ghcr.io/aquasecurity/trivy-java-db:1])
61+
--java-db-repository strings OCI repository(ies) to retrieve trivy-java-db in order of priority (default [mirror.gcr.io/aquasec/trivy-java-db:1,ghcr.io/aquasecurity/trivy-java-db:1])
6262
--license-confidence-level float specify license classifier's confidence level (default 0.9)
6363
--license-full eagerly look for licenses in source code headers and license files
6464
--list-all-pkgs output all packages in the JSON report regardless of vulnerability

‎docs/docs/references/configuration/cli/trivy_sbom.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ trivy sbom [flags] SBOM_PATH
2424
--cache-ttl duration cache TTL when using redis as cache backend
2525
--compliance string compliance report to generate
2626
--custom-headers strings custom headers in client mode
27-
--db-repository strings OCI repository(ies) to retrieve trivy-db in order of priority (default [ghcr.io/aquasecurity/trivy-db:2])
27+
--db-repository strings OCI repository(ies) to retrieve trivy-db in order of priority (default [mirror.gcr.io/aquasec/trivy-db:2,ghcr.io/aquasecurity/trivy-db:2])
2828
--detection-priority string specify the detection priority:
2929
- "precise": Prioritizes precise by minimizing false positives.
3030
- "comprehensive": Aims to detect more security findings at the cost of potential false positives.
@@ -41,7 +41,7 @@ trivy sbom [flags] SBOM_PATH
4141
--ignore-unfixed display only fixed vulnerabilities
4242
--ignored-licenses strings specify a list of license to ignore
4343
--ignorefile string specify .trivyignore file (default ".trivyignore")
44-
--java-db-repository strings OCI repository(ies) to retrieve trivy-java-db in order of priority (default [ghcr.io/aquasecurity/trivy-java-db:1])
44+
--java-db-repository strings OCI repository(ies) to retrieve trivy-java-db in order of priority (default [mirror.gcr.io/aquasec/trivy-java-db:1,ghcr.io/aquasecurity/trivy-java-db:1])
4545
--list-all-pkgs output all packages in the JSON report regardless of vulnerability
4646
--no-progress suppress progress bar
4747
--offline-scan do not issue API requests to identify dependencies

‎docs/docs/references/configuration/cli/trivy_server.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ trivy server [flags]
2222
```
2323
--cache-backend string [EXPERIMENTAL] cache backend (e.g. redis://localhost:6379) (default "fs")
2424
--cache-ttl duration cache TTL when using redis as cache backend
25-
--db-repository strings OCI repository(ies) to retrieve trivy-db in order of priority (default [ghcr.io/aquasecurity/trivy-db:2])
25+
--db-repository strings OCI repository(ies) to retrieve trivy-db in order of priority (default [mirror.gcr.io/aquasec/trivy-db:2,ghcr.io/aquasecurity/trivy-db:2])
2626
--download-db-only download/update vulnerability database but don't run a scan
2727
--enable-modules strings [EXPERIMENTAL] module names to enable
2828
-h, --help help for server

‎docs/docs/references/configuration/cli/trivy_vm.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ trivy vm [flags] VM_IMAGE
2727
--compliance string compliance report to generate
2828
--config-file-schemas strings specify paths to JSON configuration file schemas to determine that a file matches some configuration and pass the schema to Rego checks for type checking
2929
--custom-headers strings custom headers in client mode
30-
--db-repository strings OCI repository(ies) to retrieve trivy-db in order of priority (default [ghcr.io/aquasecurity/trivy-db:2])
30+
--db-repository strings OCI repository(ies) to retrieve trivy-db in order of priority (default [mirror.gcr.io/aquasec/trivy-db:2,ghcr.io/aquasecurity/trivy-db:2])
3131
--dependency-tree [EXPERIMENTAL] show dependency origin tree of vulnerable packages
3232
--detection-priority string specify the detection priority:
3333
- "precise": Prioritizes precise by minimizing false positives.
@@ -52,7 +52,7 @@ trivy vm [flags] VM_IMAGE
5252
--ignore-unfixed display only fixed vulnerabilities
5353
--ignorefile string specify .trivyignore file (default ".trivyignore")
5454
--include-non-failures include successes, available with '--scanners misconfig'
55-
--java-db-repository strings OCI repository(ies) to retrieve trivy-java-db in order of priority (default [ghcr.io/aquasecurity/trivy-java-db:1])
55+
--java-db-repository strings OCI repository(ies) to retrieve trivy-java-db in order of priority (default [mirror.gcr.io/aquasec/trivy-java-db:1,ghcr.io/aquasecurity/trivy-java-db:1])
5656
--list-all-pkgs output all packages in the JSON report regardless of vulnerability
5757
--misconfig-scanners strings comma-separated list of misconfig scanners to use for misconfiguration scanning (default [azure-arm,cloudformation,dockerfile,helm,kubernetes,terraform,terraformplan-json,terraformplan-snapshot])
5858
--module-dir string specify directory to the wasm modules that will be loaded (default "$HOME/.trivy/modules")

‎docs/docs/references/configuration/config-file.md

+2
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,7 @@ db:
105105

106106
# Same as '--java-db-repository'
107107
java-repository:
108+
- mirror.gcr.io/aquasec/trivy-java-db:1
108109
- ghcr.io/aquasecurity/trivy-java-db:1
109110

110111
# Same as '--skip-java-db-update'
@@ -115,6 +116,7 @@ db:
115116

116117
# Same as '--db-repository'
117118
repository:
119+
- mirror.gcr.io/aquasec/trivy-db:2
118120
- ghcr.io/aquasecurity/trivy-db:2
119121

120122
# Same as '--skip-db-update'

‎pkg/db/db.go

+5
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,10 @@ var (
2929
DefaultGHCRRepository = fmt.Sprintf("%s:%d", "ghcr.io/aquasecurity/trivy-db", db.SchemaVersion)
3030
defaultGHCRRepository = lo.Must(name.NewTag(DefaultGHCRRepository))
3131

32+
// GCR mirror
33+
DefaultGCRRepository = fmt.Sprintf("%s:%d", "mirror.gcr.io/aquasec/trivy-db", db.SchemaVersion)
34+
defaultGCRRepository = lo.Must(name.NewTag(DefaultGCRRepository))
35+
3236
Init = db.Init
3337
Close = db.Close
3438
Path = db.Path
@@ -73,6 +77,7 @@ func Dir(cacheDir string) string {
7377
func NewClient(dbDir string, quiet bool, opts ...Option) *Client {
7478
o := &options{
7579
dbRepositories: []name.Reference{
80+
defaultGCRRepository,
7681
defaultGHCRRepository,
7782
},
7883
}

‎pkg/fanal/image/registry/google/google.go

+8
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,15 @@ const gcrURLSuffix = ".gcr.io"
2828
// Google artifact registry
2929
const garURLSuffix = "-docker.pkg.dev"
3030

31+
// Google mirror registry
32+
const gmrURLDomain = "mirror.gcr.io"
33+
3134
func (g *Registry) CheckOptions(domain string, option types.RegistryOptions) (intf.RegistryClient, error) {
35+
// We assume there is no chance that `mirror.gcr.io` will require authentication.
36+
// So we need to skip `mirror.gcr.io` to avoid errors confusing users when downloading DB's.
37+
if domain == gmrURLDomain {
38+
return nil, xerrors.Errorf("mirror.gcr.io doesn't require authentication")
39+
}
3240
if domain != gcrURLDomain && !strings.HasSuffix(domain, gcrURLSuffix) && !strings.HasSuffix(domain, garURLSuffix) {
3341
return nil, xerrors.Errorf("Google registry: %w", types.InvalidURLPattern)
3442
}

‎pkg/flag/db_flags.go

+2-2
Original file line numberDiff line numberDiff line change
@@ -53,13 +53,13 @@ var (
5353
DBRepositoryFlag = Flag[[]string]{
5454
Name: "db-repository",
5555
ConfigName: "db.repository",
56-
Default: []string{db.DefaultGHCRRepository},
56+
Default: []string{db.DefaultGCRRepository, db.DefaultGHCRRepository},
5757
Usage: "OCI repository(ies) to retrieve trivy-db in order of priority",
5858
}
5959
JavaDBRepositoryFlag = Flag[[]string]{
6060
Name: "java-db-repository",
6161
ConfigName: "db.java-repository",
62-
Default: []string{javadb.DefaultGHCRRepository},
62+
Default: []string{javadb.DefaultGCRRepository, javadb.DefaultGHCRRepository},
6363
Usage: "OCI repository(ies) to retrieve trivy-java-db in order of priority",
6464
}
6565
LightFlag = Flag[bool]{

‎pkg/flag/db_flags_test.go

+10-4
Original file line numberDiff line numberDiff line change
@@ -66,10 +66,16 @@ func TestDBFlagGroup_ToOptions(t *testing.T) {
6666
{
6767
name: "multiple repos",
6868
fields: fields{
69-
SkipDBUpdate: true,
70-
DownloadDBOnly: false,
71-
DBRepository: []string{"ghcr.io/aquasecurity/trivy-db:2", "gallery.ecr.aws/aquasecurity/trivy-db:2"},
72-
JavaDBRepository: []string{"ghcr.io/aquasecurity/trivy-java-db:1", "gallery.ecr.aws/aquasecurity/trivy-java-db:1"},
69+
SkipDBUpdate: true,
70+
DownloadDBOnly: false,
71+
DBRepository: []string{
72+
"mirror.gcr.io/aquasec/trivy-db:2",
73+
"ghcr.io/aquasecurity/trivy-db:2",
74+
},
75+
JavaDBRepository: []string{
76+
"mirror.gcr.io/aquasec/trivy-java-db:1",
77+
"ghcr.io/aquasecurity/trivy-java-db:1",
78+
},
7379
},
7480
want: flag.DBOptions{
7581
SkipDBUpdate: true,

‎pkg/javadb/client.go

+3
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,9 @@ const (
2929
var (
3030
// GitHub Container Registry
3131
DefaultGHCRRepository = fmt.Sprintf("%s:%d", "ghcr.io/aquasecurity/trivy-java-db", SchemaVersion)
32+
33+
// GCR mirrors
34+
DefaultGCRRepository = fmt.Sprintf("%s:%d", "mirror.gcr.io/aquasec/trivy-java-db", SchemaVersion)
3235
)
3336

3437
var updater *Updater

0 commit comments

Comments
 (0)
Please sign in to comment.