Problem with using the --nvdDatafeed parameter

[JY1990]

Now with dependency-check version 9.3.0, the issue has been found with the parameter --nvdDatafeed
dependency-check.sh --project UI -s ./ui -o ./ui -f HTML -f JSON --retireJsUrl https://vuln-db-mirror.xxxxxxx.net/jsrepository.json --nvdDatafeed https://vuln-db-mirror.xxxxxxx.net/nvdcve-{0}.json.gz --suppression ./build/suppression.xml --hostedSuppressionsUrl https://vuln-db-mirror.axxxxxx.net/publishedSuppressions.xml --kevURL https://vuln-db-mirror.xxxxxxxx.net/known_exploited_vulnerabilities.json
Picked up _JAVA_OPTIONS:
[INFO] Checking for updates
[ERROR] Unable to download the NVD API cache.properties
org.owasp.dependencycheck.data.update.exception.UpdateException: Unable to download the NVD API cache.properties
at org.owasp.dependencycheck.data.update.NvdApiDataSource.getRemoteCacheProperties(NvdApiDataSource.java:558)
at org.owasp.dependencycheck.data.update.NvdApiDataSource.processDatafeed(NvdApiDataSource.java:130)
at org.owasp.dependencycheck.data.update.NvdApiDataSource.update(NvdApiDataSource.java:108)
at org.owasp.dependencycheck.Engine.doUpdates(Engine.java:906)
at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase(Engine.java:711)
at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:637)
at org.owasp.dependencycheck.App.runScan(App.java:262)
at org.owasp.dependencycheck.App.run(App.java:194)
at org.owasp.dependencycheck.App.main(App.java:89)
Caused by: org.owasp.dependencycheck.utils.ResourceNotFoundException: Requested resource does not exist - received a 404
at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection(HttpResourceConnection.java:233)
at org.owasp.dependencycheck.utils.HttpResourceConnection.fetch(HttpResourceConnection.java:163)
at org.owasp.dependencycheck.utils.Downloader.fetchContent(Downloader.java:183)
at org.owasp.dependencycheck.data.update.NvdApiDataSource.getRemoteCacheProperties(NvdApiDataSource.java:551)
... 8 common frames omitted
[INFO] Updating CISA Known Exploited Vulnerability list: https://vuln-db-mirror.xxxxxxxx.net/known_exploited_vulnerabilities.json
[INFO] Begin database defrag
[INFO] End database defrag (239 ms)
[WARN] Unable to update 1 or more Cached Web DataSource, using local data instead. Results may not include recent vulnerabilities.
[ERROR] Unable to continue dependency-check analysis.
[ERROR] One or more fatal errors occurred
[ERROR] Unable to download the NVD API cache.properties
[ERROR] No documents exist What can be the problem?

[jeremylong]

See #6243. Some web servers may not serve *.properties files by default. I'm likely going to stop using the cache.properties file and just rely on the traditional meta files which are being generated in 5.1.0 of the vulnz tool. But for now - I'd check the server configuration to ensure the properties file can be served by the web server.

[kirks]

Same error here. Local mirror set up, with nvd files going to a nist subdirectory. Configured similar to the cli docs https://internal.server/nist/nvdcve-{0}.json.gz. The request received by the mirror for cache.properties is GET /nist/nvdcve-/cache.properties HTTP/1.1 instead of GET /nist/cache.properties HTTP/1.1.

[jeremylong]

@kirks it's not that I don't believe you - it is just that no current code path would not create this condition.

DependencyCheck/core/src/main/java/org/owasp/dependencycheck/data/update/NvdApiDataSource.java

Lines 120 to 130 in c9c2bee

	if (nvdDataFeedUrl.endsWith(".json.gz")) {
	final int lio = nvdDataFeedUrl.lastIndexOf("/");
	pattern = nvdDataFeedUrl.substring(lio + 1);
	url = nvdDataFeedUrl.substring(0, lio);
	} else {
	url = nvdDataFeedUrl;
	}
	if (!url.endsWith("/")) {
	url += "/";
	}
	final Properties cacheProperties = getRemoteCacheProperties(url);

DependencyCheck/core/src/main/java/org/owasp/dependencycheck/data/update/NvdApiDataSource.java

Lines 547 to 549 in c9c2bee

	protected final Properties getRemoteCacheProperties(String url) throws UpdateException {
	try {
	final URL u = new URL(url + "cache.properties");

[kirks]

New param had to be quoted in powershell now after switching date param from %d to {0}.

[jeremylong]

Depending on how you are hosting the cache - the cache.properties may not be served by the HTTP server. If you are using vulnz 5.1.0 or newer to create the cache PR #6282 should resolve the issue. I'll release the updated version soon.