Skip to content

Commit

Permalink
feat: implement cache-dependency-path option to control caching depen…
Browse files Browse the repository at this point in the history 
…dency
  • Loading branch information
@itchyny
itchyny committed Jun 10, 2023
1 parent 87c1c70 commit a56fdde
Show file tree
Hide file tree
Showing 7 changed files with 108 additions and 44 deletions.
12 changes: 12 additions & 0 deletions README.md
View file
Expand Up @@ -41,6 +41,8 @@ This action allows you to work with Java and Scala projects.

- `cache`: Quick [setup caching](#caching-packages-dependencies) for the dependencies managed through one of the predifined package managers. It can be one of "maven", "gradle" or "sbt".

- `cache-dependency-path`: The path to a dependency file: pom.xml, build.gradle, build.sbt, etc. This option can be used with the `cache` option. If this option is omitted, the action searches for the dependency file in the entire repository. This option supports wildcards and a list of file names for caching multiple dependencies.

#### Maven options
The action has a bunch of inputs to generate maven's [settings.xml](https://maven.apache.org/settings.html) on the fly and pass the values to Apache Maven GPG Plugin as well as Apache Maven Toolchains. See [advanced usage](docs/advanced-usage.md) for more.

Expand Down Expand Up @@ -114,10 +116,13 @@ Currently, the following distributions are supported:

### Caching packages dependencies
The action has a built-in functionality for caching and restoring dependencies. It uses [actions/cache](https://github.com/actions/cache) under hood for caching dependencies but requires less configuration settings. Supported package managers are gradle, maven and sbt. The format of the used cache key is `setup-java-${{ platform }}-${{ packageManager }}-${{ fileHash }}`, where the hash is based on the following files:

- gradle: `**/*.gradle*`, `**/gradle-wrapper.properties`, `buildSrc/**/Versions.kt`, `buildSrc/**/Dependencies.kt`, and `gradle/*.versions.toml`
- maven: `**/pom.xml`
- sbt: all sbt build definition files `**/*.sbt`, `**/project/build.properties`, `**/project/**.scala`, `**/project/**.sbt`

When the option `cache-dependency-path` is specified, the hash is based on the matching file. This option supports wildcards and a list of file names, and is especially useful for monorepos.

The workflow output `cache-hit` is set to indicate if an exact match was found for the key [as actions/cache does](https://github.com/actions/cache/tree/main#outputs).

The cache input is optional, and caching is turned off by default.
Expand All @@ -131,6 +136,9 @@ steps:
distribution: 'temurin'
java-version: '17'
cache: 'gradle'
cache-dependency-path: | # optional
sub-project/*.gradle*
sub-project/**/gradle-wrapper.properties
- run: ./gradlew build --no-daemon
```

Expand All @@ -143,6 +151,7 @@ steps:
distribution: 'temurin'
java-version: '17'
cache: 'maven'
cache-dependency-path: 'sub-project/pom.xml' # optional
- name: Build with Maven
run: mvn -B package --file pom.xml
```
Expand All @@ -156,6 +165,9 @@ steps:
distribution: 'temurin'
java-version: '17'
cache: 'sbt'
cache-dependency-path: | # optional
sub-project/build.sbt
sub-project/project/build.properties
- name: Build with SBT
run: sbt package
```
Expand Down
63 changes: 55 additions & 8 deletions __tests__/cache.test.ts
View file
Expand Up @@ -6,6 +6,7 @@ import * as fs from 'fs';
import * as os from 'os';
import * as core from '@actions/core';
import * as cache from '@actions/cache';
import * as glob from '@actions/glob';

describe('dependency cache', () => {
const ORIGINAL_RUNNER_OS = process.env['RUNNER_OS'];
Expand Down Expand Up @@ -64,13 +65,18 @@ describe('dependency cache', () => {
ReturnType<typeof cache.restoreCache>,
Parameters<typeof cache.restoreCache>
>;
let spyGlobHashFiles: jest.SpyInstance<
ReturnType<typeof glob.hashFiles>,
Parameters<typeof glob.hashFiles>
>;

beforeEach(() => {
spyCacheRestore = jest
.spyOn(cache, 'restoreCache')
.mockImplementation((paths: string[], primaryKey: string) =>
Promise.resolve(undefined)
);
spyGlobHashFiles = jest.spyOn(glob, 'hashFiles');
spyWarning.mockImplementation(() => null);
});

Expand All @@ -93,6 +99,7 @@ describe('dependency cache', () => {

await restore('maven');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith('**/pom.xml');
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('maven cache is not found');
});
Expand All @@ -110,6 +117,7 @@ describe('dependency cache', () => {

await restore('gradle');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith('**/*.gradle*\n**/gradle-wrapper.properties\nbuildSrc/**/Versions.kt\nbuildSrc/**/Dependencies.kt\ngradle/*.versions.toml');
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
});
Expand All @@ -118,6 +126,7 @@ describe('dependency cache', () => {

await restore('gradle');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith('**/*.gradle*\n**/gradle-wrapper.properties\nbuildSrc/**/Versions.kt\nbuildSrc/**/Dependencies.kt\ngradle/*.versions.toml');
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
});
Expand All @@ -127,18 +136,20 @@ describe('dependency cache', () => {

await restore('gradle');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith('**/*.gradle*\n**/gradle-wrapper.properties\nbuildSrc/**/Versions.kt\nbuildSrc/**/Dependencies.kt\ngradle/*.versions.toml');
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
});
});
it('downloads cache based on buildSrc/Versions.kt', async () => {
createDirectory(join(workspace, 'buildSrc'));
createFile(join(workspace, 'buildSrc', 'Versions.kt'));
it('downloads cache based on buildSrc/Versions.kt', async () => {
createDirectory(join(workspace, 'buildSrc'));
createFile(join(workspace, 'buildSrc', 'Versions.kt'));

await restore('gradle');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
await restore('gradle');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith('**/*.gradle*\n**/gradle-wrapper.properties\nbuildSrc/**/Versions.kt\nbuildSrc/**/Dependencies.kt\ngradle/*.versions.toml');
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
});
});
describe('for sbt', () => {
it('throws error if no build.sbt found', async () => {
Expand All @@ -153,6 +164,7 @@ describe('dependency cache', () => {

await restore('sbt');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith('**/*.sbt\n**/project/build.properties\n**/project/**.scala\n**/project/**.sbt');
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('sbt cache is not found');
});
Expand All @@ -179,6 +191,41 @@ describe('dependency cache', () => {
expect(firstCall).not.toBe(thirdCall);
});
});
describe('cache-dependency-path', () => {
it('throws error if no matching dependency file found', async () => {
createFile(join(workspace, 'build.gradle.kts'));
await expect(restore('gradle', 'sub-project/**/build.gradle.kts')).rejects.toThrow(
`No file in ${projectRoot(
workspace
)} matched to [sub-project/**/build.gradle.kts], make sure you have checked out the target repository`
);
});
it('downloads cache based on the specified pattern', async () => {
createFile(join(workspace, 'build.gradle.kts'));
createDirectory(join(workspace, 'sub-project1'));
createFile(join(workspace, 'sub-project1', 'build.gradle.kts'));
createDirectory(join(workspace, 'sub-project2'));
createFile(join(workspace, 'sub-project2', 'build.gradle.kts'));

await restore('gradle', 'build.gradle.kts');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith('build.gradle.kts');
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');

await restore('gradle', 'sub-project1/**/*.gradle*\n');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith('sub-project1/**/*.gradle*');
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');

await restore('gradle', '*.gradle*\nsub-project2/**/*.gradle*\n');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith('*.gradle*\nsub-project2/**/*.gradle*');
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
});
});
});
describe('save', () => {
let spyCacheSave: jest.SpyInstance<
Expand Down
22 changes: 12 additions & 10 deletions dist/cleanup/index.js
View file
Expand Up @@ -68438,28 +68438,29 @@ function findPackageManager(id) {
/**
* A function that generates a cache key to use.
* Format of the generated key will be "${{ platform }}-${{ id }}-${{ fileHash }}"".
* If there is no file matched to {@link PackageManager.path}, the generated key ends with a dash (-).
* @see {@link https://docs.github.com/en/actions/guides/caching-dependencies-to-speed-up-workflows#matching-a-cache-key|spec of cache key}
*/
function computeCacheKey(packageManager) {
function computeCacheKey(packageManager, cacheDependencyPath) {
return __awaiter(this, void 0, void 0, function* () {
const hash = yield glob.hashFiles(packageManager.pattern.join('\n'));
return `${CACHE_KEY_PREFIX}-${process.env['RUNNER_OS']}-${packageManager.id}-${hash}`;
const pattern = cacheDependencyPath ? cacheDependencyPath.trim().split('\n') : packageManager.pattern;
const fileHash = yield glob.hashFiles(pattern.join('\n'));
if (!fileHash) {
throw new Error(`No file in ${process.cwd()} matched to [${pattern}], make sure you have checked out the target repository`);
}
return `${CACHE_KEY_PREFIX}-${process.env['RUNNER_OS']}-${packageManager.id}-${fileHash}`;
});
}
/**
* Restore the dependency cache
* @param id ID of the package manager, should be "maven" or "gradle"
* @param cacheDependencyPath The path to a dependency file
*/
function restore(id) {
function restore(id, cacheDependencyPath) {
return __awaiter(this, void 0, void 0, function* () {
const packageManager = findPackageManager(id);
const primaryKey = yield computeCacheKey(packageManager);
const primaryKey = yield computeCacheKey(packageManager, cacheDependencyPath);
core.debug(`primary key is ${primaryKey}`);
core.saveState(STATE_CACHE_PRIMARY_KEY, primaryKey);
if (primaryKey.endsWith('-')) {
throw new Error(`No file in ${process.cwd()} matched to [${packageManager.pattern}], make sure you have checked out the target repository`);
}
// No "restoreKeys" is set, to start with a clear cache after dependency update (see https://github.com/actions/setup-java/issues/269)
const matchedKey = yield cache.restoreCache(packageManager.path, primaryKey);
if (matchedKey) {
Expand Down Expand Up @@ -68636,7 +68637,7 @@ else {
"use strict";

Object.defineProperty(exports, "__esModule", ({ value: true }));
exports.DISTRIBUTIONS_ONLY_MAJOR_VERSION = exports.INPUT_MVN_TOOLCHAIN_VENDOR = exports.INPUT_MVN_TOOLCHAIN_ID = exports.MVN_TOOLCHAINS_FILE = exports.MVN_SETTINGS_FILE = exports.M2_DIR = exports.STATE_GPG_PRIVATE_KEY_FINGERPRINT = exports.INPUT_JOB_STATUS = exports.INPUT_CACHE = exports.INPUT_DEFAULT_GPG_PASSPHRASE = exports.INPUT_DEFAULT_GPG_PRIVATE_KEY = exports.INPUT_GPG_PASSPHRASE = exports.INPUT_GPG_PRIVATE_KEY = exports.INPUT_OVERWRITE_SETTINGS = exports.INPUT_SETTINGS_PATH = exports.INPUT_SERVER_PASSWORD = exports.INPUT_SERVER_USERNAME = exports.INPUT_SERVER_ID = exports.INPUT_CHECK_LATEST = exports.INPUT_JDK_FILE = exports.INPUT_DISTRIBUTION = exports.INPUT_JAVA_PACKAGE = exports.INPUT_ARCHITECTURE = exports.INPUT_JAVA_VERSION_FILE = exports.INPUT_JAVA_VERSION = exports.MACOS_JAVA_CONTENT_POSTFIX = void 0;
exports.DISTRIBUTIONS_ONLY_MAJOR_VERSION = exports.INPUT_MVN_TOOLCHAIN_VENDOR = exports.INPUT_MVN_TOOLCHAIN_ID = exports.MVN_TOOLCHAINS_FILE = exports.MVN_SETTINGS_FILE = exports.M2_DIR = exports.STATE_GPG_PRIVATE_KEY_FINGERPRINT = exports.INPUT_JOB_STATUS = exports.INPUT_CACHE_DEPENDENCY_PATH = exports.INPUT_CACHE = exports.INPUT_DEFAULT_GPG_PASSPHRASE = exports.INPUT_DEFAULT_GPG_PRIVATE_KEY = exports.INPUT_GPG_PASSPHRASE = exports.INPUT_GPG_PRIVATE_KEY = exports.INPUT_OVERWRITE_SETTINGS = exports.INPUT_SETTINGS_PATH = exports.INPUT_SERVER_PASSWORD = exports.INPUT_SERVER_USERNAME = exports.INPUT_SERVER_ID = exports.INPUT_CHECK_LATEST = exports.INPUT_JDK_FILE = exports.INPUT_DISTRIBUTION = exports.INPUT_JAVA_PACKAGE = exports.INPUT_ARCHITECTURE = exports.INPUT_JAVA_VERSION_FILE = exports.INPUT_JAVA_VERSION = exports.MACOS_JAVA_CONTENT_POSTFIX = void 0;
exports.MACOS_JAVA_CONTENT_POSTFIX = 'Contents/Home';
exports.INPUT_JAVA_VERSION = 'java-version';
exports.INPUT_JAVA_VERSION_FILE = 'java-version-file';
Expand All @@ -68655,6 +68656,7 @@ exports.INPUT_GPG_PASSPHRASE = 'gpg-passphrase';
exports.INPUT_DEFAULT_GPG_PRIVATE_KEY = undefined;
exports.INPUT_DEFAULT_GPG_PASSPHRASE = 'GPG_PASSPHRASE';
exports.INPUT_CACHE = 'cache';
exports.INPUT_CACHE_DEPENDENCY_PATH = 'cache-dependency-path';
exports.INPUT_JOB_STATUS = 'job-status';
exports.STATE_GPG_PRIVATE_KEY_FINGERPRINT = 'gpg-private-key-fingerprint';
exports.M2_DIR = '.m2';
Expand Down
25 changes: 14 additions & 11 deletions dist/setup/index.js
View file
Expand Up @@ -103643,28 +103643,29 @@ function findPackageManager(id) {
/**
* A function that generates a cache key to use.
* Format of the generated key will be "${{ platform }}-${{ id }}-${{ fileHash }}"".
* If there is no file matched to {@link PackageManager.path}, the generated key ends with a dash (-).
* @see {@link https://docs.github.com/en/actions/guides/caching-dependencies-to-speed-up-workflows#matching-a-cache-key|spec of cache key}
*/
function computeCacheKey(packageManager) {
function computeCacheKey(packageManager, cacheDependencyPath) {
return __awaiter(this, void 0, void 0, function* () {
const hash = yield glob.hashFiles(packageManager.pattern.join('\n'));
return `${CACHE_KEY_PREFIX}-${process.env['RUNNER_OS']}-${packageManager.id}-${hash}`;
const pattern = cacheDependencyPath ? cacheDependencyPath.trim().split('\n') : packageManager.pattern;
const fileHash = yield glob.hashFiles(pattern.join('\n'));
if (!fileHash) {
throw new Error(`No file in ${process.cwd()} matched to [${pattern}], make sure you have checked out the target repository`);
}
return `${CACHE_KEY_PREFIX}-${process.env['RUNNER_OS']}-${packageManager.id}-${fileHash}`;
});
}
/**
* Restore the dependency cache
* @param id ID of the package manager, should be "maven" or "gradle"
* @param cacheDependencyPath The path to a dependency file
*/
function restore(id) {
function restore(id, cacheDependencyPath) {
return __awaiter(this, void 0, void 0, function* () {
const packageManager = findPackageManager(id);
const primaryKey = yield computeCacheKey(packageManager);
const primaryKey = yield computeCacheKey(packageManager, cacheDependencyPath);
core.debug(`primary key is ${primaryKey}`);
core.saveState(STATE_CACHE_PRIMARY_KEY, primaryKey);
if (primaryKey.endsWith('-')) {
throw new Error(`No file in ${process.cwd()} matched to [${packageManager.pattern}], make sure you have checked out the target repository`);
}
// No "restoreKeys" is set, to start with a clear cache after dependency update (see https://github.com/actions/setup-java/issues/269)
const matchedKey = yield cache.restoreCache(packageManager.path, primaryKey);
if (matchedKey) {
Expand Down Expand Up @@ -103740,7 +103741,7 @@ function isProbablyGradleDaemonProblem(packageManager, error) {
"use strict";

Object.defineProperty(exports, "__esModule", ({ value: true }));
exports.DISTRIBUTIONS_ONLY_MAJOR_VERSION = exports.INPUT_MVN_TOOLCHAIN_VENDOR = exports.INPUT_MVN_TOOLCHAIN_ID = exports.MVN_TOOLCHAINS_FILE = exports.MVN_SETTINGS_FILE = exports.M2_DIR = exports.STATE_GPG_PRIVATE_KEY_FINGERPRINT = exports.INPUT_JOB_STATUS = exports.INPUT_CACHE = exports.INPUT_DEFAULT_GPG_PASSPHRASE = exports.INPUT_DEFAULT_GPG_PRIVATE_KEY = exports.INPUT_GPG_PASSPHRASE = exports.INPUT_GPG_PRIVATE_KEY = exports.INPUT_OVERWRITE_SETTINGS = exports.INPUT_SETTINGS_PATH = exports.INPUT_SERVER_PASSWORD = exports.INPUT_SERVER_USERNAME = exports.INPUT_SERVER_ID = exports.INPUT_CHECK_LATEST = exports.INPUT_JDK_FILE = exports.INPUT_DISTRIBUTION = exports.INPUT_JAVA_PACKAGE = exports.INPUT_ARCHITECTURE = exports.INPUT_JAVA_VERSION_FILE = exports.INPUT_JAVA_VERSION = exports.MACOS_JAVA_CONTENT_POSTFIX = void 0;
exports.DISTRIBUTIONS_ONLY_MAJOR_VERSION = exports.INPUT_MVN_TOOLCHAIN_VENDOR = exports.INPUT_MVN_TOOLCHAIN_ID = exports.MVN_TOOLCHAINS_FILE = exports.MVN_SETTINGS_FILE = exports.M2_DIR = exports.STATE_GPG_PRIVATE_KEY_FINGERPRINT = exports.INPUT_JOB_STATUS = exports.INPUT_CACHE_DEPENDENCY_PATH = exports.INPUT_CACHE = exports.INPUT_DEFAULT_GPG_PASSPHRASE = exports.INPUT_DEFAULT_GPG_PRIVATE_KEY = exports.INPUT_GPG_PASSPHRASE = exports.INPUT_GPG_PRIVATE_KEY = exports.INPUT_OVERWRITE_SETTINGS = exports.INPUT_SETTINGS_PATH = exports.INPUT_SERVER_PASSWORD = exports.INPUT_SERVER_USERNAME = exports.INPUT_SERVER_ID = exports.INPUT_CHECK_LATEST = exports.INPUT_JDK_FILE = exports.INPUT_DISTRIBUTION = exports.INPUT_JAVA_PACKAGE = exports.INPUT_ARCHITECTURE = exports.INPUT_JAVA_VERSION_FILE = exports.INPUT_JAVA_VERSION = exports.MACOS_JAVA_CONTENT_POSTFIX = void 0;
exports.MACOS_JAVA_CONTENT_POSTFIX = 'Contents/Home';
exports.INPUT_JAVA_VERSION = 'java-version';
exports.INPUT_JAVA_VERSION_FILE = 'java-version-file';
Expand All @@ -103759,6 +103760,7 @@ exports.INPUT_GPG_PASSPHRASE = 'gpg-passphrase';
exports.INPUT_DEFAULT_GPG_PRIVATE_KEY = undefined;
exports.INPUT_DEFAULT_GPG_PASSPHRASE = 'GPG_PASSPHRASE';
exports.INPUT_CACHE = 'cache';
exports.INPUT_CACHE_DEPENDENCY_PATH = 'cache-dependency-path';
exports.INPUT_JOB_STATUS = 'job-status';
exports.STATE_GPG_PRIVATE_KEY_FINGERPRINT = 'gpg-private-key-fingerprint';
exports.M2_DIR = '.m2';
Expand Down Expand Up @@ -105565,6 +105567,7 @@ function run() {
const packageType = core.getInput(constants.INPUT_JAVA_PACKAGE);
const jdkFile = core.getInput(constants.INPUT_JDK_FILE);
const cache = core.getInput(constants.INPUT_CACHE);
const cacheDependencyPath = core.getInput(constants.INPUT_CACHE_DEPENDENCY_PATH);
const checkLatest = util_1.getBooleanInput(constants.INPUT_CHECK_LATEST, false);
let toolchainIds = core.getMultilineInput(constants.INPUT_MVN_TOOLCHAIN_ID);
core.startGroup('Installed distributions');
Expand Down Expand Up @@ -105600,7 +105603,7 @@ function run() {
core.info(`##[add-matcher]${path.join(matchersPath, 'java.json')}`);
yield auth.configureAuthentication();
if (cache && util_1.isCacheFeatureAvailable()) {
yield cache_1.restore(cache);
yield cache_1.restore(cache, cacheDependencyPath);
}
}
catch (error) {
Expand Down

0 comments on commit a56fdde

Please sign in to comment.