Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement cache-dependency-path option to control caching dependency #499

Merged
merged 1 commit into from
Nov 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
93 changes: 93 additions & 0 deletions .github/workflows/e2e-cache-dependency-path.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
name: Validate cache with cache-dependency-path option

on:
push:
branches:
- main
- releases/*
paths-ignore:
- '**.md'
pull_request:
paths-ignore:
- '**.md'

defaults:
run:
shell: bash

jobs:
gradle1-save:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [macos-latest, windows-latest, ubuntu-latest]
steps:
- name: Checkout
uses: actions/checkout@v3
MaksimZhukov marked this conversation as resolved.
Show resolved Hide resolved
- name: Run setup-java with the cache for gradle
uses: ./
id: setup-java
with:
distribution: 'adopt'
java-version: '11'
cache: gradle
cache-dependency-path: __tests__/cache/gradle1/*.gradle*
- name: Create files to cache
# Need to avoid using Gradle daemon to stabilize the save process on Windows
# https://github.com/actions/cache/issues/454#issuecomment-840493935
run: |
gradle downloadDependencies --no-daemon -p __tests__/cache/gradle1
if [ ! -d ~/.gradle/caches ]; then
echo "::error::The ~/.gradle/caches directory does not exist unexpectedly"
exit 1
fi
gradle1-restore:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [macos-latest, windows-latest, ubuntu-latest]
needs: gradle1-save
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Run setup-java with the cache for gradle
uses: ./
id: setup-java
with:
distribution: 'adopt'
java-version: '11'
cache: gradle
cache-dependency-path: __tests__/cache/gradle1/*.gradle*
- name: Confirm that ~/.gradle/caches directory has been made
run: |
if [ ! -d ~/.gradle/caches ]; then
echo "::error::The ~/.gradle/caches directory does not exist unexpectedly"
exit 1
fi
ls ~/.gradle/caches/
gradle2-restore:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [macos-latest, windows-latest, ubuntu-latest]
needs: gradle1-save
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Run setup-java with the cache for gradle
uses: ./
id: setup-java
with:
distribution: 'adopt'
java-version: '11'
cache: gradle
cache-dependency-path: __tests__/cache/gradle2/*.gradle*
- name: Confirm that ~/.gradle/caches directory has not been made
run: |
if [ -d ~/.gradle/caches ]; then
echo "::error::The ~/.gradle/caches directory exists unexpectedly"
exit 1
fi
2 changes: 1 addition & 1 deletion .github/workflows/e2e-cache.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:
# Need to avoid using Gradle daemon to stabilize the save process on Windows
# https://github.com/actions/cache/issues/454#issuecomment-840493935
run: |
gradle downloadDependencies --no-daemon -p __tests__/cache/gradle
gradle downloadDependencies --no-daemon -p __tests__/cache/gradle1
if [ ! -d ~/.gradle/caches ]; then
echo "::error::The ~/.gradle/caches directory does not exist unexpectedly"
exit 1
Expand Down
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@ This action allows you to work with Java and Scala projects.

- `cache`: Quick [setup caching](#caching-packages-dependencies) for the dependencies managed through one of the predefined package managers. It can be one of "maven", "gradle" or "sbt".

- `cache-dependency-path`: The path to a dependency file: pom.xml, build.gradle, build.sbt, etc. This option can be used with the `cache` option. If this option is omitted, the action searches for the dependency file in the entire repository. This option supports wildcards and a list of file names for caching multiple dependencies.

#### Maven options
The action has a bunch of inputs to generate maven's [settings.xml](https://maven.apache.org/settings.html) on the fly and pass the values to Apache Maven GPG Plugin as well as Apache Maven Toolchains. See [advanced usage](docs/advanced-usage.md) for more.

Expand Down Expand Up @@ -115,10 +117,13 @@ Currently, the following distributions are supported:

### Caching packages dependencies
The action has a built-in functionality for caching and restoring dependencies. It uses [toolkit/cache](https://github.com/actions/toolkit/tree/main/packages/cache) under hood for caching dependencies but requires less configuration settings. Supported package managers are gradle, maven and sbt. The format of the used cache key is `setup-java-${{ platform }}-${{ packageManager }}-${{ fileHash }}`, where the hash is based on the following files:

- gradle: `**/*.gradle*`, `**/gradle-wrapper.properties`, `buildSrc/**/Versions.kt`, `buildSrc/**/Dependencies.kt`, `gradle/*.versions.toml`, and `**/versions.properties`
- maven: `**/pom.xml`
- sbt: all sbt build definition files `**/*.sbt`, `**/project/build.properties`, `**/project/**.scala`, `**/project/**.sbt`

When the option `cache-dependency-path` is specified, the hash is based on the matching file. This option supports wildcards and a list of file names, and is especially useful for monorepos.

The workflow output `cache-hit` is set to indicate if an exact match was found for the key [as actions/cache does](https://github.com/actions/cache/tree/main#outputs).

The cache input is optional, and caching is turned off by default.
Expand All @@ -132,6 +137,9 @@ steps:
distribution: 'temurin'
java-version: '17'
cache: 'gradle'
cache-dependency-path: | # optional
sub-project/*.gradle*
sub-project/**/gradle-wrapper.properties
- run: ./gradlew build --no-daemon
```

Expand All @@ -144,6 +152,7 @@ steps:
distribution: 'temurin'
java-version: '17'
cache: 'maven'
cache-dependency-path: 'sub-project/pom.xml' # optional
- name: Build with Maven
run: mvn -B package --file pom.xml
```
Expand All @@ -157,6 +166,9 @@ steps:
distribution: 'temurin'
java-version: '17'
cache: 'sbt'
cache-dependency-path: | # optional
sub-project/build.sbt
sub-project/project/build.properties
- name: Build with SBT
run: sbt package
```
Expand Down
108 changes: 87 additions & 21 deletions __tests__/cache.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import * as fs from 'fs';
import * as os from 'os';
import * as core from '@actions/core';
import * as cache from '@actions/cache';
import * as glob from '@actions/glob';

describe('dependency cache', () => {
const ORIGINAL_RUNNER_OS = process.env['RUNNER_OS'];
Expand Down Expand Up @@ -64,25 +65,30 @@ describe('dependency cache', () => {
ReturnType<typeof cache.restoreCache>,
Parameters<typeof cache.restoreCache>
>;
let spyGlobHashFiles: jest.SpyInstance<
ReturnType<typeof glob.hashFiles>,
Parameters<typeof glob.hashFiles>
>;

beforeEach(() => {
spyCacheRestore = jest
.spyOn(cache, 'restoreCache')
.mockImplementation((paths: string[], primaryKey: string) =>
Promise.resolve(undefined)
);
spyGlobHashFiles = jest.spyOn(glob, 'hashFiles');
spyWarning.mockImplementation(() => null);
});

it('throws error if unsupported package manager specified', () => {
return expect(restore('ant')).rejects.toThrow(
return expect(restore('ant', '')).rejects.toThrow(
'unknown package manager specified: ant'
);
});

describe('for maven', () => {
it('throws error if no pom.xml found', async () => {
await expect(restore('maven')).rejects.toThrow(
await expect(restore('maven', '')).rejects.toThrow(
`No file in ${projectRoot(
workspace
)} matched to [**/pom.xml], make sure you have checked out the target repository`
Expand All @@ -91,15 +97,16 @@ describe('dependency cache', () => {
it('downloads cache', async () => {
createFile(join(workspace, 'pom.xml'));

await restore('maven');
await restore('maven', '');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith('**/pom.xml');
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('maven cache is not found');
});
});
describe('for gradle', () => {
it('throws error if no build.gradle found', async () => {
await expect(restore('gradle')).rejects.toThrow(
await expect(restore('gradle', '')).rejects.toThrow(
`No file in ${projectRoot(
workspace
)} matched to [**/*.gradle*,**/gradle-wrapper.properties,buildSrc/**/Versions.kt,buildSrc/**/Dependencies.kt,gradle/*.versions.toml,**/versions.properties], make sure you have checked out the target repository`
Expand All @@ -108,41 +115,53 @@ describe('dependency cache', () => {
it('downloads cache based on build.gradle', async () => {
createFile(join(workspace, 'build.gradle'));

await restore('gradle');
await restore('gradle', '');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith(
'**/*.gradle*\n**/gradle-wrapper.properties\nbuildSrc/**/Versions.kt\nbuildSrc/**/Dependencies.kt\ngradle/*.versions.toml\n**/versions.properties'
);
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
});
it('downloads cache based on build.gradle.kts', async () => {
createFile(join(workspace, 'build.gradle.kts'));

await restore('gradle');
await restore('gradle', '');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith(
'**/*.gradle*\n**/gradle-wrapper.properties\nbuildSrc/**/Versions.kt\nbuildSrc/**/Dependencies.kt\ngradle/*.versions.toml\n**/versions.properties'
);
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
});
it('downloads cache based on libs.versions.toml', async () => {
createDirectory(join(workspace, 'gradle'));
createFile(join(workspace, 'gradle', 'libs.versions.toml'));

await restore('gradle');
await restore('gradle', '');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith(
'**/*.gradle*\n**/gradle-wrapper.properties\nbuildSrc/**/Versions.kt\nbuildSrc/**/Dependencies.kt\ngradle/*.versions.toml\n**/versions.properties'
);
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
});
});
it('downloads cache based on buildSrc/Versions.kt', async () => {
createDirectory(join(workspace, 'buildSrc'));
createFile(join(workspace, 'buildSrc', 'Versions.kt'));
it('downloads cache based on buildSrc/Versions.kt', async () => {
createDirectory(join(workspace, 'buildSrc'));
createFile(join(workspace, 'buildSrc', 'Versions.kt'));

await restore('gradle');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
await restore('gradle', '');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith(
'**/*.gradle*\n**/gradle-wrapper.properties\nbuildSrc/**/Versions.kt\nbuildSrc/**/Dependencies.kt\ngradle/*.versions.toml\n**/versions.properties'
);
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
});
});
describe('for sbt', () => {
it('throws error if no build.sbt found', async () => {
await expect(restore('sbt')).rejects.toThrow(
await expect(restore('sbt', '')).rejects.toThrow(
`No file in ${projectRoot(
workspace
)} matched to [**/*.sbt,**/project/build.properties,**/project/**.scala,**/project/**.sbt], make sure you have checked out the target repository`
Expand All @@ -151,8 +170,11 @@ describe('dependency cache', () => {
it('downloads cache', async () => {
createFile(join(workspace, 'build.sbt'));

await restore('sbt');
await restore('sbt', '');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith(
'**/*.sbt\n**/project/build.properties\n**/project/**.scala\n**/project/**.sbt'
);
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('sbt cache is not found');
});
Expand All @@ -161,19 +183,19 @@ describe('dependency cache', () => {
createDirectory(join(workspace, 'project'));
createFile(join(workspace, 'project/DependenciesV1.scala'));

await restore('sbt');
await restore('sbt', '');
const firstCall = spySaveState.mock.calls.toString();

spySaveState.mockClear();
await restore('sbt');
await restore('sbt', '');
const secondCall = spySaveState.mock.calls.toString();

// Make sure multiple restores produce the same cache
expect(firstCall).toBe(secondCall);

spySaveState.mockClear();
createFile(join(workspace, 'project/DependenciesV2.scala'));
await restore('sbt');
await restore('sbt', '');
const thirdCall = spySaveState.mock.calls.toString();

expect(firstCall).not.toBe(thirdCall);
Expand All @@ -182,11 +204,55 @@ describe('dependency cache', () => {
it('downloads cache based on versions.properties', async () => {
createFile(join(workspace, 'versions.properties'));

await restore('gradle');
await restore('gradle', '');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith(
'**/*.gradle*\n**/gradle-wrapper.properties\nbuildSrc/**/Versions.kt\nbuildSrc/**/Dependencies.kt\ngradle/*.versions.toml\n**/versions.properties'
);
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
});
describe('cache-dependency-path', () => {
it('throws error if no matching dependency file found', async () => {
createFile(join(workspace, 'build.gradle.kts'));
await expect(
restore('gradle', 'sub-project/**/build.gradle.kts')
).rejects.toThrow(
`No file in ${projectRoot(
workspace
)} matched to [sub-project/**/build.gradle.kts], make sure you have checked out the target repository`
);
});
it('downloads cache based on the specified pattern', async () => {
createFile(join(workspace, 'build.gradle.kts'));
createDirectory(join(workspace, 'sub-project1'));
createFile(join(workspace, 'sub-project1', 'build.gradle.kts'));
createDirectory(join(workspace, 'sub-project2'));
createFile(join(workspace, 'sub-project2', 'build.gradle.kts'));

await restore('gradle', 'build.gradle.kts');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith('build.gradle.kts');
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');

await restore('gradle', 'sub-project1/**/*.gradle*\n');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith(
'sub-project1/**/*.gradle*'
);
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');

await restore('gradle', '*.gradle*\nsub-project2/**/*.gradle*\n');
expect(spyCacheRestore).toHaveBeenCalled();
expect(spyGlobHashFiles).toHaveBeenCalledWith(
'*.gradle*\nsub-project2/**/*.gradle*'
);
expect(spyWarning).not.toHaveBeenCalled();
expect(spyInfo).toHaveBeenCalledWith('gradle cache is not found');
});
});
});
describe('save', () => {
let spyCacheSave: jest.SpyInstance<
Expand Down
12 changes: 12 additions & 0 deletions __tests__/cache/gradle2/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
.gradle
**/build/
!src/**/build/

# Ignore Gradle GUI config
gradle-app.setting

# Avoid ignoring Gradle wrapper jar file (.jar files are usually ignored)
!gradle-wrapper.jar

# Cache of project
.gradletasknamecache