Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improvements to caching #265

Merged
merged 4 commits into from
Feb 13, 2024
Merged

Improvements to caching #265

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f6c64a6
Bump actions/cache to v4 to suppress node warnings
jorgenpt Feb 4, 2024
07e9bc8
Respect `PUB_CACHE` from the environment
jorgenpt Feb 4, 2024
7f76ed3
Cache pub-cache separately from flutter
jorgenpt Feb 4, 2024
080acc0
Fix README example of pub-cache-path and document `PUB_CACHE`
jorgenpt Feb 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 5 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,11 +171,13 @@ steps:
cache: true
cache-key: 'flutter-:os:-:channel:-:version:-:arch:-:hash:' # optional, change this to force refresh cache
cache-path: '${{ runner.tool_cache }}/flutter/:channel:-:version:-:arch:' # optional, change this to specify the cache path
pub-cache-key: 'flutter-pub:os:-:channel:-:version:-:arch:-:hash:' # optional, change this to force refresh cache of dart pub get dependencies
cache-path: '${{ runner.tool_cache }}/flutter/:channel:-:version:-:arch:' # optional, change this to specify the cache path
jorgenpt marked this conversation as resolved.
Show resolved Hide resolved
architecture: x64 # optional, x64 or arm64
- run: flutter --version
```

Note: `cache-key` and `cache-path` has support for several dynamic values:
Note: `cache-key`, `pub-cache-key`, and `cache-path` has support for several dynamic values:

- `:os:`
- `:channel:`
Expand All @@ -199,5 +201,7 @@ steps:
echo CHANNEL=${{ steps.flutter-action.outputs.CHANNEL }}
echo VERSION=${{ steps.flutter-action.outputs.VERSION }}
echo ARCHITECTURE=${{ steps.flutter-action.outputs.ARCHITECTURE }}
echo PUB-CACHE-PATH=${{ steps.flutter-action.outputs.PUB-CACHE-PATH }}
echo PUB-CACHE-KEY=${{ steps.flutter-action.outputs.PUB-CACHE-KEY }}
shell: bash
```
25 changes: 20 additions & 5 deletions action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,15 @@ inputs:
cache:
description: 'Cache the Flutter SDK'
required: false
default: false
default: 'false'
cache-key:
description: 'Identifier for the Flutter SDK cache'
required: false
default: 'flutter-:os:-:channel:-:version:-:arch:-:hash:'
pub-cache-key:
description: 'Identifier for the Dart .pub-cache cache'
required: false
default: 'flutter-pub:os:-:channel:-:version:-:arch:-:hash:'
cache-path:
description: 'Flutter SDK cache path'
required: false
Expand All @@ -40,21 +44,32 @@ outputs:
value: '${{ steps.flutter-action.outputs.VERSION }}'
ARCHITECTURE:
value: '${{ steps.flutter-action.outputs.ARCHITECTURE }}'
PUB-CACHE-KEY:
value: '${{ steps.flutter-action.outputs.PUB-CACHE-KEY }}'
PUB-CACHE-PATH:
value: '${{ steps.flutter-action.outputs.PUB-CACHE-PATH }}'
runs:
using: 'composite'
steps:
- run: chmod +x $GITHUB_ACTION_PATH/setup.sh
shell: bash
- id: flutter-action
run: $GITHUB_ACTION_PATH/setup.sh -p -c '${{ inputs.cache-path }}' -k '${{ inputs.cache-key }}' -n '${{ inputs.flutter-version }}' -a '${{ inputs.architecture }}' ${{ inputs.channel }}
run: $GITHUB_ACTION_PATH/setup.sh -p -c '${{ inputs.cache-path }}' -k '${{ inputs.cache-key }}' -d '${{ inputs.pub-cache-key }}' -n '${{ inputs.flutter-version }}' -a '${{ inputs.architecture }}' ${{ inputs.channel }}
shell: bash
- if: ${{ inputs.cache == 'true' }}
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: ${{ steps.flutter-action.outputs.CACHE-PATH }}
key: ${{ steps.flutter-action.outputs.CACHE-KEY }}-${{ hashFiles('**/pubspec.lock') }}
key: ${{ steps.flutter-action.outputs.CACHE-KEY }}
jorgenpt marked this conversation as resolved.
Show resolved Hide resolved
restore-keys: |
${{ steps.flutter-action.outputs.CACHE-KEY }}-${{ hashFiles('**/pubspec.lock') }}
${{ steps.flutter-action.outputs.CACHE-KEY }}
- if: ${{ inputs.cache == 'true' }}
uses: actions/cache@v4
with:
path: ${{ steps.flutter-action.outputs.PUB-CACHE-PATH }}
key: ${{ steps.flutter-action.outputs.PUB-CACHE-KEY }}-${{ hashFiles('**/pubspec.lock') }}
restore-keys: |
${{ steps.flutter-action.outputs.PUB-CACHE-KEY }}-${{ hashFiles('**/pubspec.lock') }}
${{ steps.flutter-action.outputs.PUB-CACHE-KEY }}
- run: $GITHUB_ACTION_PATH/setup.sh -c '${{ steps.flutter-action.outputs.CACHE-PATH }}' -n '${{ steps.flutter-action.outputs.VERSION }}' -a '${{ steps.flutter-action.outputs.ARCHITECTURE }}' ${{ steps.flutter-action.outputs.CHANNEL }}
shell: bash
15 changes: 12 additions & 3 deletions setup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,15 +66,17 @@ download_archive() {

CACHE_PATH=""
CACHE_KEY=""
PUB_CACHE_KEY=""
PRINT_ONLY=""
TEST_MODE=false
ARCH=""
VERSION=""

while getopts 'tc:k:pa:n:' flag; do
while getopts 'tc:k:d:pa:n:' flag; do
case "$flag" in
c) CACHE_PATH="$OPTARG" ;;
k) CACHE_KEY="$OPTARG" ;;
d) PUB_CACHE_KEY="$OPTARG" ;;
p) PRINT_ONLY=true ;;
t) TEST_MODE=true ;;
a) ARCH="$(echo "$OPTARG" | awk '{print tolower($0)}')" ;;
Expand All @@ -91,6 +93,8 @@ CHANNEL="${ARR_CHANNEL[0]}"
[[ -z $ARCH ]] && ARCH=x64
[[ -z $CACHE_PATH ]] && CACHE_PATH="$RUNNER_TEMP/flutter/:channel:-:version:-:arch:"
[[ -z $CACHE_KEY ]] && CACHE_KEY="flutter-:os:-:channel:-:version:-:arch:-:hash:"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove -:hash: from here? It makes self-hosted runners to re-download flutter from cache every time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vladimir-goldobin I'm not clear as to why :hash: is a problem here? As I understand it, that's the hash from the manifest (e.g. https://storage.googleapis.com/flutter_infra_release/releases/releases_macos.json) and the hash of the release, independent of the runner.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think :hash: shouldn't be a problem here.

[[ -z $PUB_CACHE_KEY ]] && PUB_CACHE_KEY="flutter-pub-:os:-:channel:-:version:-:arch:-:hash:"
[[ -z $PUB_CACHE ]] && PUB_CACHE="$HOME/.pub-cache"
jorgenpt marked this conversation as resolved.
Show resolved Hide resolved

if [[ "$TEST_MODE" == true ]]; then
RELEASE_MANIFEST=$(cat "$(dirname -- "${BASH_SOURCE[0]}")/test/$MANIFEST_JSON_PATH")
Expand Down Expand Up @@ -127,6 +131,7 @@ expand_key() {
}

CACHE_KEY=$(expand_key "$CACHE_KEY")
PUB_CACHE_KEY=$(expand_key "$PUB_CACHE_KEY")
CACHE_PATH=$(expand_key "$(transform_path "$CACHE_PATH")")

if [[ "$PRINT_ONLY" == true ]]; then
Expand All @@ -142,6 +147,8 @@ if [[ "$PRINT_ONLY" == true ]]; then
echo "ARCHITECTURE=$info_architecture"
echo "CACHE-KEY=$CACHE_KEY"
echo "CACHE-PATH=$CACHE_PATH"
echo "PUB-CACHE-KEY=$PUB_CACHE_KEY"
echo "PUB-CACHE-PATH=$PUB_CACHE"
exit 0
fi

Expand All @@ -151,6 +158,8 @@ if [[ "$PRINT_ONLY" == true ]]; then
echo "ARCHITECTURE=$info_architecture"
echo "CACHE-KEY=$CACHE_KEY"
echo "CACHE-PATH=$CACHE_PATH"
echo "PUB-CACHE-KEY=$PUB_CACHE_KEY"
echo "PUB-CACHE-PATH=$PUB_CACHE"
} >>"$GITHUB_OUTPUT"

exit 0
Expand All @@ -171,11 +180,11 @@ fi

{
echo "FLUTTER_ROOT=$CACHE_PATH"
echo "PUB_CACHE=$CACHE_PATH/.pub-cache"
echo "PUB_CACHE=$PUB_CACHE"
} >>"$GITHUB_ENV"

{
echo "$CACHE_PATH/bin"
echo "$CACHE_PATH/bin/cache/dart-sdk/bin"
echo "$CACHE_PATH/.pub-cache/bin"
echo "$PUB_CACHE/bin"
} >>"$GITHUB_PATH"