New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
POC: Speed optimize class-loading for levels tests #2916
Changes from 9 commits
57bcbd2
9d04fe6
11ffaa8
37e325c
4eb6d67
030b8c0
b025c05
45b12b0
36f26cf
9adf76d
947ddb7
6185833
5fe67ce
6dc056d
0bcd114
b77b48a
f5aeead
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
<?php | ||
|
||
$testList = shell_exec('php vendor/bin/phpunit --list-tests'); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. currently the I have opened a feature request about that, so we may can get rid/simplify this script in the future. |
||
|
||
if (!is_string($testList)) { | ||
throw new RuntimeException('Error while listing tests'); | ||
} | ||
|
||
$testFilters = []; | ||
foreach(explode("\n", $testList) as $line) { | ||
$cleanedLine = trim($line, ' -'); | ||
|
||
if ($cleanedLine === '') { | ||
continue; | ||
} | ||
|
||
if ( | ||
!str_contains($cleanedLine, 'PHPStan\Generics\GenericsIntegrationTest') && | ||
!str_contains($cleanedLine, 'PHPStan\Levels\LevelsIntegrationTest') | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should check the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. to make this work based on the phpdoc, I would have needed to reflect on classes and read all class-comments. instead I updated phpunit to 10.x for the matrix job and used |
||
) { | ||
continue; | ||
} | ||
|
||
$testFilters[] = $cleanedLine; | ||
} | ||
|
||
if ($testFilters === []) { | ||
throw new RuntimeException('No tests found'); | ||
} | ||
|
||
echo json_encode($testFilters); |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -98,11 +98,45 @@ jobs: | |
- name: "Tests" | ||
run: "make tests-integration" | ||
|
||
tests-levels-matrix: | ||
name: "Determine levels tests matrix" | ||
runs-on: ubuntu-latest | ||
timeout-minutes: 60 | ||
|
||
steps: | ||
- name: "Checkout" | ||
uses: actions/checkout@v3 | ||
|
||
- name: "Install PHP" | ||
uses: "shivammathur/setup-php@v2" | ||
with: | ||
coverage: "none" | ||
php-version: "8.1" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's do 8.3 here :) |
||
tools: pecl | ||
extensions: ds,mbstring | ||
ini-file: development | ||
ini-values: memory_limit=1G | ||
|
||
- name: "Install dependencies" | ||
run: "composer install --no-interaction --no-progress" | ||
|
||
- id: set-matrix | ||
run: echo "matrix=$(php .github/workflows/tests-levels-matrix.php)" >> $GITHUB_OUTPUT | ||
|
||
outputs: | ||
matrix: ${{ steps.set-matrix.outputs.matrix }} | ||
|
||
tests-levels: | ||
needs: tests-levels-matrix | ||
|
||
name: "Levels tests" | ||
runs-on: ubuntu-latest | ||
timeout-minutes: 60 | ||
|
||
strategy: | ||
matrix: | ||
test-filter: "${{fromJson(needs.tests-levels-matrix.outputs.matrix)}}" | ||
|
||
steps: | ||
- name: "Checkout" | ||
uses: actions/checkout@v3 | ||
|
@@ -121,7 +155,7 @@ jobs: | |
run: "composer install --no-interaction --no-progress" | ||
|
||
- name: "Tests" | ||
run: "make tests-levels" | ||
run: "php vendor/bin/phpunit --filter '${{ matrix.test-filter }}' --group levels" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I just did some local testing and in this format PHPUnit will still execute all data providers from all tests. It would be good to also add the file in which the filtered test is. I think we can get even faster that way. See the comparison:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
|
||
tests-old-phpunit: | ||
name: "Tests with old PHPUnit" | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
declare strict types please