Many JUnit 4 tests not executed

[pkubowicz]

Current 'master' branch contains many tests that should fail, but don't because they are not executed:

:pact-jvm-server:test
:pact-specification-test:test
:provider:pact-jvm-provider-scalatest_2.12:test
:provider:pact-jvm-provider-scalasupport_2.12:test
:provider:pact-jvm-provider-specs2_2.12:test

This is because the code mixes JUnit 4 and JUnit 5 but not all subprojects have a dependency on JUnit Vintage runner.

To see the problem add

testRuntime "org.junit.vintage:junit-vintage-engine:${project.junit5Version}"

to dependencies in root build.gradle and execute gradle test --continue. Alternatively, update junit5Version to 5.4.2 in gradle.properties.

Something weird happens with pact-specification-test: it already has a dependency on junit-vintage-engine, tests are executed but nothing happens (the test report is empty). The problem is only visible after updating JUnit 5. I suppose old JUnit has a bug where an exception during test initialization does not fail the test. Tests in pact-specification-test would not even compile in a static language: they try to invoke a method that no longer exists in Message class.

I haven't checked which subprojects contain tests that are never executed currently but pass once they are executed - this would require more work than just running a Gradle build.

[uglyog]

Well this is annoying. Thanks for reporting it. I've got all the tests passing expect for the Scala ones that don't work. I've just ignored those for now.