Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stored Procedure Parse Exception with Spring Boot 3.1.5 and HSQLDB 2.7.2 #3782

Closed
pacphi opened this issue Nov 1, 2023 · 5 comments
Closed
Labels
s: blocked Blocked on 3rd party work

Comments

@pacphi
Copy link

pacphi commented Nov 1, 2023

Which version and edition of Flyway are you using?

The curated version with Spring Boot 3.1.5 which is 9.16.3

If this is not the latest version, can you reproduce the issue with the latest one as well? (Many bugs are fixed in newer releases and upgrading will often resolve the issue)

Yes, using Flyway 9.22.3 also fails the same way

Which client are you using? (Command-line, Java API, Maven plugin, Gradle plugin)

Java

Which database are you using? (Type & version)

HSQLDB 2.7.2

Which operating system are you using?

Mac OS

What did you do? (Please include the content causing the issue, any relevant configuration settings, the SQL statement(s) that failed (if any), and the command you ran)

Created a unit test

This is the Flyway asset

Ran mvn install

JDK 17
Maven 3.9.4

What did you expect to see?

Passing test, successful parsing

What did you see instead?

This is the error

Caused by: org.flywaydb.core.api.FlywayException: Unable to parse statement in db/hsqldb/V1_1__add_test_sproc.sql at line 3 col 1. See https://rd.gt/3ipi7Pm for more information: Incomplete statement at line 3 col 1: -- Tested w/ HSQLDB only

CREATE PROCEDURE GET_ATTRIBUTES_CREATED_BEFORE (IN CREATED_TIME TIMESTAMP)
  READS SQL DATA
  DYNAMIC RESULT SETS 1
  BEGIN ATOMIC
          DECLARE results SCROLL CURSOR WITH HOLD WITH RETURN FOR
          SELECT NAME, CREATED_TIME FROM ATTRIBUTE WHERE UNIX_MILLIS(CREATED_TIME) < UNIX_MILLIS(createdTime);
          OPEN results;
  END;
        at org.flywaydb.core.internal.parser.Parser.getNextStatement(Parser.java:306)
        at org.flywaydb.core.internal.parser.Parser$ParserSqlStatementIterator.<init>(Parser.java:715)
        at org.flywaydb.core.internal.parser.Parser.parse(Parser.java:130)
        at org.flywaydb.core.internal.sqlscript.ParserSqlScript.parse(ParserSqlScript.java:72)
        at org.flywaydb.core.internal.sqlscript.ParserSqlScript.validate(ParserSqlScript.java:120)
        at org.flywaydb.core.internal.sqlscript.ParserSqlScript.executeInTransaction(ParserSqlScript.java:187)
        at org.flywaydb.core.internal.resolver.sql.SqlMigrationExecutor.canExecuteInTransaction(SqlMigrationExecutor.java:73)
        at org.flywaydb.core.internal.command.DbMigrate.isExecuteGroupInTransaction(DbMigrate.java:305)
        at org.flywaydb.core.internal.command.DbMigrate.applyMigrations(DbMigrate.java:270)
        at org.flywaydb.core.internal.command.DbMigrate.migrateGroup(DbMigrate.java:247)
        at org.flywaydb.core.internal.command.DbMigrate.lambda$migrateAll$0(DbMigrate.java:142)
        at org.flywaydb.core.internal.jdbc.TableLockingExecutionTemplate$1.call(TableLockingExecutionTemplate.java:36)
        at org.flywaydb.core.internal.jdbc.TransactionalExecutionTemplate.execute(TransactionalExecutionTemplate.java:55)
        at org.flywaydb.core.internal.jdbc.TableLockingExecutionTemplate.execute(TableLockingExecutionTemplate.java:31)
        at org.flywaydb.core.internal.database.base.Connection.lock(Connection.java:102)
        at org.flywaydb.core.internal.schemahistory.JdbcTableSchemaHistory.lock(JdbcTableSchemaHistory.java:144)
        at org.flywaydb.core.internal.command.DbMigrate.migrateAll(DbMigrate.java:142)
        at org.flywaydb.core.internal.command.DbMigrate.migrate(DbMigrate.java:99)
        at org.flywaydb.core.Flyway.lambda$migrate$0(Flyway.java:188)
        at org.flywaydb.core.FlywayExecutor.execute(FlywayExecutor.java:190)
        at org.flywaydb.core.Flyway.migrate(Flyway.java:140)
        at org.springframework.boot.autoconfigure.flyway.FlywayMigrationInitializer.afterPropertiesSet(FlywayMigrationInitializer.java:66)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1817)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1766)
        ... 96 more
Caused by: org.flywaydb.core.api.FlywayException: Incomplete statement at line 3 col 1: -- Tested w/ HSQLDB only

CREATE PROCEDURE GET_ATTRIBUTES_CREATED_BEFORE (IN CREATED_TIME TIMESTAMP)
  READS SQL DATA
  DYNAMIC RESULT SETS 1
  BEGIN ATOMIC
          DECLARE results SCROLL CURSOR WITH HOLD WITH RETURN FOR
          SELECT NAME, CREATED_TIME FROM ATTRIBUTE WHERE UNIX_MILLIS(CREATED_TIME) < UNIX_MILLIS(createdTime);
          OPEN results;
  END;
        at org.flywaydb.core.internal.parser.Parser.getNextStatement(Parser.java:241)
        ... 119 more
@JasonLuo-Redgate
Copy link

JasonLuo-Redgate commented Nov 1, 2023

Hi @pacphi , according to the doc here https://documentation.red-gate.com/fd/hsqldb-184127598.html, the latest supported version for HSQLDB is 2.4 with Standard SQL syntax with statement delimiter. So, I think the error is caused by "DYNAMIC RESULT SETS" not supported.

@pacphi
Copy link
Author

pacphi commented Nov 1, 2023

Two things:

  1. Forcing dependency on hsqldb to 2.4.1, you will see:
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Flyway Teams Edition or HSQLDB upgrade required: HSQLDB 2.4 is no longer supported by Flyway Community Edition, but still supported by Flyway Teams Edition.
  1. The community edition additional dependency mentioned in the doc link (i.e., flyway-database-hsqldb) seems to be available only in 10.0.0, see https://repo1.maven.org/maven2/org/flywaydb/flyway-database-hsqldb/.

So when I tried that, I got...

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flyway' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Failed to instantiate [org.flywaydb.core.Flyway]: Factory method 'flyway' threw exception with message: 'org.flywaydb.core.api.configuration.FluentConfiguration org.flywaydb.core.api.configuration.FluentConfiguration.licenseKey(java.lang.String)'
        at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:654) ~[spring-beans-6.0.13.jar:6.0.13]
        at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:642) ~[spring-beans-6.0.13.jar:6.0.13]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1332) ~[spring-beans-6.0.13.jar:6.0.13]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1162) ~[spring-beans-6.0.13.jar:6.0.13]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:560) ~[spring-beans-6.0.13.jar:6.0.13]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:520) ~[spring-beans-6.0.13.jar:6.0.13]
        at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:325) ~[spring-beans-6.0.13.jar:6.0.13]
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-6.0.13.jar:6.0.13]
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323) ~[spring-beans-6.0.13.jar:6.0.13]
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[spring-beans-6.0.13.jar:6.0.13]
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:312) ~[spring-beans-6.0.13.jar:6.0.13]
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[spring-beans-6.0.13.jar:6.0.13]
        at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1166) ~[spring-context-6.0.13.jar:6.0.13]
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:940) ~[spring-context-6.0.13.jar:6.0.13]
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:616) ~[spring-context-6.0.13.jar:6.0.13]
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:738) ~[spring-boot-3.1.5.jar:3.1.5]
        at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:440) ~[spring-boot-3.1.5.jar:3.1.5]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:316) ~[spring-boot-3.1.5.jar:3.1.5]
        at org.springframework.boot.test.context.SpringBootContextLoader.lambda$loadContext$3(SpringBootContextLoader.java:137) ~[spring-boot-test-3.1.5.jar:3.1.5]
        at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:58) ~[spring-core-6.0.13.jar:6.0.13]
        at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:46) ~[spring-core-6.0.13.jar:6.0.13]
        at org.springframework.boot.SpringApplication.withHook(SpringApplication.java:1406) ~[spring-boot-3.1.5.jar:3.1.5]
        at org.springframework.boot.test.context.SpringBootContextLoader$ContextLoaderHook.run(SpringBootContextLoader.java:545) ~[spring-boot-test-3.1.5.jar:3.1.5]
        at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:137) ~[spring-boot-test-3.1.5.jar:3.1.5]
        at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:108) ~[spring-boot-test-3.1.5.jar:3.1.5]
        at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:187) ~[spring-test-6.0.13.jar:6.0.13]
        at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:119) ~[spring-test-6.0.13.jar:6.0.13]
        ... 73 common frames omitted
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.flywaydb.core.Flyway]: Factory method 'flyway' threw exception with message: 'org.flywaydb.core.api.configuration.FluentConfiguration org.flywaydb.core.api.configuration.FluentConfiguration.licenseKey(java.lang.String)'
        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:171) ~[spring-beans-6.0.13.jar:6.0.13]
        at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:650) ~[spring-beans-6.0.13.jar:6.0.13]
        ... 99 common frames omitted
Caused by: java.lang.NoSuchMethodError: 'org.flywaydb.core.api.configuration.FluentConfiguration org.flywaydb.core.api.configuration.FluentConfiguration.licenseKey(java.lang.String)'
        at org.springframework.boot.autoconfigure.flyway.FlywayAutoConfiguration$FlywayConfiguration.configureProperties(FlywayAutoConfiguration.java:254) ~[spring-boot-autoconfigure-3.1.5.jar:3.1.5]
        at org.springframework.boot.autoconfigure.flyway.FlywayAutoConfiguration$FlywayConfiguration.flyway(FlywayAutoConfiguration.java:150) ~[spring-boot-autoconfigure-3.1.5.jar:3.1.5]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
        at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:139) ~[spring-beans-6.0.13.jar:6.0.13]
        ... 100 common frames omitted

I suppose I need to wait for Spring Boot 3.2.0 release for compatibility reasons.

Others will be similarly stuck with a use case like this.

The question is, is this one of those use cases where you need a Pro or Enterprise Flyway license (and a different artifact dependency), in order to get appropriate runtime support for hsqldb?

@pacphi
Copy link
Author

pacphi commented Nov 2, 2023

Looks like ultimately this issue will get resolved when this issue is addressed: spring-projects/spring-boot#38164.

@JasonLuo-Redgate
Copy link

Yes, @pacphi , the combability error now depending on the spring-boot ticket you posted.

@JasonLuo-Redgate JasonLuo-Redgate added s: blocked Blocked on 3rd party work and removed s: waiting for feedback labels Nov 2, 2023
fastnsilver added a commit to fastnsilver/grivet that referenced this issue Nov 10, 2023
* Major upgrade
  * to Spring Boot 3.1
  * upgrade all dependencies and plugins to latest available
  * fix test configuration properties
  * fix references to test-resources
  * json schema validator changed groupId
  * auth0 provider changed to okta-spring-boot-starter
  * security configuration implementation changes - remain untested, just enough to compile

* Round 2 of updates
* still finding a path thru the woods
* Had to adjust Spring Boot Maven Plugin configuration for deployables
  * @see https://docs.spring.io/spring-boot/docs/current/maven-plugin/reference/htmlsingle/#packaging.examples.local-artifact
  * Had to adjust Spring Cloud Streams b/c there's quite a bit of change from 2.x to 3.x
    * @see https://dev.to/habutre/migrating-spring-cloud-stream-v2-to-v3-5dip
* Fix broken validation assertion logi in Ingest service stub
* Adjusted test configuration for Ingest; added disable flags and exclude autoconfigs

* More updates
  * Replace mysql-connector-java with more recent db driver myql-connector-j
  * Whitespace removal
  * Don't override Spring Boot 3.1.x parent's management of Flyway dependency version
  * Remove traces of javax.* and jsr305 dependencies and properties

* Temporarily disable Sproc testing
* due to open issue flyway/flyway#3782
* also temporarily add H2 console support to test in @BeforeAll annotated method
  * need to figure out why stored entity attribute values are getting truncated; either on persistence or retrieval

* First full successful build
  * but we're not done yet
  * had to plumb some new Flyway placeholders to support quirkiness of H2 and Java 17 wrt nanosecond precision of TIMESTAMP
  * had to update the casted type to return not just the first character but the full value persisted from tables participating in all_entity_values view
  * improve logging levels for troubleshooting in test-resources/application.yml
  * override and upgrade H2 to 2.2.224
  * add columnDefinition attribute to all @Column-annotated filed with type LocalDateTime
  * Fix some test logic in test cases within JdbcEntityRepositoryTest
  * Adjust mapRows method logic in JdbcEntityRepository so we're not casting, but taking advantage of new method signatures in SqlRowSet

* Upgrade all  Java 8 Dockerfile images
  * to Java 21

* Resurrect and refactor shared-config/application.yml
  * we'll likely have to tune this file when we really start integration testing

* Swap API Gateway: Phase out Zuul, bring in Spring Cloud Gateway
  * taking inspiration from https://spring.io/guides/gs/gateway/

* Doc update: DEV_NOTES.md
  * reflect the switch from Zuul to Spring Cloud Gateway

* Updates to Dockerfile and Docker Compose infra and setup
  * mainly upgrading to latest available images
  * needs testing!

* Roadmap updates

* Fix shell warning

* Resurrect Spring Boot Admin for application observability
  * update Docker Compose setup
  * add child module to support POM

* Fix Maven plugin Docker builds for deployables and support infra
  * and fix ref to .jar file in manifest.yml

* Update scripts to invoke docker compose as plugin

* Miscellaneous fixes

* Remove milestone and snapshot dependency and plugin repositories resolution for Spring managed artifacts
* Fallback to Micrometer 1.11.5
* Update Git Commit Plugin config
  * we now see the abbreviated form of commit id in banner text

* Rename observability module and remove shared-config/application.yml
  * because it will be sourced from a Git repo along with individual configuration for each Spring Boot microservice via a Spring Cloud Config Server instance
  * adjust Docker Compose configuration

* Addressing breaking changes wrt bootstrap config
  * @see https://github.com/spring-cloud/spring-cloud-release/wiki/Spring-Cloud-2022.0-Release-Notes#breaking-changes
  * @see https://stackoverflow.com/questions/64994034/bootstrap-yml-configuration-not-processed-anymore-with-spring-cloud-2020-0

Coalesce or rename boostrap.yml properties into application.yml.

* Override snakeyaml version

* address CVE

* Spring Cloud Config Server runs more efficiently with Caffeine

* Address fallout from having colliding application.yml on classpath
  * remove occurrences of src/main/resources/application.yml from deployables child modules
  * rely on application.yml in test-resources for unit tests
  * occurrences of spring.application.name property will be defined in individual application configuration files hosted in Git repo

* Final updates to Docker Compose scripts
  * and prune redundant application.yml from discoverable support infra services

* More refinements
  * Docker Compose updates (image versions and config)
  * Mysql adjustments (pin version to 8.1.0 since this is the latest Flyway supports)
  * Fix show-log.sh script
  * Fix Config Server client properties

* More tuning and tweaks
  * Override and upgrade Flyway to 9.22.3
  * include flyway-mysql dependency in repositories
  * Adjust heap size for Elasticsearch (it's greedy)
    * update DEV_NOTES and provision.sh script
  * Revert to Mysql 8.1 (both driver and Docker image used)
  * Adjust support child modules' dependencies
    * config server gets embedded application.yml
    * discovery service gets embedded application.yml
  * Minor adjustments to sh scripts

* Swap docker-machine for multipass setup
  * simplify how DOCKER_IP is obtained in sh scripts
  * scrub and refresh DEV_NOTES

* Corrections to DEV_NOTES.md and provision.sh

* Fix how DOCKER_IP is resolved for Linux
  * in sh scrip[ts

* More heavy-lifting
  * Update key-value pairs in phpmyadmin/admin.env
  * Refactor Docker Compose config
    * move to v3.8
    * Swap ELK for Signoz
  * Plumb all services with OTEL_* key-values in environment stanza
    * still need to troubleshoot enabling metrics and tracing
    * all Spring Boot apps need to set SPRING_APPLICATION_NAME and SPRING_ACTIVE_PROFILES accordingly
    * subtract no longer actively developed watchtower
  * Improve DEV_NOTES
    * declare and update application ports
  * All Dockerfile in deployable and support trees responsible for packaging Spring Boot apps are now setup to install an javaagent for use with Signoz
  * Shell scripts improved to take into consideration these new "infra services" - need to start them before starting everything else

* Miscellaneous updates
  * Progress on CF manifests for deployables
  * Updates to ROADMAP
  * Comments added to Token Management section of WORKFLOW
  * Maven wrapper updated to 3.9.5
  * DEV_NOTES adjustments to toolset prereq versions
  * Addition of Github Action  for CI

* Update badges

* Additional tuning and tweaks to infra services
  * add extra_hosts to grivet-standalone service
  * add alertmanager service to docker-compose
  * enhance prometheus service config, set metrics collection
  * set api-gateway service's host post to 80 in api-gateway.yml
    * update DEV_NOTES to reflect this change
  * override signoz config
    * setup additional metrics endpoint collection
    * subtract hotrod and load-hotrod services from out-of-the-box setup
  * alter ROADMAP
  * add Caffeine as cache dependency to all support modules
  * subtract brave and zipkin dependencies, we already ship trace spans to signoz
  * enhance shutdown.sh script to clean up unused volumes, orphaned images, and unused networks

* Oops! Prometheus wants to consume /actuator/prometheus endpoint instead

* Need to activate pipeline or standalone profile for api-gateway service
  * in docker-compose-*.yml

* Update WORKFLOW

* Spring Cloud Gateway routes set to delegate to Eureka registered services
  * and it's host port is set to 80 via Docker Compose

* Update DEV_NOTES with additional services when in pipeline mode
  * and add @service annotation to IngestService

---------
Co-authored-by: Chris Phillipson <christopher.a.phillipson@gmail.com>
@JasonLuo-Redgate
Copy link

The Sprint boot ticket is now fixed spring-projects/spring-boot#38164

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
s: blocked Blocked on 3rd party work
Projects
None yet
Development

No branches or pull requests

2 participants