Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

table_exists? returns false when its query gets canceled #2106

Closed
gbrlcustodio opened this issue Dec 15, 2023 · 2 comments
Closed

table_exists? returns false when its query gets canceled #2106

gbrlcustodio opened this issue Dec 15, 2023 · 2 comments

Comments

@gbrlcustodio
Copy link

gbrlcustodio commented Dec 15, 2023

Complete Description of Issue

When executing alter table operations concurrently, the DB.table_exists? call incorrectly returns false in situations where the underlying table exists but is locked by an external alter table operation. This issue arises when the #table_exists?'s underlying SELECT query gets canceled due to statement timeout. Consequently, a PG::QueryCanceled error is logged for the PostgreSQL adapter, while the method itself returns false.

This is particularly interesting for us because as part of our data processing pipeline, we execute many alter tables concurrently, which can sometimes result in the table being locked exclusively for a longer period than the statement timeout. Since there are strong ordering guarantees for processing incoming events we consider a missing target table as a data inconsistency issue and skip its execution. Therefore, before executing the next alter table operation, we have to check whether the target table already exists to decide whether to move it to the DLQ.

Simplest Possible Self-Contained Example Showing the Bug

  1. Given there's a previously created table called customers
  2. Lock that table with Access Exclusive lock: LOCK TABLE customers IN ACCESS EXCLUSIVE MODE;
  3. Try issuing a DB.table_exists? call with the customers table parameter

Full Backtrace of Exception (if any)

No response

SQL Log (if any)

ERROR -- : PG::QueryCanceled: ERROR: canceling statement due to statement timeout: SELECT NULL AS "nil" FROM "customers" LIMIT 1

Ruby Version

3.2.0

Sequel Version

5.74.0

@jeremyevans
Copy link
Owner

Thanks for the report. I suppose in the PostgreSQL support we could check for that condition in Database#table_exists?. I'll work on that.

@gbrlcustodio
Copy link
Author

Awesome, thanks for taking that on! Let me know if you need any more info or testing assistance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants