New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make sqlite3.Open thread safe #790
base: master
Are you sure you want to change the base?
Conversation
* sqlite3_open_v2 implicitly calls sqlite3_initialize * sqlite3_initialize is not thread safe in the prologue * concurrent calls of sqlite3.Open even on different files may trigger mutex initialization race and crash * Fix: use go channel with a single message in it as a barrier letting only single-threaded sqlite3_open_v2 for the first time * Reproducibility: very low, couple hours of concurrent sqlite3.Open calls trigger a single crash
This repo is a Go wrapper around the SQLite C library. If you believe you have found a bug in the C library, you should report it on their mailing list and let one of the SQLite maintainers make the proper fix. https://www.sqlite.org/support.html |
I understand. Comments above ‘sqlite3_initialize’ clearly state that it is a caller’s responsibility to init SQLite in a thread safe manner.
… On Feb 27, 2020, at 19:35, rittneje ***@***.***> wrote:
This repo is a Go wrapper around the SQLite C library. If you believe you have found a bug in the C library, you should report it on their mailing list and let one of the SQLite maintainers make the proper fix. https://www.sqlite.org/support.html
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
The docs say otherwise. https://www.sqlite.org/c3ref/initialize.html
|
It might be thread safe in master - some memory barriers are added to sync up caches and memory. In the same time the comment above the function:
Feel free to close, your call. The only thing go-sqlite3 1.10 (with sqlite 3.25.2) crashes on concurrent |
I believe that comment is describing what the implementation of |
@tsachiherman Care to elaborate on the downvote? I still contend this change should not be merged, given that the claim is that there is a bug in the SQLite library itself. |
@pzbitskiy Why do you believe those lines you posted from sqlite3-bindings.c to be relevant? They aren't the line numbers from your crash dump. Did you come to those lines from some deeper investigating you did? Also, can you post how you are calling |
rv := C._sqlite3_open_v2(name, &db, | ||
mutex|C.SQLITE_OPEN_READWRITE|C.SQLITE_OPEN_CREATE, | ||
nil) | ||
if ok { | ||
close(initCh) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This means you will close the channel even if sqlite3_open_v2
were to fail, which means your init barrier will not work as expected on the next invocation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you are right, good catch
Unfortunately I can't remember all the details after 5 months. But here is what I recalled.
They are:
151936 rc = sqlite3MutexInit();
151937 if( rc ) return rc;
24757 SQLITE_PRIVATE int sqlite3MutexInit(void){
24758 int rc = SQLITE_OK;
...
24768 if( sqlite3GlobalConfig.bCoreMutex ){
Now follow the path:
I think I had explanation about it but can't figure it out. |
@rittneje, try to look on this issue from an application perspective - the applicative stack would look like this: So, from workaround perspective:
I'm not claiming that it shouldn't be fixed in the sqlite3 library itself. I think it should. However, from practical perspective, even if the sqlite3 library would get updated, it will take a long time before the fix reaches everyone. That's why I think that in the case of such a trivial fix, it should be patched in the driver right away. With that, the bug would no longer be applicable, and the driver would be safe for usage. This workaround can be removed from the driver once the sqlite3 library would confirm to its own documentation. Another note - even if you were to report that to the sqlite3 library team, they might fix it by updating their documentation.. which would leave you at the same place. |
may trigger mutex initialization race and crash
letting only single-threaded sqlite3_open_v2 for the first time
trigger a single crash
Sample crash dump:
Relevant code for sqlite3-bindings.c