Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RustBuffer regression with nested Errors #2108

Closed
OtaK opened this issue May 14, 2024 · 5 comments · Fixed by #2119
Closed

RustBuffer regression with nested Errors #2108

OtaK opened this issue May 14, 2024 · 5 comments · Fixed by #2119

Comments

@OtaK
Copy link

OtaK commented May 14, 2024

Hi!

I'm having a problem I quite can't solve even after investigating.

I have the kinda following structure

crate 1, which exposes a couple of flat error enum:

#[derive(Debug, thiserror::Error, uniffi::Error)]
#[uniffi(flat_error)]
pub enum ForeignError {
	#[error("Error A: {0}")]
	ErrorA(#[from] ...),
	#[error("Error B: {0}")]
	ErrorB(#[from] ...),
}

#[derive(Debug, thiserror::Error, uniffi::Error)]
#[uniffi(flat_error)]
pub enum OtherForeignError {
	#[error("Error A: {0}")]
	ErrorA(#[from] ...),
	#[error("Error B: {0}")]
	ErrorB(#[from] ...),
}

crate 2, which consumes those 2 errors in a normal error enum:

#[derive(Debug, thiserror::Error, uniffi::Error)]
pub enum FfiError {
	#[error(transparent)]
	ForeignError(#[from] crate1::ForeignError),
	#[error(transparent)]
	OtherError(#[from] crate1::OtherForeignError),
}

type FfiResult<T> = Result<T, FfiError>;

On both Kotlin and Swift bindings I get those errors whenever FfiError is passed around:

Kotlin: java.lang.RuntimeException: junk remaining in buffer after lifting, something is very wrong!!
Swift: UniffiInternalError.incompleteData "The buffer still has data after lifting its containing value"

Any idea what causes this? This used to work before I upgraded to 0.27.x, but I can't poinpoint exactly which commit causes this regression.

@OtaK OtaK changed the title RustBuffer problem with nested Errors RustBuffer regression with nested Errors May 14, 2024
@mhammond
Copy link
Member

Nothing springs to mind here.

On both Kotlin and Swift bindings I get those errors whenever FfiError is passed around

What does "passed around" mean here exactly? I assume you mean as function args/results rather than via errors?

Regardless, I think we will probably need a complete example - the easiest for us would be to reproduce it in, say, https://github.com/mozilla/uniffi-rs/tree/main/fixtures/ext-types/lib - that's our fixture explicitly for external types.

@OtaK
Copy link
Author

OtaK commented May 15, 2024

What does "passed around" mean here exactly? I assume you mean as function args/results rather than via errors?

I mean passed around in the FFI via a Result carrying FfiError yes.

I'll try to whip up something, but what I can say about my investigation, is that if I turn the FfiError in my example to a #[uniffi(flat_error)] the problem completely disappears. I think it might be a problem with uniffi_bindgen::interface::ComponentInterface::should_generate_error_read.

It's entirely possible that my use case (non-flat error with externally-defined flat-error members) falls into the cracks of this test function

@mhammond
Copy link
Member

thanks for the info - so it might be that it's not related to "external" types at all - so maybe it might be easier to repro in https://github.com/mozilla/uniffi-rs/tree/main/fixtures/error-types

OtaK added a commit to wireapp/uniffi-rs that referenced this issue May 15, 2024
@OtaK
Copy link
Author

OtaK commented May 15, 2024

@mhammond I added a fixture testcase at https://github.com/wireapp/uniffi-rs/tree/fixture/issue-2108

I get the following error traces (I couldn't make the python work, seems my example also makes it generate non-compiling code)

Kotlin

java.lang.RuntimeException: junk remaining in buffer after lifting, something is very wrong!!
	at uniffi.error_types.FfiConverter$DefaultImpls.liftFromRustBuffer(error_types.kt:201)
	at uniffi.error_types.FfiConverterRustBuffer$DefaultImpls.liftFromRustBuffer(error_types.kt:211)
	at uniffi.error_types.FfiConverterTypeError.liftFromRustBuffer(error_types.kt:2617)
	at uniffi.error_types.FfiConverterTypeError.liftFromRustBuffer(error_types.kt:2617)
	at uniffi.error_types.FfiConverterRustBuffer$DefaultImpls.lift(error_types.kt:212)
	at uniffi.error_types.FfiConverterTypeError.lift(error_types.kt:2617)
	at uniffi.error_types.FfiConverterTypeError.lift(error_types.kt:2617)
	at uniffi.error_types.Exception$ErrorHandler.lift(error_types.kt:2611)
	at uniffi.error_types.Exception$ErrorHandler.lift(error_types.kt:2610)
	at uniffi.error_types.Error_typesKt.uniffiCheckCallStatus(error_types.kt:275)
	at uniffi.error_types.Error_typesKt.access$uniffiCheckCallStatus(error_types.kt:1)
	at uniffi.error_types.Error_typesKt.flatVariantAEnum(error_types.kt:2888)
	at Test.<init>(test.kts:23)
test uniffi_foreign_language_testcase_test_kts ... FAILED
test uniffi_foreign_language_testcase_test_swift ... FAILED

failures:

---- uniffi_foreign_language_testcase_test_kts stdout ----
Creating testing out_dir: /home/otak/Dev/OSS/uniffi-rs/target/tmp/uniffi-fixture-error-types-b67ac23c8ab46687
Error: running `kotlinc` failed

Swift

Swift/ErrorType.swift:200: Fatal error: Error raised at top level: error_types.(unknown context at $7f9ad5412e90).UniffiInternalError.incompleteData
Stack dump:
0.	Program arguments: /home/otak/.swiftenv/versions/5.10-RELEASE/usr/bin/swift-frontend -frontend -interpret /home/otak/Dev/OSS/uniffi-rs/fixtures/error-types/tests/bindings/test.swift -disable-objc-interop -I /home/otak/Dev/OSS/uniffi-rs/target/tmp/uniffi-fixture-error-types-c70d57d3e6faa04f -color-diagnostics -new-driver-path /home/otak/.swiftenv/versions/5.10-RELEASE/usr/bin/swift-driver -empty-abi-descriptor -resource-dir /home/otak/.swiftenv/versions/5.10-RELEASE/usr/lib/swift -Xcc -fmodule-map-file=/home/otak/Dev/OSS/uniffi-rs/target/tmp/uniffi-fixture-error-types-c70d57d3e6faa04f/combined.modulemap -module-name test -plugin-path /home/otak/.swiftenv/versions/5.10-RELEASE/usr/lib/swift/host/plugins -plugin-path /home/otak/.swiftenv/versions/5.10-RELEASE/usr/local/lib/swift/host/plugins -ltestmod_error_types -luniffi_error_types
1.	Swift version 5.10 (swift-5.10-RELEASE)
2.	Compiling with the current language version
3.	While running user code "/home/otak/Dev/OSS/uniffi-rs/fixtures/error-types/tests/bindings/test.swift"
 #0 0x000055b5707a0273 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/otak/.swiftenv/versions/5.10-RELEASE/usr/bin/swift-frontend+0x6259273)
 #1 0x000055b57079e19e llvm::sys::RunSignalHandlers() (/home/otak/.swiftenv/versions/5.10-RELEASE/usr/bin/swift-frontend+0x625719e)
 #2 0x000055b5707a05ef SignalHandler(int) Signals.cpp:0:0
 #3 0x00007f9ad5c58960 (/lib64/libc.so.6+0x3a960)
 #4 0x00007f9ad716456f $ss17_assertionFailure__4file4line5flagss5NeverOs12StaticStringV_SSAHSus6UInt32VtF (/home/otak/.swiftenv/versions/5.10-RELEASE/usr/lib/swift/linux/libswiftCore.so+0x16456f)
 #5 0x00007f9ad71cb32e (/home/otak/.swiftenv/versions/5.10-RELEASE/usr/lib/swift/linux/libswiftCore.so+0x1cb32e)
 #6 0x00007f9ad5ba4903 
 #7 0x000055b56b32fc1f llvm::orc::runAsMain(int (*)(int, char**), llvm::ArrayRef<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, llvm::Optional<llvm::StringRef>) (/home/otak/.swiftenv/versions/5.10-RELEASE/usr/bin/swift-frontend+0xde8c1f)
 #8 0x000055b56b20a2a8 swift::SwiftJIT::runMain(llvm::ArrayRef<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>) (/home/otak/.swiftenv/versions/5.10-RELEASE/usr/bin/swift-frontend+0xcc32a8)
 #9 0x000055b56b207ec3 swift::RunImmediately(swift::CompilerInstance&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>> const&, swift::IRGenOptions const&, swift::SILOptions const&, std::unique_ptr<swift::SILModule, std::default_delete<swift::SILModule>>&&) (/home/otak/.swiftenv/versions/5.10-RELEASE/usr/bin/swift-frontend+0xcc0ec3)
#10 0x000055b56b1b12c6 processCommandLineAndRunImmediately(swift::CompilerInstance&, std::unique_ptr<swift::SILModule, std::default_delete<swift::SILModule>>&&, llvm::PointerUnion<swift::ModuleDecl*, swift::SourceFile*>, swift::FrontendObserver*, int&) FrontendTool.cpp:0:0
#11 0x000055b56b1ac4ca performCompileStepsPostSILGen(swift::CompilerInstance&, std::unique_ptr<swift::SILModule, std::default_delete<swift::SILModule>>, llvm::PointerUnion<swift::ModuleDecl*, swift::SourceFile*>, swift::PrimarySpecificPaths const&, int&, swift::FrontendObserver*) FrontendTool.cpp:0:0
#12 0x000055b56b1ab5bb swift::performCompileStepsPostSema(swift::CompilerInstance&, int&, swift::FrontendObserver*) (/home/otak/.swiftenv/versions/5.10-RELEASE/usr/bin/swift-frontend+0xc645bb)
#13 0x000055b56b1bf21a withSemanticAnalysis(swift::CompilerInstance&, swift::FrontendObserver*, llvm::function_ref<bool (swift::CompilerInstance&)>, bool) FrontendTool.cpp:0:0
#14 0x000055b56b1af930 performCompile(swift::CompilerInstance&, int&, swift::FrontendObserver*) FrontendTool.cpp:0:0
#15 0x000055b56b1ad43b swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) (/home/otak/.swiftenv/versions/5.10-RELEASE/usr/bin/swift-frontend+0xc6643b)
#16 0x000055b56b043ab0 swift::mainEntry(int, char const**) (/home/otak/.swiftenv/versions/5.10-RELEASE/usr/bin/swift-frontend+0xafcab0)
#17 0x00007f9ad5c42310 (/lib64/libc.so.6+0x24310)
#18 0x00007f9ad5c423c9 __libc_start_main (/lib64/libc.so.6+0x243c9)
#19 0x000055b56b042c0a _start (/home/otak/.swiftenv/versions/5.10-RELEASE/usr/bin/swift-frontend+0xafbc0a)

💣 Program crashed: Illegal instruction at 0x00007f9ad716456f

Thread 0 "swift-frontend" crashed:

0 0x00007f9ad716456f _assertionFailure(_:_:file:line:flags:) + 351 in libswiftCore.so
1 0x00007f9ad71cb32e swift_errorInMain + 829 in libswiftCore.so
2 0x00007f9ad5ba4903

Backtrace took 0.02s

Python

Traceback (most recent call last):
  File "/home/otak/Dev/OSS/uniffi-rs/fixtures/error-types/tests/bindings/test.py", line 7, in <module>
    from error_types import *
  File "/home/otak/Dev/OSS/uniffi-rs/target/tmp/uniffi-fixture-error-types-e25336f2910a891c/error_types.py", line 32, in <module>
    from .flat_procmacro_errors import FlatErrorA
ImportError: attempted relative import with no known parent package
test uniffi_foreign_language_testcase_test_py ... FAILED

@mhammond
Copy link
Member

That's great, thank you!

mhammond added a commit to mhammond/uniffi-rs that referenced this issue May 17, 2024
mhammond added a commit to mhammond/uniffi-rs that referenced this issue May 17, 2024
mhammond added a commit to mhammond/uniffi-rs that referenced this issue May 20, 2024
When a `#[derive(uniffi::Error)]` didn't appear in any function signatures,
the FfiConverter for the item was written to assume an error, but the
bindings FfiConverter treated the item as a plain-old Enum. This caused
runtime errors when trying to unpack the rustbuffers due to the
disagreement about the buffer format.

As part of fixing this, the `Option<bool>` to try and represent the
flatness of Enums/Errors was replaced with an `EnumFlatness` enum to
make these states clearer and so the ComponentInterface can better
understand the layout.

Fixes mozilla#2108
mhammond added a commit to mhammond/uniffi-rs that referenced this issue May 24, 2024
When a `#[derive(uniffi::Error)]` didn't appear in any function signatures,
the FfiConverter for the item was written to assume an error, but the
bindings FfiConverter treated the item as a plain-old Enum. This caused
runtime errors when trying to unpack the rustbuffers due to the
disagreement about the buffer format.

As part of fixing this, the `Option<bool>` to try and represent the
flatness of Enums/Errors was replaced with an `EnumShape` enum to
make these states clearer and so the ComponentInterface can better
understand the layout.

Fixes mozilla#2108
mhammond added a commit that referenced this issue May 24, 2024
When a `#[derive(uniffi::Error)]` didn't appear in any function signatures,
the FfiConverter for the item was written to assume an error, but the
bindings FfiConverter treated the item as a plain-old Enum. This caused
runtime errors when trying to unpack the rustbuffers due to the
disagreement about the buffer format.

As part of fixing this, the `Option<bool>` to try and represent the
flatness of Enums/Errors was replaced with an `EnumShape` enum to
make these states clearer and so the ComponentInterface can better
understand the layout.

Fixes #2108
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants