Introduce language backend #853

fredszaq · 2023-06-30T14:38:39Z

As discussed in #846 this introduces the notion of language backend in order to facilitate the addition of new supported languages to cbindgen

Notable changes:

new LanguageBackend trait containing functions to write headers and footers in a source file
2 LanguageBackend impls : CLikeLanguageBackend and CythonLanguageBackend
the Source trait is now generic on a LanguageBackend and gets the language backend instead of a config as an argument ( both backend impls, end up wrapping a config)
The implementations of Source for the various structs of the ir have been moved to the corresponding backend module as there is now one implementation of Source per LanguageBackend for each relevant ir struct.
The implementations of Source for both LanguageBackends come from the existing impls and have been cleaned to contain only the parts relevant to the language of the Backend for which the impl is for

All tests continue to pass without any modification \o/ (those were quite useful during the refacto)

yanchith · 2023-07-02T07:05:25Z

I am not familiar with the codebase, so I can't really review the patch itself. However, looking at clike.rs and cython.rs, this would give me a solid form to fill out to add the C# backend I need. Looking forward 👍

fredszaq · 2023-07-10T12:34:04Z

@emilio Did you have time to take a look ? I'm planning to work on the java-jna backend this week, would be nice to have a confirmation we want to go in this direction ;)

This builds on mozilla#853 and adds a new language backend capable of generating java bindings. The generated bindings use [JNA](https://github.com/java-native-access/jna) (I used the last version 5.13.0 while developping but this will probably work with earlier ones as well) Most of the new code lives in `bindgen/language_backend/java_jna.rs` and is pretty straightfowards generation as most on the complicated things were done in the previous refactoring. The test suite has been updated to generate and compile the bindings for java when running. On nice thing gained with this is the possibilibty to put a configuration file specific to a language alongside the stardard one and the test will take this one instead of the default one if it exists (quite a few test use the `header` config to add some more code inside the generated bindings, C or preprocessor code doesn't work well in a java source file :P) All tests are green, but that doesn't mean all the generated bindings are correct, some cases may fail at runtime as I didn't test all cases in a real program. I did however generate and use the bindings on a non trivial lib I'm developping and they worked as expected.

emilio

This looks generally reasonable, but I think if instead of relying on Source<LB> for T being implemented we had stuff like:

trait LanguageBackend {
  fn write_enum<W: Write>(&self, enum: &Enum, out: &mut SourceWriter<W>);
  fn ...;
}

etc, then all the generic soup goes away, right?

src/bindgen/bindings.rs

src/bindgen/ir/generic_path.rs

src/bindgen/bindings.rs

emilio · 2023-08-25T15:06:48Z

Sorry for the lag getting to this btw :/

fredszaq · 2023-08-25T21:02:09Z

Thanks for the review ! I think I tried the write_xxx functions in the language backend trait first but was blocked by something. I cannot remember what though. I'll take a look when I'm back from holidays in september

fredszaq · 2023-09-06T15:23:32Z

Hi @emilio ! I just rebased the branch and did the changes you asked, this effectively removes the source trait.

This builds on mozilla#853 and adds a new language backend capable of generating java bindings. The generated bindings use [JNA](https://github.com/java-native-access/jna) (I used the last version 5.13.0 while developping but this will probably work with earlier ones as well) Most of the new code lives in `bindgen/language_backend/java_jna.rs` and is pretty straightfowards generation as most on the complicated things were done in the previous refactoring. The test suite has been updated to generate and compile the bindings for java when running. On nice thing gained with this is the possibilibty to put a configuration file specific to a language alongside the stardard one and the test will take this one instead of the default one if it exists (quite a few test use the `header` config to add some more code inside the generated bindings, C or preprocessor code doesn't work well in a java source file :P) All tests are green, but that doesn't mean all the generated bindings are correct, some cases may fail at runtime as I didn't test all cases in a real program. I did however generate and use the bindings on a non trivial lib I'm developping and they worked as expected.

src/bindgen/bindings.rs

This builds on mozilla#853 and adds a new language backend capable of generating java bindings. The generated bindings use [JNA](https://github.com/java-native-access/jna) (I used the last version 5.13.0 while developping but this will probably work with earlier ones as well) Most of the new code lives in `bindgen/language_backend/java_jna.rs` and is pretty straightfowards generation as most on the complicated things were done in the previous refactoring. The test suite has been updated to generate and compile the bindings for java when running. On nice thing gained with this is the possibilibty to put a configuration file specific to a language alongside the stardard one and the test will take this one instead of the default one if it exists (quite a few test use the `header` config to add some more code inside the generated bindings, C or preprocessor code doesn't work well in a java source file :P) All tests are green, but that doesn't mean all the generated bindings are correct, some cases may fail at runtime as I didn't test all cases in a real program. I did however generate and use the bindings on a non trivial lib I'm developping and they worked as expected.

ZhaoXiangXML · 2024-01-24T07:23:38Z

Hello, is this MR going to be merged?

fredszaq · 2024-01-24T10:22:09Z

Ping @emilio, I would love to see this merged (and #857) as well. Please tell be what I can do to facilitate this. All the remarks that have been raised till now have been addressed and I'll be pleased to make any other change deemed necessary.

kang-sw · 2024-01-30T14:31:50Z

Oh Wow, this was exactly what I was looking for and trying to implement on my own from this night. God thanks I tried searching 'dynamic' on issue list and found this. Great appreciation for your effort and wish it'd be merged onto mainstream as soon as possible, thanks!

src/bindgen/mod.rs

This builds on mozilla#853 and adds a new language backend capable of generating java bindings. The generated bindings use [JNA](https://github.com/java-native-access/jna) (I used the last version 5.13.0 while developping but this will probably work with earlier ones as well) Most of the new code lives in `bindgen/language_backend/java_jna.rs` and is pretty straightfowards generation as most on the complicated things were done in the previous refactoring. The test suite has been updated to generate and compile the bindings for java when running. On nice thing gained with this is the possibilibty to put a configuration file specific to a language alongside the stardard one and the test will take this one instead of the default one if it exists (quite a few test use the `header` config to add some more code inside the generated bindings, C or preprocessor code doesn't work well in a java source file :P) All tests are green, but that doesn't mean all the generated bindings are correct, some cases may fail at runtime as I didn't test all cases in a real program. I did however generate and use the bindings on a non trivial lib I'm developping and they worked as expected.

fredszaq mentioned this pull request Jun 30, 2023

Support C# and/or expose intermediate representation? #846

Open

fredszaq force-pushed the language-backend branch from 230574e to 96cea22 Compare June 30, 2023 14:42

fredszaq mentioned this pull request Jul 21, 2023

Add java support (using JNA) #857

Open

fredszaq mentioned this pull request Aug 17, 2023

Support for custom languages #867

Draft

emilio requested changes Aug 25, 2023

View reviewed changes

src/bindgen/bindings.rs Outdated Show resolved Hide resolved

src/bindgen/ir/generic_path.rs Outdated Show resolved Hide resolved

src/bindgen/bindings.rs Outdated Show resolved Hide resolved

fredszaq force-pushed the language-backend branch 2 times, most recently from 2ad1a19 to 9d84f34 Compare September 6, 2023 15:21

fredszaq force-pushed the language-backend branch from 9d84f34 to 314c47d Compare September 7, 2023 12:45

fredszaq requested a review from emilio September 7, 2023 13:03

fredszaq force-pushed the language-backend branch from 314c47d to e3da696 Compare October 4, 2023 11:24

Tacodiva reviewed Nov 16, 2023

View reviewed changes

src/bindgen/bindings.rs Outdated Show resolved Hide resolved

fredszaq force-pushed the language-backend branch from e3da696 to d550cf7 Compare November 17, 2023 11:12

kang-sw reviewed Jan 31, 2024

View reviewed changes

src/bindgen/mod.rs Show resolved Hide resolved

kang-sw mentioned this pull request Jan 31, 2024

Implement runtime binding generation #923

Draft

5 tasks

fredszaq added 2 commits March 12, 2024 13:42

introduce the concept of language backend

e50871e

make source trait generic on language backend

e592860

fredszaq added 5 commits March 12, 2024 13:42

remove generic source trait

7cc1833

remove useless box

1276a76

do not clone config

eaa4ac2

add write_bindings to LanguageBackend for more flexible LB impls

83346c9

remove write_with_backend from public api

c531e42

fredszaq force-pushed the language-backend branch from fd5497f to c531e42 Compare March 12, 2024 12:44

emilio mentioned this pull request Apr 14, 2024

Language back-end #942

Merged

emilio closed this in #942 Apr 14, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce language backend #853

Introduce language backend #853

fredszaq commented Jun 30, 2023

yanchith commented Jul 2, 2023

fredszaq commented Jul 10, 2023

emilio left a comment

emilio commented Aug 25, 2023

fredszaq commented Aug 25, 2023

fredszaq commented Sep 6, 2023

ZhaoXiangXML commented Jan 24, 2024

fredszaq commented Jan 24, 2024

kang-sw commented Jan 30, 2024

Introduce language backend #853

Introduce language backend #853

Conversation

fredszaq commented Jun 30, 2023

yanchith commented Jul 2, 2023

fredszaq commented Jul 10, 2023

emilio left a comment

Choose a reason for hiding this comment

emilio commented Aug 25, 2023

fredszaq commented Aug 25, 2023

fredszaq commented Sep 6, 2023

ZhaoXiangXML commented Jan 24, 2024

fredszaq commented Jan 24, 2024

kang-sw commented Jan 30, 2024