Document new jOOQ-refaster module

[lukaseder]

The work from #9630 needs documentation in the manual

[knutwannheden]

I think I have now finally figured out how to use Errorprone Refaster both for Java 8 and 11. I doubt that there is any solution for Java 6.

[lukaseder]

I doubt that there is any solution for Java 6.

We don't need a Java 6 solution, that's fine. The jOOQ-checker module also doesn't support Java 6

[lukaseder]

... A lot of this stuff depends on JSR-308, which shipped with Java 8

[knutwannheden]

We need to make the compiled .refaster files available. We can of course add them to the ZIP distribution, but we should also make them available for download from the website and link them from the manual. Any suggestions for where in the ZIP and where on the web site to add the files?

[lukaseder]

Can we load them from the classpath, too? Ideally, the files would be in a .jar file, so we could upload the .jar file to Maven Central like any other...

[knutwannheden]

We could also commit the .refaster files to Git and link to the files in GitHub. But that would only work for .refaster files which we also include as part of the OSS edition.

Can we load them from the classpath, too? Ideally, the files would be in a .jar file, so we could upload the .jar file to Maven Central like any other...

So far I have not found any way to do that. Maybe I find something if I study the source code. I tested with URLs, but that doesn't work: google/error-prone#1513.

[lukaseder]

We could also commit the .refaster files to Git and link to the files in GitHub. But that would only work for .refaster files which we also include as part of the OSS edition.

Hmm, not sure. We'd depend on GitHub's directory structures and the way they present URLs of files. I believe they have changed that a few times in the past.

I'd prefer a directory like www.jooq.org/refaster, similar to the XSD files. Also, we'd include them in the ZIP files somewhere, but I don't have an idea yet.

[knutwannheden]

I'd prefer a directory like www.jooq.org/refaster, similar to the XSD files. Also, we'd include them in the ZIP files somewhere, but I don't have an idea yet.

Yes, that would also be my preferred solution. As with the XSDs we will want to have versioned files. So something like https://jooq.org/refaster/deprecation-3.13.0.refaster.

[lukaseder]

Yes, that would also be my preferred solution. As with the XSDs we will want to have versioned files. So something like https://jooq.org/refaster/deprecation-3.13.0.refaster.

Exactly

[knutwannheden]

Since Errorprone currently cannot be configured to load and apply multiple .refaster templates at once, I think it would for the time being make more sense to only produce a single .refaster file. At the moment I was planning to have separate deprecation.refaster and performance.refaster files. I think we should merge them into something like jooq.refaster or similar.

[lukaseder]

I don't agree. We need separation of concerns. For example, everyone might agree with the deprecation transformations, but not with the performance ones. It is better for users to be able to apply these files individually and see the resulting patch output separately, rather than to have everything combined and then not be sure why some change was suggested.

[knutwannheden]

OK. It is just really a shame that multiple files can't be applied at once. IIRC @Stephan202 said that he had a patch to address this. Possibly he could be coerced into providing a build for this 😉

[lukaseder]

Well, I see your point... how about publishing both? 1) a set of individual files and 2) the combined file? I just want to make sure users can use the individual files as well.

[lukaseder]

The combined file would be called all.refaster, for clarity.

[knutwannheden]

Well, I see your point... how about publishing both? 1) a set of individual files and 2) the combined file?

That would also be an option. Due to another limitation of Refaster, a .refaster file can currently only be based on a single .java source file. So to create an all.refaster I would have to create a new .java source containing both of the other two .java sources. Also not very nice...

[lukaseder]

I think we have enough preprocessing infrastructure to produce such Java files by now 😉

[Stephan202]

Very nice to see this take shape!

I think we have enough preprocessing infrastructure to produce such Java files by now 😉

That's probably the most pragmatic approach for now. Some alternatives for completeness:

• An alternative is to load all files using ObjectInputStreams, wrap them in a CompositeCodeTransformer and write them back out to using an ObjectOutputStream.
• The fork provides this out of the box by including the changes of RefasterRuleCompiler ignores all but one input file google/error-prone#552.

I'd prefer a directory like www.jooq.org/refaster

Food for thought:

• Since the .refaster files are (presumably) produced as part of a Maven build, you could also attach them as regular artifacts, e.g. using the build-helper-maven-plugin. This way they'll be hosted on Maven Central and keeping them in sync with the release cycles will come for "free".
• It would be nice to (eventually) produce a Maven Central-hosted JAR in which these files are on the "classpath". More on that below :)

OK. It is just really a shame that multiple files can't be applied at once. IIRC @Stephan202 said that he had a patch to address this.

I checked and it seems I don't have a patch against Error Prone for this. (The fork includes google/error-prone#947, but while that is a step in the right direction, it is not enough.) Currently within Picnic we rely on the grouping provided by google/error-prone#552. The next step is to finalize the code I have described under point 2 in google/error-prone#552 (comment). This is reason I'm interested in the classpath-based distribution mechanism mentioned above: once the RefasterCheck is open-source users can not only apply Refaster rules, but they can also treat them as regular Error prone checks.

I'm swamped with work these weeks, but I can see how it'd be nice to finally open-source a first version of the code. Let me see what I can do in the upcoming weeks.

[knutwannheden]

@Stephan202 Thanks for your feedback. There are indeed a few open questions and issues surrounding Refaster. I hope Google will at some point provide some feedback.

• An alternative is to load all files using ObjectInputStreams, wrap them in a CompositeCodeTransformer and write them back out to using an ObjectOutputStream.

I just did this a few minutes ago 🤣

• Since the .refaster files are (presumably) produced as part of a Maven build, you could also attach them as regular artifacts, e.g. using the build-helper-maven-plugin. This way they'll be hosted on Maven Central and keeping them in sync with the release cycles will come for "free".

Good idea. I will look into this. But I would really also prefer a solution which allows loading the .refaster files from the classpath.

[knutwannheden]

Documentation has been published along with .refaster files. See https://www.jooq.org/doc/3.13/manual/tools/refaster/ and https://www.jooq.org/refaster/.

[lukaseder]

• Since the .refaster files are (presumably) produced as part of a Maven build, you could also attach them as regular artifacts, e.g. using the build-helper-maven-plugin. This way they'll be hosted on Maven Central and keeping them in sync with the release cycles will come for "free".

Good idea. I will look into this. But I would really also prefer a solution which allows loading the .refaster files from the classpath.

I'm not really a fan of this idea for these reasons:

• It is very un-idiomatic to search for non jar/war/etc artifacts on Maven Central.
• It feels hacky. While it may help with the distribution model, we cannot easily include the dependency for usage. Users would have to define a weird <type>refaster</type> or whatever dependency, and then reference it on their local file system nonetheless (as they wouldn't end up on the classpath)
• It is very difficult to document to users who are not Maven gurus.
• It is very difficult to keep forwards compatible. We might just rename the files, and then people would spend endless times looking for the current file name on Maven Central, as we'd be polluting our namespace.

As always, when ideas are "clever", the caveats start feeling like show stoppers. We just may have to accept that this technology is not very mature yet, and without the vendor (Google) providing clear guidelines and better features, I prefer not to do too many clever things.

[knutwannheden]

Let's stick with the .refaster files on the web site and in the ZIP files for now. Once Error Prone can load the .refaster files from the classpath, we can revisit this.

[lukaseder]

Yes, and I hope we'll then be able to ship them in a jar file... Am Freitag, 21. Februar 2020 schrieb Knut Wannheden < notifications@github.com>:

…

Let's stick with the .refaster files on the web site and in the ZIP files for now. Once Error Prone can load the .refaster files from the classpath, we can revisit this. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#9825?email_source=notifications&email_token=AAFTLAIMVHTND7KJOJBSHXTRD64MBA5CNFSM4KTX4C52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMSNYHI#issuecomment-589618205>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFTLAN2YNPCGEORGHS5OYLRD64MBANCNFSM4KTX4C5Q> .