Maintaining a Black fork and/or derivative

[jsh9]

Hi @warsaw,

I saw your comment on Blue (grantjenks/blue#2 (comment)) that keeping Blue in sync with Black can be painful.

May I pick your brain on why it is painful and how exactly it is painful?

I recently forked Black to implement some configurable options that I need. (It's this repo that contains this discussion thread.)

And so far I find it relatively easy to keep in sync with Black's latest changes. I wrote this short instruction doc to remind myself how to do that. But I haven't been maintaining my fork long, so maybe I don't know what I don't know.

(I didn't leave this message over at Blue's issue board because I respect Blue's maintainers such as you, and I don't want to appear to be plugging my own stuff there.)

[warsaw]

Hello @jsh9 !

Blue took a different approach, which is probably why it's harder to keep in sync. Rather than actually forking black and rebasing as it progresses, blue is more of a monkeypatching wrapper around black. So black is a dependency of blue, and blue's entry point reaches into the guts of black and monkeypatches the heck out of it. We did this originally because we were hoping that we could convince the black developers to at least provide some standard hooks for the functionality we wanted to change, and then our monkeypatched functions could easily turn into hook functions. This ultimately never happened though - black's developers were reluctant to provide such hooks (and I spent quite a bit of time at a US Pycon a few years ago talking with some of them about it 😄 ). It's their project so no hard feelings of course!

Ultimately the monkeypatching approach is just too fragile because we have to reach into internal, non-public APIs and find the right place to monkeypatch. That ties blue too closely to those APIs, which of course the black maintainers are under no obligation to keep. So when those internal APIs change, blue must also change and sometimes it's difficult to find the right place to monkeypatch.

The other problem that blue has had is a dependency on flake8 for its configuration support. Since black itself doesn't support any configuration options, it doesn't support any configuration files. Blue tried to use flake8 for that but then that just ties us to another project's non-public API.

Another problem is blackd. While that's a totally reasonable approach for trying to squeeze better perceived performance out of black, we can't use it because there's no good way to monkeypatch the server.

Your approach of actually forking black probably better solves the first problem, but I'm curious how you adjust for the same refactoring of black's internal API. I suspect that your forking approach also better handles the server problem too.

[jsh9]

Hi @warsaw , thanks for the detailed reply!

My approach (forking and then adding my own configurable options) also requires doing some hacky stuff in its non-public API. Since Black announced that it had gone stable, it's reasonable to believe that there won't be big refactoring of its code base. But we'll never know for certain, so I'll cross the bridge when I get there.

The configuration options of my fork live in pyproject.toml. I think that's sufficient for me at this point, and there hasn't been such requests to make config available in tox.ini or other files. So fortunately I don't have to be dependent on flake8.

I haven't made any changes to blackd. It still remains a folder in my fork.

I really hope the authors of Ruff can be open-minded about people's requests for configurability. If Ruff doesn't offer the features I'm looking for, it would be hard for me to fork it and make changes. I don't code in Rust, at least yet.