Option to apply on an indented fragment of Python code (e.g. a single method)

[tartley]

My problem is that sometimes I'm editing some code in a project that doesn't use Black (heresy, I know). Sometimes I'll type some code, or paste into the source something like a generated dict literal, and I want to run Black on just the edited section of code.

I set up a Vim binding to do that, passing just the code from the visual selection to Black, and using the output to update the code in Vim. The result is:

1. If I select lines of module-level code, this works really well! Hooray!
2. If I select a few lines in the middle of a function, or select a whole class method, this fails, because Black (understandably) does not like that the initial line of code is indented.
3. Trivially, it also fails if the selected code is not an entire valid expression or statement. That's fine and I don't propose trying to "fix" that.

What I'd like to see:

• Black invoked normally should continue to produce an error if the initial line has indents.
• Black should gain something like a "--fragment" (or perhaps "--indented") command line flag, for working on potentially indented code fragments. When given, this causes Black to:
  • un-indent the given source sufficiently to make the first line have no indents
  • reformat as normal
  • re-indent the reformatted source by the same amount.

The alternative I'm currently considering is writing my own wrapper script to do the un-indenting and re-indenting. Not a biggie, but I thought other people might appreciate this mode of operation being supported out of the box.

[tartley]

For reference, Vim binding to call Black on the visual selection:

" Black(Python) format the visual selection
xnoremap <Leader>k :!black -q -<CR>

[SanketDG]

If indentation seems to be the only problem here not allowing to perform black as intended, then a wrapper script using textwrap (the indent/dedent functions) seems like a crude/quick way of doing that.

This can obviously be integrated in black through some argument (or maybe natively!) using the same logic.

[tartley]

I wrote a wrapper script to do the dedenting/reindenting: https://github.com/tartley/dotfiles/blob/master/other/bin/enblacken

(I don't think textwrap is any use for this particular task. We could use the .dedent() there, but then we'd have to go and manually count how many spaces it dedented by, so that we know how much to reindent by later. The reindenting is just line = " " * indent + line. So I just do it all manually.)

It's rough and ready, and in particular doesn't handle errors from Black (e.g. what if the fragment you pass to Black has a syntax error) but works for me thus far.

[ichard26]

For emac users, there is https://github.com/wbolster/emacs-python-black (according to #1076 (comment))

See also https://github.com/wbolster/black-macchiato (according to #987 (comment) and #1076 (comment))

I haven't verified that either of these options actually work.

[GreatBahram]

I don't understand @ichard26, you just closed all issues that asked for "Format Selection" including mine! All of them were saying the same thing but in different situations: one asked for a cli option, for me (#987) I asked for running it against some visualized lines in vim, other one asked for running in vscode, All of them are the same.

Is it going somewhere anysoon? Can I help to achieve this functionality?

[ambv]

Due to microsoft/vscode-black-formatter#176 we should tackle this soonish.

[akaihola]

Darker is a tool which applies Black reformatting only to lines it detects as having been modified after the latest commit (or since a given commit, or between given commits).

The way Darker does it is:

1. ask Git for a diff between the original and modified versions of a Python source file (e.g. HEAD and working directory)
2. record line numbers for modified lines in the modified version
3. get a reformatted version of the modified file using Black
4. for the modified file, create a diff comparing the file before and after reformatting
5. record contiguous line regions in the diff
6. check which of those regions intersect with modified line numbers from step 2.
7. apply only those regions of the diff to the modified file
8. verify that the AST stays intact using Black

In some edge cases, it is not possible to reliably detect which areas in the file before reformatting correspond to reformatted blocks. In that case, Darker extends the diff regions until the AST verification succeeds. This is done using bisection to quickly find the smallest successful region.

For those interested, the essential parts of this algorithm are in darker.__main__._reformat_single_file().

Doing all this would be much simpler if Black indeed grew the capability to reformat partial files! Steps 3–8 could be replaced with a simple request for Black to reformat the given line range. So I'm excited about this – and the fact that the decision to not support line ranges (in comment from @ambv in microsoft/vscode-python#134 in Apr 2018) has been reversed. 👍

[akaihola]

Also, as I mentioned in microsoft/vscode-black-formatter#176, Darker can already act as a drop-in replacement for Black as a code formatter in VSCode (and various other IDEs, see Editor integration in the README).

Currently, when using Darker as a formatter, essentially VSCode will reformat code (either with a shortcut, after a delay or after saving) but leave unmodified regions intact.

So if Black should decide to not implement reformatting of a given line range after all, we could add e.g. a --range=<first-line>-<last-line> option to Darker (when processing a single file) and propose that VSCode adds support for Darker (although @brettcannon wasn't yet fond of that idea in his comment).

[gpshead]

You may also be interested in https://github.com/google/yapf/blob/main/yapf/third_party/yapf_diff/yapf_diff.py as a reference or even its own tool, it is a stand alone little script to take a diff as input and use that to drive a tool (ie: a formatter accepting one or more --lines X-Y argument pairs) telling it which input line ranges to output changes for.

All the algorithms used for this that i've seen use basically the same obvious tactic: process a diff to discover the input file regions of interest, only accept changes that land within those (anchored to original input file line numbers) line ranges.

[yilei]

Curious, what is the status on the "reformatting of given line range(s)" support in Black? I've also read microsoft/vscode-black-formatter#176 and #2883, and there is now a vscode-black-formatter extension, but it doesn't seem to have Format Selection support? Is this feature still planned?

[leorochael]

As an out-of-the-blue idea I just had, for specifically the case of formatting an indented block, what if we just reproduced the indentation for black, instead of trying to dedent before and re-indent later?

For instance suppose I want to reformat the section marked below:

class Something(Other):

    def method(self, some_parameter):
        try:
            # The code selection starts on the line below
            if condition:
                self.do_something_else_with_parameters({
                    'complex-structure': 'containing values', 'some other key':
                        some_parameter})
            # The code selection ends on the line above
        finally:
            self.do_some_cleanup()

We could sent to black something like:

if indent_level_1:
    if indent_level_2:
        if indent_level_3:  # START
            if condition:
                self.do_something_else_with_parameters({
                    'complex-structure': 'containing values', 'some other key':
                        some_parameter})

Whatever black sends back, we get the lines after the line containing # START (result.split('# START\n', 1)[1]), and it should be correctly formatted and indented.

This should work for any python fragment if it contains complete statements (i.e. if the fragment contains a try should have the respective except or finally clauses).

And detecting how many levels of indentation are needed should be easy from the first line of the fragment, again, as long as it contains complete statements.

[yilei]

FYI-- We have patches that implement this feature (in the Pyink fork via the --pyink-lines= CLI flag), and I plan to upstream them to Black in a few months.

(Before that though, I need to fix microsoft/vscode-python#3438 since the implementation relies it.)

[hauntsaninja]

Closing since now we have #4020!