Add --unstable flag

CHANGES.md

@@ -6,22 +6,63 @@
 
 <!-- Include any especially major or disruptive changes here -->
 
+This release introduces the new 2024 stable style (#4106), stabilizing the following
+changes:
+
+- Add parentheses around `if`-`else` expressions (#2278)
+- Dummy class and function implementations consisting only of `...` are formatted more
+  compactly (#3796)
+- If an assignment statement is too long, we now prefer splitting on the right-hand side
+  (#3368)
+- Hex codes in Unicode escape sequences are now standardized to lowercase (#2916)
+- Allow empty first lines at the beginning of most blocks (#3967, #4061)
+- Add parentheses around long type annotations (#3899)
+- Standardize on a single newline after module docstrings (#3932)
+- Fix incorrect magic trailing comma handling in return types (#3916)
+- Remove blank lines before class docstrings (#3692)
+- Wrap multiple context managers in parentheses if combined in a single `with` statement
+  (#3489)
+- Fix bug in line length calculations for power operations (#3942)
+- Add trailing commas to collection literals even if there's a comment after the last
+  entry (#3393)
+- When using `--skip-magic-trailing-comma` or `-C`, trailing commas are stripped from
+  subscript expressions with more than 1 element (#3209)
+- Add extra blank lines in stubs in a few cases (#3564, #3862)
+- Accept raw strings as docstrings (#3947)
+- Split long lines in case blocks (#4024)
+- Stop removing spaces from walrus operators within subscripts (#3823)
+- Fix incorrect formatting of certain async statements (#3609)
+- Allow combining `# fmt: skip` with other comments (#3959)
+
+There are already a few improvements in the `--preview` style, which are slated for the
+2025 stable style. Try them out and
+[share your feedback](https://github.com/psf/black/issues). In the past, the preview
+style has included some features that we were not able to stabilize. This year, we're
+adding a separate `--unstable` style for features with known problems. Now, the
+`--preview` style only includes features that we actually expect to make it into next
+year's stable style.
+
 ### Stable style
 
 <!-- Changes that affect Black's stable style -->
 
-### Preview style
-
-<!-- Changes that affect Black's preview style -->
+Several bug fixes were made in features that are moved to the stable style in this
+release:
 
 - Fix comment handling when parenthesising conditional expressions (#4134)
-- Format module docstrings the same as class and function docstrings (#4095)
 - Fix bug where spaces were not added around parenthesized walruses in subscripts,
   unlike other binary operators (#4109)
 - Remove empty lines before docstrings in async functions (#4132)
 - Address a missing case in the change to allow empty lines at the beginning of all
   blocks, except immediately before a docstring (#4130)
 - For stubs, fix logic to enforce empty line after nested classes with bodies (#4141)
+
+### Preview style
+
+<!-- Changes that affect Black's preview style -->
+
+- Add `--unstable` style (#4096)
+- Format module docstrings the same as class and function docstrings (#4095)
 - Fix crash when using a walrus in a dictionary (#4155)
 - Fix unnecessary parentheses when wrapping long dicts (#4135)
 

docs/faq.md

@@ -41,9 +41,10 @@ other tools, such as `# noqa`, may be moved by _Black_. See below for more detai
 Stable. _Black_ aims to enforce one style and one style only, with some room for
 pragmatism. See [The Black Code Style](the_black_code_style/index.md) for more details.
 
-Starting in 2022, the formatting output will be stable for the releases made in the same
-year (other than unintentional bugs). It is possible to opt-in to the latest formatting
-styles, using the `--preview` flag.
+Starting in 2022, the formatting output is stable for the releases made in the same year
+(other than unintentional bugs). At the beginning of every year, the first release will
+make changes to the stable style. It is possible to opt in to the latest formatting
+styles using the `--preview` flag.
 
 ## Why is my file not formatted?
 

docs/the_black_code_style/current_style.md

@@ -449,6 +449,12 @@ file that are not enforced yet but might be in a future version of the formatter
 _Black_ will normalize line endings (`\n` or `\r\n`) based on the first line ending of
 the file.
 
+### Form feed characters
+
+_Black_ will retain form feed characters on otherwise empty lines at the module level.
+Only one form feed is retained for a group of consecutive empty lines. Where there are
+two empty lines in a row, the form feed is placed on the second line.
+
 ## Pragmatism
 
 Early versions of _Black_ used to be absolutist in some respects. They took after its

docs/the_black_code_style/future_style.md

@@ -1,117 +1,38 @@
 # The (future of the) Black code style
 
-```{warning}
-Changes to this document often aren't tied and don't relate to releases of
-_Black_. It's recommended that you read the latest version available.
-```
-
-## Using backslashes for with statements
-
-[Backslashes are bad and should be never be used](labels/why-no-backslashes) however
-there is one exception: `with` statements using multiple context managers. Before Python
-3.9 Python's grammar does not allow organizing parentheses around the series of context
-managers.
-
-We don't want formatting like:
-
-```py3
-with make_context_manager1() as cm1, make_context_manager2() as cm2, make_context_manager3() as cm3, make_context_manager4() as cm4:
-    ...  # nothing to split on - line too long
-```
-
-So _Black_ will, when we implement this, format it like this:
-
-```py3
-with \
-     make_context_manager1() as cm1, \
-     make_context_manager2() as cm2, \
-     make_context_manager3() as cm3, \
-     make_context_manager4() as cm4 \
-:
-    ...  # backslashes and an ugly stranded colon
-```
-
-Although when the target version is Python 3.9 or higher, _Black_ uses parentheses
-instead in `--preview` mode (see below) since they're allowed in Python 3.9 and higher.
-
-An alternative to consider if the backslashes in the above formatting are undesirable is
-to use {external:py:obj}`contextlib.ExitStack` to combine context managers in the
-following way:
-
-```python
-with contextlib.ExitStack() as exit_stack:
-    cm1 = exit_stack.enter_context(make_context_manager1())
-    cm2 = exit_stack.enter_context(make_context_manager2())
-    cm3 = exit_stack.enter_context(make_context_manager3())
-    cm4 = exit_stack.enter_context(make_context_manager4())
-    ...
-```
-
-(labels/preview-style)=
-
 ## Preview style
 
 Experimental, potentially disruptive style changes are gathered under the `--preview`
 CLI flag. At the end of each year, these changes may be adopted into the default style,
 as described in [The Black Code Style](index.md). Because the functionality is
 experimental, feedback and issue reports are highly encouraged!
 
-### Improved string processing
-
-_Black_ will split long string literals and merge short ones. Parentheses are used where
-appropriate. When split, parts of f-strings that don't need formatting are converted to
-plain strings. User-made splits are respected when they do not exceed the line length
-limit. Line continuation backslashes are converted into parenthesized strings.
-Unnecessary parentheses are stripped. The stability and status of this feature is
-tracked in [this issue](https://github.com/psf/black/issues/2188).
-
-### Improved line breaks
-
-For assignment expressions, _Black_ now prefers to split and wrap the right side of the
-assignment instead of left side. For example:
-
-```python
-some_dict[
-    "with_a_long_key"
-] = some_looooooooong_module.some_looooooooooooooong_function_name(
-    first_argument, second_argument, third_argument
-)
-```
-
-will be changed to:
+In the past, the preview style included some features with known bugs, so that we were
+unable to move these features to the stable style. Therefore, such features are now
+moved to the `--unstable` style. All features in the `--preview` style are expected to
+make it to next year's stable style; features in the `--unstable` style will be
+stabilized only if issues with them are fixed. If bugs are discovered in a `--preview`
+feature, it is demoted to the `--unstable` style.
 
-```python
-some_dict["with_a_long_key"] = (
-    some_looooooooong_module.some_looooooooooooooong_function_name(
-        first_argument, second_argument, third_argument
-    )
-)
-```
+Currently, the following features are included in the preview style:
 
-### Improved parentheses management
+- `hex_codes_in_unicode_sequences`: normalize casing of Unicode escape characters in
+  strings
+- `unify_docstring_detection`: fix inconsistencies in whether certain strings are
+  detected as docstrings
+- `hug_parens_with_braces_and_square_brackets`: more compact formatting of nested
+  brackets ([see below](labels/hug-parens))
 
-For dict literals with long values, they are now wrapped in parentheses. Unnecessary
-parentheses are now removed. For example:
+The unstable style additionally includes the following features:
 
-```python
-my_dict = {
-    "a key in my dict": a_very_long_variable
-    * and_a_very_long_function_call()
-    / 100000.0,
-    "another key": (short_value),
-}
-```
+- `string_processing`: split long string literals and related changes
+  ([see below](labels/string-processing))
+- `wrap_long_dict_values_in_parens`: add parentheses to long values in dictionaries
+  ([see below](labels/wrap-long-dict-values))
+- `multiline_string_handling`: more compact formatting of expressions involving
+  multiline strings ([see below](labels/multiline-string-handling))
 
-will be changed to:
-
-```python
-my_dict = {
-    "a key in my dict": (
-        a_very_long_variable * and_a_very_long_function_call() / 100000.0
-    ),
-    "another key": short_value,
-}
-```
+(labels/hug-parens)=
 
 ### Improved multiline dictionary and list indentation for sole function parameter
 
@@ -185,6 +106,46 @@ foo(
 )
 ```
 
+(labels/string-processing)=
+
+### Improved string processing
+
+_Black_ will split long string literals and merge short ones. Parentheses are used where
+appropriate. When split, parts of f-strings that don't need formatting are converted to
+plain strings. User-made splits are respected when they do not exceed the line length
+limit. Line continuation backslashes are converted into parenthesized strings.
+Unnecessary parentheses are stripped. The stability and status of this feature is
+tracked in [this issue](https://github.com/psf/black/issues/2188).
+
+(labels/wrap-long-dict-values)=
+
+### Improved parentheses management in dicts
+
+For dict literals with long values, they are now wrapped in parentheses. Unnecessary
+parentheses are now removed. For example:
+
+```python
+my_dict = {
+    "a key in my dict": a_very_long_variable
+    * and_a_very_long_function_call()
+    / 100000.0,
+    "another key": (short_value),
+}
+```
+
+will be changed to:
+
+```python
+my_dict = {
+    "a key in my dict": (
+        a_very_long_variable * and_a_very_long_function_call() / 100000.0
+    ),
+    "another key": short_value,
+}
+```
+
+(labels/multiline-string-handling)=
+
 ### Improved multiline string handling
 
 _Black_ is smarter when formatting multiline strings, especially in function arguments,
@@ -297,13 +258,51 @@ s = (  # Top comment
 )
 ```
 
-=======
+## Potential future changes
+
+This section lists changes that we may want to make in the future, but that aren't
+implemented yet.
+
+### Using backslashes for with statements
+
+[Backslashes are bad and should be never be used](labels/why-no-backslashes) however
+there is one exception: `with` statements using multiple context managers. Before Python
+3.9 Python's grammar does not allow organizing parentheses around the series of context
+managers.
+
+We don't want formatting like:
+
+```py3
+with make_context_manager1() as cm1, make_context_manager2() as cm2, make_context_manager3() as cm3, make_context_manager4() as cm4:
+    ...  # nothing to split on - line too long
+```
+
+So _Black_ will, when we implement this, format it like this:
+
+```py3
+with \
+     make_context_manager1() as cm1, \
+     make_context_manager2() as cm2, \
+     make_context_manager3() as cm3, \
+     make_context_manager4() as cm4 \
+:
+    ...  # backslashes and an ugly stranded colon
+```
+
+Although when the target version is Python 3.9 or higher, _Black_ uses parentheses
+instead in `--preview` mode (see below) since they're allowed in Python 3.9 and higher.
 
-### Form feed characters
+An alternative to consider if the backslashes in the above formatting are undesirable is
+to use {external:py:obj}`contextlib.ExitStack` to combine context managers in the
+following way:
 
-_Black_ will now retain form feed characters on otherwise empty lines at the module
-level. Only one form feed is retained for a group of consecutive empty lines. Where
-there are two empty lines in a row, the form feed will be placed on the second line.
+```python
+with contextlib.ExitStack() as exit_stack:
+    cm1 = exit_stack.enter_context(make_context_manager1())
+    cm2 = exit_stack.enter_context(make_context_manager2())
+    cm3 = exit_stack.enter_context(make_context_manager3())
+    cm4 = exit_stack.enter_context(make_context_manager4())
+    ...
+```
 
-_Black_ already retained form feed literals inside a comment or inside a string. This
-remains the case.
+(labels/preview-style)=

docs/the_black_code_style/index.md

@@ -42,9 +42,11 @@ _Black_:
   enabled by newer Python language syntax as well as due to improvements in the
   formatting logic.
 
-- The `--preview` flag is exempt from this policy. There are no guarantees around the
-  stability of the output with that flag passed into _Black_. This flag is intended for
-  allowing experimentation with the proposed changes to the _Black_ code style.
+- The `--preview` and `--unstable` flags are exempt from this policy. There are no
+  guarantees around the stability of the output with these flags passed into _Black_.
+  They are intended for allowing experimentation with proposed changes to the _Black_
+  code style. The `--preview` style at the end of a year should closely match the stable
+  style for the next year, but we may always make changes.
 
 Documentation for both the current and future styles can be found:
 

docs/usage_and_configuration/black_as_a_server.md

@@ -62,6 +62,9 @@ The headers controlling how source code is formatted are:
 - `X-Preview`: corresponds to the `--preview` command line flag. If present and its
   value is not an empty string, experimental and potentially disruptive style changes
   will be used.
+- `X-Unstable`: corresponds to the `--unstable` command line flag. If present and its
+  value is not an empty string, experimental style changes that are known to be buggy
+  will be used.
 - `X-Fast-Or-Safe`: if set to `fast`, `blackd` will act as _Black_ does when passed the
   `--fast` command line flag.
 - `X-Python-Variant`: if set to `pyi`, `blackd` will act as _Black_ does when passed the

docs/usage_and_configuration/the_basics.md

@@ -144,9 +144,18 @@ magic trailing comma is ignored.
 
 #### `--preview`
 
-Enable potentially disruptive style changes that may be added to Black's main
-functionality in the next major release. Read more about
-[our preview style](labels/preview-style).
+Enable potentially disruptive style changes that we expect to add to Black's main
+functionality in the next major release. Use this if you want a taste of what next
+year's style will look like.
+
+Read more about [our preview style](labels/preview-style).
+
+#### `--unstable`
+
+Enable all style changes in `--preview`, plus additional changes that we would like to
+make eventually, but that have known issues that need to be fixed before they can move
+back to the `--preview` style. Use this if you want to experiment with these changes and
+help fix issues with them.
 
 (labels/exit-code)=