Added support for keyword-only arguments on Python 3+

[malinoff]

Pull Request Check List

• Added tests for changed code.
• New features have been added to our Hypothesis testing strategy.
• Updated documentation for changed code.
• Documentation in .rst files is written using semantic newlines.
• Changed/added classes/methods/functions have appropriate versionadded, versionchanged, or deprecated directives.
• Changes (and possible deprecations) have news fragments in changelog.d.

Please, correct me if I checked something wrong.

[malinoff]

@hynek I decided to go with a dedicated attr.ib() argument for the sake of simplicity: I don't like to overload init with more options than a simple on/off switch.
But if you think this is a bad API, let's discuss a better approach.

[hynek]

Wow, thanks for tackling this!

Before I dive deeper and start nitpicking:

• please make it kw_only it’s more readable and I’d like to set the precedent of not smashing words together German-style. :)
• There was an example of how to solve this problem for Python 2: Add support for keyword only arguments #106 (comment) and I think it’s worth pursuing. No need to make the feature Python 3-only and it should be rather easy to add? If you run into any problems implementing it, we can always back out again.

[codecov]

Codecov Report

Merging #281 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master   #281   +/-   ##
=====================================
  Coverage     100%   100%           
=====================================
  Files           9      9           
  Lines         660    679   +19     
  Branches      138    144    +6     
=====================================
+ Hits          660    679   +19

Impacted Files	Coverage Δ
src/attr/_make.py	100% <100%> (ø)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1bb8fbd...af1a70a. Read the comment docs.

[malinoff]

• please make it kw_only

Done.

• There was an example of how to solve this problem for Python 2: Add support for keyword only arguments #106 (comment) and I think it’s worth pursuing. No need to make the feature Python 3-only and it should be rather easy to add? If you run into any problems implementing it, we can always back out again.

Or we could make Python 3 transition slightly faster :) With no jokes, although I'm against of supporting such unique Python 3 features on Python 2 I'll try to implement this fallback anyway.

[hynek]

Ohhh…so you implemented it only on a per-attribute basis instead of per-class?

[hynek]

(JFTR I have zero qualms leaving Python 2 behind in 2017 if the cost exceeds a certain threshold)

[malinoff]

Ohhh…so you implemented it only on a per-attribute basis instead of per-class?

Yes. In my opinion, if we have a per-attribute way of specifying keyword-only attributes making a whole class keyword-only is easy:

from functools import partial
kw_attrib = partial(attr.ib, kw_only=True)
@attr.s
class Foo:
    bar = kw_attrib()
    baz = kw_attrib()

But maybe an additional attr.s option (kw_only) is worth implementing. I don't know, I feel no evil with partial approach.

[malinoff]

@hynek regarding Python 2 emulation of keyword-only arguments, I've found https://github.com/pasztorpisti/kwonly-args.
Although it works to some extent, it still doesn't exactly match Python 3 behavior:

Python 2.7.14 (default, Oct  8 2017, 18:34:38)
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from kwonly_args import first_kwonly_arg, KWONLY_REQUIRED
>>> @first_kwonly_arg('b')
... def foo(a, b=KWONLY_REQUIRED, c=KWONLY_REQUIRED, d=4):
...   pass
...
>>> foo()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/malinoff/Projects/opensource/attrs/venv27/lib/python2.7/site-packages/kwonly_args/__init__.py", line 108, in wrapper
    ', '.join(sorted(missing_kwonly_args))))
TypeError: foo() missing 2 keyword-only argument(s): b, c
>>> foo(1, 2, 3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/malinoff/Projects/opensource/attrs/venv27/lib/python2.7/site-packages/kwonly_args/__init__.py", line 108, in wrapper
    ', '.join(sorted(missing_kwonly_args))))
TypeError: foo() missing 2 keyword-only argument(s): b, c

vs

Python 3.6.1 (default, May 29 2017, 09:11:03)
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> def foo(a, *, b, c, d=4):
...   pass
...
>>> foo()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: foo() missing 1 required positional argument: 'a'
>>> foo(1, 2, 3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: foo() takes 1 positional argument but 3 were given

And the implementation is scary enough for me not wanting to translate it into the list of lines used to generate the __init__ body. Maybe somebody else has courage to do so.

What do you think about adding this as a Python 3 only feature for now?

[malinoff]

Rebased onto master.

[malinoff]

@Tinche since you've been pretty active in the referenced issues, I'd like to hear your opinion too.

[hynek]

As I’ve said, I don’t mind leaving Python 2 behind if it means too much work. @Tinche is currently having a life-or-death battle with a manflu, but this won’t make 17.3 anyways, so no pressure. :)

[leezu]

@malinoff Implementing the additional @attr.s option would be make this work nicely with @attr.s(auto_attribs=True), or am I overlooking something? I.e. it would remove the need for attr.ib calls..

Quoting an usage example from the docs

>>> import typing
>>> @attr.s(auto_attribs=True)
... class AutoC:
...     cls_var: typing.ClassVar[int] = 5  # this one is ignored
...     l: typing.List[int] = attr.Factory(list)
...     x: int = 1
...     foo: str = attr.ib(
...          default="every attrib needs a type if auto_attribs=True"
...     )
...     bar: typing.Any = None

[malinoff]

@leezu having only an attr.s option locks you between def foo(a, b, c=3): and def foo(*, a, b, c=3):.
Having an attr.ib option gives you enough flexibility: def foo(a, *b, c=3): and allows to implement the corresponding attr.s option for short-cut and convenience.

Also, I've opened this PR before 17.3 have been released, so I couldn't take auto_attribs into account at that time.

@hynek @Tinche, since 17.3 was cut off, could you please take a closer took at this PR?

[RonnyPfannschmidt]

would it be acceptable for python2 support to use a **kw_somewhat_unreadable_123 name combined with a funcsigs2 signature object that would return the complete signature for correctness ?

(given that popping from the dict + asserting can work, but the signature would suck

[leezu]

@malinoff essentially I would like to avoid writing out attr.ib for every attribute (based on auto_attribs=True), but still be able to use keyword-only arguments.

Having an attr.ib option gives you enough flexibility: def foo(a, *b, c=3): and allows to implement the corresponding attr.s option for short-cut and convenience.

Do you mean the kw_attrib partial? It doesn't allow avoiding attr.ib (or well, now kw_attrib).

You can argue that using keyword-only arguments is an advanced use-case, so it should require the "more complex" attr.ib syntax instead of the type hint based syntax with auto_attribs=True. I think otherwise, but please let me know if you disagree.

having only an attr.s option locks you between def foo(a, b, c=3): and def foo(*, a, b, c=3):

What about not "only an attr.s option" but an "shortcut attr.s option" which defaults to def foo(*, a, b, c=3) and more complex use-cases can be implemented via the corresponding attr.ib calls?

[malinoff]

@leezu

What about not "only an attr.s option" but an "shortcut attr.s option" which defaults to def foo(*, a, b, c=3) and more complex use-cases can be implemented via the corresponding attr.ib calls?

That's what I mean in my previous reply.

[hynek]

So to come back to this: IIRC this would be the first option that exists only on the attributes but not for the whole class?

I think we should really allow both: in attr.ib to None (=default) and add an option to attr.s. If you don’t want to do it here, it’s fine: we can finish this one up but block 17.4. until the second part is done. At this moment, this has to be rebased.

[ronlut]

Sorry for bursting into your conversation, but I hope it's related enough.
Will this PR (or combined with anything else already implemented) allow me to use attrs for something similar to this use case:

class A:
    def __init__(self, **kwargs):
        for key in kwargs:
            setattr(self, key, kwargs[key])

I guess it's more related to your plan to add an option to attr.s and not the attr.ib one.

[veltzerdoron]

I see this is not implemented in the code yet, will it be a part of version 17.4?

[hynek]

Unlikely, 17.4 is gonna be a mostly (overdue) bug fix release that should drop next week if nothing comes up.

[alanhdu]

@malinoff Is there anything else that needs to happen with this PR? I'm happy to take over the branch to try to get it in if you don't have the time).

(Although for my own use-case, I just need the class-level kw_only flag, but not the attr.ib level flag).

[asvetlov]

See also python/cpython#6238

[malinoff]

I'm okay to finish the PR by myself, I just don't understand what else should be implemented/fixed.

[alanhdu]

@hynek Would you mind commenting on what else needs to happen here (other than resolving the merge conflicts)? Is this blocked on the design issue of the whether this should also be an attr.s option?

(FWIW @malinoff I would definitely like the attr.s option so this works nicely with auto_attribs).

[alanhdu]

Sorry to be a bother, but bump @hynek about what more needs to happen here.

[hynek]

I’m not gonna review the concrete code now, here’s the general requirements for this to be merged:

• Python 2 is completely optional for new features if it would be too kludgy to implement. I cannot stress this enough.
• There needs to be a global option like @attr.s(kw_only=True).

So it should work like this:

@attr.s(kw_only=True)
class C:
    x = attr.ib()
    y = attr.ib()

-> __init__(self, *, x, y)

@attr.s
class C:
    x = attr.ib()
    y = attr.ib(kw_only=True)

-> __init__(self, x, *, y)

I think it’s fair to expect that a non-kw_only attribute must not come after a kw_only attribute.

Am I miss something?

I’m sorry for not being very responsive.

(Please disregard my comments from before about setting attr.ib's kw_only to None – that would be inconsistent with the rest of attrs. It should be False of course.)

[hynek]

fixed by #411