RFC: #[derive(SmartPointer)]

[Darksonn]

Use custom smart pointers with trait objects.

Rendered

Tracking issue: rust-lang/rust#123430

---

Co-authored by @Darksonn and @Veykril
Thank you to @compiler-errors for the original idea

[Lokathor]

My main thought is that maybe this should just require repr(transparent), which simplifies a lot of the explanation of the requirements part.

[joshtriplett]

My main thought is that maybe this should just require repr(transparent), which simplifies a lot of the explanation of the requirements part.

I don't think there should be a hard requirement that smart pointers be transparent. But since the requirements are very similar, it might make sense to incorporate the requirements by reference, for simplicity of documentation.

[clarfonthey]

After doing a bit more research to get myself acquainted with the CoerceUnsized and DispatchFromDyn traits, I really don't think that it's a good idea to just bundle these both together as "smart pointer." Yes, all smart pointers will have these, but it's a false equivalence; having them does not make you a smart pointer.

What these traits really are about is ensuring proper variance, which is especially weird since variance is usually only exposed to programmers in Rust through weird mechanisms like PhantomData. Ultimately:

• CoerceUnsized lets you declare a type as covariant, and varying the type this way makes it more general. For example, this lets you vary a mutable reference into an immutable reference or an array into a slice.
• DispatchFromDyn lets you declare a type as contravariant, and varying the type this way makes it more specific. This would be the opposite, like varying a slice into an array or a trait object into a concrete type.

Smart pointers require both of these just by the nature of how references in Rust work in general: because autoderef lets us borrow things mostly automatically, we expect this automatic borrowing to happen even if a smart pointer is between us and our data. However, it's important to realise that this is just about variance and we only tie it specifically to smart pointers because unsized types in Rust today have to be used behind pointers.

If we allow unsized locals, whose RFC was accepted, now these types become even more just about variance. You should always be able to pass arrays into functions which accept unsized slices, return unsized trait objects by returning a concrete object, etc., etc.

Honestly, thinking of this more specifically as a way of managing variance, I do question whether the current method of manually implementing these traits is the right approach, and whether we should rely more on something like repr(transparent), as mentioned by @Lokathor. This falls closer to what currently is done for managing variance in other ways, via PhantomData.

Of course, we can't just use repr(transparent), for the following reasons:

1. Reference-counting necessitates adding extra stuff alongside your original data, and requiring repr(transparent) would forbid this.
2. The allocator API also requires extra stuff, assuming the allocator is nonempty, and that also messes with things.

Actually, the allocator API runs amok with this current proposal, too, since it would require that an allocator be zero-sized. If we reframe the proposal from the perspective of unsized types (treating smart pointers as implicitly always sized), this "single field" instead turns into "last field," like the current unsized requirements. I believe that zero-sized types are also not allowed to be after the unsized field of a struct, but we could probably amend this pretty easily.

Maybe we could have some kind of repr(variant) for smart pointers and repr(contravariant) for cells and those could be used instead, requiring that either:

1. The last field of these structs is something that is also repr(same-variant) in the same type parameter.
2. The last field of these structs is a value of that type parameter.

Basically ensuring that everything works as you'd expect.

---

A few footnotes:

• I am aware that CoerceUnsized is specifically for references/pointers and that a pure impl<T: Unsize<U>> CoerceUnsized<U> for T does not exist, but I'm going to pretend that it does for the sake of simplicity here.
• Unsized locals are still unstable, so, no one knows what their final form will be. I'm mostly operating on assumptions here.
• DispatchFromDyn is also very specifically limited to dyn dyspatch (unsurprisingly) and other cases, like being able to pass arrays into unsized slice arguments are completely unaccounted for in the compiler at the moment.
• I was right about the "last field" requirement for unsized types; adding zero-sized fields after totally break this. (playground link)

[compiler-errors]

@clarfonthey: A few things regarding your last comment--

It's not correct to call the relationship between [T; N] and [T] subtyping (which I know you're not doing explicitly, but you are kinda implicitly doing so by mentioning variance), and saying that CoerceUnsize and DispatchFromDyn are about variance is a bit misleading.

Their relationship is called unsizing in Rust, which is an explicit coercion operation inserted into the MIR and involves actually changing the type layout of pointer.

Also, Box<T> is covariant in its T parameter, but implements DispatchFromDyn, so I'm not certain what you mean by it enforcing contravariance in its argument.

DispatchFromDyn really is just a trait that instructs typecheck what's valid so that later codegen can peel off layers of a type to look for the "data" part to pass into the vtable method for, e.g., Rc<Self> or Box<Self>; it has slightly different requirements that try to (somewhat generally) capture what needs to happen when codegenning a dynamic call.¹

Also the last field requirement you mention is a limitation of data being Unsized, but not pointers being CoerceUnsized, which are kinda two different operations. That is to say, SmartPointer<T> doesn't need its NonNull<T> field to be its last field, but Struct<[i32; 1]> needs that array located at the end (or as we call in the compiler, the tail) of the struct for it to be unsized. But this is mostly related to the limitation that the only struct field that is not required to be SIzed is the last one.

(disclaimer that i've been too busy to read the actual rfc yet, just wanted to clarify some stuff before people started also responding and a whole thread gets generated based off of false premises)

Footnotes

1. the way that codegen actually works is why DispatchFromDyn for Box and Rc don't support custom allocators -- this could work, but it would need to piece back together the allocator on the receiver end.² ↩

2. though, wow, the codegen for DispatchFromDyn is kind... interesting. I actually wonder why it doesn't just rip off the metadata from the ScalarPair and pass just the "sized" version of the data.... 🤔 ↩

[clarfonthey]

Oh, huh, I was under the impression that DispatchFromDyn effectively had to be able to do more than just remove the metadata from a pointer, since like mentioned the presence of extra data on Arc and Rc effectively mandates this, but I guess that theoretically "could" work?

I just assumed that allocators were kind of in the same boat, where they could be present like the counts for references but would be peeled away somehow, also like those.

In terms of my mention of contravariance: I definitely am jumbling up terms here since generally what people mean by variance is about subtyping and supertyping, whereas I'm using it to mean literally varying the type via the unsize and dynamic dispatch operations you describe. It's a sloppy analogy for sure, but the main reason I used it is to point out that it shouldn't be explicitly derived and instead inferred somehow like variance currently is.

I guess in that sense, it's closer to the mathematical definitions of stuff like homomorphisms, where instead of stating that two things have identical properties, you have the existence of operations which can be applied while retaining certain properties. Subtle differences but similar analogies.

[kennytm]

The last-field limitation actually means nothing ABI-wise since the Rust layout allows reordering the fields

use std::{ptr::NonNull, mem::offset_of};

struct MySmartPointer<T: ?Sized> {
    extra: u16,
    interior: NonNull<T>,
}

fn main() {
    type P = MySmartPointer<usize>;
    assert_eq!(offset_of!(P, interior), 0);
    assert_eq!(offset_of!(P, extra), 8);
}

So, to call fn method(self: MySmartPointer<Self>, ...), the DispatchFromDyn needs to strip the pointer-metadata in the middle of the struct anyway.

MySmartPointer<T>
    +---+---+---+---+---+---+---+---+---+---+
    | interior                      | extra |
    +---+---+---+---+---+---+---+---+---+---+

            |                       ^
            |                       |
        CoerceUnsized       DispatchFromDyn
            |                       |
            v                       |

    +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
    | interior as *const ()         | ptr::metadata(interior)       | extra |
    +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
MySmartPointer<dyn Trait>

However, since CoerceUnsized can perfectly insert the metadata into the middle of struct, I don't see any theoretical reason why the reverse operation in DispatchFromDyn can't be done. IMO the pointer-shape requirement is just an artificial limitation of the current rustc implementation, and once lifted the two traits DispatchFromDyn and CoerceUnsized can be made equivalent.

[programmerjake]

The last-field limitation actually means nothing ABI-wise since the Rust layout allows reordering the fields

the Rust compiler is what chooses what order the fields are in, which also means the Rust compiler can always leave the unsizable field at the end, as I'm guessing it currently does. This means conversion from/to dyn Trait doesn't need to insert/remove the vtable in the middle and move everything after, and that conversion from dyn Trait back to the underlying type can just use a prefix of the struct, which in some cases may mean just changing pointer types.

[Darksonn]

What these traits really are about is ensuring proper variance, which is especially weird since variance is usually only exposed to programmers in Rust through weird mechanisms like PhantomData. Ultimately:

• CoerceUnsized lets you declare a type as covariant, and varying the type this way makes it more general. For example, this lets you vary a mutable reference into an immutable reference or an array into a slice.
• DispatchFromDyn lets you declare a type as contravariant, and varying the type this way makes it more specific. This would be the opposite, like varying a slice into an array or a trait object into a concrete type.

I can see how CoerceUnsized is a form of covariance, though as @compiler-errors mentions, it is not the same subtyping relationship as the one with which we usually use the word "covariance" with in Rust.

But I disagree with calling DispatchFromDyn contravariance. When something is contravariant, you must be able to coerce it to any subtype. When you have a fn(&'short T), you can cast that to fn(&'long T) for any lifetime 'long that is longer than 'short. But in the case of DispatchFromDyn, we can't do that. It's not up to you whether you want to cast your Box<dyn MyTrait> to Box<MyFirstStruct> or Box<MySecondStruct>. Only one of those conversions can be valid, but contravariance means that both must be okay.

[Urhengulas]

Is there a reason to use the fully qualified path of Sized for U? For T the prelude is used.

[kennytm]

@programmerjake

the Rust compiler is what chooses what order the fields are in, which also means the Rust compiler can always leave the unsizable field at the end, as I'm guessing it currently does.

My example code in #3621 (comment) already showed it does not. We are not talking about maybe-unsized struct tail here, but a pointer to a maybe-unsized type which can appear anywhere.

struct X<T: ?Sized> {
    extra: u16,
    tail: T,  // yes, this field is guaranteed to be at the end of X<T>
}

/* layout of X<T>:

    +---------------+
0   | extra         |
    +---------------+
2   |x(padding)x x x|
    | x x x x x x x |
4   |x x x x x x x x|
    | x x x x x x x |
6   |x x x x x x x x|
    +---------------+
8   | tail          |
    | (*actual      |
A   |   offset      |
    |   depends on  |
C   |   align_of T) |
    |               |
    :               :

*/

struct P<T: ?Sized> {
    extra: u16,
    ptr: *const T, // note that it is a pointer here, there is no guarantee about offset of `ptr` in P<T>
}

/* layout of P<T>:

    +---------------+
0   | ptr           |
    |             ~~~~~~~~~~~>  +---------------+
2   |               |           | *ptr          |
    |               |           |               |
4   |               |           :               :
    |               |
6   |               |
    +---------------+
8   | extra         |
    +---------------+
A   |x(padding)x x x|
    | x x x x x x x |
C   |x x x x x x x x|
    | x x x x x x x |
E   |x x x x x x x x|
    +---------------+

*/

This means conversion from/to dyn Trait doesn't need to insert/remove the vtable in the middle and move everything after

Coercion from P<T> to P<dyn Trait> is already inserting a vtable in the middle of the struct (next to the ptr).

[RalfJung]

It's actually worse than a transmute of these values, we're effectively transmuting the function pointer. This means the two types do not just have to be layout compatible, they have to be function call ABI compatible. That's also why allowing extra fields in these types is pretty difficult.

[RalfJung]

I don't think there should be a hard requirement that smart pointers be transparent.

We actually currently rely on them being the equivalent of transparent. The only reason DispatchFromDyn does not just check the repr is that we want it to be implemented on Box, and Box cannot be repr(transparent) because of the allocator -- but DispatchFromDyn is only implemented for Box<_, Global> which is effectively repr(transparent).

The way we call arbitrary-self receiver functions via vtables relies on repr(transparent) -- specifically, it relies on ABI compatibility. There's a subtle invariant here that all instances of repr(Rust) types that follow the repr(transparent) rules are indeed ABI-compatible to their one non-1-ZST field even if the repr(transparent) attribute is not present. I hope all our ABI adjustment logic ensures this invariant (and we have tests covering the basics), but this is easy to get wrong. This would be much easier if we could rely on repr(transparent)...

[clarfonthey]

I don't think there should be a hard requirement that smart pointers be transparent.

We actually currently rely on them being the equivalent of transparent. The only reason DispatchFromDyn does not just check the repr is that we want it to be implemented on Box, and Box cannot be repr(transparent) because of the allocator -- but DispatchFromDyn is only implemented for Box<_, Global> which is effectively repr(transparent).

The way we call arbitrary-self receiver functions via vtables relies on repr(transparent) -- specifically, it relies on ABI compatibility. There's a subtle invariant here that all instances of repr(Rust) types that follow the repr(transparent) rules are indeed ABI-compatible to their one non-1-ZST field even if the repr(transparent) attribute is not present. I hope all our ABI adjustment logic ensures this invariant (and we have tests covering the basics), but this is easy to get wrong. This would be much easier if we could rely on repr(transparent)...

Is there any plan to make this work with allocators at all, or are we basically stuck with this design for the foreseeable future? Since as I mentioned, the current RFC proposal would also be unable to do anything with non-zero-sized allocators for reasons mentioned. I'd guess that kernel development would at best be dealing with ZSTs anyway since storing an extra pointer or something like it would be undesirable for performance reasons, but it's still a question that feels worth asking.

What I'd imagine would happen here is the fields for the allocator would always be after the pointee to ensure that you can still cast the pointer and have it Just Work, but that requires a bit of compiler help.

[RalfJung]

Extending DispatchFromDyn to be able to cover more cases (i.e., to handle types that are not just newtyped pointers) is a non-trivial topic on its own, I'd rather not derail this RFC by going there.

[davidhewitt]

One question I have is about FFI smart pointers, which were one of the key use cases for arbitrary self types.

If I understand correctly, these smart pointers cannot easily be e.g. CppRef<dyn MyTrait> or Py<dyn MyTrait> because the pointee is always a thin pointer from FFI so there is no space for the dyn metadata.

It seems a little awkward to call this derive SmartPointer if an important category of smart pointers cannot implement it.

Is there perhaps a way where the FFI smart pointers could be made compatible with this? By having dyn metadata (manually?) placed in a second field, perhaps?

[Darksonn]

@davidhewitt I think you underestimate the extent to which something like CppRef could support this today. I'm not so familiar with the CppRef type, but the kernel also has a smart pointer for FFI. It's called ARef<T>, where T is some C struct with refcounting built in.

pub struct ARef<T: AlwaysRefCounted> {
    ptr: NonNull<T>,
    _p: PhantomData<T>,
}

pub unsafe trait AlwaysRefCounted {
    fn inc_ref(&self);
    unsafe fn dec_ref(obj: NonNull<Self>);
}

Making this work with ARef<dyn MyTrait> probably requires MyTrait to be a sub-trait of AlwaysRefCounted, so AlwaysRefCounted must be object safe. It isn't right now because of the raw pointer argument, but it could be made object safe if we wrap NonNull in a #[derive(SmartPointer)] struct.

All that said, mixing FFI and dyn Trait isn't something the Linux Kernel needs right now. But I think it would work.

[davidhewitt]

I see, I think you might also be able to do something like trait MyTraitRefCounted: MyTrait + AlwaysRefCounted when it's not possible to modify MyTrait directly.

For Py<T> in PyO3 we don't need a trait on T so we don't suffer the same problem. However the pointer we store is always NonNull<PyObject> hence my wondering about where to fit the metadata.

[programmerjake]

For Py<T> in PyO3 we don't need a trait on T so we don't suffer the same problem. However the pointer we store is always NonNull<PyObject> hence my wondering about where to fit the metadata.

I expect you can just store NonNull<T> but it's really always *mut PyObject in disguise, you'd always cast pointer types when creating or accessing the pointer.

[clarfonthey]

I tried typing this up before on my phone but GitHub ate my message. -_-

Basically translating what I was proposing into a more concrete proposal. The tl;dr is that I think that instead of derive(SmartPointer), this should be repr(smart_pointer). The #[pointee] attribute remains the same. Going over a few of the details:

• With repr, it's clearer that the layout of the type is constrained, whereas with a derive macro this could emit some weird errors. It also lets us rely a bit more heavily on repr(transparent) while technically not actually making the type repr(transparent), allowing for us to change the representation later.
• If we go with repr, the situation on whether CoerceUnsized and DispatchFromDyn should be derived separately goes away. We just have a separate repr for each combination of the two, but for now, we don't need to offer the other two combinations (just CoerceUnsized or just DispatchFromDyn).
• This also means that smart_pointer is a decent name for deriving both, since it seems pretty fair that only smart pointers would have both. It also avoids bikeshedding the name.

This also converts my nebulous and partially wrong analysis into an actionable proposal.

[Darksonn]

Continuing #3621 (comment) by @clarfonthey in a thread.

I'm not convinced about #[repr(smart_pointer)]. It's pretty weird for a repr to implement traits on the type. And like I explained in this new section, I think it is unlikely that we will not end up using traits for this functionality.

I think we can get pretty good errors from the derive macro. The DispatchFromDyn type already has special built-in checks in the compiler, and we can just adjust the errors that it emits to get reasonable errors.

Other than that, did you see commit f7758256? It's not correct that only smart pointers implement both traits.

[clarfonthey]

Note that them existing as traits is mostly just a quirk of the existing implementation. Since they're unstable, the traits might as well not exist.

[Darksonn]

Of course, but I imagine that #[derive(SmartPointer)] will not be the end of evolution of this feature. At some point we will hopefully stabilize the underlying traits as well, and I think it is quite likely that we still want them to be traits when we do so.

[clarfonthey]

Honestly, I'm not quite sure how that would be useful, but I'll ask anyway: what kind of generic code would you expect to be able to use these traits for?

[Darksonn]

It's not so much that I think it would be useful for generic code. Rather, I think it is useful to use traits because it gives us a lot of control over when the special traits are implemented.

Imagine we go for the #[repr(smart_pointer)] solution and also have a #[repr(transparent_container)] for the case of Cell. Then each type has essentially three possible choices:

1. No repr annotation. The type is never usable with dynamic dispatch.
2. The #[repr(smart_pointer)] annotation. The type is always usable with dynamic dispatch.
3. The #[repr(transparent_container)] annotation. The type is usable with dynamic dispatch whenever its contents are.

However, these choices are not sufficiently flexible due to the Pin issue. If we go with the StableDeref solution, which is the most elegant solution, then we need a fourth option of "the type is usable with dynamic dispatch whenever the inner smart pointer implements StableDeref".

Using traits, this is no issue. You just add T: StableDeref as a trait bound to the implementation and that's it.

[clarfonthey]

Ah, I see. From that perspective, you're definitely right that having traits would be helpful long-term, even though it's still mostly a compiler detail. I think this analysis would be good to add to that section of the RFC; I think that deriving would make sense here, but that then leads back to the original issue about the combined deriving.

Of course, I guess that derive macros could ultimately be deprecated anyway if we decide to split them up later. Although it's unprecedented for the standard library, proc_macros can do whatever they want, so, we already see individual crates doing stuff like this.

[Darksonn]

I did try to explain this in the section I added yesterday, but I will elaborate in more details.

[clarfonthey]

(I also hadn't read the section before you mentioned it, so, I'll double check I didn't just miss the details there.)