Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support slice DSTs and metadata fix-up in transmute_ref! et al #1817

Open
joshlf opened this issue Oct 4, 2024 · 0 comments
Open

Support slice DSTs and metadata fix-up in transmute_ref! et al #1817

joshlf opened this issue Oct 4, 2024 · 0 comments

Comments

@joshlf
Copy link
Member

joshlf commented Oct 4, 2024

This design is prototyped in #1924. An older, more limited design is prototyped in transmute-slice-experiment.

Outline

  • Expand transmute_ref! et al to support &Src -> &Dst where Src and Dst are both?Sized + KnownLayout and their KnownLayout::LAYOUT.size_infos are identical
  • Same as above, but where LAYOUT.size_infos are not identical, but support a runtime metadata fix-up operation

Design

Currently, transmute_ref! validates:

  • Src and Dst have the same size
  • Src has alignment at least as large as Dst

However, the more general condition is that Src and Dst have the same size as defined by KnownLayout::LAYOUT.size_info. Since these macros already only support being called in a const context, we can check this condition via PME without creating a footgun (namely, errors will always be surfaced immediately at the usage site).

It should be trivial to enforce this via a const fn, although it will require Src: KnownLayout and Dst: KnownLayout bounds, meaning it won't work on our MSRV, so we'll have to gate it on Rust toolchain version.

We may also be able to use the same technique to detect when LAYOUT.size_infos are not identical, but when reference transmutes can be performed infallibly via a metadata fix-up operation. For example, &[u16] -> &[u8] is possible by doubling the slice length.

Old design

Here is an old design that only supported slices, not arbitrary slice DSTs

For slices, when transmuting [Src] into [Dst], the conditions are identical:

  • The fat pointer cast from [Src] to [Dst] preserves length if Src and Dst have the same size
  • [T] has the same alignment as T

If we could figure out a way to get transmute_ref! to infer Src and Dst when those are the transmuted types and Src and Dst when the transmuted types are [Src] and [Dst], then we could get it to support both sized and slice transmutations.

As a bonus, we could also support size_of::<Src>() being a multiple of size_of::<Dst>(), although this would require runtime code to update the slice metadata.

Design

We could do this by introducing a trait like the following:

trait TransmuteRefHelper {
    type Elem;
}

impl<T: Sized> TransmuteRefHelper for T {
    type Elem = T;
}

impl<T: Sized> TransmuteRefHelper for [T] {
    type Elem = T;
}

We'd need to figure out some way of coaxing the correct inference. Currently we do the following:

zerocopy/src/macros.rs

Lines 214 to 217 in d204727

// `t` is inferred to have type `T` because it's assigned to `e` (of
// type `&T`) as `&t`.
let mut t = loop {};
e = &t;

Instead we need to make this more generic. Naively, we could do:

fn from_elem<T: TransmuteRefHelper>(e: T::Elem) -> &'static T

Then we could modify the code in transmute_ref! something like:

// Infers `t` as either equal to the sized source value or
// the element type of the source slice.
let mut t = loop {}; 
e = from_elem(t);

This works, but it means we can't support const fn on our MSRV, which doesn't permit trait bounds in const fn. If we can figure out another way to accomplish the same inference, then we could avoid breaking const support.

joshlf added a commit that referenced this issue Oct 16, 2024
TODO:
- Make it so that `Sized` bounds are still enforced on older toolchains
  that don't support slice DST transmutes
- Write safety comments
- Get rid of now-unused macros and functions in util::macro_util

Makes progress on #1817

gherrit-pr-id: Ib4bc62202e0b3b09d155333b525087f7aa8f02c2
@joshlf joshlf changed the title Support slices in transmute_ref! et al Support slice DSTs and metadata fix-up in transmute_ref! et al Oct 16, 2024
joshlf added a commit that referenced this issue Oct 16, 2024
TODO:
- Figure out how to make this backwards-compatible (since current macros
  work with `T: Sized`, but `T: Sized` does not imply `T: KnownLayout`)
- Make it so that `Sized` bounds are still enforced on older toolchains
  that don't support slice DST transmutes
- Write safety comments
- Get rid of now-unused macros and functions in util::macro_util

Makes progress on #1817

gherrit-pr-id: Ib4bc62202e0b3b09d155333b525087f7aa8f02c2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant