-
Notifications
You must be signed in to change notification settings - Fork 1.7k
repr(ordered_fields) #3845
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
repr(ordered_fields) #3845
Conversation
|
Not to add too many extra colours to the list, but (Note: those three things should cover every case I've seen that uses Whereas Also, while it may be more technical than most users need to understand, it would be helpful if the RFC reiterated the current issues with |
|
Just as a small point of style the Guide Level Explanation is usually "what would be written in the rust tutorial book", and the Reference Level Explanation is "what could be written into the Rust Reference". This isn't a strict requirement, but personally I'd like to see the Reference Level part written out. Using the present tense, as if the RFC was accepted and implemented. |
add more unresolved questions
Rework struct layout description.
text/3845-repr-ordered-fields.md
Outdated
| # Guide-level explanation | ||
| [guide-level-explanation]: #guide-level-explanation | ||
|
|
||
| `repr(ordered_fields)` is a new representation that can be applied to `struct`, `enum`, and `union` to give them a consistent, cross-platform, and predictable in memory layout. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| `repr(ordered_fields)` is a new representation that can be applied to `struct`, `enum`, and `union` to give them a consistent, cross-platform, and predictable in memory layout. | |
| `repr(ordered_fields)` is a new representation that can be applied to `struct`, `enum`, and `union` to give them a consistent, cross-platform, and predictable in-memory layout. |
"cross-platform" -- the layout will differ when there are different layouts for struct members' types, in particular primitive types can have different alignments which changes the amount of padding.
e.g., #[repr(ordered_fields)] struct S(u8, f64); doesn't have the same layout on x86_64 and i686
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, this will need to be documented as a hazard in the ordered_fields docs. However, the repr itself will be cross-platform. For example, #[repr(ordered_fields)] struct Cross([u8; 3], SomeEnum); will be truly cross-platform (given that SomeEnum is!).
Co-authored-by: Jacob Lifshay <programmerjake@gmail.com>
Just voicing support for |
|
Nominating this so that we can do a preliminary vibe-check on it in a lang triage meeting. |
| Currently `repr(C)` serves two roles | ||
| 1. Provide a consistent, cross-platform, predictable layout for a given type | ||
| 2. Match the target C compiler's struct/union layout algorithm and ABI |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Big fan of doing this split, especially for structs. (It's less obvious what choices to make for other things, IMHO, but at least for structs this is something I've wanted for ages, so that for example Layout::extend can talk about it instead of C.)
Pondering the bikeshed: declaration_order or something could also be used to directly say what you're getting.
(This could be contrasted with other potential reprs that I wouldn't expect this RFC to add, but could consider as future work, like a deterministic_by_size_and_alignment where some restricted set of optimizations are allowed but you can be sure that usize and NonNull<String> can be mixed between different types while still getting the "same" field offsets, for example.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is also useful for unions, so we don't need to rely on repr(C) to ensure that all fields of a union are at offset 0.
This could be contrasted with other potential reprs that I wouldn't expect this RFC to add...
This also works as an argument against names like repr(consistent), since there are multiple consistent and useful repr, making it not descriptive enough.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do think that declaration_order or ordered_fields is a bit weird on a union, because of course they're not really in any "order".
It makes me ponder whether we should just have repr(offset_zero) for unions to be explicit about it, or something.
(Which makes me think of other things like addressing rust-lang/unsafe-code-guidelines#494 by having a different constructs for "bag of maybeuninit stuff that overlap" vs "distinct options with an active-variant rule for enum-but-without-stored-discriminant". But those are definitely not this RFC.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't mind spelling it as repr(offset_zero) for unions if that helps get this RFC accepted 😄. However, I have a sneaking suspicion that this isn't the contentious part of this RFC.
I know the name isn't optimal (intentionally). This can be hashed out after the RFC is accepted (or even give a different name for all of struct, union, and enum).
The most important bit for me is just that we do the split (for all of struct, union, and enum, to be consistent).
|
I've updated how enums's tags are specified, now they just defer to whatever |
text/3845-repr-ordered-fields.md
Outdated
|
|
||
| `repr(C)` in edition <= 2024 is an alias for `repr(ordered_fields)` and in all other editions, it matches the default C compiler for the given target for structs, unions, and field-less enums. Enums with fields will be laid out as if they are a union of structs with the corresponding fields. | ||
|
|
||
| Using `repr(C)` in editions <= 2024 triggers a lint to use `repr(ordered_fields)` as a future compatibility lint with a machine-applicable fix. If you are using `repr(C)` for FFI, then you may silence this lint. If you are using `repr(C)` for anything else, please switch over to `repr(ordered_fields)` so updating to future editions doesn't change the meaning of your code. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is too noisy. Most code out there using repr(C) is probably fine - IIUC, if you're not targeting Windows or AIX, maybe definitely fine? - and having a bunch of allow(...) across a bunch of projects seems unfortunate.
Maybe we can either (a) only enable the lint for migration, i.e., the next edition's cargo fix would add allows for you or (b) we find some new name... C2 for the existing repr(C) usage to avoid allows. But (b) also seems too noisy to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe it could just be an optional edition compatibility lint, so if someone enables e.g. rust_20xx_compatibility it shows up but otherwise not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(a) only enable the lint for migration
That was the intention, hence the name edition_2024_repr_c. I'll make this more clear, that this is intended to be a migration lint.
Rustfix would update to #[repr(ordered_fields)] to preserve the current behavior. For the FFI crates, #![allow(edition_2024_repr_c)] at the top of lib.rs would suffice. If you have a mix of FFI and non-FFI uses of repr(C), then you'll have to do the work to figure out which is which, no matter what option is chosen to update repr(C) - even adding repr(C2), since then the FFI use case would need to update all their reprs to repr(C2).
Overall, I think this scheme only significantly burdens those who have a mix of FFI and non-FFI uses of repr(C). But they were going to be burdened no matter what option was chosen.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the new wording/lints too noisy still?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the new wording is still too noisy. We shouldn't assume that most people using repr(C) are using it for ordering rather than FFI.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That wasn't my intention, but I don't see another way to do all of the following in the next edition:
- make
repr(C)mean - same layout/ABI as what the standard C compiler does - make
repr(ordered_fields)- the same algorithm that's listed forrepr(C)in the Rust reference - ensure that everyone who upgrades to the next edition gets the layout they need (as long as they read the warnings and follow the given advice)
- make it as painless as possible for people who don't mix FFI and stable ordering cases (which I suspect is the vast majority of people). In other words, each crate currently uses
repr(C)either exclusively for FFI or exclusively for some stable layout. - for people who do mix FFI and stable ordering cases in one crate, at least the warning should give them all the places they need to double-check, and they can silence the warning on a case-by-case basis.
I'm open to suggestions on how to handle the diagnostics. Within these constraints, I think my solution is the only real option we have. If there are some objections to these constraints, I would like to hear those too, maybe I missed the mark with these constraints, and missed a potential solution because of it.
Co-authored-by: Ralf Jung <post@ralfj.de>
This post and its neighbors discuss some of that. |
…enkov repr(transparent): do not consider repr(C) types to be 1-ZST Context: rust-lang/unsafe-code-guidelines#552 This experiments with a [suggestion](rust-lang/rfcs#3845 (comment)) by `@RustyYato` to stop considering repr(C) types as 1-ZST for the purpose of repr(transparent). If we go with rust-lang/rfcs#3845 (or another approach for fixing repr(C)), they will anyway not be ZST on all targets any more, so this removes a portability hazard. Furthermore, zero-sized repr(C) structs [may have to be treated](rust-lang/unsafe-code-guidelines#552 (comment)) as non-ZST for the win64 ABI (at least that's what gcc/clang do), so allowing them to be ignored in repr(transparent) types is not entirely coherent. Turns out we already have an FCW for repr(transparent), namely rust-lang#78586. This extends that lint to also check for repr(C).
…enkov repr(transparent): do not consider repr(C) types to be 1-ZST Context: rust-lang/unsafe-code-guidelines#552 This experiments with a [suggestion](rust-lang/rfcs#3845 (comment)) by ``@RustyYato`` to stop considering repr(C) types as 1-ZST for the purpose of repr(transparent). If we go with rust-lang/rfcs#3845 (or another approach for fixing repr(C)), they will anyway not be ZST on all targets any more, so this removes a portability hazard. Furthermore, zero-sized repr(C) structs [may have to be treated](rust-lang/unsafe-code-guidelines#552 (comment)) as non-ZST for the win64 ABI (at least that's what gcc/clang do), so allowing them to be ignored in repr(transparent) types is not entirely coherent. Turns out we already have an FCW for repr(transparent), namely rust-lang#78586. This extends that lint to also check for repr(C).
…enkov repr(transparent): do not consider repr(C) types to be 1-ZST Context: rust-lang/unsafe-code-guidelines#552 This experiments with a [suggestion](rust-lang/rfcs#3845 (comment)) by ```@RustyYato``` to stop considering repr(C) types as 1-ZST for the purpose of repr(transparent). If we go with rust-lang/rfcs#3845 (or another approach for fixing repr(C)), they will anyway not be ZST on all targets any more, so this removes a portability hazard. Furthermore, zero-sized repr(C) structs [may have to be treated](rust-lang/unsafe-code-guidelines#552 (comment)) as non-ZST for the win64 ABI (at least that's what gcc/clang do), so allowing them to be ignored in repr(transparent) types is not entirely coherent. Turns out we already have an FCW for repr(transparent), namely rust-lang#78586. This extends that lint to also check for repr(C).
Rollup merge of #147185 - RalfJung:repr-c-not-zst, r=petrochenkov repr(transparent): do not consider repr(C) types to be 1-ZST Context: rust-lang/unsafe-code-guidelines#552 This experiments with a [suggestion](rust-lang/rfcs#3845 (comment)) by ```@RustyYato``` to stop considering repr(C) types as 1-ZST for the purpose of repr(transparent). If we go with rust-lang/rfcs#3845 (or another approach for fixing repr(C)), they will anyway not be ZST on all targets any more, so this removes a portability hazard. Furthermore, zero-sized repr(C) structs [may have to be treated](rust-lang/unsafe-code-guidelines#552 (comment)) as non-ZST for the win64 ABI (at least that's what gcc/clang do), so allowing them to be ignored in repr(transparent) types is not entirely coherent. Turns out we already have an FCW for repr(transparent), namely #78586. This extends that lint to also check for repr(C).
repr(transparent): do not consider repr(C) types to be 1-ZST Context: rust-lang/unsafe-code-guidelines#552 This experiments with a [suggestion](rust-lang/rfcs#3845 (comment)) by ```@RustyYato``` to stop considering repr(C) types as 1-ZST for the purpose of repr(transparent). If we go with rust-lang/rfcs#3845 (or another approach for fixing repr(C)), they will anyway not be ZST on all targets any more, so this removes a portability hazard. Furthermore, zero-sized repr(C) structs [may have to be treated](rust-lang/unsafe-code-guidelines#552 (comment)) as non-ZST for the win64 ABI (at least that's what gcc/clang do), so allowing them to be ignored in repr(transparent) types is not entirely coherent. Turns out we already have an FCW for repr(transparent), namely rust-lang/rust#78586. This extends that lint to also check for repr(C).
|
@RustyYato So, an chance you could incorporate that kind of motivation into the RFC? Also let us know if you'd like help with editing the RFC. |
text/3845-repr-ordered-fields.md
Outdated
| `FooUnion` has the same layout as `u32`, since `u32` has both the biggest size and alignment. | ||
| ### enum | ||
| The enum's tag type is the same type that is used for `repr(C)` in edition <= 2024, and the discriminants are assigned the same way as `repr(C)` (in edition <= 2024). This means the discriminants are assigned such that each variant without an explicit discriminant is exactly one more than the previous variant in declaration order. | ||
| This does mean that the tag type will be platform-specific. To alleviate this concern, using `repr(ordered_fields)` on an enum without an explicit `repr(uN)`/`repr(iN)` will trigger a warning (name TBD). This warning should suggest the smallest integer type that can hold the discriminant values (preferring signed integers to break ties). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
GCC also provides a compiler flag for changing this size, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see why GCC is relevant to how we spec repr(ordered_fields). And are you referring to -fshort-enums in GCC?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, -fshort-enums. I misread this section as about repr(C).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, makes sense. However I'm not suggesting we change repr(C) in this RFC. Any reference to it is purely to ensure compatibility on existing editions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll reword these sections to prevent that misunderstanding. Thanks for the feedback!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've gone ahead and updated the text, to try to prevent this misunderstanding. Do you think this would have helped you when reading this section?
| Code in case 1 generally falls into one of these buckets: | ||
| * rely on the exact layout being consistent across platforms | ||
| * for example, zero-copy deserialization (see [rkyv](https://crates.io/crates/rkyv)) | ||
| * manually calculating the offsets of fields | ||
| * This is common in code written before the stabilization of `offset_of` (currently only stabilized for `struct`)/`&raw const`/`&raw mut` | ||
| * But sometimes this is still required if you are manually doing type-erasure and handling DSTs (for example, implementing [`erasable::Erasable`](https://docs.rs/erasable/1.3.0/erasable/trait.Erasable.html)) | ||
| * manually calculating the layout for a DST, to prepare an allocation (see [slice-dst](https://crates.io/crates/slice-dst), specifically [here](https://github.com/CAD97/pointer-utils/blob/0fe399f8f7e519959224069360f3900189086683/crates/slice-dst/src/lib.rs#L162-L163)) | ||
| * match layouts of two different types (or even, two different monomorphizations of the same generic type) | ||
| * see [here](https://github.com/rust-lang/rust/pull/68099), where in `alloc` this is done for `Rc` and `Arc` to give a consistent layout for all `T` | ||
|
|
||
| So, providing any fix for case 2 would subtly break any users of case 1. This breakage cannot be checked easily since it affects unsafe code making assumptions about data layouts. Making it difficult to fix within a single edition/existing editions. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@RalfJung I believe this section covers why "just fix repr(C)" is a nonstarter.
I would love any help editing any part of this document 🙇♂️
clarify that we are describing the behavior of repr(ordered_fields) clarify differences with `repr(iN)`
teor2345
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some typo fixes, and an idea for a repr name
Co-authored-by: teor <teor@riseup.net> Co-authored-by: Jacob Lifshay <programmerjake@gmail.com>
Co-authored-by: Jules Bertholet <jules.bertholet@gmail.com>
Add
repr(ordered_fields)and provide a migration path to switch users fromrepr(C)torepr(ordered_fields), then change the meaning ofrepr(C)in the next edition.This RFC is meant to be an MVP, and any extensions (for example, adding more
reprs) are not in scope. This is done to make it as easy as possible to accept this RFC and make progress on the issue ofrepr(C)serving two opposing roles.Rendered
To avoid endless bikeshedding, I'll make a poll if this RFC is accepted with all the potential names for the new
repr. If you have a new name, I'll add it to the list of names in the unresolved questions section, and will include it in the poll.