feature: implement Relink, Don't Rebuild (RDR) #151265

davidbarsky · 2026-01-17T18:16:20Z

Note: This is not ready for review. I accidentally opened this pull request before it was ready.

Hello! I didn't mean to convert this to a PR (let alone a draft PR), but this branch implements "Relink, Don't Rebuild", which was approved in this major change proposal. The idea behind RDR is to treat changes to non-public items as not a factor in considering whether dependent crates also need to be rebuilt. The trick, of course, is defining what "non-public" items are. This change considers two classes:

Spans, which I've split out into a separate .spans file. By storing spans separately from the main crate metadata, changes to span information (line numbers shifting, reformatting, etc.) no longer affect the crate hash. This approach was suggested by David Lattimore in this Zulip post.
Items, where I've changed the crate hash to only incorporate items reachable from the public API. "Reachable" uses rustc's existing EffectiveVisibilities analysis, which includes pub items, items leaked through public signatures, and reexports. For function bodies specifically, they're only included when they affect downstream compilation: generic functions (which get monomorphized) and #[inline] functions (which get inlined across crate boundaries). Trait implementations are always included since they affect trait resolution. Changes to truly private items, or even non-generic, non-inlinable function bodies, no longer invalidate dependent crates.

This is accomplished by changing how the Strict Version Hash (SVH) is computed. The SVH was originally introduced¹ to identify a crate's "ABI/API/publicly reachable state," but at the time was implemented as a hash of the entire AST, which the original commit notes was "obviously incorrect." This implementation follows through on that original intent and the design outlined in the project goal. Additionally, the hash is resilient to item reordering: since items are sorted by their DefPathHash prior hashing, moving items around in source doesn't change the SVH.

This feature is currently exposed as -Zstable-crate-hash, but it would probably make sense to split this into separate feature flags—one for span separation and one for the public-item-only hashing—to allow for more granular testing and rollout. I didn't do that before opening this as a draft pull request because, well, I accidentally published this pull request.

On my work project, this brought down incremental cargo check times from 4 seconds to:

1.8 seconds without any Cargo scheduler changes.
0.81 seconds with Cargo scheduler changes.

For rust-analyzer, whose compile times have been optimized to a ridiculous degree, RDR reduced cargo check times (after changing a non-public item in hir-ty) from 3 seconds to 1.5 seconds.

AI Disclosure: I used Claude Code (Opus 4.5) and Codex (GPT-5.2-codex) to implement this.

Thanks to @bjorn3 for digging that commit up! ↩

Part of the RDR (Relink, Don't Rebuild) work to separate span data from .rmeta files. This commit converts Span fields to SpanRef in two areas: 1. ExpnData (hygiene.rs): Convert call_site and def_site from Span to SpanRef. The constructors convert incoming Spans internally, keeping the API stable. 2. rmeta tables (mod.rs, encoder.rs, decoder.rs, cstore_impl.rs): - Predicate tables: (Clause/PolyTraitRef, Span) → SpanRef - Direct span tables: def_span, def_ident_span, proc_macro_quoted_spans - Type-span table: assumed_wf_types_for_rpitit - Added ProcessQueryValue impls and macro variants for SpanRef decoding

rustbot · 2026-01-17T18:16:23Z

rustc_error_messages was changed

cc @davidtwco, @TaKO8Ki

Some changes occurred in compiler/rustc_codegen_cranelift

cc @bjorn3

rustbot · 2026-01-17T18:16:25Z

r? @nnethercote

rustbot has assigned @nnethercote.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

bjorn3 · 2026-01-17T18:30:41Z

compiler/rustc_incremental/src/persist/work_product.rs

+///
+/// The `spans_hash` parameter stores a hash of the spans at the time this work product was
+/// created. For metadata work products compiled with `-Z separate-spans`, this hash is used
+/// to determine whether diagnostics should be replayed when reusing the metadata.


Why specifically metadata work products? Codegen can also result in diagnostics. And why does reusing the metadata have any effect on diagnostics?

Sorry, I accidentally published this as a pull request. this is not yet ready for review!

Add IdentRef type that stores identifiers with SpanRef instead of Span, preserving SyntaxContext for hygiene while avoiding absolute byte positions in metadata. Changes: - rustc_span/symbol.rs: New IdentRef struct with name/span fields, bidirectional conversion methods (to_ident_ref/ident), and proper PartialEq/Hash using SyntaxContext - rustc_span/lib.rs: Re-export IdentRef - rmeta: Change fn_arg_idents table from LazyArray<Option<Ident>> to LazyArray<Option<IdentRef>>, with encoding/decoding support

Update diagnostic types to serialize Span fields as SpanRef to avoid storing absolute byte positions in metadata. Unlike other span-containing types which store SpanRef internally, diagnostic types keep Span internally for performance (diagnostics are created/modified frequently; serialization is rare). Custom Encodable/Decodable impls convert at serialization boundaries: - MultiSpan: primary_spans and span_labels - SubstitutionPart: span field - TrimmedSubstitutionPart: original_span and span - DiagInner: sort_span field

When compiling with -Zseparate-spans, the source_map is now written to a separate .spans file instead of being embedded in .rmeta. This allows .rmeta to remain stable when only span positions change (e.g., adding whitespace or comments), enabling Relink, Don't Rebuild (RDR). Changes: - SpanFileRoot: new struct with source_map field for .spans files - CrateRoot: conditionally excludes source_map when separate_spans enabled - Encoder: writes source_map to .spans file with required_source_files - Decoder: loads source_map from .spans file when present - locator.rs: get_span_metadata_section() to load .spans files - CrateMetadata: stores span_blob and span_source_map for decoding

Load .spans files on-demand during span resolution instead of eagerly when crates are loaded. This avoids I/O overhead for crates where spans are never actually resolved. Changes: - CrateMetadata: stores span_file_path instead of loaded blob - SpanFileData: new struct for lazily-loaded span blob and source_map - span_file_data: OnceLock for lazy initialization - CrateMetadata::span_file_data(): loads span file on first access - creader.rs: only checks if span file exists, does not load it

Handle missing .spans files gracefully by returning dummy spans with preserved SyntaxContext, allowing diagnostics to proceed without precise location info. Changes: - ImportedSourceFileState enum: Loaded/Unavailable variants to track three states (not yet loaded, loaded, unavailable) - imported_source_file(): returns Option<ImportedSourceFile> - Returns DUMMY_SP with preserved SyntaxContext when source files are unavailable - Caches unavailable state to avoid repeated load attempts

Replace the derived HashStable_Generic on SpanRef with a custom impl that respects hash_spans(). When hash_spans() returns false (e.g., with -Zincremental-ignore-spans), SpanRef contributes nothing to the hash. This is essential for RDR where span-only changes should not trigger downstream rebuilds. The implementation mirrors Span's HashStable behavior: - Early return when !ctx.hash_spans() - When hashing, include discriminant tag and all fields - SyntaxContext is hashed for hygiene correctness Also adds -Zseparate-spans flag and comprehensive incremental tests: - rdr_add_println_private: adding println to private fn - rdr_add_private_fn: adding a private function - rdr_metadata_spans: metadata span stability - rdr_private_span_changes: private span changes - rdr_separate_spans: separate span file handling - rdr_span_hash_cross_crate: cross-crate span hashing

Add the core infrastructure for RDR span resolution across crates: SpanBlob/SpanBlobDecodeContext: Wrapper for span file data with header validation and root decoding, parallel to BlobDecodeContext. CrateMetadata additions: - span_file_data(): lazily loads and caches span file - load_span_file(): loads span file and validates against rmeta hash TyCtxtSpanResolver: Implements SpanResolver trait - resolve_span_ref(): converts SpanRef to Span via source file lookups - span_ref_from_span(): converts Span to SpanRef with file-relative offsets find_and_import_source_file_by_stable_id(): Searches external crates for source files needed to resolve spans. Also includes tests for include_str!/include_bytes! handling with separate spans.

Add safety mechanism to prevent diagnostics from being emitted after codegen begins, when span files may be unavailable for relinking. Changes: - Session::diagnostics_allowed(): tracks whether diagnostics can be emitted - Session::enter_codegen_phase(): marks transition to codegen - Dep graph integration: force diagnostics before codegen starts - Codegen backends: call enter_codegen_phase() at appropriate points - SpanRef resolution: check diagnostics_allowed() before resolving - Work product handling: track span file dependencies Run-make tests: - rdr-diagnostic-gating: verify gating works correctly - rdr-hygiene-hash-collision: test hygiene context handling - rdr-missing-spans-fallback: test graceful degradation

Implement "Relink, Don't Rebuild" (RDR) by modifying the Strict Version Hash (SVH) computation to only include publicly-visible API elements, rather than the entire HIR. Previously, any change to a crate (including private function bodies) would change the SVH, causing all downstream crates to rebuild. Now, the SVH only changes when the public API changes. The new `public_api_hash` query hashes: - Public item signatures (types, bounds, generics, predicates) - Inlinable function bodies (#[inline] and generic functions) - Trait implementations Private function bodies are excluded from the hash, so modifying them no longer triggers downstream rebuilds. Internal incremental compilation continues to work correctly because it uses the query system's fine-grained dependency tracking, which is separate from the SVH.

Add run-make tests to verify RDR works correctly with: 1. Panic locations (rdr-panic-location): - Verifies panic messages show correct file:line:col - Tests public functions, private functions, and format strings - Ensures span resolution works for panic messages 2. Debug info (rdr-debuginfo): - Verifies DWARF debug info contains correct source references - Tests with -Cdebuginfo=2 and -Zseparate-spans - Checks reproducibility of debug line tables 3. Coverage instrumentation (rdr-coverage): - Verifies coverage profiling works with -Zseparate-spans - Tests generation of profraw data - Exercises various control flow patterns (branches, loops)

The RDR changes now correctly hash syntax context even with -Zincremental-ignore-spans. This is a correctness fix: hygiene information must always be considered to avoid false cache hits when macro expansion contexts differ. For change_return_impl_trait, the trait bounds (Clone vs Copy) are semantic differences that should invalidate typeck regardless of span handling settings. The previous test expectation was incorrect - it relied on the old behavior where spans contributed nothing to the hash with -Zincremental-ignore-spans.

This allows using RDR without `-Zincremental-ignore-spans`.

- Fix decoder to use span_file_data source map for -Zstable-crate-hash crates - Add preload_all_source_files for deterministic BytePos ordering - Enable -Zstable-crate-hash for library builds and copy .spans to sysroot - Encode fn_arg_idents with SpanRef - Add transform_span_ref_for_export to convert local stable_ids to exported stable_ids when encoding predicate queries to metadata Refactor span conversion into SpanConversionState and add lazy_array_with and lazy_array_with_mut helpers that scope borrows per-item. This avoids intermediate Vec allocations and HashMap cloning when encoding arrays that need span transformation, while avoiding borrow conflicts with encode(&mut self).

rust-log-analyzer · 2026-01-19T01:05:01Z

The job pr-check-2 failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

REPOSITORY                                   TAG       IMAGE ID       CREATED       SIZE
ghcr.io/dependabot/dependabot-updater-core   latest    63ba4a2ff346   10 days ago   775MB
=> Removing docker images...
Deleted Images:
untagged: ghcr.io/dependabot/dependabot-updater-core:latest
untagged: ghcr.io/dependabot/dependabot-updater-core@sha256:830c840ea4b8c27b6919bdd47c017030adeb538f226e1cdfd386490a622b9218
deleted: sha256:63ba4a2ff3467a98ac6c8610fece2cdc9893e2af6c18111e57618f8949749b82
deleted: sha256:77617fa265a7311b1c0502e01ff6157d25228d91f6cddc21d7901eb440d0adee
deleted: sha256:0bd8abdbb1f7c0453b4bc76cce6ecf0a3b9e4b5d0c8ceeb63652e91a63cab4a1
deleted: sha256:4e9913397e051a70593edc1662de159778ce1bf65132295639da0e119913b53f
deleted: sha256:3e54edb2fef4b1c1393a2928bfa1017c22cc578f5f3eef677580662e8d2e79ea
---
error[E0308]: mismatched types
   --> src/tools/clippy/clippy_lints/src/implied_bounds_in_impls.rs:249:21
    |
249 |                     predicates,
    |                     ^^^^^^^^^^ expected `&[(Clause<'_>, Span)]`, found `&[(Clause<'_>, SpanRef)]`
    |
    = note: expected reference `&[(rustc_middle::ty::Clause<'_>, rustc_span::Span)]`
               found reference `&[(rustc_middle::ty::Clause<'_>, SpanRef)]`

[RUSTC-TIMING] pulldown_cmark test:false 0.926
    Checking filetime v0.2.26
[RUSTC-TIMING] filetime test:false 0.087
[RUSTC-TIMING] askama_derive test:false 3.838

rust-bors · 2026-01-21T19:29:36Z

☔ The latest upstream changes (presumably #151459) made this pull request unmergeable. Please resolve the merge conflicts.

jackh726 · 2026-01-21T21:45:35Z

Note: This is not ready for review. I accidentally opened this pull request before it was ready.

Hello! I didn't mean to convert this to a PR (let alone a draft PR)

Given this, I'm just going to go ahead and unassign Nick from this.

David, this is neat :) When you're more ready with this, would be good to split this into separate PRs to manage the review burden. It's a big PR as-is.

rustbot assigned nnethercote Jan 17, 2026

davidbarsky marked this pull request as draft January 17, 2026 18:16

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 17, 2026

davidbarsky changed the title ~~Push ukyuqwxxwmxr~~ feature: implement Relink, Don't Build (RDR) Jan 17, 2026

This comment has been minimized.

Sign in to view

bjorn3 reviewed Jan 17, 2026

View reviewed changes

davidbarsky force-pushed the push-ukyuqwxxwmxr branch 2 times, most recently from 6836b37 to f445e3d Compare January 18, 2026 00:56

This comment has been minimized.

Sign in to view

Veykril changed the title ~~feature: implement Relink, Don't Build (RDR)~~ feature: implement Relink, Don't Rebuild (RDR) Jan 18, 2026

davidbarsky added 10 commits January 18, 2026 13:59

davidbarsky added 4 commits January 18, 2026 14:00

rdr: consolidate span loading

b7bad85

rdr: add a bunch of tests exercising rdr

7d02a56

rdr: rename -Zseparate-spans to -Zstable-crate-hash

f0233d8

rdr: add a test that shows adding a private item doesn't cause rebuilds

20f7d4f

davidbarsky force-pushed the push-ukyuqwxxwmxr branch from f445e3d to acc634f Compare January 18, 2026 22:14

rustbot added the T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) label Jan 18, 2026

This comment has been minimized.

Sign in to view

davidbarsky added 2 commits January 18, 2026 14:42

rdr: migrate some queries from Span to SpanRef

770d5fe

This allows using RDR without `-Zincremental-ignore-spans`.

davidbarsky force-pushed the push-ukyuqwxxwmxr branch from acc634f to 69ccd5a Compare January 18, 2026 22:44

This comment has been minimized.

Sign in to view

rdr: add a test that exercises RDR with -Zshare-generics

8c20bd8

jackh726 unassigned nnethercote Jan 21, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feature: implement Relink, Don't Rebuild (RDR) #151265

feature: implement Relink, Don't Rebuild (RDR) #151265

Uh oh!

davidbarsky commented Jan 17, 2026 •

edited

Loading

Uh oh!

rustbot commented Jan 17, 2026

Uh oh!

rustbot commented Jan 17, 2026

Uh oh!

This comment has been minimized.

bjorn3 Jan 17, 2026

Uh oh!

davidbarsky Jan 17, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

rust-log-analyzer commented Jan 19, 2026

Uh oh!

rust-bors bot commented Jan 21, 2026

Uh oh!

jackh726 commented Jan 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Uh oh!

feature: implement Relink, Don't Rebuild (RDR) #151265

Are you sure you want to change the base?

feature: implement Relink, Don't Rebuild (RDR) #151265

Uh oh!

Conversation

davidbarsky commented Jan 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Footnotes

Uh oh!

rustbot commented Jan 17, 2026

Uh oh!

rustbot commented Jan 17, 2026

Uh oh!

This comment has been minimized.

bjorn3 Jan 17, 2026

Choose a reason for hiding this comment

Uh oh!

davidbarsky Jan 17, 2026

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

rust-log-analyzer commented Jan 19, 2026

Uh oh!

rust-bors bot commented Jan 21, 2026

Uh oh!

jackh726 commented Jan 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

davidbarsky commented Jan 17, 2026 •

edited

Loading