Skip to content

Conversation

@taiki-e
Copy link
Owner

@taiki-e taiki-e commented Jul 18, 2024

Originally proposed by @orlp. https://discord.com/channels/273534239310479360/273541522815713281/1263250421653438625

        --random <NUM_SAMPLES>
            Performs with random feature combinations up to the number specified per crate.

            This flag can only be used together with --feature-powerset flag.

There are a few things missing as I have just roughly implemented the idea:

  • Resolve todos in code comments:

    cargo-hack/src/features.rs

    Lines 232 to 236 in d6b3164

    // TODO:
    // - If duplicates are found, they should be de-duplicated and regenerated.
    // - Same for filtered case.
    // - If the total number of possible combinations is less than num_samples,
    // then we should use normal powerset().
  • Add a seeding option (Add --random option #255 (comment))

Example:

$ cargo hack check --feature-powerset --no-dev-deps --optional-deps --random 10
info: --no-dev-deps modifies real `Cargo.toml` while cargo-hack is running and restores it when finished
info: running `cargo check --no-default-features --features b,c,default,member1` on real (1/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.04s

info: running `cargo check --no-default-features --features c,default` on real (2/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.00s

info: running `cargo check --no-default-features --features a,b,default,member1` on real (3/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.03s

info: running `cargo check --no-default-features --features c,default` on real (4/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.00s

info: running `cargo check --no-default-features --features default,member1` on real (5/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.03s

info: running `cargo check --no-default-features --features a,b` on real (6/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.00s

info: running `cargo check --no-default-features --features c` on real (7/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.03s

info: running `cargo check --no-default-features --features a,b,c,member1` on real (8/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.03s

info: running `cargo check --no-default-features --features a,c,default` on real (9/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.00s

info: running `cargo check --no-default-features --features a,b,default` on real (10/10)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.00s

@taiki-e taiki-e added the A-features Area: features (--feature-powerset, --each-feature, etc.) label Jul 18, 2024
@elichai
Copy link

elichai commented Jun 4, 2025

I might be misunderstanding this feature, or missing something about the code changes,
but why not just call fastrand::shuffle on the Vec<Vec<Feature>> from fn powerset and then take the first num_samples?

@taiki-e
Copy link
Owner Author

taiki-e commented Jun 4, 2025

The way to calculate the powerset first and then select specific combinations is fine if the number of feature combinations is small, but as the number of combinations increases, performance and memory usage issues arise, eventually leading to OOM.

@orlp
Copy link

orlp commented Jun 4, 2025

Can there be a seeding option?

That is, --random N without specifying a seed picks a random seed and proceeds. Should an error occur it will report the error as well as the seed used (for later reproduction).

Then --random N --random-seed S can be used to reproduce that error and/or do deterministic runs if desired.

@taiki-e
Copy link
Owner Author

taiki-e commented Jun 7, 2025

I hadn't thought about that, but indeed supporting seeding seems to be very useful.

Added it to the todo list in this PR's description.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-features Area: features (--feature-powerset, --each-feature, etc.)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants