feat: Add Task Groups and Tasks-Include from External Toml

[garylvov]

Description

Hi Pixi devs,

I took an initial crack at some functionality I'd love to see in Pixi based on our discussion at #4971 (thank you for the great feedback!) , while also doing a small sidequest to try to take care of #5276 .

This PR adds support for specifying task groups, associated CLI stuff to make it convenient, and being able to include external task tomls to clean up the main pixi.toml

Please feel free to close this PR ( I promise it won't hurt my feelings and I'll keep using Pixi ;) ), as there are two pretty solid reasons to do so:

• I don't know Rust well, and Claude did all of the implementation, I only guided high level
• If the maintenance and/or review burden of this new functionality is too much to be worth it for the feature, that's totally understandable. I dislike hard to maintain code and large PRs in my own codebase so I totally get it

That being said, I'd be happy to make any adjustments to this PR if we decide that it's worth it!

There is an overview of the proposed functionality in advanced_tasks/#task-groups and advanced_tasks/#including-tasks-from-external-files

Here are some local tests I did:

Fixes #4971, #5276

How Has This Been Tested?

I tested locally with the build release version, added some small Python/Rust tests, and ran the test suite locally

AI Disclosure

• This PR contains AI-generated content.
  • I have tested any AI-generated content in my PR.
  • I take responsibility for any AI-generated content in my PR.

Tools: Opus 4.5 within Claude Code 2.1.9

Checklist:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added sufficient tests to cover my changes.
• I have verified that changes that would impact the JSON schema have been made in schema/model.py.

[baszalmstra]

Thats really cool, thanks for taking the time to make this!

Ill leave reviewing this to @ruben-arts who is currently on a skiing trip, he will be back next week.

Maybe it would be good to split the two functionalities into two separate PRs so we can review them more easily?

[garylvov]

Sounds great! I can split it, but also I think crates/pixi_cli/src/run.rs is the only file changed related to #5276 , in which the diff is <30 lines, while also depending on this PR. Let me know if you still want me to split it!

[baszalmstra]

If possible it would be nice to split the functionality in:

• task groups
• include additional task files

I feel the first should be easy for us to merge while the second might need some design.

Feel free to add the fix to #5276 in either of those.

[garylvov]

I will try to split up task groups and the additional task files soon thank you!

[bollwyvl]

Woo! This has the potential to be really good for large projects... or (if tasks
were packageable), very small projects that pack a lot of punch without much TOML.

I have verified that changes that would impact the JSON schema have been made in schema/model.py.

Definitely applicable here, and would argue this new file format needs its own,
dedicated $schema (though maybe #1089 would be at the front of the queue).

I might even suggest a way to offload more of the metadata to the file:

# foo.tasks.toml
"$schema" = "https://pixi.sh/v0.XX.X/schema/tasks/schema.json"

name = "foo"
version = "0.1.0"           # while we're here
description = """yep"""

[tasks.foo]
# all the tasks, but directly copy-pasta-able from `pixi.toml`

Then perhaps thinking about the interplay with feature:

# pixi.toml
[workspace]
task-includes = ["foo.tasks.toml"]

# equivalent to:
# [feature.default]
# task-includes = ["foo.tasks.toml"]

# some craazier stuff
[feature.bar]
task-includes = [
    { file = "./bar.tasks.toml", name = "something-other-than-bar-for-some-reason" }
]

As for group = "whatever" sprinkled all over: spitballing another approach
like [environments], semi-incompatbile with the above:

# pixi.toml
[task-groups]
build = ["build"]   # a list of tasks declared in this file
foo = "./foo.tasks.toml"    # a single path

# a fat group
[task-groups.bar]
title = "Bar"
description = "..."
tasks = ["bar-in-this-file", { file = "bar.tasks.toml" }] # mixed

Of course, what I'd want to do is package tasks so they can be reused in a
non-copy-pasta-tastic way:

# pixi.toml
[task-groups]
bar = "./.pixi/envs/tasks/share/pixi-tasks/bar/bar.tasks.toml"

[feature.tasks.dependencies]
pixi-tasks-bar = "==1.2.3"

[enviroments]
tasks = ["tasks"]

That might be worth its own typographic (and potentially file layout) convention.

# pixi.toml
[task-groups]
bar = { env="tasks", file = "bar/bar.tasks.toml" }

Having a "well-known" path could be most excellent:

$> pixi task-group discover --environment tasks

Available task groups in the `tasks` environment:

Group           bar
Title           Bar         
File            bar/bar.tasks.toml
Description     Some Bar Tasks
Tasks           bar0, bar1, bar2


Task       Args           Description
=========  =============  ===================================
bar0       arg0, arg1     A nicely described task
#....

[garylvov]

I'm closing this now that we have #5318 (broke out groups only) for the sake of cleanliness. I'm still down to keep the conversation about external TOMLs going here thank you for your insight @bollwyvl ! I still really want the external toml functionality as my current research project manifest is approaching 2k lines. I will try to PR a prototype soon for the external TOMLs separately, I'll open it in my fork, and PR it here once groups only is merged. I think having a single unified tasks-groups that can do the include makes a lot of sense.