Skip to content

Warn of process over-allocation #105

@phargogh

Description

@phargogh

I have a case right now where I'm running taskgraph graph on a slurm cluster, which requires that I request the number of CPU cores that I think I'm going to need.

Looking at glances or top while I'm running the graph, I can see that the most tasks that I'm actually executing in parallel are about 5 at a time. It would be really great if taskgraph could warn be about egregious over-allocation of workers, like if only 2 or 3 processes are ever used at a time but n_workers was set to 20 or something, then we could see a warning in the logs.

The specific case I'm encountering is that there are 20 CPUs allocated, but the most I'm ever using (as far as I can tell) is 12.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions