Skip to content

Improper device setting when using neuron.device=cuda:<id> #63

@ifrit98

Description

@ifrit98

Using --neuron.device cuda:N fails with device placement mismatches.

e.g. When starting with
pm2 start run.sh --name openvalidators_autoupdate -- --wallet.name <your-wallet-name> --wallet.hotkey <your-wallet-hot-key> --neuron.device cuda:1

eventually fails with cuda:0 vs cuda:1 mismatch in forward() See: https://github.com/opentensor/validators/blob/main/openvalidators/forward.py#L297

Gets 1 tensor on cuda:0 and the other on cuda:1 set from the command line.

Suggestion is to simply recommend away from this arg until we find a fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions