Skip to content

intuition behind trainable weights and bias for losses #25

@DikshaMeghwal

Description

@DikshaMeghwal

Hey
I was wondering if you could shed some light on why did you add learnable weights and biases to the sync and identity losses? To me it seemed like you were possibly trying to scale and shift but I don't understand why the model doesn't train without it.
Also if you put learnable weights and biases, whats stopping the weight to be 0 and making loss 0?
I am using voxceleb dataset for training.

Below are the loss curves for when I removed the weights. The sync loss seems to be stagnant while the identity loss is increasing.
Screenshot 2024-04-15 at 12 46 35 PM
Screenshot 2024-04-15 at 12 46 51 PM

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions