How to use negative samples?

[sunjian2015]

The paper mentions that three negative samples are generated during the training process, which one to use when calculating loss? How is it determined?

[madeline-past]

When calculating the loss, all three negative samples are considered during the training process, but we do not select just one for the loss calculation. Instead, the loss is computed by comparing the positive sample (win) with all three negative samples (lose) , with each negative sample being compared against the positive sample.

[sunjian2015]

When calculating the loss, all three negative samples are considered during the training process, but we do not select just one for the loss calculation. Instead, the loss is computed by comparing the positive sample (win) with all three negative samples (lose) , with each negative sample being compared against the positive sample.

Thanks for your reply, and there is another question that I don’t understand, why convert the v-predict result to x0 when caculate loss?

[madeline-past]

1. The win/lose labels come from human judgment on the decoded videos – i.e., how good the underlying clean latent (x_0) is.
   If we directly compare the model’s output in v-space, the error depends on the particular parameterization, whereas comparing in the “denoised latent” space is closer to “how close is this sample to the target video.”

2. For a fixed clean latent (x_0), the v-prediction at different timesteps (t) is a different linear combination of (x_0) and noise. If we define the loss directly on v, the same visual error can have very different magnitudes at different timesteps.
   By converting back to a denoised quantity (x₀-space), we make the loss comparable across timesteps, which is important when we sample many different (t).

[sunjian2015]

1. The win/lose labels come from human judgment on the decoded videos – i.e., how good the underlying clean latent (x_0) is.
   If we directly compare the model’s output in v-space, the error depends on the particular parameterization, whereas comparing in the “denoised latent” space is closer to “how close is this sample to the target video.”
2. For a fixed clean latent (x_0), the v-prediction at different timesteps (t) is a different linear combination of (x_0) and noise. If we define the loss directly on v, the same visual error can have very different magnitudes at different timesteps.
   By converting back to a denoised quantity (x₀-space), we make the loss comparable across timesteps, which is important when we sample many different (t).

Thanks, and another small question, do you use cfg when getting negative latents?

[madeline-past]

We don’t use CFG when collecting the negative latents.

[Hzj199]

When calculating the loss, all three negative samples are considered during the training process, but we do not select just one for the loss calculation. Instead, the loss is computed by comparing the positive sample (win) with all three negative samples (lose) , with each negative sample being compared against the positive sample.

Thanks for your reply, and there is another question that I don’t understand, why convert the v-predict result to x0 when caculate loss?

When calculating the loss, all three negative samples are considered during the training process, but we do not select just one for the loss calculation. Instead, the loss is computed by comparing the positive sample (win) with all three negative samples (lose) , with each negative sample being compared against the positive sample.

Thanks for your reply, and there is another question that I don’t understand, why convert the v-predict result to x0 when caculate loss?

Where in the code can I see the v-prediction being converted to x0?
Looking at the code in real_dpo_trainer.py lines 270-281, it seems that the loss is being calculated directly using
v-prediction.