fix: grad norm check for automodel gpt oss nightly

[hemildesai]

fixes the end grad norm check after #1693

Summary by CodeRabbit

• Tests
  • Updated metric validation thresholds in model training test suite to reflect revised performance expectations.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

📝 Walkthrough

Walkthrough

Test configuration file updated to modify gradient norm metric validation. The check for train/grad_norm at step 50 transitions from a single upper bound (< 2.5) to a bounded range (10.0 ≤ value ≤ 17.5), altering the acceptance criteria for model training validation.

Changes

Cohort / File(s)	Summary
Test metric validation tests/test_suites/llm/sft-gpt-oss-20b-1n8g-fsdp8ep8-automodel.sh	Modified train/grad_norm check at step 50 from single upper bound (< 2.5) to range check (10.0 ≤ value ≤ 17.5). Adds lower bound and raises upper bound threshold.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Suggested labels

CI:L1, Run CICD

Suggested reviewers

• terrykong
• yuki-97

Pre-merge checks and finishing touches

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix: grad norm check for automodel gpt oss nightly' directly and specifically describes the main change—adjusting the grad norm check for the automodel GPT OSS nightly test.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Test Results For Major Changes	✅ Passed	PR contains minor test metric threshold adjustments for grad_norm validation, directly supporting prior PR #1693 changes. No major features or breaking changes present.

✨ Finishing touches

• 📝 Generate docstrings

---

📜 Recent review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c8d6569 and 7795224.

📒 Files selected for processing (1)

• tests/test_suites/llm/sft-gpt-oss-20b-1n8g-fsdp8ep8-automodel.sh

🧰 Additional context used

📓 Path-based instructions (4)

**/*.sh

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

**/*.sh: Use uv run instead of python to execute scripts
Follow the Google Shell Style Guide for shell scripts

Files:

• tests/test_suites/llm/sft-gpt-oss-20b-1n8g-fsdp8ep8-automodel.sh

tests/test_suites/**/*.sh

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

tests/test_suites/**/*.sh: When adding support for a new model, create a corresponding driver shell script under tests/test_suites/ in the matching domain
Driver shell scripts should match the YAML base name with .sh extension and invoke training entrypoint with uv run

Files:

• tests/test_suites/llm/sft-gpt-oss-20b-1n8g-fsdp8ep8-automodel.sh

!(**/tests/**|**/test_*.py|**/test_*.sh)

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

Add the NVIDIA copyright header to all Python files and shell scripts (excluding tests). The header should include the current year

Files:

• tests/test_suites/llm/sft-gpt-oss-20b-1n8g-fsdp8ep8-automodel.sh

**/*.{py,sh}

📄 CodeRabbit inference engine (CODING_GUIDELINES.md)

The NVIDIA copyright header should appear at the top of all Python files and shell scripts (excluding tests)

Files:

• tests/test_suites/llm/sft-gpt-oss-20b-1n8g-fsdp8ep8-automodel.sh

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)

• GitHub Check: Lint check
• GitHub Check: Post submodule check comment / Comment on PR
• GitHub Check: Post automodel integration comment / Comment on PR

🔇 Additional comments (1)

tests/test_suites/llm/sft-gpt-oss-20b-1n8g-fsdp8ep8-automodel.sh (1)

39-40: Ensure PR description documents empirical justification for gradient norm bounds change.

The gradient norm bounds change (from < 2.5 to [10.0, 17.5]) represents a significant shift in expected training behavior. Per the learnings on convergence-impacting changes, the PR description should include evidence demonstrating that this adjustment is empirically grounded and does not introduce regressions.

Confirm that the PR description includes:

1. Actual gradient norm values observed during training runs post-PR fix: grad norm calculation for dtensor v2 #1693
2. Justification for the specific bounds [10.0, 17.5]
3. Verification that this change does not regress convergence or final metrics

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}