-
Notifications
You must be signed in to change notification settings - Fork 209
fix: Add debug parameter to reduce verbose output #1664
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
fix: Add debug parameter to reduce verbose output #1664
Conversation
📝 WalkthroughWalkthroughA debug parameter is added to Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
nemo_rl/data/llm_message_utils.py (1)
557-574: Global state mutation may cause issues in concurrent execution.The function uses
get_formatted_message_log._debug_printedas a persistent flag to print debug output only once across all invocations. This global mutable state is not thread-safe and could cause issues in:
- Concurrent/parallel execution (race conditions)
- Ray remote contexts (per coding guidelines: functions with
@ray.remoterun in separate processes)- Testing scenarios where isolation is needed
Consider one of these alternatives:
- Pass a mutable container (e.g.,
debug_state: dict[str, bool] = None) to track state across a batch- Accept that debug output may appear multiple times and remove the flag
- Use thread-local storage if single output is critical
🔎 Option 1: Remove persistent flag (simplest)
- if debug and not getattr(get_formatted_message_log, "_debug_printed", False): + if debug: if i == 0: # Print header only at the start of first message print("\n" + "=" * 80) print("DEBUG: Individual message turns from apply_chat_template") print("=" * 80) print(f"\n[Turn {i + 1}/{len(message_log_strs)}] Role: {message['role']}") print("-" * 40) print("Extracted message chunk:") print(repr(message_chunk)) # Using repr to show special characters print(f"Raw text (len={len(message_chunk)}):") print(message_chunk) print("-" * 40) if i == len(message_log_strs) - 1: - # Mark as printed after processing all turns of the first sample - get_formatted_message_log._debug_printed = True print("\n" + "=" * 80) print("DEBUG: Complete formatted conversation:") print("-" * 80) print(formatted_message) print("=" * 80 + "\n")This allows debug output for every call when
debug=True, which may be more intuitive.
🧹 Nitpick comments (2)
nemo_rl/data/llm_message_utils.py (1)
463-463: Clarify "first sample" in docstring.The docstring states "print debug information for the first sample," but doesn't clarify this is the first sample globally across all calls (due to the persistent
_debug_printedflag on line 557), not the first sample in each call. Consider updating to: "Whether to print debug information for the first sample processed (persists across function calls)."tests/unit/data/test_llm_message_utils.py (1)
806-807: Manual flag reset indicates design coupling.Both tests manually reset the
_debug_printedfunction attribute to ensure test isolation. This is a code smell indicating that the tests are tightly coupled to the implementation's global state mechanism. If the global state issue innemo_rl/data/llm_message_utils.py(line 557) is addressed, these manual resets can be removed, improving test maintainability.Also applies to: 828-829
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
nemo_rl/data/llm_message_utils.py(3 hunks)tests/unit/data/test_llm_message_utils.py(1 hunks)
🧰 Additional context used
📓 Path-based instructions (4)
**/*.py
📄 CodeRabbit inference engine (CODING_GUIDELINES.md)
**/*.py: Conform code to Python 3.12+
Indent code with 4 spaces. Do not use tabs
Use snake_case for file names
Use PascalCase for class names
Use snake_case for function and method names
Use snake_case for local variables
Prefix variable names that start with a number with 'k' (e.g., k_99th_percentile)
Use upper snake_case with 'G' prefix for global variables (e.g., G_MY_GLOBAL)
Use upper snake_case for constants
Avoid shadowing variables declared in an outer scope
Initialize all externally visible members of a class in the constructor
Prefer docstrings over comments for interfaces that may be used outside a file
Reserve comments for code within a function or interfaces that are local to a file
If a piece of code is commented out, include a comment describing its usage and why it's commented out. Remove debug comments before merging
Use Google style docstrings for classes and functions in Python, which can be parsed by Sphinx
Avoid using reflection when functionality can be easily achieved without reflection
When using try-except blocks, limit the except clause to the smallest set of specific errors possible
When using try-except blocks for duck-typing, keep the body of the try as small as possible and use the else block for logic
YAML is the single source of truth for configuration defaults. Do not set non-None defaults in code for configuration values
For required configuration attributes, access config directly and expect presence (e.g., policy_cfg['precision']) without hidden defaults
Use typing.NotRequired to mark optional attributes in TypedDict for configuration
When adding a new config key to a TypedDict subclass, document the key's purpose, valid values/types, and recommended default, and reflect the default in exemplar YAMLs under examples/configs/*.yaml
Follow the Google Python Style Guide for Python code
Files:
tests/unit/data/test_llm_message_utils.pynemo_rl/data/llm_message_utils.py
!(**/tests/**|**/test_*.py|**/test_*.sh)
📄 CodeRabbit inference engine (CODING_GUIDELINES.md)
Add the NVIDIA copyright header to all Python files and shell scripts (excluding tests). The header should include the current year
Files:
tests/unit/data/test_llm_message_utils.pynemo_rl/data/llm_message_utils.py
**/*.{py,sh}
📄 CodeRabbit inference engine (CODING_GUIDELINES.md)
The NVIDIA copyright header should appear at the top of all Python files and shell scripts (excluding tests)
Files:
tests/unit/data/test_llm_message_utils.pynemo_rl/data/llm_message_utils.py
nemo_rl/**/*.py
📄 CodeRabbit inference engine (CODING_GUIDELINES.md)
For any source file under nemo_rl/*.py that defines a class or function decorated with @ray.remote, add a coverage pragma (# pragma: no cover) because these run in separate Ray processes
Files:
nemo_rl/data/llm_message_utils.py
🧬 Code graph analysis (1)
tests/unit/data/test_llm_message_utils.py (2)
nemo_rl/data/interfaces.py (1)
TaskDataSpec(53-86)nemo_rl/data/llm_message_utils.py (1)
get_formatted_message_log(443-661)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: Post submodule check comment / Comment on PR
- GitHub Check: Post automodel integration comment / Comment on PR
🔇 Additional comments (3)
nemo_rl/data/llm_message_utils.py (1)
451-451: LGTM! Parameter follows conventions.The
debug: bool = Falseparameter addition aligns with the PR objectives and follows Python naming conventions.tests/unit/data/test_llm_message_utils.py (2)
797-817: Test correctly validates default behavior.The test verifies that debug output is suppressed by default, which aligns with the PR objectives to reduce verbose output.
819-840: Test correctly validates debug=True behavior.The test verifies that debug output appears when explicitly enabled with
debug=True, checking for expected debug markers in the output.
…o reduce verbose output Signed-off-by: Sahger Lad <lad.sahger@gmail.com>
0217870 to
fe9ad84
Compare
What does this PR do ?
Add debug parameter to
get_formatted_message_logwithdefault=Falseto reduce verbose outputSummary by CodeRabbit
New Features
Tests
✏️ Tip: You can customize this high-level summary in your review settings.