Skip to content

Conversation

@Acepresso
Copy link
Contributor

@Acepresso Acepresso commented Dec 17, 2025

User description

Restructure the workflow so that Test and Acceptance jobs save coverage as artifacts, then a new Upload job waits for both to complete before uploading all reports together, ensuring Codecov receives complete data in a single batch instead of calculating status prematurely.


PR Type

Enhancement


Description

  • Restructure workflow to defer Codecov uploads until both jobs complete

  • Test and Acceptance jobs now save coverage as artifacts instead of uploading

  • New Upload job waits for both jobs, downloads artifacts, and uploads together

  • Prevents premature Codecov status calculation with incomplete coverage data


Diagram Walkthrough

flowchart LR
  Test["Test Job<br/>Save coverage artifacts"]
  Acceptance["Acceptance Job<br/>Save coverage artifacts"]
  Upload["Upload Job<br/>Download artifacts & upload to Codecov"]
  Test -- "coverage-test artifact" --> Upload
  Acceptance -- "coverage-acceptance artifact" --> Upload
Loading

File Walkthrough

Relevant files
Configuration changes
checks-codecov.yaml
Restructure workflow for batched Codecov uploads                 

.github/workflows/checks-codecov.yaml

  • Removed direct Codecov uploads from Test job, replaced with artifact
    upload for all three coverage files
  • Removed direct Codecov upload from Acceptance job, replaced with
    artifact upload
  • Added new Upload job that depends on both Test and Acceptance jobs
  • Upload job downloads artifacts and performs all Codecov uploads with
    proper file paths
+73/-27 

@codecov
Copy link

codecov bot commented Dec 17, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.

❗ There is a different number of reports uploaded between BASE (9f4e9bf) and HEAD (9baeb52). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (9f4e9bf) HEAD (9baeb52)
acceptance 1 0
Flag Coverage Δ
acceptance ?
generative 18.88% <ø> (-0.09%) ⬇️
integration 22.82% <ø> (-5.62%) ⬇️
unit 58.46% <ø> (-9.51%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 52 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions github-actions bot added size: M and removed size: XS labels Dec 17, 2025
@Acepresso Acepresso changed the title Wait for all code coverage reports before calculating status Fix premature Codecov status checks Dec 17, 2025
@Acepresso Acepresso force-pushed the codecov-tuning branch 2 times, most recently from 0865e3a to 382cb7d Compare December 18, 2025 12:28
@github-actions github-actions bot added size: L and removed size: M labels Dec 18, 2025
@github-actions github-actions bot added size: XL and removed size: L labels Dec 18, 2025
@Acepresso Acepresso force-pushed the codecov-tuning branch 2 times, most recently from 1b01ee3 to 3119e28 Compare December 21, 2025 10:53
@Acepresso Acepresso force-pushed the codecov-tuning branch 2 times, most recently from 51f9eee to 473a364 Compare December 21, 2025 13:19
The Checks pipeline has 2 separate jobs that upload codecov results - Test and
Acceptance. Test job always finishes first, so Codecov calculates status before Acceptance finishes, showing a red status (i.e. reduced coverage) that later turns green.

To fix that, upload all coverage together after both Test and Acceptance
jobs complete.

Ref: https://issues.redhat.com/browse/EC-1591
Assisted-by: Claude
@Acepresso Acepresso force-pushed the codecov-tuning branch 3 times, most recently from 9baeb52 to 3f15f98 Compare December 29, 2025 14:29
@Acepresso
Copy link
Contributor Author

I've verified this fix successfully.
This is how I verified: I removed some tests to reduce the code coverage above the threshold, and saw that the codecov check fails and the Codecov report comment is updated. See screenshots below.

Screenshot From 2025-12-29 16-28-45 Screenshot From 2025-12-29 16-28-30

@Acepresso Acepresso marked this pull request as ready for review December 29, 2025 14:43
@qodo-code-review
Copy link
Contributor

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
🟢
No security concerns identified No security vulnerabilities detected by AI analysis. Human verification advised for critical code.
Ticket Compliance
🎫 No ticket provided
  • Create ticket/issue
Codebase Duplication Compliance
Codebase context is not defined

Follow the guide to enable codebase context checks.

Custom Compliance
🟢
Generic: Comprehensive Audit Trails

Objective: To create a detailed and reliable record of critical system actions for security analysis
and compliance.

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Meaningful Naming and Self-Documenting Code

Objective: Ensure all identifiers clearly express their purpose and intent, making code
self-documenting

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Secure Error Handling

Objective: To prevent the leakage of sensitive system information through error messages while
providing sufficient detail for internal debugging.

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Secure Logging Practices

Objective: To ensure logs are useful for debugging and auditing without exposing sensitive
information like PII, PHI, or cardholder data.

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

Generic: Security-First Input Validation and Data Handling

Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent
vulnerabilities

Status: Passed

Learn more about managing compliance generic rules or creating your own custom rules

🔴
Generic: Robust Error Handling and Edge Case Management

Objective: Ensure comprehensive error handling that provides meaningful context and graceful
degradation

Status:
Invalid YAML indentation: The new actions/upload-artifact step appears to have name/path/retention-days mis-indented
relative to with:, which can break the workflow rather than failing with actionable
context.

Referred Code
- name: Upload test coverage artifacts
  uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.0
  with:
    name: coverage-test
    path: |
      ./coverage-unit.out
      ./coverage-generative.out
      ./coverage-integration.out
    retention-days: 1

Learn more about managing compliance generic rules or creating your own custom rules

Compliance status legend 🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

@qodo-code-review
Copy link
Contributor

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
Possible issue
Correct YAML indentation for acceptance upload

Correct the YAML indentation for the disable_search and flags parameters in the
acceptance test coverage upload step to ensure they are correctly applied.

.github/workflows/checks-codecov.yaml [202-205]

 with:
   files: ./coverage/coverage-acceptance.out
- disable_search: true
- flags: acceptance
+  disable_search: true
+  flags: acceptance
  • Apply / Chat
Suggestion importance[1-10]: 8

__

Why: This suggestion correctly identifies and fixes a YAML syntax error (incorrect indentation) that would cause the disable_search and flags parameters to be ignored, which is a critical bug fix for the workflow.

Medium
General
Consolidate multiple Codecov uploads into one

Consolidate the four separate Codecov upload steps into a single step by using
the directory option. Also, add fail_ci_if_error: true to ensure the job fails
if the upload does.

.github/workflows/checks-codecov.yaml [171-205]

-- name: Upload unit test coverage report
+- name: Upload coverage reports
   uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
   env:
     CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
   with:
-    files: ./coverage/coverage-unit.out
+    directory: ./coverage
     disable_search: true
-    flags: unit
+    fail_ci_if_error: true
 
-- name: Upload generative test coverage report
-  uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
-  env:
-    CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
-  with:
-    files: ./coverage/coverage-generative.out
-    disable_search: true
-    flags: generative
-
-- name: Upload integration test coverage report
-  uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
-  env:
-    CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
-  with:
-    files: ./coverage/coverage-integration.out
-    disable_search: true
-    flags: integration
-
-- name: Upload acceptance test coverage report
-  uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
-  env:
-    CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
-  with:
-    files: ./coverage/coverage-acceptance.out
-    disable_search: true
-    flags: acceptance
-
  • Apply / Chat
Suggestion importance[1-10]: 7

__

Why: This is a valuable refactoring that simplifies the workflow by consolidating four upload steps into one, improving maintainability. It also correctly suggests adding fail_ci_if_error: true for robustness.

Medium
Enforce CI failure on upload errors

Add fail_ci_if_error: true to the with block of the unit test coverage upload
step to ensure the CI job fails upon an upload error.

.github/workflows/checks-codecov.yaml [175-178]

 with:
   files: ./coverage/coverage-unit.out
   disable_search: true
   flags: unit
+  fail_ci_if_error: true
  • Apply / Chat
Suggestion importance[1-10]: 5

__

Why: This is a good suggestion for improving CI reliability, but it only applies the change to one of the four upload steps. The improvement should be applied to all codecov-action steps for consistency.

Low
  • More

Copy link
Contributor

@st3penta st3penta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@Acepresso Acepresso merged commit 77868e3 into conforma:main Dec 29, 2025
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants