enhance: evaluation result handling- include resource key address #262

AkashS0510 · 2025-06-04T12:08:06Z

Pull Request Template

This pull request improves the handling of metadata and addresses in evaluation results and provider outputs. The changes ensure better compatibility, clarity, and backward support by explicitly managing the address field and enhancing how metadata is processed.

Enhancements to metadata and address handling:

generate_evaluator_result in src/tirith/core/core.py: Updated to include the meta field only if it exists and is not None. Additionally, the address field is now explicitly handled, either directly from the input or as a fallback from the meta dictionary for backward compatibility. The message field is updated to include the address when available.
get_value in src/tirith/providers/json/handler.py: Modified to add the address field as a separate property in the result dictionary instead of embedding it in meta. This change improves clarity and aligns with the new handling approach.

Description

Add Resource Address to Evaluation Results
Summary
This PR adds support for including resource addresses in evaluation results, making it easier to identify which specific resources are passing or failing evaluations.

Changes
Modified core to include resource addresses in evaluation results and messages
Updated the JSON provider to include key paths as address fields
Enhanced the pretty printer to display resource addresses in CLI output
Made address a top-level field in evaluation results for easier programmatic access

What changes are being made?
Why are these changes necessary?
Which issues or tickets does this PR close or relate to?
- Related issue: #

Type of Change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Documentation update
Other (please specify):

Checklist

Code has been linted.
Documentation has been updated (if needed).
Tests have been added or updated (if needed).
Any breaking changes have been communicated to the team.

Screenshots or Recordings (if applicable)

Additional Information

codecov · 2025-06-04T12:08:11Z

Codecov Report

❌ Patch coverage is 75.00000% with 15 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/tirith/providers/terraform_plan/handler.py	80.95%	5 Missing and 3 partials ⚠️
src/tirith/prettyprinter.py	0.00%	5 Missing ⚠️
src/tirith/core/core.py	75.00%	1 Missing and 1 partial ⚠️

Files with missing lines	Coverage Δ
src/tirith/providers/json/handler.py	`81.39% <100.00%> (+1.90%)`	⬆️
src/tirith/core/core.py	`75.00% <75.00%> (-8.15%)`	⬇️
src/tirith/prettyprinter.py	`0.00% <0.00%> (-51.49%)`	⬇️
src/tirith/providers/terraform_plan/handler.py	`63.70% <80.95%> (+1.64%)`	⬆️

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

refeed

Thanks. I think you forgot to include terraform_plan provider related changes here :)

src/tirith/core/core.py

AkashS0510 · 2025-06-09T13:16:13Z

Thanks. I think you forgot to include terraform_plan provider related changes here :)

You mean in the PR description?

Copilot

Pull Request Overview

This PR enhances the handling of evaluation results and provider outputs by explicitly including a top-level "address" field. Key changes include:

Updating the core evaluator result handling to copy and display the "address" in messages.
Modifying the Terraform plan handler to conditionally add the "address" field in multiple output cases.
Adjusting the JSON provider to separate the "address" from the metadata.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
src/tirith/providers/terraform_plan/handler.py	Updates to conditionally add an "address" field to result dictionaries in various operation paths.
src/tirith/providers/json/handler.py	Refactors result creation to include a separate "address" property for JSON outputs.
src/tirith/core/core.py	Enhances evaluator result generation to include and display the "address" in the output message.

src/tirith/providers/terraform_plan/handler.py

src/tirith/core/core.py

refeed

Hi Akash, thanks, could you also make sure that every result["addresses"] is always a list of strings. Example in the suggestions

src/tirith/providers/json/handler.py

src/tirith/providers/terraform_plan/handler.py

src/tirith/core/core.py

src/tirith/prettyprinter.py

src/tirith/providers/terraform_plan/handler.py

refeed · 2025-06-12T13:47:29Z

src/tirith/providers/terraform_plan/handler.py

                continue
            for reference in expressions_val.get("references", []):
                # Only get the resource type
                resource_references.add(reference.split(".")[0])


Please use the address from the reference

addresses.append(reference)

@refeed I am unable to understand why do we need to do this?

Because that's the real addresses the optype is processing. The resource var that's placed in the meta key was actually not placed correctly, it only contains one resource even though this optype is processing more than one resources

refeed

Thanks, put some comments

refeed · 2025-06-13T09:07:20Z

src/tirith/core/core.py

+
+            if "addresses" in evaluator_input:
+                # Add addresses directly
+                # TODO: We need to make a model class for the `evaluator_input` and move this validation there


Please add the type assertion mentioned in #262 (comment) here

if not isinstance(addresses, list): raise Exception("`addresses` should be a list")

Move the comment to the top of line 94

# TODO: We need to make a model class for the `evaluator_input` and move this validation there if not isinstance(addresses, list): raise Exception("`addresses` should be a

refeed · 2025-06-13T09:13:09Z

src/tirith/providers/terraform_plan/handler.py

+                address = resource_change.get("address")
+                addresses = [address] if address is not None else []


Please create a helper method that creates addresses from resource_change dict as I'm seeing this pattern often

We are just repeating it twice or thrice atmost. I dont think we need a helper class for this

Repeating it twice is already a good candidate for making a helper method. It doesn't hurt much and improves readability :)

sonarqubecloud · 2025-06-16T07:02:06Z

Quality Gate passed

Issues
3 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

refeed · 2025-06-17T02:32:22Z

src/tirith/core/core.py

+
+            if "addresses" in evaluator_input:
+                # Add addresses directly
+                # TODO: We need to make a model class for the `evaluator_input` and move this validation there


Move the comment to the top of line 94

refeed · 2025-06-17T02:33:11Z

src/tirith/core/core.py

+
+            if "addresses" in evaluator_input:
+                # Add addresses directly
+                # TODO: We need to make a model class for the `evaluator_input` and move this validation there


# TODO: We need to make a model class for the `evaluator_input` and move this validation there if not isinstance(addresses, list): raise Exception("`addresses` should be a

refeed · 2025-06-17T03:04:54Z

src/tirith/providers/terraform_plan/handler.py

+                address = resource_change.get("address")
+                addresses = [address] if address is not None else []


Repeating it twice is already a good candidate for making a helper method. It doesn't hurt much and improves readability :)

refeed · 2025-09-26T10:47:25Z

I'll continue to work on this

sonarqubecloud · 2025-09-26T11:08:21Z

Quality Gate passed

Issues
3 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

AkashS0510 added 2 commits June 4, 2025 17:34

Enhance evaluation result handling: include resource key address

99b46bd

lint fix

afb9e1a

AkashS0510 requested review from Akshat0694, arunim2405 and refeed as code owners June 4, 2025 12:08

AkashS0510 requested a review from Copilot June 4, 2025 12:09

This comment was marked as outdated.

Sign in to view

refeed requested changes Jun 9, 2025

View reviewed changes

src/tirith/core/core.py Outdated Show resolved Hide resolved

AkashS0510 added 4 commits June 10, 2025 18:10

add meta in json provider even if its null

4806062

lint fix

85dd547

move resource_key addition logic to respective handlers

e3a1fe7

lint fix

625701e

refeed requested a review from Copilot June 11, 2025 07:50

Copilot AI reviewed Jun 11, 2025

View reviewed changes

src/tirith/providers/terraform_plan/handler.py Outdated Show resolved Hide resolved

refeed reviewed Jun 11, 2025

View reviewed changes

src/tirith/providers/terraform_plan/handler.py Outdated Show resolved Hide resolved

src/tirith/providers/terraform_plan/handler.py Outdated Show resolved Hide resolved

src/tirith/core/core.py Outdated Show resolved Hide resolved

AkashS0510 added 3 commits June 12, 2025 15:26

Update

52a1875

change address to addresses

866f20c

lint fix

ce9960a

refeed requested changes Jun 12, 2025

View reviewed changes

src/tirith/providers/json/handler.py Outdated Show resolved Hide resolved

src/tirith/providers/terraform_plan/handler.py Outdated Show resolved Hide resolved

AkashS0510 added 3 commits June 12, 2025 16:52

make addresses list

fbf9aa1

lint fix

c40ff7b

fix

8786453

refeed requested changes Jun 12, 2025

View reviewed changes

Refactor

539bfdd

refeed requested changes Jun 13, 2025

View reviewed changes

Refactor

ac4bd7e

refeed requested changes Jun 17, 2025

View reviewed changes

Add address extraction helper and refactor address handling in providers

e63e462

		address = resource_change.get("address")
		addresses = [address] if address is not None else []

enhance: evaluation result handling- include resource key address #262

Are you sure you want to change the base?

enhance: evaluation result handling- include resource key address #262

Conversation

AkashS0510 commented Jun 4, 2025

Pull Request Template

Enhancements to metadata and address handling:

Description

What changes are being made?

Why are these changes necessary?

Type of Change

Checklist

Screenshots or Recordings (if applicable)

Additional Information

Uh oh!

codecov bot commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

This comment was marked as outdated.

Uh oh!

refeed left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AkashS0510 commented Jun 9, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

refeed left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

refeed left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud bot commented Jun 16, 2025

Quality Gate passed

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

refeed commented Sep 26, 2025

Uh oh!

codecov bot commented Jun 4, 2025 •

edited

Loading