Allow for different metrics in NPV calculation #1027
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1027 +/- ##
==========================================
- Coverage 82.09% 81.77% -0.32%
==========================================
Files 53 53
Lines 7310 7376 +66
Branches 7310 7376 +66
==========================================
+ Hits 6001 6032 +31
- Misses 1019 1054 +35
Partials 290 290 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| let (metric_precedence, metric) = match annual_fixed_cost.value() { | ||
| // If AFC is zero, use total surplus as the metric (strictly better than nonzero AFC) | ||
| 0.0 => (0, -profitability_index.total_annualised_surplus.value()), | ||
| // If AFC is non-zero, use profitability index as the metric | ||
| _ => (1, -profitability_index.value().value()), |
There was a problem hiding this comment.
I'm not fully convinced by this. the profitability index is dimensionless, but the annualised surplus is Money. Even though it does not matter from the typing perspective since you are getting the underlying value in both cases, which is float, I wonder if this choice makes sense from a logic perspective.
Not that I've a better suggestion.
|
|
||
| // calculate metric and precedence depending on asset parameters | ||
| // note that metric will be minimised so if larger is better, we negate the value | ||
| let (metric_precedence, metric) = match annual_fixed_cost.value() { |
There was a problem hiding this comment.
Thinking again on this, I think this logic (whatever it becomes, see my other comment) should be within the ProfitabilityIndex.value, also adding a ProfitabilityIndex.precedence method that returns 0 or 1 depending on the value of AFC.
tsmbland
left a comment
tsmbland
left a comment
There was a problem hiding this comment.
I think this is ok, I have an alternative suggestion though.
The idea would be introduce a new trait for appraisal metrics:
pub trait MetricTrait {
fn value(&self) -> f64;
fn compare(&self, other: &Self) -> Ordering;
}
pub struct AppraisalOutput {
pub metric: Box<dyn MetricTrait>,
// ...
}
You could add this trait to your ProfitabilityIndex struct, and add a custom compare method here. You'd also have to make an equivalent struct for LCOX - it would probably be very simple, although there may be some edge cases we haven't thought of yet. I think this would help to contain the comparison logic and make the code cleaner. We'd also no longer have to make the profitability index negative as the custom compare method could be written to look for the maximum - I always found this a bit hacky and it makes the output files confusing
Or this: I think there are various pros and cons of each option which I don't fully understand. I think possibly the latter is better if you don't need to store |
alexdewar
left a comment
alexdewar
left a comment
There was a problem hiding this comment.
I've taken the liberty of jumping in for a review here @Aurashk 🙃. Hope that's ok!
I agree with @tsmbland's suggestion that it would be better to use traits for this instead though -- I just think it will make it a bit clearer and more maintainable.
I'm wondering if it might be best to define a supertrait instead (that's just an alias for a combination of traits). In our case, we just need things which can be compared (Ord) and written to file (Serialize). We did talk about having an Ord implementation for unit types (#717) and I think I've actually done that somewhere, but didn't open a PR as we didn't need it, but I can do if that would be useful! That unit types would automatically define the supertrait.
I think the problem with having a value() method returning f64, as @tsmbland suggested, is that it wouldn't be obvious which value was being returned for the NPV case.
E.g.:
trait ComparisonMetric: Ord + Serialize {}
pub struct AppraisalOutput {
pub metric: Box<dyn ComparisonMetric>,
// ...
}What do people think?
| /// Where there is more than one possible metric for comparing appraisals, this integer | ||
| /// indicates the precedence of the metric (lower values have higher precedence). | ||
| /// Only metrics with the same precedence should be compared. | ||
| pub metric_precedence: u8, |
There was a problem hiding this comment.
I'd probably make this a u32 instead. I know we won't ever need more than 256 different values here (if we did, that would be horrible!), but I think it's best to use 32-bit integers as a default, unless there's a good reason not to.
| // Calculate profitability index for the hypothetical investment | ||
| let annual_fixed_cost = annual_fixed_cost(asset); | ||
| if annual_fixed_cost.value() < 0.0 { | ||
| bail!("The current NPV calculation does not support negative annual fixed costs"); |
There was a problem hiding this comment.
Can this actually happen? I'm struggling to think... @tsmbland?
If it's more of a sanity check instead (still worth doing!) then I'd change this to an assert! instead.
There was a problem hiding this comment.
If it is then something has gone badly wrong! Agree, better to change to assert!
There was a problem hiding this comment.
From my understanding of this comment from Adam, it may be possible in the future. And in that case a negative AFC isn't necessarily wrong, just the resulting profitability index will be wrong.
There was a problem hiding this comment.
Ah ok. I'd think I'd still make it a panic! for now though, because if it happens it's a coding bug, not a user-facing error.
|
PS -- this isn't super important, but for the NPV metric, I'd include the numerator and denominator in the output CSV file (e.g. "1.0/2.0" or something) rather than just saving one of the two values. We want users to be able to see why the algo has made whatever decisions it has |
|
Actually, on reflection, my suggestion won't work after all 😞. I still think we should use traits, but it's not as simple as adding a single supertrait. The problem is that I think what you want is the pub trait AppraisalMetric {
fn compare(&self, other: &dyn Any) -> Ordering;
// ...
}
impl AppraisalMetric for LCOXMetric {
fn compare(&self, other: &dyn Any) -> Ordering {
let other = other.downcast_ref::<Self>().expect("Cannot compare metrics of different types");
// ...
}
}(This assumes you have an I still think it might make sense to have a supertrait which is It's a little convoluted but that's the downside of using a statically typed language 😛 |
|
Btw, I know I'm a bit late to the party on this one, but I'm not sure about the I'm not sure why we need to go ensure consistent ordering for assets with identical appraisal outputs. If they're identical -- or approximately identical -- then, by definition, we should just return I'm only raising it because I think the added complexity will make @Aurashk's life harder here and I can't really see what benefit it brings. Is there something I'm missing? |
It's not actually that unlikely:
I agree with you that it feels a little hacky for identical metrics not to return |
Ok, good point.
Can I just check what the motivation for this is? On reflection, I'm guessing that the idea was that it would make it easier to figure out why a particular asset was chosen over another. Is that right? If so, that seems reasonable. Initially I was thinking that it was to make choosing between two assets with identical metrics less arbitrary which I'm less convinced about. A lot of things in MUSE2 are arbitrary, e.g. how HiGHS distributes activity across time slices, and the results fluctuate as we change the code anyway, so it seemed overkill to try to make guarantees to users about this when we can't do the same for so much of the rest of the model. Anyway, in terms of the code, I think the problem is that it's not a good separation of concerns. It would be better if the |
That's part of it at least. E.g. Before working on this I didn't previously consider that multiple existing assets from different commission years might have the same metric. If it's going to have to pick one over the other, I'd at least like some consistency so that decision is explainable.
I don't think we can/should try guarantee to users that the model is completely unarbitrary, but we can still do our best and if there's an easy way to make certain behaviours just a little bit more predictable/explainable then I don't think there's an excuse not to.
I think that's a good idea, do you want to give it a try? |
|
I think your other point is that the warning that we're currently raising if it ultimately does have to make an arbitrary decision isn't really worthy of a warning that users should have to be concerned about. I think that's fair, so happy if you'd rather change that to a debug message |
|
@tsmbland Ok cool. Seems like we're on the same page now. I'll have a go at the refactoring. |
|
Sorry just catching up with the discussion. I could do with a bit more convincing on the the traits approach to help me understand the benefits. I do see the elegance of it (particularly over the existing approach), but in the end we have just have three possible comparable metrics - the LCOX one and the two NPV ones (profitability index and total annualised surplus) - and adding others would be a rare occasion. It seems like a metric for our purposes is always going to be a number and one of three (maybe more in the future) labels, we only need to compare like-for-like labels on the same logical path and we are always comparing an Based on the requirements above, it feels overly-general to make the metric an arbitrary data type that has the property of being comparable. If the implementation was really simple I might feel differently, but it feels like you're having to navigate through too much abstraction for a relatively simple bit of logic. I realise I'm outvoted on this and happy to go with the majority, just want to understand the reasoning better. |
You're right that we won't be adding new metrics all that often. There is an open issue to add support for "equivalent annual cost" (#524), but I guess the output of that will be similar to the other two. I think the main rationale for using traits was to have clearer code with better separation of concerns, so you'd have a dedicated function to handle the logic for comparing NPV outputs (for example). Another upside is that we could represent these metrics in more intuitive ways in the output file (currently we output negative NPV, which is potentially pretty confusing), but I don't think this is super important. I hear what you're saying about it being clunky and overcomplex though. Another way to do this would be to keep your current approach, but have a generic |
I'd probably still favour the traits approach, for the reasons that @alexdewar mentioned, but if it's too much work then don't worry about it (I didn't think it would be too difficult when I suggested it here, but maybe I'm missing something). We're also going to have to revisit this when we come to allow multiple objectives, so no point over-engineering things right now |
I don't think it's actually too difficult -- there's just slightly more boilerplate. Maybe try it and see how it looks @Aurashk? If it's a pain, let me know and I can try to help or we can go with the other approach.
Good point! |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| fn compare(&self, other: &dyn MetricTrait) -> Ordering { | ||
| let other = other | ||
| .as_any() | ||
| .downcast_ref::<Self>() | ||
| .expect("Cannot compare metrics of different types"); | ||
|
|
||
| // Handle comparison based on fixed cost status | ||
| match (self.is_zero_fixed_cost(), other.is_zero_fixed_cost()) { | ||
| // Both have zero fixed cost: compare total surplus (higher is better) | ||
| (true, true) => { | ||
| let self_surplus = self.profitability_index.total_annualised_surplus; | ||
| let other_surplus = other.profitability_index.total_annualised_surplus; | ||
|
|
||
| if approx_eq!(Money, self_surplus, other_surplus) { | ||
| Ordering::Equal | ||
| } else { | ||
| other_surplus.partial_cmp(&self_surplus).unwrap() | ||
| } | ||
| } | ||
| // Both have non-zero fixed cost: compare profitability index (higher is better) | ||
| (false, false) => { | ||
| let self_pi = self.profitability_index.value(); | ||
| let other_pi = other.profitability_index.value(); | ||
|
|
||
| if approx_eq!(Dimensionless, self_pi, other_pi) { | ||
| Ordering::Equal | ||
| } else { | ||
| other_pi.partial_cmp(&self_pi).unwrap() | ||
| } | ||
| } | ||
| // Zero fixed cost is always better than non-zero fixed cost | ||
| (true, false) => Ordering::Less, | ||
| (false, true) => Ordering::Greater, | ||
| } | ||
| } |
There was a problem hiding this comment.
The critical new behavior of NPVMetric comparison with zero fixed costs lacks test coverage. The new logic handles three scenarios: both with zero fixed cost, both with non-zero fixed cost, and mixed cases. Only the panic behavior on calling .value() with zero cost is tested. Consider adding unit tests for NPVMetric.compare() that verify: 1) Comparison of two metrics with zero fixed cost correctly uses total surplus, 2) Comparison of two metrics with non-zero fixed cost correctly uses profitability index, and 3) Zero fixed cost metrics are correctly preferred over non-zero fixed cost metrics.
There was a problem hiding this comment.
I'll wait for your review @alexdewar but I do think some tests would be nice
There was a problem hiding this comment.
Agreed.
I think the logic for this method is a bit convoluted as it is. There will be different ways you could go about it, but here's how I'd split it up:
- Make a helper function for doing approx comparisons (as you're doing this in a bunch of places), returning Ordering::Equal if two values are approx equal. (You might need to make this generic, in which case I think
Twill have trait boundsPartialEq + ApproxEq<Margin=F64>.) - Write a method which tries to compare the metrics based on profitability index, returning
Noneif either has an AFC that's approx zero. - Then this method can just call this method, falling back on comparing AFC, e.g.:
// Using `compare_approx` as I mentioned in 1 and newtypes
self.try_compare_pi(other).unwrap_or_else(|| compare_approx(self.0.annualised_fixed_cost, other.0.annualised_fixed_cost)Does this make sense?
Thanks @alexdewar, I thinks it's ready for another review but you might also want to have a look at how I addressed copilot and Tom's comments |
alexdewar
left a comment
alexdewar
left a comment
There was a problem hiding this comment.
I think this is mostly there now, but I've made a few suggestions. In particular, I think NPVMetric::compare() could be made much simpler; it's hard to read as it is.
We could also define Serialize for the metric structs so we can have custom formatting in the CSV file, but you can open an issue for that and do it later if you prefer.
| /// Trait for appraisal metrics that can be compared. | ||
| /// | ||
| /// Implementers define how their values should be compared to determine | ||
| /// which investment option is preferable through the `compare` method. | ||
| pub trait MetricTrait: Any + Send + Sync { | ||
| /// Returns the numeric value of this metric. | ||
| fn value(&self) -> f64; | ||
|
|
||
| /// Compares this metric with another of the same type. | ||
| /// | ||
| /// Returns `Ordering::Less` if `self` is better than `other`, | ||
| /// `Ordering::Greater` if `other` is better, or `Ordering::Equal` | ||
| /// if they are approximately equal. | ||
| /// | ||
| /// # Panics | ||
| /// | ||
| /// Panics if `other` is not the same concrete type as `self`. | ||
| fn compare(&self, other: &dyn MetricTrait) -> Ordering; | ||
|
|
||
| /// Helper for downcasting to enable type-safe comparison. | ||
| fn as_any(&self) -> &dyn Any; | ||
| } |
There was a problem hiding this comment.
It is an interesting point, but I think the traits approach will be a bit cleaner when we get to the custom Serialize implementations. Sure, you can still do this with an enum, but I think it'll probably look cleaner to separate these things out for the different metric types.
|
|
||
| /// Returns true if this metric represents a zero fixed cost case. | ||
| fn is_zero_fixed_cost(&self) -> bool { | ||
| self.profitability_index.annualised_fixed_cost == Money(0.0) |
There was a problem hiding this comment.
That's a good point. It probably makes sense to use approx_eq! instead, so that we don't get subtly different behaviours depending on small rounding errors (see also #893).
I don't think we need this as a separate helper method though, as it's only used it one place. See my comment below about refactoring.
| #[derive(Debug, Clone)] | ||
| pub struct NPVMetric { | ||
| /// Profitability index data for this NPV metric | ||
| pub profitability_index: ProfitabilityIndex, |
There was a problem hiding this comment.
You could also make this a newtype, i.e. not give the field a name, which would make things less verbose:
pub struct NPVMetric(ProfitabilityIndex);You can then get at the field with my_metric.0.
| } | ||
|
|
||
| impl MetricTrait for NPVMetric { | ||
| fn value(&self) -> f64 { |
There was a problem hiding this comment.
I think we should implement the Serialize trait for NPVMetric and LCOXMetric rather than having a method for getting a scalar like this. Then we can write the outputs in a custom way, without having to restrict ourselves to a single number, e.g. for NPV, we could write both the numerator and the denominator.
I think the easiest way to implement this is with a supertrait (without extra methods): https://doc.rust-lang.org/rust-by-example/trait/supertraits.html
So we'd have:
- A trait for comparing two metrics, like the current
MetricTrait(maybe rename toComparableMetricor something) - A supertrait which is defined as:
pub trait MetricTrait: ComparableMetric + Serialize {}Does that make sense?
That said, adding custom Serialize implementations is something we could do later, so if you'd rather open an issue for it instead, feel free.
| fn compare(&self, other: &dyn MetricTrait) -> Ordering { | ||
| let other = other | ||
| .as_any() | ||
| .downcast_ref::<Self>() | ||
| .expect("Cannot compare metrics of different types"); | ||
|
|
||
| // Handle comparison based on fixed cost status | ||
| match (self.is_zero_fixed_cost(), other.is_zero_fixed_cost()) { | ||
| // Both have zero fixed cost: compare total surplus (higher is better) | ||
| (true, true) => { | ||
| let self_surplus = self.profitability_index.total_annualised_surplus; | ||
| let other_surplus = other.profitability_index.total_annualised_surplus; | ||
|
|
||
| if approx_eq!(Money, self_surplus, other_surplus) { | ||
| Ordering::Equal | ||
| } else { | ||
| other_surplus.partial_cmp(&self_surplus).unwrap() | ||
| } | ||
| } | ||
| // Both have non-zero fixed cost: compare profitability index (higher is better) | ||
| (false, false) => { | ||
| let self_pi = self.profitability_index.value(); | ||
| let other_pi = other.profitability_index.value(); | ||
|
|
||
| if approx_eq!(Dimensionless, self_pi, other_pi) { | ||
| Ordering::Equal | ||
| } else { | ||
| other_pi.partial_cmp(&self_pi).unwrap() | ||
| } | ||
| } | ||
| // Zero fixed cost is always better than non-zero fixed cost | ||
| (true, false) => Ordering::Less, | ||
| (false, true) => Ordering::Greater, | ||
| } | ||
| } |
There was a problem hiding this comment.
Agreed.
I think the logic for this method is a bit convoluted as it is. There will be different ways you could go about it, but here's how I'd split it up:
- Make a helper function for doing approx comparisons (as you're doing this in a bunch of places), returning Ordering::Equal if two values are approx equal. (You might need to make this generic, in which case I think
Twill have trait boundsPartialEq + ApproxEq<Margin=F64>.) - Write a method which tries to compare the metrics based on profitability index, returning
Noneif either has an AFC that's approx zero. - Then this method can just call this method, falling back on comparing AFC, e.g.:
// Using `compare_approx` as I mentioned in 1 and newtypes
self.try_compare_pi(other).unwrap_or_else(|| compare_approx(self.0.annualised_fixed_cost, other.0.annualised_fixed_cost)Does this make sense?
| pub fn value(&self) -> Dimensionless { | ||
| self.total_annualised_surplus / self.annualised_fixed_cost |
There was a problem hiding this comment.
Another way to do this would be to have value() return None in the case that AFC==0. That might compose nicely with the refactoring I suggested above. Up to you though -- panicking is also fine.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
src/simulation/investment/appraisal.rs:40
- The PR description mentions adding a
metric_precedencefield toAppraisalOutputto rank metrics and filter them inselect_best_assets, but this implementation does not include such a field. Instead, the comparison logic is embedded within theNPVMetric::comparemethod. The documentation should be updated to accurately reflect the actual implementation approach, or the mentioned feature should be implemented if it's still needed.
pub struct AppraisalOutput {
/// The asset being appraised
pub asset: AssetRef,
/// The hypothetical capacity to install
pub capacity: Capacity,
/// Time slice level activity of the asset
pub activity: IndexMap<TimeSliceID, Activity>,
/// The hypothetical unmet demand following investment in this asset
pub unmet_demand: DemandMap,
/// The comparison metric to compare investment decisions
pub metric: Box<dyn MetricTrait>,
/// Capacity and activity coefficients used in the appraisal
pub coefficients: ObjectiveCoefficients,
/// Demand profile used in the appraisal
pub demand: DemandMap,
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| /// Represents the profitability index of an investment | ||
| /// in terms of its annualised components. | ||
| #[derive(Debug, Clone, Copy)] | ||
| pub struct ProfitabilityIndex { | ||
| /// The total annualised surplus of an asset | ||
| pub total_annualised_surplus: Money, | ||
| /// The total annualised fixed cost of an asset | ||
| pub annualised_fixed_cost: Money, | ||
| } | ||
|
|
||
| impl ProfitabilityIndex { | ||
| /// Calculates the value of the profitability index. | ||
| pub fn value(&self) -> Dimensionless { | ||
| assert!( | ||
| self.annualised_fixed_cost != Money(0.0), | ||
| "Annualised fixed cost cannot be zero when calculating profitability index." | ||
| ); | ||
| self.total_annualised_surplus / self.annualised_fixed_cost | ||
| } | ||
| } |
There was a problem hiding this comment.
The ProfitabilityIndex::value() method panics when annualised_fixed_cost is zero, but this condition can legitimately occur as described in the PR. The NPVMetric::value() method correctly handles this case by returning the total surplus instead, but calling profitability_index.value() directly elsewhere in the codebase could cause panics. Consider adding documentation to ProfitabilityIndex that warns users about this limitation and directs them to check the fixed cost before calling value(), or use NPVMetric wrapper instead.
|
@alexdewar As I understand it the best we can do here is make both |
|
@Aurashk Ah, I see! That's annoying... sorry for sending you down that path. So it seems like the issue is that Alternatively we can skip the serialisation stuff for now and just review the other bits so we can get this merged, then circle back. Up to you. |
Add serialize implementations to metric types
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 6 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| (true, true) => { | ||
| let self_surplus = self.0.total_annualised_surplus; | ||
| let other_surplus = other.0.total_annualised_surplus; | ||
|
|
||
| if approx_eq!(Money, self_surplus, other_surplus) { | ||
| Ordering::Equal | ||
| } else { | ||
| other_surplus.partial_cmp(&self_surplus).unwrap() | ||
| } |
There was a problem hiding this comment.
When comparing NPVMetrics with zero fixed costs (lines 170-178), the code compares total_annualised_surplus values where higher is considered better. However, if both assets have negative surplus (losing money), this logic would still prefer the one with the higher (less negative) loss. While this might be intentional, consider whether assets with negative surplus should be rejected earlier in the appraisal process, or document this behavior explicitly if it's intended.
| // Zero fixed cost is always better than non-zero fixed cost | ||
| (true, false) => Ordering::Less, | ||
| (false, true) => Ordering::Greater, |
There was a problem hiding this comment.
The comparison logic for NPVMetric when comparing across different fixed cost statuses (zero vs non-zero) may lead to unintuitive behavior. When one asset has zero fixed costs and another has non-zero fixed costs, the code always selects the zero fixed cost option (lines 192-193). However, this doesn't account for the case where the non-zero fixed cost asset might actually have a much higher profitability index that could make it economically superior.
Consider adding a check to ensure that when comparing zero vs non-zero fixed cost cases, the total annualised surplus of the zero-cost option is actually positive and economically meaningful before declaring it always better.
| // Zero fixed cost is always better than non-zero fixed cost | |
| (true, false) => Ordering::Less, | |
| (false, true) => Ordering::Greater, | |
| // Mixed fixed cost: prefer zero fixed cost only if its surplus is positive | |
| (true, false) => { | |
| let self_surplus = self.0.total_annualised_surplus; | |
| if self_surplus.value() > 0.0 { | |
| // Zero fixed cost with positive surplus is preferred | |
| Ordering::Less | |
| } else { | |
| // Non-positive surplus: prefer the non-zero fixed cost option | |
| Ordering::Greater | |
| } | |
| } | |
| (false, true) => { | |
| let other_surplus = other.0.total_annualised_surplus; | |
| if other_surplus.value() > 0.0 { | |
| // Zero fixed cost with positive surplus is preferred | |
| Ordering::Greater | |
| } else { | |
| // Non-positive surplus: prefer the non-zero fixed cost option | |
| Ordering::Less | |
| } | |
| } |
| let other = other | ||
| .as_any() | ||
| .downcast_ref::<Self>() | ||
| .expect("Cannot compare metrics of different types"); |
There was a problem hiding this comment.
The panic will occur if different metric types (LCOXMetric vs NPVMetric) are compared. Since all appraisals in a given investment round use the same objective type (either LCOX or NPV), this should not happen in normal operation. However, consider adding a more descriptive panic message that includes the actual types being compared to aid debugging if this ever occurs unexpectedly.
| let other = other | |
| .as_any() | |
| .downcast_ref::<Self>() | |
| .expect("Cannot compare metrics of different types"); | |
| let other_any = other.as_any(); | |
| let other = if let Some(other) = other_any.downcast_ref::<Self>() { | |
| other | |
| } else { | |
| panic!( | |
| "Cannot compare metrics of different types: expected {}, got {}", | |
| std::any::type_name::<Self>(), | |
| std::any::type_name_of_val(other_any), | |
| ); | |
| }; |
| let other = other | ||
| .as_any() | ||
| .downcast_ref::<Self>() | ||
| .expect("Cannot compare metrics of different types"); |
There was a problem hiding this comment.
The panic will occur if different metric types (LCOXMetric vs NPVMetric) are compared. Since all appraisals in a given investment round use the same objective type (either LCOX or NPV), this should not happen in normal operation. However, consider adding a more descriptive panic message that includes the actual types being compared to aid debugging if this ever occurs unexpectedly.
| let other = other | |
| .as_any() | |
| .downcast_ref::<Self>() | |
| .expect("Cannot compare metrics of different types"); | |
| let other_any = other.as_any(); | |
| let other = other_any | |
| .downcast_ref::<Self>() | |
| .unwrap_or_else(|| { | |
| panic!( | |
| "Cannot compare metrics of different types: self={}, other={}", | |
| std::any::type_name::<Self>(), | |
| std::any::type_name_of_val(other_any), | |
| ) | |
| }); |
| fn compare(&self, other: &dyn ComparableMetric) -> Ordering { | ||
| let other = other | ||
| .as_any() | ||
| .downcast_ref::<Self>() | ||
| .expect("Cannot compare metrics of different types"); | ||
|
|
||
| if approx_eq!(MoneyPerActivity, self.cost, other.cost) { | ||
| Ordering::Equal | ||
| } else { | ||
| self.metric.partial_cmp(&other.metric).unwrap() | ||
| // Lower cost is better | ||
| self.cost.partial_cmp(&other.cost).unwrap() | ||
| } | ||
| } | ||
|
|
||
| fn as_any(&self) -> &dyn Any { | ||
| self | ||
| } | ||
| } |
There was a problem hiding this comment.
The LCOXMetric comparison logic lacks test coverage. While this metric type is simpler than NPVMetric, it would be valuable to have unit tests that verify the comparison behavior, especially around the approximate equality check and the ordering logic.
| #[test] | ||
| #[should_panic(expected = "Annualised fixed cost cannot be zero")] | ||
| fn profitability_index_panics_on_zero_cost() { | ||
| let result = profitability_index( | ||
| Capacity(0.0), | ||
| MoneyPerCapacity(100.0), | ||
| &indexmap! {}, | ||
| &indexmap! {}, | ||
| ); | ||
| result.value(); |
There was a problem hiding this comment.
The new test profitability_index_panics_on_zero_cost tests that calling .value() on a ProfitabilityIndex with zero annualised fixed cost panics. However, this test uses Capacity(0.0) with MoneyPerCapacity(100.0), which results in zero annualised fixed cost. Consider adding a comment to clarify this calculation or restructure the test to use MoneyPerCapacity(0.0) with non-zero capacity to make the intent clearer.
Thanks for that. This was relatively straightforward to add, you just need the concrete structs lcox and npv metric to be serializable and the supertrait to use the ErasedSerialize trait. I think implementing the ErasedSerialize trait just provides a wrapper implementation for each concrete struct to call the actual serialize method so it's being called polymorphically at runtime like Is the future goal here to make AppraisalOutput Serializable or is this data written to a different data structure before it's outputted? It's outside the scope of this PR but just to note I think you need to add: above If making |
|
@Aurashk Actually, would you be able to sort the serialisation stuff in this PR too 👼? You just need to update the code in |
I've had a go at this but I'm a bit confused about what argument to apply to the because the writer works row by row. Am I missing something obvious here? In the latest commit I tried what seems like the simplest solution - you can serialize them to json, then just write the json as a string. It requires another dependency but wasn't able to find anything more straightforward. We could alternatively explicitly add the fields (profitability index etc) to the results row and downcast but it seems like that defeats the object of doing the |
| fn compare(&self, other: &dyn ComparableMetric) -> Ordering { | ||
| let other = other | ||
| .as_any() | ||
| .downcast_ref::<Self>() | ||
| .expect("Cannot compare metrics of different types"); | ||
|
|
||
| // Handle comparison based on fixed cost status | ||
| match (self.is_zero_fixed_cost(), other.is_zero_fixed_cost()) { | ||
| // Both have zero fixed cost: compare total surplus (higher is better) | ||
| (true, true) => { | ||
| let self_surplus = self.0.total_annualised_surplus; | ||
| let other_surplus = other.0.total_annualised_surplus; | ||
|
|
||
| compare_approx(other_surplus, self_surplus) | ||
| } | ||
| // Both have non-zero fixed cost: compare profitability index (higher is better) | ||
| (false, false) => { | ||
| let self_pi = self.0.value(); | ||
| let other_pi = other.0.value(); | ||
|
|
||
| compare_approx(other_pi, self_pi) | ||
| } | ||
| // Zero fixed cost is always better than non-zero fixed cost | ||
| (true, false) => Ordering::Less, | ||
| (false, true) => Ordering::Greater, | ||
| } | ||
| } |
There was a problem hiding this comment.
The new comparison logic in NPVMetric::compare handles an important edge case (zero fixed costs) with different comparison semantics than normal NPV comparison. However, there are no tests added to verify this behavior works correctly. Consider adding tests that verify:
- Two NPVMetrics with zero fixed cost are compared correctly by surplus
- Two NPVMetrics with non-zero fixed cost are compared correctly by profitability index
- An NPVMetric with zero fixed cost is always considered better than one with non-zero fixed cost (lines 203-204)
- The comparison correctly returns
Ordering::Equalfor approximately equal values
| ensure!( | ||
| number_of_years[&(commodity_id.clone(), region_id.clone())] | ||
| == required_years.len().try_into().unwrap(), | ||
| == u32::try_from(required_years.len()).unwrap(), |
There was a problem hiding this comment.
This refactoring from .try_into().unwrap() to u32::try_from(...).unwrap() improves code clarity but appears unrelated to the PR's main purpose of allowing different metrics in NPV calculation. Consider keeping such unrelated refactorings in separate commits or PRs to maintain a clear change history.
| == u32::try_from(required_years.len()).unwrap(), | |
| == required_years.len().try_into().unwrap(), |
| assert!( | ||
| annual_fixed_cost >= MoneyPerCapacity(0.0), | ||
| "The current NPV calculation does not support negative annual fixed costs" | ||
| ); |
There was a problem hiding this comment.
The assertion that checks annual fixed cost is non-negative only runs in debug builds. If negative annual fixed costs could occur in production (even if they shouldn't), this would silently pass in release builds and could lead to incorrect behavior. Consider using ensure! or returning a Result error instead of assert! to enforce this constraint in all builds.
| assert!( | ||
| self.annualised_fixed_cost != Money(0.0), | ||
| "Annualised fixed cost cannot be zero when calculating profitability index." | ||
| ); |
There was a problem hiding this comment.
The assertion that checks annualised fixed cost is non-zero only runs in debug builds. In release builds, if this condition is violated, the division on line 49 would produce infinity or NaN, leading to incorrect behavior. Consider using a runtime check (like ensure!) that returns an error instead of assert! to enforce this constraint in all builds.
| assert!( | ||
| !(self.metric.is_nan() || other.metric.is_nan()), | ||
| !(self.metric.value().is_nan() || other.metric.value().is_nan()), | ||
| "Appraisal metric cannot be NaN" | ||
| ); |
There was a problem hiding this comment.
The assertion checking that metric values are not NaN only runs in debug builds. In release builds, if NaN values occur, the comparison logic could produce unexpected results. Consider using a runtime check that returns an error or handles NaN values explicitly to ensure correct behavior in all builds.
Description
If annual fixed costs (AFC) are zero, this currently makes the metric in appraisal comparisons NaN. This PR modifies the behaviour so that we use the total annualised surplus to appraise assets in this case instead. Since there are two different possible metrics in this case, with one strictly better (AFC == 0 always better than AFC > 0). I've added a metric_precedence to
AppraisalOutputwhich ranks the metrics by the order which they can be used. Then,select_best_assetswill disregard all appraisals which have a precedence higher that the minimum. Another way of doing this may be to make the metric itself a struct and implement more sophisticated comparison logic, but since lcox uses the same struct it might end up getting too involvedFixes #1012
Type of change
Key checklist
$ cargo test$ cargo docFurther checks