-
Notifications
You must be signed in to change notification settings - Fork 25
Open
Description
Currently get_pairwise_comparison() will stop you if you have two models:
scoringutils/R/pairwise-comparisons.R
Lines 161 to 171 in 158ba6c
| # check there are enough comparators | |
| if (length(setdiff(comparators, baseline)) < 2) { | |
| #nolint start: keyword_quote_linter | |
| cli_abort( | |
| c( | |
| "!" = "More than one non-baseline model is needed to compute | |
| pairwise compairisons." | |
| ) | |
| ) | |
| #nolint end | |
| } |
While it's fair enough to say that you don't really need relative scores if you just have two models, there is not a conceptual problem with computing a comparison between two models. (Also if you are running more models and setting up the infrastructure with just the first one!)
So I think this should be a warning rather than an abort.
(At the moment, my solution is to create a fake model with one row, append it on, then remove it afterwards.)
Metadata
Metadata
Assignees
Labels
No labels