allow for custom loss function to be passed as measure
argument
#13
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Allows for a custom loss function to be supplied to the measure argument. Must take truth, response, and prob as arguments (as they are output from the
predict_learner()
function followed by the few modifications at the beginning of thecompute_loss()
function (which themselves might be further modified by a call to themodify_trp()
function). Returns a single vector (which may be a one-element vector) representing the loss (i.e., 1 - model skill measure).Addresses some of the need described in #10.
I added an example to the help file that shows an implementation of Matthew's correlation coefficient. I included a note in that example about MCC being a batch-level skill measure and so that statistical approach currently implemented in this package isn't yet set up to handle that. The work around is to have the user set up a series of
test_data
data frames ahead of time, which may mimic a resampling strategy (like repeated cross-validation, with some number of iterations and folds defined), then to use that series of batch-level CPI values to conduct an external significance test.All build tests and checks pass. DESCRIPTION and cpi.Rd files changed using {roxygen2}.