Skip to content

Commit

Permalink
Merge 7c17eb7 into c418c6a
Browse files Browse the repository at this point in the history
  • Loading branch information
kelly-sovacool authored May 20, 2022
2 parents c418c6a + 7c17eb7 commit 3095791
Show file tree
Hide file tree
Showing 80 changed files with 849 additions and 142 deletions.
1 change: 1 addition & 0 deletions .Rbuildignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,4 @@
^cran-comments\.md$
^revdep$
^CRAN-RELEASE$
^CRAN-SUBMISSION$
6 changes: 3 additions & 3 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Package: mikropml
Title: User-Friendly R Package for Supervised Machine Learning Pipelines
Version: 1.2.2.9000
Date: 2022-02-03
Version: 1.3.0
Date: 2022-05-19
Authors@R:
c(person(given = "Begüm",
family = "Topçuoğlu",
Expand Down Expand Up @@ -91,4 +91,4 @@ VignetteBuilder:
Encoding: UTF-8
LazyData: true
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.1.1
RoxygenNote: 7.1.2
2 changes: 2 additions & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ export(":=")
export(.data)
export(calc_perf_metrics)
export(combine_hp_performance)
export(compare_models)
export(contr.ltfr)
export(define_cv)
export(get_caret_processed_df)
Expand All @@ -19,6 +20,7 @@ export(get_perf_metric_name)
export(get_performance_tbl)
export(get_tuning_grid)
export(group_correlated_features)
export(permute_p_value)
export(plot_hp_performance)
export(plot_model_performance)
export(preprocess_data)
Expand Down
2 changes: 1 addition & 1 deletion NEWS.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# development version
# mikropml 1.3.0

- mikropml now requires R version 4.1.0 or greater due to an update in the randomForest package (#292).
- New function `compare_models()` compares the performance of two models with a permutation test (#295, @courtneyarmour).
Expand Down
16 changes: 7 additions & 9 deletions R/compare_models.R
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
#' Average metric difference
#'
#' Calculate the difference in the mean of the metric for two groups
#'
#' @param sub_data subset of the merged performance data frame for two groups
Expand All @@ -8,7 +6,7 @@
#'
#' @return numeric difference in the average metric between the two groups
#'
#' @export
#' @noRd
#' @author Courtney Armour, \email{armourc@@umich.edu}
#'
#' @examples
Expand Down Expand Up @@ -38,7 +36,7 @@ get_difference <- function(sub_data, group_name, metric) {
#' @param col_name column name to shuffle
#'
#' @return `dat` with the rows of `col_name` shuffled
#' @export
#' @noRd
#' @author Courtney R Armour, \email{armourc@@umich.edu}
#'
#' @examples
Expand All @@ -62,7 +60,6 @@ shuffle_group <- function(dat, col_name) {
return(data_shuffled)
}


#' Calculated a permuted p-value comparing two models
#'
#' @inheritParams compare_models
Expand Down Expand Up @@ -124,16 +121,17 @@ permute_p_value <- function(merged_data, metric, group_name, group_1, group_2, n
}


#' Compute all pairs of comparisons
#' calculate permuted p-value across all pairs of group variable.
#' wrapper for `permute_p_value`
#' Perform permutation tests to compare the performance metric
#' across all pairs of a group variable.
#'
#' A wrapper for `permute_p_value()`.
#'
#' @param merged_data the concatenated performance data from `run_ml`
#' @param metric metric to compare, must be numeric
#' @param group_name column with group variables to compare
#' @param nperm number of permutations, default=10000
#'
#' @return a table of p-values for all pairs of group varible
#' @return a table of p-values for all pairs of group variable
#' @export
#' @author Courtney R Armour, \email{armourc@@umich.edu}
#'
Expand Down
10 changes: 5 additions & 5 deletions R/data.R
Original file line number Diff line number Diff line change
Expand Up @@ -32,19 +32,19 @@
#' Cross validation on `train_data_mini` with grouped features.
"otu_mini_cv"

#' Results from running the pipline with L2 logistic regression on `otu_mini_bin` with feature importance and grouping
#' Results from running the pipeline with L2 logistic regression on `otu_mini_bin` with feature importance and grouping
"otu_mini_bin_results_glmnet"

#' Results from running the pipline with random forest on `otu_mini_bin`
#' Results from running the pipeline with random forest on `otu_mini_bin`
"otu_mini_bin_results_rf"

#' Results from running the pipline with svmRadial on `otu_mini_bin`
#' Results from running the pipeline with svmRadial on `otu_mini_bin`
"otu_mini_bin_results_svmRadial"

#' Results from running the pipline with xbgTree on `otu_mini_bin`
#' Results from running the pipeline with xbgTree on `otu_mini_bin`
"otu_mini_bin_results_xgbTree"

#' Results from running the pipline with rpart2 on `otu_mini_bin`
#' Results from running the pipeline with rpart2 on `otu_mini_bin`
"otu_mini_bin_results_rpart2"

#' Results from running the pipeline with glmnet on `otu_mini_bin` with `Otu00001`
Expand Down
20 changes: 12 additions & 8 deletions _pkgdown.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,15 +29,22 @@ reference:
- mikropml
- preprocess_data
- run_ml
- title: Plotting & evalutation helpers
- title: Plotting helpers
desc: >
Visualize & evalutate performance to help you tune hyperparameters and choose model methods.
Visualize results to help you tune hyperparameters and choose model methods.
contents:
- compare_models
- starts_with('plot')
- tidy_perf_data
- get_hp_performance
- combine_hp_performance
- title: Model evaluation
desc: >
Evaluate and interpret models.
contents:
- get_feature_importance
- get_performance_tbl
- compare_models
- permute_p_value
- title: Package Data
- subtitle: datasets
contents:
Expand All @@ -54,9 +61,8 @@ reference:
- replace_spaces
- title: Pipeline customization
desc: >
These are functions called by preprocess_data() or run_ml().
We make them available in case you would like to customize various steps
of the pipeline beyond the arguments provided by the main functions.
Customize various steps of the pipeline beyond the arguments provided by
run_ml() and preprocess_data().
contents:
- remove_singleton_columns
- get_caret_processed_df
Expand All @@ -70,6 +76,4 @@ reference:
- get_perf_metric_fn
- train_model
- calc_perf_metrics
- get_performance_tbl
- get_feature_importance
- group_correlated_features
1 change: 0 additions & 1 deletion cran-comments.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
This patch fixes a test failure on the no long doubles platform.

## Test environments

Expand Down
2 changes: 1 addition & 1 deletion docs/404.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion docs/CODE_OF_CONDUCT.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion docs/CONTRIBUTING.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion docs/LICENSE-text.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion docs/LICENSE.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion docs/SUPPORT.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion docs/articles/index.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 8 additions & 3 deletions docs/articles/introduction.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions docs/articles/paper.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 7 additions & 2 deletions docs/articles/parallel.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions docs/articles/preprocess.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 3095791

Please sign in to comment.