Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

give glmnet the $importance slot #28

Open
mb706 opened this issue Jul 14, 2019 · 2 comments
Open

give glmnet the $importance slot #28

mb706 opened this issue Jul 14, 2019 · 2 comments

Comments

@mb706
Copy link
Contributor

mb706 commented Jul 14, 2019

because then it could be used in combination with FilterEmbedded in mlr3featsel for feature selection in order of L1 inclusion. Importance could be the (approximate) lambda value at which a feature is first included and can easily be calculated from the model.

@mb706
Copy link
Contributor Author

mb706 commented Jul 14, 2019

Some code I used for something similar (using "old" mlr). This only gets the order by which the features are introduced; I think the approximate lamba value would be more informative.

# orders features by in what order they are introduced when decreasing shrinkage in L1 regression.
slfun <- function(task, nselect, alpha = 1, ...) {
  xy <- getTaskData(task, target.extra = TRUE)
  if (getTaskType(task) == "regr") {
    family <- "gaussian"
  } else {
    family <- if (length(levels(xy$target)) > 2) "multinomial" else "binomial"
  }
  fit <- glmnet(x = as.matrix(xy$data), y = xy$target, alpha = alpha,
    lambda.min.ratio = 1e-4, family = family)
  captured <- integer(0)
  for (col in seq_len(ncol(fit$beta))) {
    curcols <- which(fit$beta[, col] != 0)
    newcols <- setdiff(curcols, captured)
    captured <- c(captured, newcols)
  }
  captured <- c(captured, setdiff(seq_len(getTaskNFeats(task)), captured))
  res <- -order(captured)
  names(res) <- getTaskFeatureNames(task)
  res
}

@be-marc
Copy link
Member

be-marc commented Dec 10, 2019

@mb706 Like this 7594ad7 ?
Do you know how we can handle multi class tasks? We get for each target class a different beta matrix and the positions at which the features are introduced varies between them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants