Implement an ensembler of MetaLearners

[kklein]

sklearn provides a BaseEnsemble class which can be used to ensemble various Estimators.

Unfortunately, sklearn's BaseEnsemble does not work out of the box with a MetaLearner from metalearners due to differences in predict and fit signatures.

In order to facilitate the ensembling of CATE estimates from various MetaLearners, it would be useful to implement helpers.

Some open questions:

• Should the ensemble be given trained MetaLearners or train the MetaLearners itself?
• Should the ensemble require all MetaLearners to have been trained on exactly the same data?
• Should the ensemble work with both, in-sample and out-of-sample data, too?

[FrancescMartiEscofetQC]

If we want to make it work with in-sample data, we obviously need that they have been trained on exactly the same data. I think the best option for this is that the user provides already initialized metalearners (fitted or unfitted) and then we implement a fit method which calls fit for all the metalearners with the same parameters. To avoid issues with copies and stuff I would suggest implementing a clone method for the MetaLearner which initializes a metalearner with the same parameters.
I think implementing it for both in-sample and oos data is not much more work than implementing it only for in-sample and this allows the user to choose which option they want to use.

I think that if the user wants to use only for oos data and metalearners with different training data, they can easily and it does not require a lot of work.

[erikcs]

Cool package. Nie & Wager’s R-loss gives you an approach for ensembling CATE estimators: stack many final-stage CATE estimators and minimize that loss. They discuss this in section 4.2 of the R-learner paper. Here’s a paper trying it out in case it’s helpful: https://arxiv.org/abs/2202.12445. On a general note, you can take the same ensembling approach to estimate nuisance components $E[W_i|X_i], E[Y_i|X_i]$ for metalearners too, but then by minimizing the standard predictive loss (that’s what van der Laan typically refers to as superlearning in TMLE).

[kklein]

Hi @erikcs - apologies for the super late reply.
Thanks a lot for the reference (and the kind words :)). We'll take a look asap!