Autoemulate.compare: proposed fix for issue #222

[MaxBalmus]

No description provided.

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
tests
test_compare.py
Project Total

This report was generated by python-coverage-comment-action

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.19%. Comparing base (2f1d439) to head (e6445e2).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #224      +/-   ##
==========================================
+ Coverage   91.14%   91.19%   +0.05%     
==========================================
  Files          45       45              
  Lines        2393     2407      +14     
==========================================
+ Hits         2181     2195      +14     
  Misses        212      212              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[mastoffel]

Thanks for spotting and fixing this @MaxBalmus !

Also, would you mind adding a test for this to test_compare.py ? I think something like this should work:

# --------------- test correct hyperparameter updating ------------------
def test_param_search_updates_models(ae, Xy):
    X, y = Xy
    ae.setup(X, y, model_subset=["RandomForest"], param_search=True,
             param_search_iters=5)
    params_before = ae.models[0].get_params() # just one model, so index with 0
    ae.compare()
    params_after = ae.models[0].get_params()
    assert params_before != params_after

def test_model_params_equal_wo_param_search(ae, Xy):
    X, y = Xy
    ae.setup(X, y, model_subset=["RandomForest"])
    params_before = ae.models[0].get_params()
    ae.compare()
    params_after = ae.models[0].get_params()
    assert params_before == params_after

tests can be run with pytest: pytest tests/test_compare.py::test_param_search_updates_models

[MaxBalmus]

Silly question: how do I run the tests locally?

[mastoffel]

Silly question: how do I run the tests locally?

not a silly question! Run them with pytest in the terminal in the AutoEmulate directory:

pytest tests/test_compare.py -> run all tests in the file
pytest tests/test_compare.py::specific_test -> run specific_test in that file
pytest -s tests/test_compare.py -> show print statements in tests

let me know if something doesn't work!

[mastoffel]

@all-contributors please add @MaxBalmus for code, bug

[allcontributors]

@mastoffel

I've put up a pull request to add @MaxBalmus! 🎉

[github-actions]

[blackfmt] _{reported by reviewdog 🐶}

Suggested change

	def test_model_params_equal_wo_param_search(ae, Xy):
	def test_model_params_equal_wo_param_search(ae, Xy):

[github-actions]

[blackfmt] _{reported by reviewdog 🐶}

Suggested change

	assert params_before == params_after
	assert params_before == params_after

[mastoffel]

looks great!