Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import scipy.special._cdflib in build #723

Merged
merged 4 commits into from
Apr 3, 2024
Merged

Conversation

rly
Copy link
Collaborator

@rly rly commented Apr 3, 2024

Fix #720

The log says:

771 DEBUG: Collecting submodules for scipy.special.cython_special
1152 DEBUG: collect_submodules - scipy.special.cython_special is not a package.
1320 DEBUG: Collecting submodules for scipy.special._cdflib
1486 DEBUG: collect_submodules - scipy.special._cdflib is not a package

Alternative solution:

hiddenimports = [
    'email_validator',
    *collect_submodules('scipy.special'),
    *os.path.join(os.path.dirname(scipy.__file__), '.libs')
]

This results in many more imports:

680 DEBUG: Collecting submodules for scipy.special
186 DEBUG: collect_submodules - scanning (sub)package scipy.special
514 DEBUG: collect_submodules - scanning (sub)package scipy.special in location(s): ['/Users/rly/mambaforge/envs/nwb-guide/lib/python3.9/site-packages/scipy/special']
515 DEBUG: collect_submodules - scanning (sub)package scipy.special.tests
516 DEBUG: collect_submodules - scanning (sub)package scipy.special.tests in location(s): ['/Users/rly/mambaforge/envs/nwb-guide/lib/python3.9/site-packages/scipy/special/tests']
516 DEBUG: collect_submodules - scanning (sub)package scipy.special.tests.data
517 DEBUG: collect_submodules - scanning (sub)package scipy.special.tests.data in location(s): ['/Users/rly/mambaforge/envs/nwb-guide/lib/python3.9/site-packages/scipy/special/tests/data']
517 DEBUG: collect_submodules - scanning (sub)package scipy.special._precompute
517 DEBUG: collect_submodules - scanning (sub)package scipy.special._precompute in location(s): ['/Users/rly/mambaforge/envs/nwb-guide/lib/python3.9/site-packages/scipy/special/_precompute']
1405 DEBUG: collect_submodules - found submodules: ['scipy.special', 'scipy.special._add_newdocs', 'scipy.special._basic', 'scipy.special._cdflib', 'scipy.special._comb', 'scipy.special._ellip_harm', 'scipy.special._ellip_harm_2', 'scipy.special._lambertw', 'scipy.special._logsumexp', 'scipy.special._mptestutils', 'scipy.special._orthogonal', 'scipy.special._precompute', 'scipy.special._precompute.cosine_cdf', 'scipy.special._precompute.expn_asy', 'scipy.special._precompute.gammainc_asy', 'scipy.special._precompute.gammainc_data', 'scipy.special._precompute.lambertw', 'scipy.special._precompute.loggamma', 'scipy.special._precompute.struve_convergence', 'scipy.special._precompute.utils', 'scipy.special._precompute.wright_bessel', 'scipy.special._precompute.wright_bessel_data', 'scipy.special._precompute.wrightomega', 'scipy.special._precompute.zetac', 'scipy.special._sf_error', 'scipy.special._specfun', 'scipy.special._spfun_stats', 'scipy.special._spherical_bessel', 'scipy.special._support_alternative_backends', 'scipy.special._test_internal', 'scipy.special._testutils', 'scipy.special._ufuncs', 'scipy.special._ufuncs_cxx', 'scipy.special.add_newdocs', 'scipy.special.basic', 'scipy.special.cython_special', 'scipy.special.orthogonal', 'scipy.special.sf_error', 'scipy.special.specfun', 'scipy.special.spfun_stats', 'scipy.special.tests', 'scipy.special.tests.data', 'scipy.special.tests.test_basic', 'scipy.special.tests.test_bdtr', 'scipy.special.tests.test_boxcox', 'scipy.special.tests.test_cdflib', 'scipy.special.tests.test_cdft_asymptotic', 'scipy.special.tests.test_cosine_distr', 'scipy.special.tests.test_cython_special', 'scipy.special.tests.test_data', 'scipy.special.tests.test_dd', 'scipy.special.tests.test_digamma', 'scipy.special.tests.test_ellip_harm', 'scipy.special.tests.test_erfinv', 'scipy.special.tests.test_exponential_integrals', 'scipy.special.tests.test_faddeeva', 'scipy.special.tests.test_gamma', 'scipy.special.tests.test_gammainc', 'scipy.special.tests.test_hyp2f1', 'scipy.special.tests.test_hypergeometric', 'scipy.special.tests.test_kolmogorov', 'scipy.special.tests.test_lambertw', 'scipy.special.tests.test_log_softmax', 'scipy.special.tests.test_loggamma', 'scipy.special.tests.test_logit', 'scipy.special.tests.test_logsumexp', 'scipy.special.tests.test_mpmath', 'scipy.special.tests.test_nan_inputs', 'scipy.special.tests.test_ndtr', 'scipy.special.tests.test_ndtri_exp', 'scipy.special.tests.test_orthogonal', 'scipy.special.tests.test_orthogonal_eval', 'scipy.special.tests.test_owens_t', 'scipy.special.tests.test_pcf', 'scipy.special.tests.test_pdtr', 'scipy.special.tests.test_powm1', 'scipy.special.tests.test_precompute_expn_asy', 'scipy.special.tests.test_precompute_gammainc', 'scipy.special.tests.test_precompute_utils', 'scipy.special.tests.test_round', 'scipy.special.tests.test_sf_error', 'scipy.special.tests.test_sici', 'scipy.special.tests.test_specfun', 'scipy.special.tests.test_spence', 'scipy.special.tests.test_spfun_stats', 'scipy.special.tests.test_sph_harm', 'scipy.special.tests.test_spherical_bessel', 'scipy.special.tests.test_support_alternative_backends', 'scipy.special.tests.test_trig', 'scipy.special.tests.test_wright_bessel', 'scipy.special.tests.test_wrightomega', 'scipy.special.tests.test_zeta']

Generating the test data works fine with this change as well. It seems like we do not need to import all of these, but maybe it is better to preemptively include them just in case they are used by some other package?

App size is 1,379,222,782 bytes with importing just scipy.special._cdflib and 1,381,365,267 bytes importing all scipy.special, a difference of 2.14 MB.

@CodyCBakerPhD CodyCBakerPhD enabled auto-merge April 3, 2024 19:11
@CodyCBakerPhD CodyCBakerPhD merged commit d70d73a into main Apr 3, 2024
14 checks passed
@CodyCBakerPhD CodyCBakerPhD deleted the fix_scipy_import_build branch April 3, 2024 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Generating test data in local Mac build has error
3 participants