Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Runs slowly #1

Closed
ericbarefoot opened this issue Jun 28, 2019 · 1 comment · Fixed by #36
Closed

Runs slowly #1

ericbarefoot opened this issue Jun 28, 2019 · 1 comment · Fixed by #36
Milestone
Acceptable Runtime

Comments

@ericbarefoot
Copy link
Collaborator

This code is very slow. We should do some parts in C.
amoodie pushed a commit that referenced this issue Apr 13, 2020
@elbeejay
Merge pull request #1 from amoodie/develop
abcaadb 
import tests relative
amoodie referenced this issue in amoodie/pyDeltaRCM Apr 18, 2020
@elbeejay
Merge pull request #1 from amoodie/develop
7926aa2 
import tests relative
amoodie referenced this issue in amoodie/pyDeltaRCM Apr 23, 2020
@ericbarefoot
Merge pull request #1 from amoodie/separate_bmi
bf77a5c 
Change logic, add error message, add tests
@amoodie
Copy link
Member

amoodie commented May 7, 2020

I profiled the code to see where to target optimization. The following records one iteration of delta.update(), and does not include any initialization or finalization of the model. The output is sorted by cumtime.

@ericbarefoot

         5243306 function calls (4858106 primitive calls) in 8.873 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    8.873    8.873 {built-in method builtins.exec}
        1    0.000    0.000    8.873    8.873 <string>:1(<module>)
        1    0.000    0.000    8.873    8.873 deltaRCM_driver.py:53(update)
        1    0.000    0.000    8.873    8.873 deltaRCM_tools.py:31(run_one_timestep)
        1    0.038    0.038    6.803    6.803 water_tools.py:143(run_water_iteration)
       29    0.066    0.002    5.612    0.194 water_tools.py:167(<listcomp>)
    37333    1.942    0.000    5.546    0.000 water_tools.py:392(get_weight)
678739/293539    0.658    0.000    3.371    0.000 {built-in method numpy.core._multiarray_umath.implement_array_function}
   192405    0.108    0.000    3.129    0.000 <__array_function__ internals>:2(nansum)
   192405    0.163    0.000    2.880    0.000 nanfunctions.py:559(nansum)
    56420    0.361    0.000    2.282    0.000 shared_tools.py:12(random_pick)
        1    0.000    0.000    1.688    1.688 sed_tools.py:27(sed_route)
     1000    0.605    0.001    1.674    0.002 sed_tools.py:174(sed_parcel)
   192405    0.535    0.000    1.449    0.000 nanfunctions.py:70(_replace_nan)
   193791    0.112    0.000    1.275    0.000 <__array_function__ internals>:2(sum)
   193791    0.182    0.000    1.056    0.000 fromnumeric.py:2092(sum)
        1    0.001    0.001    0.852    0.852 sed_tools.py:277(mud_route)
   193983    0.225    0.000    0.839    0.000 fromnumeric.py:73(_wrapreduction)
        1    0.001    0.001    0.835    0.835 sed_tools.py:235(sand_route)
    58000    0.025    0.000    0.583    0.000 water_tools.py:173(<lambda>)
    37333    0.558    0.000    0.558    0.000 water_tools.py:364(step_update)
   194063    0.514    0.000    0.514    0.000 {method 'reduce' of 'numpy.ufunc' objects}
   192406    0.102    0.000    0.455    0.000 <__array_function__ internals>:2(copyto)
   387895    0.393    0.000    0.393    0.000 {built-in method numpy.array}
        1    0.363    0.363    0.369    0.369 water_tools.py:240(free_surf)
    56450    0.328    0.000    0.328    0.000 {built-in method builtins.sum}
       29    0.012    0.000    0.325    0.011 water_tools.py:201(check_for_loops)
    59420    0.034    0.000    0.308    0.000 <__array_function__ internals>:2(cumsum)
       29    0.300    0.010    0.306    0.011 water_tools.py:203(<listcomp>)
    59420    0.041    0.000    0.240    0.000 fromnumeric.py:2405(cumsum)
    59420    0.239    0.000    0.239    0.000 {method 'uniform' of 'numpy.random.mtrand.RandomState' objects}
    59805    0.033    0.000    0.199    0.000 fromnumeric.py:55(_wrapfunc)
    58000    0.024    0.000    0.195    0.000 water_tools.py:170(<lambda>)
    37333    0.063    0.000    0.171    0.000 water_tools.py:382(calculate_new_ind)
     9556    0.064    0.000    0.163    0.000 sed_tools.py:138(mud_dep_ero)
    59420    0.153    0.000    0.153    0.000 {method 'cumsum' of 'numpy.ndarray' objects}
     8498    0.107    0.000    0.134    0.000 sed_tools.py:41(deposit)
   192405    0.049    0.000    0.112    0.000 _asarray.py:88(asanyarray)
    37523    0.015    0.000    0.109    0.000 <__array_function__ internals>:2(ravel_multi_index)
     9531    0.049    0.000    0.106    0.000 sed_tools.py:100(sand_dep_ero)
   193983    0.079    0.000    0.079    0.000 fromnumeric.py:74(<dictcomp>)
    59420    0.069    0.000    0.069    0.000 {method 'searchsorted' of 'numpy.ndarray' objects}
   193797    0.036    0.000    0.036    0.000 {built-in method builtins.isinstance}
     3000    0.006    0.000    0.031    0.000 shared_tools.py:30(random_pick_inlet)
   192435    0.027    0.000    0.027    0.000 {built-in method builtins.issubclass}
     9463    0.026    0.000    0.026    0.000 sed_tools.py:86(update_u)
   192405    0.025    0.000    0.025    0.000 nanfunctions.py:555(_nansum_dispatcher)
   192406    0.022    0.000    0.022    0.000 multiarray.py:1043(copyto)
        1    0.001    0.001    0.022    0.022 water_tools.py:146(<listcomp>)
   193983    0.022    0.000    0.022    0.000 {method 'items' of 'dict' objects}
   193791    0.019    0.000    0.019    0.000 fromnumeric.py:2087(_sum_dispatcher)
      965    0.013    0.000    0.016    0.000 sed_tools.py:63(erode)
    59997    0.013    0.000    0.013    0.000 {built-in method builtins.getattr}
        1    0.000    0.000    0.013    0.013 water_tools.py:112(finalize_water_iteration)
        1    0.005    0.005    0.012    0.012 water_tools.py:308(update_water)
       29    0.011    0.000    0.011    0.000 water_tools.py:164(<listcomp>)
    19306    0.010    0.000    0.010    0.000 {built-in method builtins.min}
     2219    0.001    0.000    0.007    0.000 <__array_function__ internals>:2(unravel_index)
    59420    0.007    0.000    0.007    0.000 fromnumeric.py:2401(_cumsum_dispatcher)
   120865    0.007    0.000    0.007    0.000 {built-in method builtins.len}
        1    0.000    0.000    0.006    0.006 sed_tools.py:241(<listcomp>)
        1    0.000    0.000    0.005    0.005 sed_tools.py:283(<listcomp>)
      385    0.000    0.000    0.002    0.000 <__array_function__ internals>:2(round_)
      192    0.000    0.000    0.002    0.000 <__array_function__ internals>:2(amax)
    37523    0.002    0.000    0.002    0.000 multiarray.py:929(ravel_multi_index)
      192    0.000    0.000    0.002    0.000 fromnumeric.py:2551(amax)
      385    0.000    0.000    0.002    0.000 fromnumeric.py:3590(round_)
      385    0.000    0.000    0.002    0.000 <__array_function__ internals>:2(around)
      385    0.000    0.000    0.001    0.000 fromnumeric.py:3130(around)
       29    0.001    0.000    0.001    0.000 water_tools.py:189(check_for_boundary)
        2    0.000    0.000    0.001    0.000 water_tools.py:78(flooding_correction)
      380    0.001    0.000    0.001    0.000 {method 'round' of 'numpy.generic' objects}
     3000    0.001    0.000    0.001    0.000 shared_tools.py:39(<listcomp>)
        1    0.000    0.000    0.001    0.001 water_tools.py:126(init_water_iteration)
        5    0.000    0.000    0.001    0.000 <__array_function__ internals>:2(pad)
        5    0.000    0.000    0.000    0.000 arraypad.py:534(pad)
        1    0.000    0.000    0.000    0.000 deltaRCM_tools.py:54(finalize_timestep)
        1    0.000    0.000    0.000    0.000 sed_tools.py:257(topo_diffusion)
      412    0.000    0.000    0.000    0.000 {built-in method builtins.max}
     2219    0.000    0.000    0.000    0.000 multiarray.py:990(unravel_index)
        6    0.000    0.000    0.000    0.000 filters.py:652(convolve)
        6    0.000    0.000    0.000    0.000 filters.py:596(_correlate_or_convolve)
       80    0.000    0.000    0.000    0.000 {method 'any' of 'numpy.ndarray' objects}
        5    0.000    0.000    0.000    0.000 arraypad.py:459(_as_pairs)
       80    0.000    0.000    0.000    0.000 _methods.py:44(_any)
        6    0.000    0.000    0.000    0.000 _ni_support.py:71(_get_output)
        4    0.000    0.000    0.000    0.000 water_tools.py:429(build_weight_array)
        5    0.000    0.000    0.000    0.000 arraypad.py:88(_pad_simple)
        1    0.000    0.000    0.000    0.000 water_tools.py:28(update_flow_field)
        6    0.000    0.000    0.000    0.000 {built-in method scipy.ndimage._nd_image.correlate}
        2    0.000    0.000    0.000    0.000 water_tools.py:470(get_wet_mask_nh)
      770    0.000    0.000    0.000    0.000 fromnumeric.py:3126(_around_dispatcher)
       10    0.000    0.000    0.000    0.000 arraypad.py:156(_get_edges)
        6    0.000    0.000    0.000    0.000 _dtype.py:333(_name_get)
       10    0.000    0.000    0.000    0.000 arraypad.py:131(_set_pad_area)
        1    0.000    0.000    0.000    0.000 water_tools.py:63(update_velocity_field)
       22    0.000    0.000    0.000    0.000 _asarray.py:16(asarray)
       11    0.000    0.000    0.000    0.000 {built-in method numpy.zeros}
        6    0.000    0.000    0.000    0.000 _dtype.py:319(_name_includes_bit_suffix)
       29    0.000    0.000    0.000    0.000 water_tools.py:481(check_size_of_indices_matrix)
        6    0.000    0.000    0.000    0.000 numerictypes.py:365(issubdtype)
       10    0.000    0.000    0.000    0.000 arraypad.py:60(_view_roi)
      192    0.000    0.000    0.000    0.000 fromnumeric.py:2546(_amax_dispatcher)
       12    0.000    0.000    0.000    0.000 numerictypes.py:293(issubclass_)
        5    0.000    0.000    0.000    0.000 {method 'astype' of 'numpy.generic' objects}
        6    0.000    0.000    0.000    0.000 _ni_support.py:55(_normalize_sequence)
        6    0.000    0.000    0.000    0.000 <__array_function__ internals>:2(iscomplexobj)
        2    0.000    0.000    0.000    0.000 <__array_function__ internals>:2(where)
        6    0.000    0.000    0.000    0.000 {built-in method numpy.empty}
       40    0.000    0.000    0.000    0.000 arraypad.py:35(_slice_at_axis)
       15    0.000    0.000    0.000    0.000 arraypad.py:112(<genexpr>)
        1    0.000    0.000    0.000    0.000 numeric.py:159(ones)
        5    0.000    0.000    0.000    0.000 {method 'ravel' of 'numpy.generic' objects}
       15    0.000    0.000    0.000    0.000 arraypad.py:123(<genexpr>)
        6    0.000    0.000    0.000    0.000 type_check.py:280(iscomplexobj)
        6    0.000    0.000    0.000    0.000 {method 'format' of 'str' objects}
       12    0.000    0.000    0.000    0.000 filters.py:50(_invalid_origin)
        5    0.000    0.000    0.000    0.000 {method 'round' of 'numpy.ndarray' objects}
        6    0.000    0.000    0.000    0.000 filters.py:603(<listcomp>)
        6    0.000    0.000    0.000    0.000 _dtype.py:36(_kind_name)
        6    0.000    0.000    0.000    0.000 {method 'copy' of 'numpy.ndarray' objects}
        6    0.000    0.000    0.000    0.000 {built-in method builtins.hasattr}
        6    0.000    0.000    0.000    0.000 _ni_support.py:38(_extend_mode_to_code)
        1    0.000    0.000    0.000    0.000 deltaRCM_tools.py:86(record_stratigraphy)
        1    0.000    0.000    0.000    0.000 deltaRCM_driver.py:81(time_step)
        1    0.000    0.000    0.000    0.000 deltaRCM_tools.py:152(apply_subsidence)
        1    0.000    0.000    0.000    0.000 deltaRCM_tools.py:168(output_data)
        6    0.000    0.000    0.000    0.000 type_check.py:207(_is_type_dispatcher)
        2    0.000    0.000    0.000    0.000 multiarray.py:312(where)
        5    0.000    0.000    0.000    0.000 {built-in method builtins.callable}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        5    0.000    0.000    0.000    0.000 arraypad.py:526(_pad_dispatcher)

@amoodie amoodie mentioned this issue May 7, 2020
Closed
@ericbarefoot ericbarefoot linked a pull request May 22, 2020 that will close this issue
BUMBANUMBA #36
Merged
@ericbarefoot ericbarefoot added this to the Acceptable Runtime milestone Jun 2, 2020
@ericbarefoot ericbarefoot mentioned this issue Jun 3, 2020
Closed
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
Acceptable Runtime
Development

Successfully merging a pull request may close this issue.

BUMBANUMBA ericbarefoot/pyDeltaRCM
2 participants
@amoodie @ericbarefoot