Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Release] v0.2.1 #546

Merged
merged 29 commits into from
Oct 26, 2023
Merged

[Release] v0.2.1 #546

merged 29 commits into from
Oct 26, 2023

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Oct 26, 2023

No description provided.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 26, 2023
@vmoens vmoens added the ciflow/binaries/all Build all wheels label Oct 26, 2023
@vmoens vmoens temporarily deployed to pytorchbot-env October 26, 2023 20:46 — with GitHub Actions Inactive
@vmoens vmoens temporarily deployed to pytorchbot-env October 26, 2023 20:46 — with GitHub Actions Inactive
@vmoens vmoens temporarily deployed to pytorchbot-env October 26, 2023 20:46 — with GitHub Actions Inactive
@vmoens vmoens temporarily deployed to pytorchbot-env October 26, 2023 20:46 — with GitHub Actions Inactive
@github-actions
Copy link

github-actions bot commented Oct 26, 2023

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 105. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_plain_set_nested 40.2000μs 23.0203μs 43.4399 KOps/s 44.2633 KOps/s $\color{#d91a1a}-1.86\%$
test_plain_set_stack_nested 0.2710ms 0.2138ms 4.6773 KOps/s 4.6946 KOps/s $\color{#d91a1a}-0.37\%$
test_plain_set_nested_inplace 49.0000μs 26.9815μs 37.0624 KOps/s 37.0675 KOps/s $\color{#d91a1a}-0.01\%$
test_plain_set_stack_nested_inplace 0.2955ms 0.2520ms 3.9684 KOps/s 3.9680 KOps/s $+0.01\%$
test_items 32.1010μs 4.1113μs 243.2329 KOps/s 220.2629 KOps/s $\textbf{\color{#35bf28}+10.43\%}$
test_items_nested 0.4803ms 0.4317ms 2.3164 KOps/s 2.3956 KOps/s $\color{#d91a1a}-3.31\%$
test_items_nested_locked 3.2906ms 0.4423ms 2.2611 KOps/s 2.3799 KOps/s $\color{#d91a1a}-4.99\%$
test_items_nested_leaf 0.3549ms 0.2603ms 3.8413 KOps/s 3.9431 KOps/s $\color{#d91a1a}-2.58\%$
test_items_stack_nested 4.5555ms 2.4248ms 412.3984 Ops/s 434.7827 Ops/s $\textbf{\color{#d91a1a}-5.15\%}$
test_items_stack_nested_leaf 2.3095ms 2.0858ms 479.4385 Ops/s 480.8236 Ops/s $\color{#d91a1a}-0.29\%$
test_items_stack_nested_locked 2.8837ms 1.1378ms 878.8913 Ops/s 881.6696 Ops/s $\color{#d91a1a}-0.32\%$
test_keys 38.3000μs 6.0503μs 165.2823 KOps/s 164.4347 KOps/s $\color{#35bf28}+0.52\%$
test_keys_nested 0.8609ms 0.2144ms 4.6649 KOps/s 4.3183 KOps/s $\textbf{\color{#35bf28}+8.03\%}$
test_keys_nested_locked 0.2988ms 0.2107ms 4.7458 KOps/s 4.7173 KOps/s $\color{#35bf28}+0.60\%$
test_keys_nested_leaf 0.3446ms 0.2050ms 4.8790 KOps/s 4.9140 KOps/s $\color{#d91a1a}-0.71\%$
test_keys_stack_nested 2.1798ms 2.1088ms 474.1955 Ops/s 469.2373 Ops/s $\color{#35bf28}+1.06\%$
test_keys_stack_nested_leaf 2.3253ms 2.1284ms 469.8304 Ops/s 469.8283 Ops/s $+0.00\%$
test_keys_stack_nested_locked 1.0797ms 0.9293ms 1.0760 KOps/s 1.0586 KOps/s $\color{#35bf28}+1.65\%$
test_values 36.0000μs 1.9508μs 512.6132 KOps/s 518.8797 KOps/s $\color{#d91a1a}-1.21\%$
test_values_nested 0.1537ms 73.7098μs 13.5667 KOps/s 13.3300 KOps/s $\color{#35bf28}+1.78\%$
test_values_nested_locked 0.1053ms 73.7483μs 13.5596 KOps/s 13.5675 KOps/s $\color{#d91a1a}-0.06\%$
test_values_nested_leaf 94.7010μs 66.5086μs 15.0357 KOps/s 15.1584 KOps/s $\color{#d91a1a}-0.81\%$
test_values_stack_nested 2.0853ms 1.8413ms 543.1051 Ops/s 540.0210 Ops/s $\color{#35bf28}+0.57\%$
test_values_stack_nested_leaf 1.9258ms 1.8299ms 546.4670 Ops/s 542.2628 Ops/s $\color{#35bf28}+0.78\%$
test_values_stack_nested_locked 0.8004ms 0.7410ms 1.3495 KOps/s 1.3373 KOps/s $\color{#35bf28}+0.91\%$
test_membership 61.9000μs 2.1367μs 468.0139 KOps/s 461.2628 KOps/s $\color{#35bf28}+1.46\%$
test_membership_nested 38.5010μs 4.1262μs 242.3565 KOps/s 242.4620 KOps/s $\color{#d91a1a}-0.04\%$
test_membership_nested_leaf 63.6000μs 4.1279μs 242.2551 KOps/s 243.1117 KOps/s $\color{#d91a1a}-0.35\%$
test_membership_stacked_nested 39.2010μs 17.0406μs 58.6833 KOps/s 59.0637 KOps/s $\color{#d91a1a}-0.64\%$
test_membership_stacked_nested_leaf 44.5000μs 16.9715μs 58.9223 KOps/s 59.7624 KOps/s $\color{#d91a1a}-1.41\%$
test_membership_nested_last 38.0000μs 8.8798μs 112.6147 KOps/s 115.0570 KOps/s $\color{#d91a1a}-2.12\%$
test_membership_nested_leaf_last 32.6000μs 8.7189μs 114.6939 KOps/s 114.7679 KOps/s $\color{#d91a1a}-0.06\%$
test_membership_stacked_nested_last 0.2945ms 0.2636ms 3.7932 KOps/s 3.8760 KOps/s $\color{#d91a1a}-2.14\%$
test_membership_stacked_nested_leaf_last 50.7000μs 19.8055μs 50.4909 KOps/s 50.8974 KOps/s $\color{#d91a1a}-0.80\%$
test_nested_getleaf 43.3000μs 18.0518μs 55.3962 KOps/s 56.4180 KOps/s $\color{#d91a1a}-1.81\%$
test_nested_get 74.3000μs 17.1964μs 58.1516 KOps/s 59.4429 KOps/s $\color{#d91a1a}-2.17\%$
test_stacked_getleaf 1.1368ms 1.0098ms 990.2988 Ops/s 990.5458 Ops/s $\color{#d91a1a}-0.02\%$
test_stacked_get 1.0564ms 0.9656ms 1.0356 KOps/s 1.0413 KOps/s $\color{#d91a1a}-0.55\%$
test_nested_getitemleaf 46.5000μs 18.0782μs 55.3152 KOps/s 56.6015 KOps/s $\color{#d91a1a}-2.27\%$
test_nested_getitem 42.7000μs 17.1202μs 58.4106 KOps/s 59.6495 KOps/s $\color{#d91a1a}-2.08\%$
test_stacked_getitemleaf 1.1369ms 1.0135ms 986.7265 Ops/s 991.8312 Ops/s $\color{#d91a1a}-0.51\%$
test_stacked_getitem 1.6047ms 0.9726ms 1.0282 KOps/s 1.0429 KOps/s $\color{#d91a1a}-1.40\%$
test_lock_nested 78.4047ms 1.8004ms 555.4187 Ops/s 584.1835 Ops/s $\color{#d91a1a}-4.92\%$
test_lock_stack_nested 0.1059s 21.8392ms 45.7892 Ops/s 41.8976 Ops/s $\textbf{\color{#35bf28}+9.29\%}$
test_unlock_nested 76.8562ms 1.8067ms 553.4890 Ops/s 552.2399 Ops/s $\color{#35bf28}+0.23\%$
test_unlock_stack_nested 0.1056s 22.3591ms 44.7246 Ops/s 40.9212 Ops/s $\textbf{\color{#35bf28}+9.29\%}$
test_flatten_speed 1.1910ms 1.1522ms 867.8763 Ops/s 856.9269 Ops/s $\color{#35bf28}+1.28\%$
test_unflatten_speed 2.1933ms 2.0499ms 487.8314 Ops/s 482.4780 Ops/s $\color{#35bf28}+1.11\%$
test_common_ops 1.5732ms 1.2713ms 786.6056 Ops/s 773.3183 Ops/s $\color{#35bf28}+1.72\%$
test_creation 48.9000μs 7.2678μs 137.5924 KOps/s 136.7991 KOps/s $\color{#35bf28}+0.58\%$
test_creation_empty 34.8010μs 15.4416μs 64.7603 KOps/s 64.5234 KOps/s $\color{#35bf28}+0.37\%$
test_creation_nested_1 62.2010μs 28.5058μs 35.0805 KOps/s 35.8214 KOps/s $\color{#d91a1a}-2.07\%$
test_creation_nested_2 61.7000μs 30.5654μs 32.7167 KOps/s 32.9820 KOps/s $\color{#d91a1a}-0.80\%$
test_clone 0.1611ms 28.6055μs 34.9583 KOps/s 34.9033 KOps/s $\color{#35bf28}+0.16\%$
test_getitem[int] 60.4000μs 32.1817μs 31.0736 KOps/s 30.9756 KOps/s $\color{#35bf28}+0.32\%$
test_getitem[slice_int] 0.1013ms 63.2666μs 15.8061 KOps/s 16.0118 KOps/s $\color{#d91a1a}-1.28\%$
test_getitem[range] 0.1688ms 93.4215μs 10.7042 KOps/s 10.4062 KOps/s $\color{#35bf28}+2.86\%$
test_getitem[tuple] 81.8010μs 52.5997μs 19.0115 KOps/s 18.9510 KOps/s $\color{#35bf28}+0.32\%$
test_getitem[list] 0.1679ms 88.1293μs 11.3470 KOps/s 11.0078 KOps/s $\color{#35bf28}+3.08\%$
test_setitem_dim[int] 61.8000μs 40.2633μs 24.8365 KOps/s 25.3228 KOps/s $\color{#d91a1a}-1.92\%$
test_setitem_dim[slice_int] 0.1003ms 70.3028μs 14.2242 KOps/s 14.1553 KOps/s $\color{#35bf28}+0.49\%$
test_setitem_dim[range] 0.2175ms 93.7884μs 10.6623 KOps/s 10.4727 KOps/s $\color{#35bf28}+1.81\%$
test_setitem_dim[tuple] 90.3010μs 59.3308μs 16.8546 KOps/s 16.9232 KOps/s $\color{#d91a1a}-0.40\%$
test_setitem 0.2255ms 36.9298μs 27.0784 KOps/s 27.0111 KOps/s $\color{#35bf28}+0.25\%$
test_set 0.1754ms 35.6127μs 28.0799 KOps/s 28.2267 KOps/s $\color{#d91a1a}-0.52\%$
test_set_shared 0.4465ms 0.2320ms 4.3110 KOps/s 4.2715 KOps/s $\color{#35bf28}+0.92\%$
test_update 0.1897ms 40.2169μs 24.8651 KOps/s 24.7470 KOps/s $\color{#35bf28}+0.48\%$
test_update_nested 0.2672ms 59.1657μs 16.9017 KOps/s 16.7394 KOps/s $\color{#35bf28}+0.97\%$
test_set_nested 0.1904ms 39.6278μs 25.2348 KOps/s 25.4578 KOps/s $\color{#d91a1a}-0.88\%$
test_set_nested_new 0.2200ms 62.2218μs 16.0715 KOps/s 16.2159 KOps/s $\color{#d91a1a}-0.89\%$
test_select 0.3082ms 0.1155ms 8.6565 KOps/s 8.8057 KOps/s $\color{#d91a1a}-1.69\%$
test_unbind_speed 0.9075ms 0.7583ms 1.3187 KOps/s 1.3081 KOps/s $\color{#35bf28}+0.82\%$
test_unbind_speed_stack0 87.9049ms 9.6406ms 103.7280 Ops/s 96.2097 Ops/s $\textbf{\color{#35bf28}+7.81\%}$
test_unbind_speed_stack1 23.3010μs 1.3176μs 758.9277 KOps/s 781.0773 KOps/s $\color{#d91a1a}-2.84\%$
test_creation[device0] 0.6102ms 0.5220ms 1.9156 KOps/s 1.8501 KOps/s $\color{#35bf28}+3.54\%$
test_creation_from_tensor 4.6012ms 0.5871ms 1.7032 KOps/s 1.6851 KOps/s $\color{#35bf28}+1.07\%$
test_add_one[memmap_tensor0] 2.0249ms 37.4351μs 26.7129 KOps/s 26.0772 KOps/s $\color{#35bf28}+2.44\%$
test_contiguous[memmap_tensor0] 37.7010μs 10.0104μs 99.8960 KOps/s 95.0482 KOps/s $\textbf{\color{#35bf28}+5.10\%}$
test_stack[memmap_tensor0] 92.1010μs 30.7005μs 32.5727 KOps/s 32.5588 KOps/s $\color{#35bf28}+0.04\%$
test_memmaptd_index 0.3918ms 0.3548ms 2.8184 KOps/s 2.8068 KOps/s $\color{#35bf28}+0.41\%$
test_memmaptd_index_astensor 1.4849ms 1.4017ms 713.4397 Ops/s 706.0255 Ops/s $\color{#35bf28}+1.05\%$
test_memmaptd_index_op 3.1126ms 3.0376ms 329.2120 Ops/s 326.7405 Ops/s $\color{#35bf28}+0.76\%$
test_reshape_pytree 0.1059ms 37.6767μs 26.5416 KOps/s 26.1144 KOps/s $\color{#35bf28}+1.64\%$
test_reshape_td 76.7010μs 46.9243μs 21.3109 KOps/s 21.1157 KOps/s $\color{#35bf28}+0.92\%$
test_view_pytree 0.1111ms 37.2894μs 26.8173 KOps/s 26.4216 KOps/s $\color{#35bf28}+1.50\%$
test_view_td 28.6010μs 10.0674μs 99.3307 KOps/s 99.2168 KOps/s $\color{#35bf28}+0.11\%$
test_unbind_pytree 92.0010μs 43.1792μs 23.1593 KOps/s 22.7582 KOps/s $\color{#35bf28}+1.76\%$
test_unbind_td 0.2236ms 0.1117ms 8.9543 KOps/s 8.7743 KOps/s $\color{#35bf28}+2.05\%$
test_split_pytree 64.4010μs 42.7520μs 23.3907 KOps/s 23.3248 KOps/s $\color{#35bf28}+0.28\%$
test_split_td 0.9026ms 0.1235ms 8.0962 KOps/s 7.9114 KOps/s $\color{#35bf28}+2.34\%$
test_add_pytree 0.1615ms 53.4716μs 18.7015 KOps/s 18.1590 KOps/s $\color{#35bf28}+2.99\%$
test_add_td 0.1300ms 87.1465μs 11.4749 KOps/s 11.1497 KOps/s $\color{#35bf28}+2.92\%$
test_distributed 47.5000μs 10.4242μs 95.9307 KOps/s 89.1762 KOps/s $\textbf{\color{#35bf28}+7.57\%}$
test_tdmodule 1.6398ms 32.9766μs 30.3245 KOps/s 30.0002 KOps/s $\color{#35bf28}+1.08\%$
test_tdmodule_dispatch 0.3013ms 60.7187μs 16.4694 KOps/s 15.9474 KOps/s $\color{#35bf28}+3.27\%$
test_tdseq 66.9000μs 37.0373μs 26.9998 KOps/s 26.3267 KOps/s $\color{#35bf28}+2.56\%$
test_tdseq_dispatch 0.2229ms 74.8168μs 13.3660 KOps/s 13.1118 KOps/s $\color{#35bf28}+1.94\%$
test_instantiation_functorch 86.7907ms 2.1736ms 460.0634 Ops/s 510.1854 Ops/s $\textbf{\color{#d91a1a}-9.82\%}$
test_instantiation_td 9.7119ms 1.6078ms 621.9866 Ops/s 616.9465 Ops/s $\color{#35bf28}+0.82\%$
test_exec_functorch 0.2748ms 0.2264ms 4.4161 KOps/s 4.2544 KOps/s $\color{#35bf28}+3.80\%$
test_exec_td 0.2677ms 0.2149ms 4.6527 KOps/s 4.4770 KOps/s $\color{#35bf28}+3.92\%$
test_vmap_mlp_speed[True-True] 7.7528ms 1.5284ms 654.2878 Ops/s 700.0495 Ops/s $\textbf{\color{#d91a1a}-6.54\%}$
test_vmap_mlp_speed[True-False] 7.2059ms 0.7648ms 1.3074 KOps/s 1.2830 KOps/s $\color{#35bf28}+1.90\%$
test_vmap_mlp_speed[False-True] 1.5805ms 1.1874ms 842.1525 Ops/s 819.2306 Ops/s $\color{#35bf28}+2.80\%$
test_vmap_mlp_speed[False-False] 6.7835ms 0.5671ms 1.7634 KOps/s 1.7321 KOps/s $\color{#35bf28}+1.81\%$

# Conflicts:
#	.github/scripts/m1_script.sh
@vmoens vmoens merged commit c3caa76 into main Oct 26, 2023
@vmoens vmoens deleted the release/0.2.1 branch October 26, 2023 20:56
@vmoens vmoens temporarily deployed to pytorchbot-env October 26, 2023 20:57 — with GitHub Actions Inactive
@vmoens vmoens temporarily deployed to pytorchbot-env October 26, 2023 20:57 — with GitHub Actions Inactive
@vmoens vmoens temporarily deployed to pytorchbot-env October 26, 2023 20:57 — with GitHub Actions Inactive
@vmoens vmoens temporarily deployed to pytorchbot-env October 26, 2023 20:57 — with GitHub Actions Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ciflow/binaries/all Build all wheels CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants