Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Repo Refactor #622

Draft
wants to merge 686 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 129 commits
Commits
Show all changes
686 commits
Select commit Hold shift + click to select a range
bd668c8
Remove outdated runtime files (#1006)
KateUnger Aug 21, 2023
c2e7bfc
Delete include (#1007)
KateUnger Aug 21, 2023
55e8de3
further draft substitution
wmdi Aug 21, 2023
4f0e4d2
format
wmdi Aug 21, 2023
550eb3c
Merge remote-tracking branch 'upstream/repo-refactor' into substitutions
wmdi Aug 22, 2023
58a304c
Added missing iterator include (#1015)
stelleg Aug 23, 2023
77727a7
Rename DeviceSpecificArg to DeviceSpecific (#1009)
KateUnger Aug 23, 2023
fc12962
Per lib build checks (#1022)
aquan9 Aug 23, 2023
cc5837b
minor fix
wmdi Aug 23, 2023
2e16668
Fix test errors in utils
lockshaw Aug 25, 2023
4e3029e
Format
lockshaw Aug 25, 2023
45de184
Add missing constructor
lockshaw Aug 25, 2023
dbf912a
Merge remote-tracking branch 'origin/repo-refactor' into lambda-utils…
lockshaw Aug 25, 2023
d1aa92f
refactor the pattern graph to be OutputLabelledOpenMultiDiGraph
wmdi Aug 25, 2023
a5e111e
format
wmdi Aug 25, 2023
2fb2c7d
minor fix
wmdi Aug 28, 2023
a6ed3b5
Small set of graph fixes
lockshaw Aug 28, 2023
a1bffc5
updates
wmdi Aug 28, 2023
efb0112
Address lambda comments
lockshaw Aug 28, 2023
59839b7
Format
lockshaw Aug 28, 2023
395df2f
Fix strange nccl build issue from -w flag
lockshaw Aug 29, 2023
ee9f7ca
readme for substitutions
wmdi Aug 29, 2023
af92caa
fix the cmake
lambda7xx Aug 29, 2023
dede2c7
implement the constructor in cc
lambda7xx Aug 29, 2023
c65c4ac
format
lambda7xx Aug 29, 2023
08dd3fe
format
wmdi Aug 30, 2023
82e2c2c
check substitution validity
wmdi Aug 30, 2023
8682892
Fix fmt bugs
lockshaw Aug 31, 2023
9051bf6
Merge remote-tracking branch 'refs/remotes/lockshaw/lambda-utils-test…
lockshaw Aug 31, 2023
a0cac29
Revert lambda lib/CMakeLists.txt changes
lockshaw Aug 31, 2023
ae97d59
initialize tests for substitutions
wmdi Aug 31, 2023
c1b5f4a
Fix bug in pcg build
lockshaw Aug 31, 2023
cee33f3
Merge pull request #940 from lockshaw/lambda-utils-testing
lambda7xx Sep 1, 2023
066eb1d
Bump c++ version to 17 (#1067)
lockshaw Sep 1, 2023
e0559cb
Merge remote-tracking branch 'upstream/repo-refactor' into substitutions
wmdi Sep 1, 2023
c2513c6
fix
wmdi Sep 1, 2023
c2b6b04
format
wmdi Sep 1, 2023
21b8549
remove output tensor computation
wmdi Sep 4, 2023
193b44a
start to implement the softmax_kernel
lambda7xx Sep 7, 2023
e776068
add API inferface
lambda7xx Sep 7, 2023
47d7d90
softmax kernels version0.1
lambda7xx Sep 7, 2023
204d0b9
combine
KateUnger Sep 7, 2023
6c32d50
concat
KateUnger Sep 8, 2023
cf7cf05
add empty method
lambda7xx Sep 8, 2023
bd63d52
copy some old code and implement topK version 0.1
lambda7xx Sep 8, 2023
19e20b1
add API method
lambda7xx Sep 8, 2023
06e21c3
transpose version0.1
lambda7xx Sep 8, 2023
244a0ec
start to do the Repartition
lambda7xx Sep 8, 2023
6f0af2a
modify the backward and forward
lambda7xx Sep 8, 2023
c5df897
start to implement the init
lambda7xx Sep 8, 2023
c2780b2
start measure_operator_cost
lambda7xx Sep 8, 2023
a43ff0f
partition version 0.1
lambda7xx Sep 8, 2023
e3b633f
implement get_operator_attrs
wmdi Sep 8, 2023
3041dc7
get parallel operator attributes & minor fix
wmdi Sep 8, 2023
ea1e8e5
format
wmdi Sep 8, 2023
8e28dd0
concat
KateUnger Sep 11, 2023
676f247
combine
KateUnger Sep 12, 2023
98a3c67
format
KateUnger Sep 12, 2023
bc12e40
Conv 2D Op (#1112)
KateUnger Sep 13, 2023
1508050
match open graphs
wmdi Sep 13, 2023
c0d3513
conv2d typo (#1135)
KateUnger Sep 14, 2023
6b64ae6
cuda
KateUnger Sep 14, 2023
2138358
format
KateUnger Sep 14, 2023
d764545
format and cuda
KateUnger Sep 14, 2023
116d162
Merge branch 'repo-refactor' into concat_new
KateUnger Sep 14, 2023
c2bdb7a
Merge branch 'repo-refactor' of https://github.com/flexflow/FlexFlow …
KateUnger Sep 14, 2023
f315b74
delete comments
KateUnger Sep 14, 2023
faa2411
format and fix cc
KateUnger Sep 15, 2023
58a7c62
Merge remote-tracking branch 'upstream/repo-refactor' into substitutions
wmdi Sep 15, 2023
b5ce50c
format
KateUnger Sep 15, 2023
322d945
minor fix
wmdi Sep 17, 2023
4fadbcf
Implement substitutions (#1011)
wmdi Sep 19, 2023
76a86ed
update the softmax
lambda7xx Sep 24, 2023
b17dae9
Serialize jobs in per-lib-checks workflow (#1149)
goliaro Sep 24, 2023
d684286
update the topk
lambda7xx Sep 24, 2023
c114bc4
update the transpose
lambda7xx Sep 24, 2023
cecc7f8
add update and leave task to implement
lambda7xx Sep 24, 2023
06c96ad
update the reduce
lambda7xx Sep 24, 2023
675a22b
use exceptions
lambda7xx Sep 27, 2023
e4c90cf
use exceptions
lambda7xx Sep 27, 2023
986af8f
use exceptions in partition.cc
lambda7xx Sep 27, 2023
dc73c86
fix signatures and bind_arg
KateUnger Sep 28, 2023
cdd5b36
Merge branch 'repo-refactor' into no_op
reyna-abhyankar Sep 30, 2023
990276c
fix
KateUnger Sep 30, 2023
98539d4
fix type error
lambda7xx Sep 30, 2023
4359386
refine the softmax by binding new thing in init
lambda7xx Oct 3, 2023
9e17b38
leave the index
lambda7xx Oct 3, 2023
53a6b2a
format the code
lambda7xx Oct 3, 2023
5d3cee7
format repo-refactor (#1168)
KateUnger Oct 4, 2023
069e94c
Merge branch 'repo-refactor' of https://github.com/flexflow/FlexFlow …
KateUnger Oct 4, 2023
0f083a3
combine
KateUnger Oct 4, 2023
b20afc8
Merge branch 'repo-refactor' of https://github.com/flexflow/FlexFlow …
KateUnger Oct 4, 2023
0b5704c
finish concat
KateUnger Oct 4, 2023
a09b61f
Merge branch 'concat_new' of github.com:KateUnger/FlexFlow into conca…
KateUnger Oct 4, 2023
94a1bd9
concat
KateUnger Oct 4, 2023
0dfbed8
add delete
KateUnger Oct 4, 2023
2b8894b
conv2d typo fix
KateUnger Oct 4, 2023
dd9c2c3
Merge branch 'repo-refactor' of https://github.com/flexflow/FlexFlow …
KateUnger Oct 4, 2023
a79b37c
finish element_binary
KateUnger Oct 4, 2023
63627fa
element_binary
KateUnger Oct 4, 2023
b71b73e
conv2d
KateUnger Oct 4, 2023
931d8e4
Merge branch 'repo-refactor' into no_op
reyna-abhyankar Oct 5, 2023
49f811d
concat
KateUnger Oct 5, 2023
e388534
concat
KateUnger Oct 5, 2023
cdb29f2
finish concat
KateUnger Oct 5, 2023
630da82
Fix signature return
reyna-abhyankar Oct 5, 2023
16ca54c
Comment CHECK_FMTABLE
reyna-abhyankar Oct 5, 2023
2e39499
Add signature for other ops
reyna-abhyankar Oct 5, 2023
8145d0f
Merge pull request #1139 from KateUnger/no_op
reyna-abhyankar Oct 5, 2023
57edb78
Merge branch 'repo-refactor' into concat_new
reyna-abhyankar Oct 5, 2023
cf33e1e
Repo refactor ci (#1083)
lambda7xx Oct 6, 2023
b670a2c
Merge branch 'repo-refactor' into concat_new
reyna-abhyankar Oct 6, 2023
4d1bd0a
Merge branch 'repo-refactor' into element_binary
lockshaw Oct 6, 2023
76d0ee7
Merge branch 'repo-refactor' into repo-refactor-lambda-Repartition-OP
lockshaw Oct 6, 2023
bc70bdb
Merge branch 'repo-refactor' into repo-refactor-lambda-transpose-OP
lockshaw Oct 6, 2023
9a227d4
Merge branch 'repo-refactor' into repo-refactor-lambda-topK-OP
lockshaw Oct 6, 2023
6ec88c5
Merge branch 'repo-refactor' into combine_new
lockshaw Oct 6, 2023
6eb3a16
Merge pull request #1114 from KateUnger/concat_new
reyna-abhyankar Oct 6, 2023
b1806b3
Merge branch 'repo-refactor' into repo-refactor-lambda-softmax-OP
lockshaw Oct 6, 2023
5a38012
Merge branch 'repo-refactor' into conv2d
reyna-abhyankar Oct 6, 2023
7d52ca8
Merge branch 'repo-refactor' into repo-refactor-lambda-Repartition-OP
lockshaw Oct 6, 2023
aadaff7
Merge branch 'repo-refactor' into conv2d
reyna-abhyankar Oct 6, 2023
16c9c7c
Merge pull request #1169 from KateUnger/conv2d
reyna-abhyankar Oct 6, 2023
0376cae
Call fwd sig
reyna-abhyankar Oct 6, 2023
ac38e58
Remove namespace std
reyna-abhyankar Oct 6, 2023
bc35da0
Merge branch 'repo-refactor' into combine_new
reyna-abhyankar Oct 6, 2023
3dc4093
Format
reyna-abhyankar Oct 7, 2023
d926516
Merge pull request #1113 from KateUnger/combine_new
reyna-abhyankar Oct 7, 2023
95dd194
Fix signature
reyna-abhyankar Oct 7, 2023
e34245c
Merge branch 'repo-refactor' into element_binary
reyna-abhyankar Oct 7, 2023
329c0e5
Merge pull request #1136 from KateUnger/element_binary
reyna-abhyankar Oct 7, 2023
277e107
Dropout Op (#1134)
KateUnger Oct 7, 2023
dd39cc8
Flat Operator (#1137)
KateUnger Oct 7, 2023
e1b1be2
Batch Norm Op (#1110)
KateUnger Oct 7, 2023
8c424a9
fix the init_task
lambda7xx Oct 10, 2023
13a0dc5
add topk
lambda7xx Oct 10, 2023
bcd6bf4
Merge branch 'repo-refactor-lambda-topK-OP' of https://github.com/lam…
lambda7xx Oct 10, 2023
9a8c3fb
add allocator to allocate memory for index_ptr
lambda7xx Oct 10, 2023
e7dc8c0
remove the old comment
lambda7xx Oct 10, 2023
58ecd89
Merge branch 'repo-refactor-lambda-transpose-OP' of https://github.co…
lambda7xx Oct 10, 2023
89dd105
fix the input_tensor
lambda7xx Oct 10, 2023
31a55ca
start to update the kernel
lambda7xx Oct 10, 2023
1039372
fix the transpose_kernels cu
lambda7xx Oct 11, 2023
56f931a
fix the transpose op
lambda7xx Oct 11, 2023
e750c93
subsitutions build
wmdi Oct 11, 2023
4daf4d9
fmt
wmdi Oct 11, 2023
77e85ad
Merge remote-tracking branch 'upstream/repo-refactor' into test-subst…
wmdi Oct 12, 2023
a44b0cb
fmt
wmdi Oct 12, 2023
561a2f7
fmt
wmdi Oct 12, 2023
8a1fb69
fix issues caused by merge
wmdi Oct 12, 2023
d096ba1
Purge MOE operators (#1177)
reyna-abhyankar Oct 12, 2023
9ef1113
Merge branch 'repo-refactor' into test-substitution
lockshaw Oct 12, 2023
7eb34b7
Cast Op (#1111)
KateUnger Oct 12, 2023
716e5b9
Merge branch 'repo-refactor' into repo-refactor-lambda-Repartition-OP
reyna-abhyankar Oct 17, 2023
1d0ef22
Update lib/runtime/src/ops/softmax.cc
reyna-abhyankar Oct 17, 2023
6c2751f
Update lib/runtime/src/ops/softmax.cc
reyna-abhyankar Oct 17, 2023
a55c54b
Update lib/runtime/src/ops/softmax.cc
reyna-abhyankar Oct 17, 2023
1771a01
Merge branch 'repo-refactor' into repo-refactor-lambda-softmax-OP
reyna-abhyankar Oct 17, 2023
e31a41d
fix
wmdi Oct 17, 2023
933981e
fix the error
lambda7xx Oct 17, 2023
24c55ef
Merge branch 'repo-refactor-lambda-Repartition-OP' of https://github.…
lambda7xx Oct 17, 2023
d1b38c8
fix the kernel
lambda7xx Oct 17, 2023
c352c52
fix the topk error and add indeices
lambda7xx Oct 17, 2023
b6f4112
fix the error
lambda7xx Oct 17, 2023
66b9591
fix the conflict
lambda7xx Oct 17, 2023
f9549b6
format the code
lambda7xx Oct 17, 2023
6e29568
format the code
lambda7xx Oct 17, 2023
5b97efa
Split OP (#1107)
lambda7xx Oct 18, 2023
1610902
Merge branch 'repo-refactor' into repo-refactor-lambda-softmax-OP
reyna-abhyankar Oct 18, 2023
4d799b7
fix the typo
lambda7xx Oct 18, 2023
fa6a929
fix the typo
lambda7xx Oct 18, 2023
5ddd801
fix the typo
lambda7xx Oct 18, 2023
c62b26c
Merge branch 'repo-refactor' into repo-refactor-lambda-topK-OP
lambda7xx Oct 19, 2023
53ed023
fix the semi
lambda7xx Oct 19, 2023
4ba60f4
fix the format
lambda7xx Oct 19, 2023
f494210
Merge pull request #1106 from lambda7xx/repo-refactor-lambda-softmax-OP
lambda7xx Oct 19, 2023
d8a3276
Merge branch 'repo-refactor' into repo-refactor-lambda-transpose-OP
reyna-abhyankar Oct 19, 2023
4a5b2dc
Replicate OP (#1101)
lambda7xx Oct 19, 2023
5e227c2
Merge branch 'repo-refactor' into repo-refactor-lambda-transpose-OP
reyna-abhyankar Oct 19, 2023
c8adea0
implement some missing functions
wmdi Oct 20, 2023
ecbb20f
format
wmdi Oct 20, 2023
8993f43
Merge branch 'repo-refactor' into repo-refactor-lambda-topK-OP
reyna-abhyankar Oct 23, 2023
04a348f
Merge pull request #1117 from lambda7xx/repo-refactor-lambda-transpos…
lambda7xx Oct 25, 2023
29cf3cd
Merge branch 'repo-refactor' into repo-refactor-lambda-topK-OP
lambda7xx Oct 25, 2023
d75a8f5
Merge pull request #1116 from lambda7xx/repo-refactor-lambda-topK-OP
lambda7xx Oct 25, 2023
28f9aea
Reshape OP (#1100)
lambda7xx Nov 1, 2023
e2ba62e
Reverse OP (#1105)
lambda7xx Nov 1, 2023
f2e1ec7
Merge branch 'repo-refactor' into repo-refactor-lambda-Repartition-OP
reyna-abhyankar Nov 1, 2023
41e15ef
Merge pull request #1119 from lambda7xx/repo-refactor-lambda-Repartit…
lambda7xx Nov 1, 2023
f29461b
Pool2D OP (#1182)
lambda7xx Nov 1, 2023
ed8b4b4
Reduce OP (#1118)
lambda7xx Nov 2, 2023
f01e13b
Reduction OP (#1120)
lambda7xx Nov 2, 2023
9372819
Update submodule (#1212)
reyna-abhyankar Nov 2, 2023
0f3c7de
substitutions tests pass
wmdi Nov 5, 2023
ec0dead
fmt
wmdi Nov 5, 2023
2940646
Batch Matmul Op (#1023)
KateUnger Dec 31, 2023
6e6779e
improve at for OutputLabelledOpenMultiDiGraph
wmdi Jan 6, 2024
0f75405
graph get_ptr fix
wmdi Jan 6, 2024
68a1bf1
fmt
wmdi Jan 6, 2024
ebfde7a
Embedding (#1256)
reyna-abhyankar Jan 7, 2024
53a9daa
update fmt
wmdi Jan 8, 2024
75d8482
Hip kernel fix (#1178)
reyna-abhyankar Jan 17, 2024
d4d9354
remove unnecessary virtual
wmdi Jan 19, 2024
b474d8d
format
wmdi Jan 19, 2024
7a87d11
Merge pull request #1189 from wmdi/test-substitution
wmdi Jan 19, 2024
bf41a4b
linear operator (#1180)
lambda7xx Feb 7, 2024
49d2ff7
LayerNorm OP draft (#1186)
lambda7xx Feb 8, 2024
3237169
Element Unary Op (#1257)
reyna-abhyankar Mar 7, 2024
74c90bf
Remove unnecessary dependencies and allow using external installs (#1…
lockshaw Mar 16, 2024
1b0c32e
Re-merge #1229 (#1346)
lockshaw Mar 30, 2024
041f338
Add external tl-expected via nix, add proj via flake instead of submo…
lockshaw Apr 4, 2024
318152a
Commit proj toml and update proj version (#1350)
lockshaw Apr 5, 2024
8e0625f
Kernel build (#1366)
reyna-abhyankar Apr 17, 2024
6a3abdf
Code Coverage Support (#1380)
Bob-Chen222 May 14, 2024
7723449
Resurrect substitution-to-dot (#1351)
lockshaw May 23, 2024
e03b996
Hip Refactor (#1359)
Bob-Chen222 May 24, 2024
19d0aba
Computation Graph and Builder (#1388)
reyna-abhyankar May 24, 2024
8c78cc2
Local allocator (#1386)
reyna-abhyankar May 28, 2024
aeee8ef
Code Coverage Support (#1396)
Bob-Chen222 May 30, 2024
411bc12
Hip Refactor for optimizer, partition and pool (#1379)
Bob-Chen222 May 31, 2024
e667278
Graph Documentation (#1391)
Marsella8 Jun 1, 2024
fe452db
Local Execution: Op refactor (#1389)
reyna-abhyankar Jun 2, 2024
c97f63f
Hip Refactor for element_binary_kernels, unary kernels, and embedding…
Bob-Chen222 Jun 3, 2024
f93e262
PCG serialization, rapidcheck, dtgen, and shape inference (#1394)
lockshaw Jun 4, 2024
89cbe93
filtering out dtgen related files in code coverage report (#1406)
Bob-Chen222 Jun 4, 2024
af1caf5
refactor for softmax, split, topk, transpose (#1404)
Bob-Chen222 Jun 5, 2024
6fe5dba
Hip refactor for loss, dropout, flat, and gather (#1373)
Bob-Chen222 Jun 5, 2024
be2aad1
hip refactor for reduce, reduction, replicate, reshape and reverse (#…
Bob-Chen222 Jun 5, 2024
35b1741
Hip refactor for attention, batch, combine, cast, conv (#1402)
Bob-Chen222 Jun 5, 2024
8633945
Local backing (#1400)
reyna-abhyankar Jun 6, 2024
1a71d4c
Add `ParallelComputationGraphBuilder` (#1411)
lockshaw Jun 19, 2024
d456733
Local Cost Estimator (#1410)
reyna-abhyankar Jun 20, 2024
7bd12e3
Run `proj dtgen` in CI (#1424)
lockshaw Jun 26, 2024
d30c8f0
Add unit tests for subset of kernels (#1384)
oOTigger Jul 10, 2024
989972a
Implementations for methods for machine_views and associated modules …
Marsella8 Jul 19, 2024
9b5902e
Add `DataflowGraph` and fix part of `substitutions` (#1449)
lockshaw Jul 31, 2024
24f8f81
Implement `get_allowed_machine_views` (#1455)
lockshaw Aug 2, 2024
8f9082f
add compiler to CI (#1457)
wmdi Aug 2, 2024
5e1f349
Local execution tests (#1418)
reyna-abhyankar Aug 22, 2024
1cfb07e
Add Transformer Model PCG (#1453)
hsdfzhsdfz Sep 3, 2024
2b4106f
Re-enable `substitutions` (#1471)
lockshaw Sep 6, 2024
ae8bbf9
Add tool for exporting and visualizing model architectures and SP dec…
lockshaw Sep 14, 2024
6455415
Add interface for differentiating inputs and weights in CG & PCG (#1493)
lockshaw Sep 16, 2024
6475ec7
Add Inception-v3 model (#1495)
Marsella8 Sep 17, 2024
dbb642a
Add Candle Uno Model PCG (#1479)
hsdfzhsdfz Sep 23, 2024
cf96db6
Add BERT model computation graph (#1488)
hsdfzhsdfz Sep 23, 2024
b77598c
Unity device mapping algorithm (#1459)
wmdi Oct 8, 2024
a9d10d7
Fix `concretize_abstract_tensor_set_movement` (#1519)
lockshaw Oct 8, 2024
65c3911
Unordered StridedRectangle, `get_allowed_machine_views` (#1458)
Marsella8 Oct 9, 2024
1d5140d
Utils: Refactor and Test Updates (#1464)
Marsella8 Oct 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
5 changes: 3 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# clang compilation db
compile_commands.json

# Compiled files
/.tools/
/python/flexflow_python
Expand Down Expand Up @@ -25,8 +28,6 @@ dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
Expand Down
18 changes: 18 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,21 @@
[submodule "deps/json"]
path = deps/json
url = https://github.com/nlohmann/json.git
[submodule "deps/spdlog"]
path = deps/spdlog
url = https://github.com/gabime/spdlog.git
[submodule "deps/rapidcheck"]
path = deps/rapidcheck
url = https://github.com/emil-e/rapidcheck.git
[submodule "deps/doctest"]
path = deps/doctest
url = https://github.com/doctest/doctest.git
[submodule "deps/visit_struct"]
path = deps/visit_struct
url = https://github.com/cbeck88/visit_struct.git
[submodule "deps/expected"]
path = deps/expected
url = https://github.com/TartanLlama/expected.git
[submodule "deps/fmt"]
path = deps/fmt
url = https://github.com/fmtlib/fmt.git
2 changes: 2 additions & 0 deletions .ignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/deps/
/substitutions/
Loading