Is it possible to run FlexFlow only like a simulator?

[OneMoreProblem]

I found a machine_config_example can I use it for a simulator to imitate a real execution, and predict execution time?

For example, I have a machine without GPU. Is it possible to simulate the execution time on a machine with 16 GPUs, with different parallelism strategies? How can I access simulation data like a memory, and forward and backward execution?

[lockshaw]

It is possible as long as you have access to a single GPU (we use the GPU to profile kernel execution times) . We're working on providing interface for users to provide an analytical cost model for GPU execution, but as of now this is not stable.

[cnjsdfcy]

hi @lockshaw , in which branch the cost model setting interface is working on ? thx~

[Rashad-CSU]

It is possible as long as you have access to a single GPU (we use the GPU to profile kernel execution times) . We're working on providing interface for users to provide an analytical cost model for GPU execution, but as of now this is not stable.

Hi,

Currently, I am using "flexflow_python "$FF_HOME"/examples/python/native/cifar10_cnn.py -ll:gpu 1 -ll:fsize 30000 -ll:zsize 3000" to run a model on one GPU. My machine has only one. What command should I use to simulate it for several (e.g. 16) GPUs, and create a tree structure?

Thanks,
Rashad

[lockshaw]

hi @lockshaw , in which branch the cost model setting interface is working on ? thx~

@cnjsdfcy #622

[lockshaw]

It is possible as long as you have access to a single GPU (we use the GPU to profile kernel execution times) . We're working on providing interface for users to provide an analytical cost model for GPU execution, but as of now this is not stable.

Hi,

Currently, I am using "flexflow_python "$FF_HOME"/examples/python/native/cifar10_cnn.py -ll:gpu 1 -ll:fsize 30000 -ll:zsize 3000" to run a model on one GPU. My machine has only one. What command should I use to simulate it for several (e.g. 16) GPUs, and create a tree structure?

Thanks, Rashad

AFAIK there isn't a convenient way to do this on #475, which I'm guessing you're using (unless @goliaro knows otherwise). We're working on making it easier to use the simulator as a separate component on #622, but it's definitely not ready for public usage yet. If you're interested in this I recommend following that branch as we're hoping to have it merged in the next couple months.