Scaling in msprime?

[David-Peede]

Hello all! I am currently trying to benchmark methods to detect introgression utilizing msprime and my overarching goal is to test under a wide parameter space that makes biological sense to a plethora of organisms. However, the issue that I am now running into is that msprime is starting to become quite slow when considering systems that are millions of generations diverged. I know it is common to apply a scaling factor to forward in time simulations and I wanted to confirm or deny that this is applicable to msprime.

Thanks in advance!
-Dave

[grahamgower]

Hi @David-Peede! In a coalescent setting, simulating the ancestry of a sample is agnostic to the time scale of the simulation. So it's probably the simulation of mutations that is the computational constraint in your case. The total number of mutations to be simulated is proportional to mu*T (where T the sum of the branch lengths in the tree(s) and mu is the mutation rate). When scaling a SLiM simulation, one generally divides the time values by the scaling factor and multiplies the mutation rate by the scaling factor, which maintains the same total diversity as an unscaled simulation. But this would imply that mu*T is held constant! So unless I'm missing something, I don't think any kind of scaling will help.

[David-Peede]

Ahh, that makes sense. I guess I may need to just rethink my current pipeline. Currently with parallelizing every replicate by passing a random seed works great for divergence times less than ~100,000 generations, but still takes multiple days to run for deeper divergence times.

Thanks for the clarification @grahamgower!

[petrelharp]

I agree with @grahamgower - another note: a good way to benchmark would be to run your simulations on a range of shorter chromosomes, and then extrapolate to figure out how long the simulations you actually want to run would take. Collecting data from simulations that take minutes to a few hours and looking at the msprime paper for suggestions of what curve to fit to the data should give you good predictions.

[David-Peede]

Thank you @petrelharp! This is very helpful and something I would not have done right off the bat!

[benjeffery]

@petrelharp Should this tip be in one of the tutorials/docs somewhere? Or maybe could even be codified in a function predict_runtime?!

[petrelharp]

Gee, that's a nice idea. At least a tutorial, yes!