How to improve tsdate accurancy?

[Bai-fengting]

I had a tree sequence consist of 2 different species that do not have reproductive isolation. According to previous report, the divergence time of the two species is about 200 kya. However, the time of root nodes refered by tsdate is mainly distribute around 96,000 ~ 102,000 ya (16000~17000 generations ago, 6 years/generations), which is too earlier than their divergence time. The ages of mutation shared by two species should older than their divergence time. My tsdate pipline are as follows:
tsdate preprocess chr1_infer.trees chr1_infer_preprocess.trees
tsdate date chr1_infer_preprocess.trees chr1_infer_preprocess_dated.trees 10000 -m 1.26e-8 -t 128 -p
What parameters can I adjust to imporve accurancy?
Thanks.

[hyanwong]

What version of tsinfer have you used. Version 0.3.0 made a substantial change that will affect the dating of root nodes, so I suggest you check that you have used this version. If not, I can provide code which will break the root into pieces, which is what the new version does. You can check because (ts.nodes_time == ts.max_time).sum() should be more than one.

@nspope is currently making substantial improvements to tsdate itself, and they may help, especially if you expect the population size of your target species to have changes over time. As part of that, I have made a proposed minor change which affects the oldest timeslice, here: #233. I haven't merged this because on very simple examples it makes dating slightly worse than the theoretical predictions, but we have observed that it generally improves matters on real data, and we will be merging it soon.

You could try installing that PR branch using

python3 -m pip install git+https://github.com/tskit-dev/tsdate.git@refs/pull/233/merge

We can't guarantee the results at the moment, but I think it should improve matters for you. Let me know if it works.

[Bai-fengting]

(oh, and what is the oldest node time before dating, found using undated_ts.max_time)

Oh, Sorry, I made a typo. I used dated_ts.max_root_time to get the time of the oldest root.

[hyanwong]

I have 2,475,927 trees generated by a chromosome. The tsinfer version I uesd is 0.2.3.

Wow, that's a lot of trees. How many sites on that chromosome, and how many samples?

That's the old tsinfer version, so you might improve the dating of the root by following the procedure in #255

[Bai-fengting]

I have 2,475,927 trees generated by a chromosome. The tsinfer version I uesd is 0.2.3.

Wow, that's a lot of trees. How many sites on that chromosome, and how many samples?

That's the old tsinfer version, so you might improve the dating of the root by following the procedure in #255

My data has 5054 individuals and 2,872,764 sites on that chromosome.
Thanks, I have followed the procedure in #255. In order to decrease running time, I choose a shortest chromosome that has 1,536,251 sites, 1,539,182 mutations and 1,340,884 trees for testing. I will feedback the result as soon as possible.

[hyanwong]

So you have a tree change almost every site, even though you only have 5054 individuals (presumably ~10,000 haploid genomes). That seems quite a lot. In the unified genealogy of humans (https://zenodo.org/record/5495535), we had about 20 sites per tree, with ~ 7000 haploid samples. What's the organism? Did you run this with mismatch (i.e. did you provide a recombination rate to tsinfer? Since your number of sites and mutations is almost the same, I suspect you didn't use mismatch at all and you have ~2900 triallelic sites)

[Bai-fengting]

The organism I study is cattle and geneotype data is whole genome resequencing .
No, I did not run with mismatch, because I do not know how to set the optimal ratio. Maybe I should refer to this Issue (tskit-dev/tsinfer#652 (comment)).