append_kv_cache's documentation is out of date

[reyoung]

I noticed in the main branch, that the append_kv_cache does not take append_indptr any more. But the document reference append_indptr. It is a little bit confusing.

And I cannot find the batch indices documentation for a detailed situation.

• Is this tensor [0, 0, 2, 3, 5, 5] a valid batch indices?
  • It happens when some prefilling sample's kv are fully cached in a prefilling batch.
  • I just gathered the input tensor before append_kv_cache when append_kv_cache takes append_indptr. Is this gathering unnecessary in the new API?
  • Can I specify batch_indices = -1 to skip some append_kv_cache ?

[zhyncs]

Hi @reyoung Recently, FlashInfer has been undergoing extensive refactoring, with tens of thousands of lines of code to be updated soon. The documentation hasn't been updated in time, and it will be once @yzh119 has the bandwidth.