Skip to content

Issues: alipay/PainlessInferenceAcceleration

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

14 Open 16 Closed
14 Open 16 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

论文里看到Table 8 Inference Latency with Lookahead for vLLM.
#30 opened Sep 3, 2024 by zjjznw123
请问：batch_indices 是什么含义？ LookaheadCache 的put、stream_put函数的最后一个参数ids是什么含义？ class Tree(): 里面的idx代表什么含义？
#29 opened Aug 5, 2024 by handsome-chips
stop_ids does not seem to be taking effect？
#28 opened Jul 4, 2024 by jianyuheng
modeling_qwen attention not use multi branch position ids & attention_mask
#27 opened Jun 5, 2024 by snippetzero
1
lookahead with do_sample=True does not take temperature, top_k, top_p
#26 opened Apr 23, 2024 by learning-chip
2
How is verification done in PAIN?
#25 opened Apr 11, 2024 by jivanph
Do lookahead and repetition_penalty conflict?
#24 opened Apr 7, 2024 by zhanweiw
1
AntRAG
#23 opened Mar 15, 2024 by nrmer
1
Changing naive attention to SDPA gives wrong result for batched llama example
#22 opened Mar 5, 2024 by learning-chip
3
是否支持Qwen 1.5?
#20 opened Mar 3, 2024 by hwang824
Counting how many forward passes/steps were done when using PAIN
#14 opened Jan 30, 2024 by jivanph
4
Consider Support CodeLlama?
#13 opened Jan 30, 2024 by RainYQ
2
In the benchmark studies, how are the draft tokens generated?
#9 opened Jan 24, 2024 by jivanph
9
How the performance VS vLLM inference（vLLM vs Lookahead）
#3 opened Jan 18, 2024 by buptygz
5
ProTip! no:milestone will show everything without a milestone.