Code

Selective merged prefill#643

Code

Open

xuechendi wants to merge 25 commits intoHabanaAI:mlperf_featuresHabanaAI/vllm-fork:mlperf_featuresfrom xuechendi:selective_merged_prefillxuechendi/vllm-fork:selective_merged_prefillCopy head branch name to clipboard

+428-31

Commits

Commits on Dec 19, 2024

• update benchmark with bucketing strategy

  Show description for 997a10a
  xuechendi

  committed
  997a10a

  Copy full SHA for 997a10a

• merge input tokens

  Show description for 79c8b8e
  xuechendi

  committed
  79c8b8e

  Copy full SHA for 79c8b8e

• Enable merged prefill

  Show description for 552e294
  xuechendi

  committed
  552e294

  Copy full SHA for 552e294

• accuracy issue is fixed

  Show description for bd87512
  xuechendi

  committed
  bd87512

  Copy full SHA for bd87512

• use logical_and_

  Show description for 1caf266
  xuechendi

  committed
  1caf266

  Copy full SHA for 1caf266

Commits on Dec 20, 2024

• update

  Show description for c528736
  xuechendi

  committed
  c528736

  Copy full SHA for c528736

• update benchmark

  Show description for 510722e
  xuechendi

  committed
  510722e

  Copy full SHA for 510722e

• update

  Show description for f6c0c84
  xuechendi

  committed
  f6c0c84

  Copy full SHA for f6c0c84

• rewrite split function to make fp8 work

  Show description for b133542
  xuechendi

  committed
  b133542

  Copy full SHA for b133542

• Fix accuracy issue

  Show description for 2d6ceb9
  xuechendi

  committed
  2d6ceb9

  Copy full SHA for 2d6ceb9

Commits on Jan 6, 2025

• clean up codes in hpu-attn

  Show description for 116dc6c
  xuechendi

  committed
  116dc6c

  Copy full SHA for 116dc6c

Commits on Jan 7, 2025

• update warming up strategy for merged_prefill

  Show description for fade386
  xuechendi

  committed
  fade386

  Copy full SHA for fade386

• fix an accuracy issue caused by selected_token_index

  Show description for 911f14b
  xuechendi

  committed
  911f14b

  Copy full SHA for 911f14b

• move tolist to llamamodel fwd

  Show description for 612abed
  xuechendi

  committed
  612abed

  Copy full SHA for 612abed

Commits on Jan 8, 2025

• use index_put with full block_indices

  Show description for a3602f2
  xuechendi

  committed
  a3602f2

  Copy full SHA for a3602f2

• use fixed length for selected_token_indices

  Show description for 97ea32b
  xuechendi

  committed
  97ea32b

  Copy full SHA for 97ea32b

• clean up hpu_attn codes

  Show description for 11ffc2f
  xuechendi

  committed
  11ffc2f

  Copy full SHA for 11ffc2f

• add CPU version attn_mask preparation

  Show description for 2ef08d6
  xuechendi

  committed
  2ef08d6

  Copy full SHA for 2ef08d6

• update CPU version attn_bias prepration and clean up

  Show description for 405243a
  xuechendi

  committed
  405243a

  Copy full SHA for 405243a

• 111

  Show description for 774c13c
  xuechendi

  committed
  774c13c

  Copy full SHA for 774c13c

Commits on Jan 9, 2025

• update hpu attn

  Show description for 3826c1d
  xuechendi

  committed
  3826c1d

  Copy full SHA for 3826c1d

• update

  Show description for a906f36
  xuechendi

  committed
  a906f36

  Copy full SHA for a906f36

• clean up codes

  Show description for 67a2923
  xuechendi

  committed
  67a2923

  Copy full SHA for 67a2923

Commits on Jan 10, 2025

• Fix fusedSDPA fp8 70B issue

  Show description for a2d3007
  xuechendi

  committed
  a2d3007

  Copy full SHA for a2d3007

Commits on Jan 12, 2025

• increase absolute value for attn_bias to get better accuracy

  Show description for c2d4c33
  xuechendi

  committed
  c2d4c33

  Copy full SHA for c2d4c33