Code

merge upstream#46

Code

Merged

l3utterfly merged 103 commits intolayla-buildl3utterfly/llama.cpp:layla-buildfrom mergel3utterfly/llama.cpp:mergeCopy head branch name to clipboardNov 27, 2024

+12,412-12,807

Commits

Commits on Nov 16, 2024

• scripts : fix missing key in compare-llama-bench.py (ggerganov#10332)

  ggerganov

  authored
  f245cc2

  Copy full SHA for f245cc2

• server: (web UI) Add samplers sequence customization (ggerganov#10255)

  Show description for bcdb7a2
  

  MaggotHATE

  and

  ngxson

  authored
  bcdb7a2

  Copy full SHA for bcdb7a2

• make : auto-determine dependencies (#0)

  ggerganov

  committed
  8ee0d09

  Copy full SHA for 8ee0d09

• llamafile : fix include path (#0)

  Show description for db4cfd5
  ggerganov

  committed
  db4cfd5

  Copy full SHA for db4cfd5

• llama/ex: remove --logdir argument (ggerganov#10339)

  JohannesGaessler

  authored
  4e54be0

  Copy full SHA for 4e54be0

• docs : vulkan build instructions to use git bash mingw64 (ggerganov#10303)

  FirstTimeEZ

  authored
  0fff7fd

  Copy full SHA for 0fff7fd

Commits on Nov 17, 2024

• scripts : update sync

  ggerganov

  committed
  5c9a8b2

  Copy full SHA for 5c9a8b2

• ggml: new optimization interface (ggml/988)

  

  JohannesGaessler

  authored and

  ggerganov

  committed
  8a43e94

  Copy full SHA for 8a43e94

• ggml : fix compile warnings (#0)

  Show description for 68fcb47
  ggerganov

  committed
  68fcb47

  Copy full SHA for 68fcb47

• tests : remove test-grad0

  ggerganov

  committed
  84274a1

  Copy full SHA for 84274a1

• make : add ggml-opt (#0)

  Show description for a4200ca
  ggerganov

  committed
  a4200ca

  Copy full SHA for a4200ca

• ggml : adapt AMX to tensor->grad removal (#0)

  Show description for 5d9e599
  ggerganov

  committed
  5d9e599

  Copy full SHA for 5d9e599

• ggml : inttypes.h -> cinttypes (#0)

  Show description for 24203e9
  ggerganov

  committed
  24203e9

  Copy full SHA for 24203e9

• ggml : fix possible buffer use after free in sched reserve (ggerganov#9930)

  slaren

  authored
  eda7e1d

  Copy full SHA for eda7e1d

• CMake: default to -arch=native for CUDA build (ggerganov#10320)

  JohannesGaessler

  authored
  467576b

  Copy full SHA for 467576b

• CUDA: remove DMMV, consolidate F16 mult mat vec (ggerganov#10318)

  JohannesGaessler

  authored
  c3ea58a

  Copy full SHA for c3ea58a

• ggml : fix undefined reference to 'getcpu' (ggerganov#10354)

  Show description for a431782
  FirstTimeEZ

  authored
  a431782

  Copy full SHA for a431782

• metal : refactor kernel args into structs (ggerganov#10238)

  Show description for cf32a9b
  ggerganov

  authored
  cf32a9b

  Copy full SHA for cf32a9b

• gitignore : ignore local run scripts [no ci]

  ggerganov

  committed
  20a780c

  Copy full SHA for 20a780c

• llama : only use default buffer types for the KV cache (ggerganov#10358)

  slaren

  authored
  be5cacc

  Copy full SHA for be5cacc

• CMake: fix typo in comment [no ci] (ggerganov#10360)

  JohannesGaessler

  authored
  ce2e59b

  Copy full SHA for ce2e59b

• CUDA: fix MMV kernel being used for FP16 src1 (ggerganov#10357)

  JohannesGaessler

  authored
  76e9e58

  Copy full SHA for 76e9e58

• docker: use GGML_NATIVE=OFF (ggerganov#10368)

  JohannesGaessler

  authored
  75207b3

  Copy full SHA for 75207b3

Commits on Nov 18, 2024

• Vulkan: Fix device info output format specifiers (ggerganov#10366)

  Show description for 9b75f03
  0cc4m

  authored
  9b75f03

  Copy full SHA for 9b75f03

• flake.lock: Update (ggerganov#10346)

  Show description for 2eb76b2
  

  ggerganov

  and

  github-actions[bot]

  authored
  2eb76b2

  Copy full SHA for 2eb76b2

• vulkan: remove use of null initializer (ggerganov#10372)

  Show description for f139d2e
  jeffbolznv

  authored
  f139d2e

  Copy full SHA for f139d2e

• Skip searching root path for cross-compile builds (ggerganov#10383)

  bandoti

  authored
  531cb1c

  Copy full SHA for 531cb1c

• cuda : only use native when supported by cmake (ggerganov#10389)

  slaren

  authored
  d3481e6

  Copy full SHA for d3481e6

Commits on Nov 19, 2024

• sycl: Revert MUL_MAT_OP support changes (ggerganov#10385)

  Alcpz

  authored
  557924f

  Copy full SHA for 557924f

• vulkan: Optimize soft_max (ggerganov#10301)

  Show description for b3e5859
  jeffbolznv

  authored
  b3e5859

  Copy full SHA for b3e5859

• sycl : Add option to set the SYCL architecture for all targets (ggerganov#10266)

  Show description for 2a1507c
  Rbiessy

  authored
  2a1507c

  Copy full SHA for 2a1507c

• llama : add OLMo November 2024 support (ggerganov#10394)

  Show description for a88ad00
  2015aroras

  authored
  a88ad00

  Copy full SHA for a88ad00

• llama : add check for KV cache shifts (ggerganov#10401)

  Show description for 8e752a7
  ggerganov

  authored
  8e752a7

  Copy full SHA for 8e752a7

• cuda : fix CUDA_FLAGS not being applied (ggerganov#10403)

  slaren

  authored
  3ee6382

  Copy full SHA for 3ee6382

• Add required ggml-base and backend libs to cmake pkg (ggerganov#10407)

  bandoti

  authored
  2a11b6b

  Copy full SHA for 2a11b6b

• cmake: force MSVC compiler charset to utf-8 (ggerganov#9989)

  shou692199

  authored
  342397d

  Copy full SHA for 342397d

• metal : add GGML_UNARY_OP_ELU kernel (ggml/1018)

  

  PABannier

  authored and

  ggerganov

  committed
  12b0ad9

  Copy full SHA for 12b0ad9

• metal : fox offset integer overflows in im2col (ggml/1015)

  Show description for 611fabd
  

  pminev

  authored and

  ggerganov

  committed
  611fabd

  Copy full SHA for 611fabd

• sync : ggml

  ggerganov

  committed
  9fe0fb0

  Copy full SHA for 9fe0fb0

• add cmake rvv support (ggerganov#10411)

  lhpqaq

  authored
  42ae10b

  Copy full SHA for 42ae10b

• Fix missing file renames in Makefile due to changes in commit ae8de6d (ggerganov#10413)

  avdg

  authored
  3952a22

  Copy full SHA for 3952a22

Commits on Nov 20, 2024

• update rel to 4040 (ggerganov#10395)

  Show description for ad21c9e
  

  NeoZhangJianyu

  and

  arthw

  authored
  ad21c9e

  Copy full SHA for ad21c9e

• vulkan: further optimize mul_mat_vec using larger loads (ggerganov#10387)

  Show description for 1bacb9f
  jeffbolznv

  authored
  1bacb9f

  Copy full SHA for 1bacb9f

• vulkan: copy iq4_nl LUT into shared memory (ggerganov#10409)

  jeffbolznv

  authored
  8fd4b7f

  Copy full SHA for 8fd4b7f

• llama : add .clang-format file (ggerganov#10415)

  slaren

  authored
  fab5d30

  Copy full SHA for fab5d30

• cmake: add link dependencies to cmake find pkg (ggerganov#10433)

  Show description for f95caa7
  bandoti

  authored
  f95caa7

  Copy full SHA for f95caa7

• vulkan: predicate max operation in soft_max shaders/soft_max (ggerganov#10437)

  Show description for 9abe9ee
  jeffbolznv

  authored
  9abe9ee

  Copy full SHA for 9abe9ee

Commits on Nov 21, 2024

• ggml-opt: fix data corruption (ggml/1022)

  

  JohannesGaessler

  authored and

  ggerganov

  committed
  02e4eaf

  Copy full SHA for 02e4eaf

• ggml/sched : do not skip views in pre-assignments

  

  slaren

  authored and

  ggerganov

  committed
  59b9172

  Copy full SHA for 59b9172

• sync : ggml

  ggerganov

  committed
  87a533b

  Copy full SHA for 87a533b

• llama : handle KV shift for recurrent models (ggerganov#10402)

  Show description for 1bb30bf
  ggerganov

  authored
  1bb30bf

  Copy full SHA for 1bb30bf

• cuda : optimize argmax (ggerganov#10441)

  Show description for a5e4759
  

  slaren

  and

  JohannesGaessler

  authored
  a5e4759

  Copy full SHA for a5e4759

Commits on Nov 22, 2024

• CANN: Support Ascend310P to accelerate F32 and F16 Model (ggerganov#10216)

  Show description for c18610b
  leo-pony

  authored
  c18610b

  Copy full SHA for c18610b

• GitHub: ask for more info in issue templates (ggerganov#10426)

  Show description for 599b3e0
  JohannesGaessler

  authored
  599b3e0

  Copy full SHA for 599b3e0

• ci: Update oneAPI runtime dll packaging (ggerganov#10428)

  Show description for 6dfcfef
  shou692199

  authored
  6dfcfef

  Copy full SHA for 6dfcfef

Commits on Nov 23, 2024

• ggml : do not use ARM features not included in the build (ggerganov#10457)

  slaren

  authored
  55ed008

  Copy full SHA for 55ed008

Commits on Nov 24, 2024

• fix gguf-py: Conversion error when multiple licenses are configured (ggerganov#9807)

  Show description for 96fa2c5
  

  mmngays

  and

  mmnga

  authored
  96fa2c5

  Copy full SHA for 96fa2c5

• convert : XLMRoberta Type Vocab Size (ggerganov#10458)

  Show description for 9336db4
  gabe-l-hart

  authored
  9336db4

  Copy full SHA for 9336db4

• llama : fix op mul check with command-r-plus (ggerganov#10476)

  slaren

  authored
  dc39012

  Copy full SHA for dc39012

• flake.lock: Update (ggerganov#10470)

  Show description for cce5a90
  

  ggerganov

  and

  github-actions[bot]

  authored
  cce5a90

  Copy full SHA for cce5a90

Commits on Nov 25, 2024

• speculative : refactor and add a simpler example (ggerganov#10362)

  Show description for d9d54e4
  ggerganov

  authored
  d9d54e4

  Copy full SHA for d9d54e4

• [SYCL] Fix building Win package for oneAPI 2025.0 update (ggerganov#10483)

  Show description for 5a89877
  

  NeoZhangJianyu

  and

  arthw

  authored
  5a89877

  Copy full SHA for 5a89877

• metal : minor code formatting

  ggerganov

  committed
  b756441

  Copy full SHA for b756441

• tests : fix compile warning

  ggerganov

  committed
  f6d12e7

  Copy full SHA for f6d12e7

• ggml : add support for dynamic loading of backends (ggerganov#10469)

  Show description for 5931c1f
  

  slaren

  and

  ggerganov

  authored
  5931c1f

  Copy full SHA for 5931c1f

• server : add speculative decoding support (ggerganov#10455)

  Show description for 9ca2e67
  ggerganov

  authored
  9ca2e67

  Copy full SHA for 9ca2e67

• Add download chat feature to server chat (ggerganov#10481)

  Show description for a9a678a
  

  brucepro

  and

  ngxson

  authored
  a9a678a

  Copy full SHA for a9a678a

• Github: update issue templates [no ci] (ggerganov#10489)

  JohannesGaessler

  authored
  1f92225

  Copy full SHA for 1f92225

• llama : accept a list of devices to use to offload a model (ggerganov#10497)

  Show description for 10bce04
  slaren

  authored
  10bce04

  Copy full SHA for 10bce04

• Rename Olmo1124 to Olmo2 (ggerganov#10500)

  2015aroras

  authored
  80acb7b

  Copy full SHA for 80acb7b

• metal : enable mat-vec kernels for bs <= 4 (ggerganov#10491)

  ggerganov

  authored
  106964e

  Copy full SHA for 106964e

• server : enable cache_prompt by default (ggerganov#10501)

  Show description for 47f931c
  ggerganov

  authored
  47f931c

  Copy full SHA for 47f931c

• server : add more information about error (ggerganov#10455)

  ggerganov

  committed
  9fd8c26

  Copy full SHA for 9fd8c26

• ci : build docker images only once daily (ggerganov#10503)

  slaren

  authored
  50d5cec

  Copy full SHA for 50d5cec

• Introduce llama-run (ggerganov#10291)

  Show description for 0cc6375
  ericcurtin

  authored
  0cc6375

  Copy full SHA for 0cc6375

Commits on Nov 26, 2024

• vulkan: Fix a vulkan-shaders-gen arugment parsing error (ggerganov#10484)

  Show description for 0eb4e12
  sparkleholic

  authored
  0eb4e12

  Copy full SHA for 0eb4e12

• CANN: RoPE and CANCAT operator optimization (ggerganov#10488)

  Show description for 7066b4c
  

  noemotiovon

  and

  noemotiovon

  authored
  7066b4c

  Copy full SHA for 7066b4c

• CANN: Improve the Inferencing Performance for Ascend NPU Device (ggerganov#10454)

  Show description for 9a4b79b
  

  3 peopleauthored
  9a4b79b

  Copy full SHA for 9a4b79b

• speculative : simplify the implementation (ggerganov#10504)

  Show description for 811872a
  ggerganov

  authored
  811872a

  Copy full SHA for 811872a

• server : fix parallel speculative decoding (ggerganov#10513)

  Show description for 84e1c33
  ggerganov

  authored
  84e1c33

  Copy full SHA for 84e1c33

• ggml-cpu: cmake add arm64 cpu feature check for macos (ggerganov#10487)

  Show description for 25669aa
  chaxu01

  authored
  25669aa

  Copy full SHA for 25669aa

• ci : add ubuntu cuda build, build with one arch on windows (ggerganov#10456)

  slaren

  authored
  c6807b3

  Copy full SHA for c6807b3

• ci : publish the docker images created during scheduled runs (ggerganov#10515)

  slaren

  authored
  7db3846

  Copy full SHA for 7db3846

• cmake : enable warnings in llama (ggerganov#10474)

  Show description for ab96610
  ggerganov

  authored
  ab96610

  Copy full SHA for ab96610

• restore the condistion to build & update pacakge when merge (ggerganov#10507)

  Show description for 0bbd226
  

  NeoZhangJianyu

  and

  arthw

  authored
  0bbd226

  Copy full SHA for 0bbd226

• server : replace behave with pytest (ggerganov#10416)

  Show description for 45abe0f
  ngxson

  authored
  45abe0f

  Copy full SHA for 45abe0f

• vulkan: fix group_norm (ggerganov#10496)

  Show description for 904109e
  jeffbolznv

  authored
  904109e

  Copy full SHA for 904109e

• mtgpu: Add MUSA_DOCKER_ARCH in Dockerfiles && update cmake and make (ggerganov#10516)

  Show description for 249cd93
  yeahdongcn

  authored
  249cd93

  Copy full SHA for 249cd93

• Fix HIP flag inconsistency & build docs (ggerganov#10524)

  Show description for be0e350
  tristandruyen

  authored
  be0e350

  Copy full SHA for be0e350

• llama : disable warnings for 3rd party sha1 dependency (ggerganov#10527)

  slaren

  authored
  30ec398

  Copy full SHA for 30ec398

• ci : remove nix workflows (ggerganov#10526)

  slaren

  authored
  5a349f2

  Copy full SHA for 5a349f2

• Add OLMo 2 model in docs (ggerganov#10530)

  Show description for de50973
  2015aroras

  authored
  de50973

  Copy full SHA for de50973

• ci : fix cuda releases (ggerganov#10532)

  slaren

  authored
  c9b00a7

  Copy full SHA for c9b00a7

Commits on Nov 27, 2024

• vulkan: optimize Q2_K and Q3_K mul_mat_vec (ggerganov#10459)

  jeffbolznv

  authored
  4a57d36

  Copy full SHA for 4a57d36

• vulkan: skip integer div/mod in get_offsets for batch_idx==0 (ggerganov#10506)

  jeffbolznv

  authored
  71a6498

  Copy full SHA for 71a6498

• vulkan: further optimize q5_k mul_mat_vec (ggerganov#10479)

  jeffbolznv

  authored
  249a790

  Copy full SHA for 249a790

• vulkan: Handle GPUs with less shared memory (ggerganov#10468)

  Show description for 5b3466b
  jeffbolznv

  authored
  5b3466b

  Copy full SHA for 5b3466b

• vulkan: define all quant data structures in types.comp (ggerganov#10440)

  jeffbolznv

  authored
  c31ed2a

  Copy full SHA for c31ed2a

• Do not include arm_neon.h when compiling CUDA code (ggml/1028)

  

  frankier

  authored and

  ggerganov

  committed
  9150f8f

  Copy full SHA for 9150f8f

• sync : ggml

  ggerganov

  committed
  fee824a

  Copy full SHA for fee824a

• metal : fix group_norm support condition (#0)

  ggerganov

  committed
  9e2301f

  Copy full SHA for 9e2301f

• ci : faster CUDA toolkit installation method and use ccache (ggerganov#10537)

  Show description for 46c69e0
  slaren

  authored
  46c69e0

  Copy full SHA for 46c69e0

• Merge branch 'layla-build' into merge

  l3utterfly

  authored
  289e208

  Copy full SHA for 289e208