Skip to content

Commit

Permalink
fix typo
Browse files Browse the repository at this point in the history
  • Loading branch information
ajindal1 committed Dec 20, 2024
1 parent 6833733 commit d9622de
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/generators.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -356,7 +356,7 @@ void Generator::GenerateNextToken() {
// TODO: Extend the solution to make it work for batch size > 1 and num beams > 1
// Phi3 model switches from short factor to long factor at 4097 (original_max_position_embeddings+1) token, needs Recomputation of Position IDs and KV Cache
// at this stage which is achieved by rewinding to zero and appending the current sequence
if (model_->config_->model.type == "phi3" && search_->params_->search.batch_size == 1 && params.search.num_beams == 1) {
if (model_->config_->model.type == "phi3" && search_->params_->search.batch_size == 1 && search_->params_->search.num_beams == 1) {
if (search_->GetSequenceLength() == 4097 && first_switch) {
first_switch = false;
auto current_seq = cpu_span<int32_t>(GetSequence(0).CpuSpan());
Expand Down

0 comments on commit d9622de

Please sign in to comment.