Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correct how periodic concurrency works in PACG #777

Merged
merged 7 commits into from
Oct 18, 2023

Conversation

nv-braf
Copy link
Contributor

@nv-braf nv-braf commented Oct 17, 2023

I've changed the PA config generator to correctly reflect how periodic concurrency is supposed to work (start:stop:range).

This incorporates all the new CLI/config fields and updates unit testing to check that the correct number (and value) of configs are generated when periodic concurrency is searched.

I've also updated the default search values to keep the number of searches under 100. The algorithm will search by powers of 2 (just like how the other searches work in brute mode).

The other change is to allow ranges to be specified using colons, versus having to create a dictionary, which would be onerous on the command line.

@nv-braf nv-braf requested a review from nv-hwoo October 17, 2023 18:18
@nv-braf nv-braf requested a review from nv-hwoo October 17, 2023 22:23
@nv-braf nv-braf marked this pull request as ready for review October 18, 2023 16:28
@nv-braf nv-braf merged commit efea104 into add-llm-mode Oct 18, 2023
3 checks passed
nv-braf added a commit that referenced this pull request Nov 3, 2023
* Adding new options for LLM (#768)

* Update README and versions for 23.09 branch (#761) (#767)

* Adding new options for LLM

* Fixing codeQL issues

* Fixing codeQL issue

---------

Co-authored-by: Misha Chornyi <[email protected]>

* Add LLM support to Brute Search (#769)

* Initial coding complete

* First unit test passing

* Adding test for prompt length

* Refactor PACG methods

* Further refactoring

* Ensure early exit isn't enabled for LLM models

* Fix type checking errors

* Attempt at fixing codeql issue

* Revert "Attempt at fixing codeql issue"

This reverts commit 2619b83.

* Attempt at codeQL fix

* Adding deepcopy back in

* Removing deepcopy in an attempt to fix codeQL errors

* Update model_analyzer/config/input/config_command_profile.py

Co-authored-by: Hyunjae Woo <[email protected]>

* Update model_analyzer/config/generate/perf_analyzer_config_generator.py

Co-authored-by: Hyunjae Woo <[email protected]>

* Update model_analyzer/config/generate/perf_analyzer_config_generator.py

Co-authored-by: Hyunjae Woo <[email protected]>

* Update model_analyzer/config/generate/perf_analyzer_config_generator.py

Co-authored-by: Hyunjae Woo <[email protected]>

* Moving location of method

* Changing parameter to inference load

* Changing parameter to inference load

* Changing prompt length to text input length

* Changing max_tokens to use request-parameter

* Fix input-data typo

* Changing non-parameter to parameter

---------

Co-authored-by: Hyunjae Woo <[email protected]>

* New LLM record types (#770)

* New measurement fields created.

* Fixing omission in llm_metric_table

* Changing name to be avg_token_to_token...

* New config options based on live run (#775)

* Added new config options and modified existing options

* Refactoring model parameter setting

* Removing magic numbers

* Capture LLM metrics from PA (#774)

* Initial code for aggregation of new LLM metrics

* New measurement fields created.

* Fixing PA unit tests

* Adding hooks in metrics to capture new LLM fields

* Fixing codeQL errors

* Fixing type checking errors

* Changes needed post-merge from other branches

* Revert naming mistake (due to merge).

* Changes uncovered during live testing

* Fixes based on hwoo review

* Fixing typo

* Change to use lists and mean()

* Changes based on hwoo review

* Correct how periodic concurrency works in PACG (#777)

* Created a new class ConfigRangeNumeric and using it for periodic-concurrency

* Fixes and defaults for periodic concurrency

* First unit test passing

* PACG chagnes complete. Unit tests updated and passing

* Removing uneeded class

* Fixing codeQL and hwoo's review suggestions

* Adding missing else

* Llm testing live run (#778)

* Created a new class ConfigRangeNumeric and using it for periodic-concurrency

* Fixes and defaults for periodic concurrency

* First unit test passing

* PACG chagnes complete. Unit tests updated and passing

* Removing uneeded class

* Changes to fix live run

* Minor refactor and cleanup

* Removing json files

* Changing to use f-string

* More cleanup from hwoo CR

* Removing stale code for request period

* Fix nit

* Changes to get LLM summary reports working (#779)

* Changes to get LLM summary reports working

* Addressing hwoo's CR

* Adding illegal LLM checks w/ unit testing + some minor cleanup (#781)

* Adding illegal LLM checks w/ unit testing + some minor cleanup

* Updated with TMA

* Misc LLM cleanup (#782)

* General cleanup

* Add ticket nums to todos

* Fix for non-LLM breaking bug introduced.

* summary table in progress

---------

Co-authored-by: Misha Chornyi <[email protected]>
Co-authored-by: Hyunjae Woo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants