Support synthetic image generation in GenAI-Perf #754
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
nv-hwoo
commented
Jul 17, 2024
src/c++/perf_analyzer/genai-perf/genai_perf/llm_inputs/synthetic_image_generator.py
Show resolved
Hide resolved
debermudez
reviewed
Jul 17, 2024
| while True: | ||
| n = int(self.rng.normal(mean, stddev)) | ||
| n = int(random.gauss(mean, stddev)) |
There was a problem hiding this comment.
Can we do this using an offset or abs() instead of a loop?
There was a problem hiding this comment.
Good point. Changed to using abs.
There was a problem hiding this comment.
The abs corrupts the gaussian distribution. The while loop truncates it at zero. The same solution is used in SyntheticPromptGenerator:
But, I'm fine with abs+offset.
There was a problem hiding this comment.
i didnt think of that when i suggested it.
We can revert if you prefer.
There was a problem hiding this comment.
@mwawrzos I think this is still a valid normal distribution, but a folded one (instead of truncated). It may affect the statistics, but I'm not too sure about the consequences of the slight shift in the statistics. But at the same time, I don't think this should be a huge concern since sampling image resolutions near zero seems like an unlikely case.
@nv-braf do you have any thoughts?
src/c++/perf_analyzer/genai-perf/genai_perf/llm_inputs/synthetic_image_generator.py
Show resolved
Hide resolved
debermudez
approved these changes
Jul 18, 2024
nv-hwoo
added a commit
that referenced
this pull request
Jul 18, 2024
* POC LLaVA VLM support (#720) * POC for LLaVA support * non-streaming request in VLM tests * image component sent in "image_url" field instead of HTML tag * generate sample image instead of loading from docs * add vision to endpoint mapping * fixes for handling OutputFormat * refactor - extract image preparation to a separate module * fixes to the refactor * replace match-case syntax with if-elseif-else * Update image payload format and fix tests * Few clean ups and tickets added for follow up tasks * Fix and add tests for vision format * Remove output format from profile data parser * Revert irrelevant code change * Revert changes * Remove unused dependency * Comment test_extra_inputs --------- Co-authored-by: Hyunjae Woo <[email protected]> * Support multi-modal input from file for OpenAI Chat Completions (#749) * add synthetic image generator (#751) * synthetic image generator * format randomization * images should be base64-encoded arbitrarly * randomized image format * randomized image shape * prepare SyntheticImageGenerator to support different image sources * read from files * python 3.10 support fixes * remove unused imports * skip sampled image sizes with negative values * formats type fix * remove unused variable * synthetic image generator encodes images to base64 * image format not randomized * sample each dimension independently Co-authored-by: Hyunjae Woo <[email protected]> * apply code-review suggestsions * update class name * deterministic synthetic image generator * add typing to SyntheticImageGenerator * SyntheticImageGenerator doesn't load files * SyntheticImageGenerator always encodes images to base64 * remove unused imports * generate gaussian noise instead of blank images --------- Co-authored-by: Hyunjae Woo <[email protected]> * Add command line arguments for synthetic image generation (#753) * Add CLI options for synthetic image generation * read image format from file when --input-file is used * move encode_image method to utils * Lazy import some modules * Support synthetic image generation in GenAI-Perf (#754) * support synthetic image generation for VLM model * add test * integrate sythetic image generator into LlmInputs * add source images for synthetic image data * use abs to get positive int --------- Co-authored-by: Marek Wawrzos <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SyntheticImageGeneratorintoLlmInputs