NNCF helpers

[KodiaqQ]

Changes

• Added helpers module to NNCF.
• Added dataset helper.

Reason for changes

• Extended data-free usage.

Related tickets

• 152550

Tests

• Added tests/common/test_helpers.py

[AlexKoff88]

I am ok with the code but two comments from my side:

• Why is it not a part of nncf.data namespace?
• Why do we need requirements file inside, especially with "-c constraints.txt"?

[KodiaqQ]

• Why is it not a part of nncf.data namespace?

Since NNCF does not have helpers namespace, it was introduced. Should I introduce nncf.data.helpers module for a corresponded namespace?

• Why do we need requirements file inside, especially with "-c constraints.txt"?

The provided code utilizes torch and transformers modules. They are not part of the main NNCF requirements, so I provided an external requirements.txt list for the helpers module.

[AlexKoff88]

• Why is it not a part of nncf.data namespace?

Since NNCF does not have helpers namespace, it was introduced. Should I introduce nncf.data.helpers module for a corresponded namespace?

I missed that part during the last review but I would like to name the function generate_text_data(model: PreTrainedModel, tokenizer: PreTrainedTokenizer, max_length: int = 32, num_samples: int = 128) -> List[str]
It is up to you how to arrange it internally. I would like to see the import like this: from nncf.data import generate_text_data.

• Why do we need requirements file inside, especially with "-c constraints.txt"?

The provided code utilizes torch and transformers modules. They are not part of the main NNCF requirements, so I provided an external requirements.txt list for the helpers module.

Ok, thanks.

[AlexanderDokuchaev]

@KodiaqQ @AlexKoff88 @alexsu52
Is planning to add other helper function to nncf.helpers?
May be will be better to locate the function in nncf.data or nncf/quantization/algorithms ?

[KodiaqQ]

function in nncf.data or nncf/quantization/algorithms ?

Please suggest the appropriate place for this function considering that this method requires transformers & torch Python packages and shouldn't be mixed with the nncf.Dataset.

[alexsu52]

I would suggest to add an example of usage generate_text_data in https://github.com/openvinotoolkit/nncf/tree/develop/examples/llm_compression. It could be done in the follow-up PR, but you need to think about convenient use of the function in this PR.

[KodiaqQ]

I would suggest to add an example of usage generate_text_data in https://github.com/openvinotoolkit/nncf/tree/develop/examples/llm_compression. It could be done in the follow-up PR, but you need to think about convenient use of the function in this PR.

An example of the non-API method usage was added.

[alexsu52]

I would suggest to add an example of usage generate_text_data in https://github.com/openvinotoolkit/nncf/tree/develop/examples/llm_compression. It could be done in the follow-up PR, but you need to think about convenient use of the function in this PR.

An example of the non-API method usage was added.

Great example! Please add test for the example.

[KodiaqQ]

@AlexKoff88, re-review an example, please.

[KodiaqQ]

Great example! Please add test for the example.

Done.

[alexsu52]

LGTM

[KodiaqQ]

Since there are some issues with the test_examples, I propose to disable this example in tests until the issue is investigated and merge this functionality.
@alexsu52, @MaximProshin, what do you think?

[MaximProshin]

@KodiaqQ

Since there are some issues with the test_examples, I propose to disable this example in tests until the issue is investigated and merge this functionality. @alexsu52, @MaximProshin, what do you think?

I agree

[alexsu52]

Since there are some issues with the test_examples, I propose to disable this example in tests until the issue is investigated and merge this functionality. @alexsu52, @MaximProshin, what do you think?

Good idea!