[On Hold] Upgrade nanoGPT

[boris-drazic]

No description provided.

[boris-drazic]

Update nanoGPT model with following:

• - store weights as TT tensors on Weka an load them from there PR #4221
• - if a TT OP is not available, record which OP and its shapes; use fallback or torch OP until TT is implemented PR #4221
• - implement entire model and make a test that compares PCC to run on CPU for a single input PR #4221
• - set all OPs to produce tensors in tile layout commit
• - implement submodules and tests for them (use batch >1 if it makes sense) Commit
• - make a test that will load real inputs from a data set, run model, and compare TT outputs to valid/expected outputs from data set; also report compile/inference times and throughput Commit
• - store all intermedaite tensors in L1 commit
• - store all weights in L1 (if they fit) commit
• - use bfp8 data format and tile layoute for all tensors

[boris-drazic]

This replaces old issue #2193

[punithsekar]

Here is the commit for Loading weights from weka path for nanogpt model Link

[punithsekar]

Currently, We have moved weights to Weka path and uplifted the model with supported TT ops.
Commit Link

[Sudharsan-V]

The task set all OPs to produce tensors in tile layout is linked to #4091

The task make a test that will load real inputs from a data set, run model, and compare TT outputs to valid/expected outputs from data set; also report compile/inference times and throughput is linked to #4092

[punithsekar]

This PR(#4221) has following

• Uplifted nanoGPT model and recorded the ops that do not have TT-ops.
• Tilized weights tensors are stored and loaded from the Weka path.
• Tests for both sub-modules and the whole model.

We face a drop in PCC (0.99 to 0.98) for the whole model while using tt_lib.tensor.softmax in the attention submodule.

[Sudharsan-V]

The tasks

 - store all intermedaite tensors in L1
 - store all weights in L1 (if they fit)
 

are linked to #4342

[boris-drazic]

Can you make a PR for this?

[punithsekar]

We were planning to create a PR for ,

 - store all intermedaite tensors in L1
 - store all weights in L1 (if they fit)

after #4221 gets merged. As this work is depended on #4221 work.

Can we wait until #4221 gets merged or can we create a PR for it?

[boris-drazic]

Yes, you can wait.

[saichandax]

PR #4221 merged. We can proceed with creating PRs for other commits.