Change the llama max_batch_size larger than default eval batch size (#…

…2283)

Summary:
Fix the PyTorch issue: pytorch/pytorch#106110 for llama dynamic batch test case.
The root-cause and discussion is as described in pytorch/pytorch#106110 (comment).

Pull Request resolved: #2283

Reviewed By: aaronenyeshi

Differential Revision: D58532244

Pulled By: xuzhao9

fbshipit-source-id: ef2a51974c77e8dcf7e041db10e71869768126a3

torchbenchmark/models/llama/model.py

@@ -19,7 +19,7 @@ class ModelArgs:
     multiple_of: int = 256  # make SwiGLU hidden layer size multiple of large power of 2
     norm_eps: float = 1e-5
 
-    max_batch_size: int = 32 # From the paper they use a batch size of 4M for training
+    max_batch_size: int = 64 # From the paper they use a batch size of 4M for training
     max_seq_len: int = 1024
 
     device: Optional[str] = None