Possible New Research Way For BenchMark

[017lead]

https://arxiv.org/pdf/2407.12734. (https://arxiv.org/abs/2407.12734). (https://youtu.be/uhKnU2dP-Bg?feature=shared) I think it's interesting that we can maybe use Minecraft to benchmark the LLms, the guy in the youtube video pointed to the way llms formed tasks in Minecraft 3d world. The lower models did not have much of success or cognitive capbllityes thus the result came out to be disappointing but as you went higher to flagship (Claude 3.5(new), o1) you had way more creativity shown by the LLM. specially the o1 which has reasoning did significantly well. Using a structured sandbox environment like Minecraft to benchmark LLMs is a novel approach, especially since Minecraft provides an interactive 3D world where models must demonstrate complex reasoning, creativity, and sequential planning. The challenges here allow a clearer comparison of "cognitive capability,". So to benchmark the reasoning of the model o1, and what Raspberry comes out to be. can be evaluated with a new tests.