You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I follow this notebook in colab with T4 GPU.
There are several problems:
Now the version of cuda is 12.2 instead of 11.8. mlc-ai-nightly-cu118 mlc-chat-nightly-cu118 should be replaced by mlc-ai-nightly-cu122 mlc-chat-nightly-cu122
The prebuilt_libs does not work for colab T4 GPU (Get a error CUDA_ERROR_NO_BINARY_FOR_GPU). I compiled the model lib on colab with command !mlc_chat compile ./dist/Llama-2-7b-chat-hf-q4f32_1-MLC/mlc-chat-config.json --device cuda -o dist/Llama-2-7b-chat-hf-q4f32_1-cuda.so and it works to me. Maybe consider update the prebuild_libs.
The text was updated successfully, but these errors were encountered:
I follow this notebook in colab with T4 GPU.
There are several problems:
mlc-ai-nightly-cu118 mlc-chat-nightly-cu118
should be replaced bymlc-ai-nightly-cu122 mlc-chat-nightly-cu122
!mlc_chat compile ./dist/Llama-2-7b-chat-hf-q4f32_1-MLC/mlc-chat-config.json --device cuda -o dist/Llama-2-7b-chat-hf-q4f32_1-cuda.so
and it works to me. Maybe consider update the prebuild_libs.The text was updated successfully, but these errors were encountered: