Install MLC LLM Python Package¶
MLC LLM Python Package can be installed directly from a prebuilt developer package, or built from source.
Option 1. Prebuilt Package¶
We provide nightly built pip wheels for MLC-LLM via pip. Select your operating system/compute platform and run the command in your terminal:
Note
❗ Whenever using Python, it is highly recommended to use conda to manage an isolated Python environment to avoid missing dependencies, incompatible versions, and package conflicts.
conda activate your-environment
python3 -m pip install --pre -U -f https://mlc.ai/wheels mlc-chat-nightly mlc-ai-nightly
conda activate your-environment
python3 -m pip install --pre -U -f https://mlc.ai/wheels mlc-chat-nightly-cu117 mlc-ai-nightly-cu117
conda activate your-environment
python3 -m pip install --pre -U -f https://mlc.ai/wheels mlc-chat-nightly-cu118 mlc-ai-nightly-cu118
conda activate your-environment
python3 -m pip install --pre -U -f https://mlc.ai/wheels mlc-chat-nightly-cu121 mlc-ai-nightly-cu121
conda activate your-environment
python3 -m pip install --pre -U -f https://mlc.ai/wheels mlc-chat-nightly-cu122 mlc-ai-nightly-cu122
conda activate your-environment
python3 -m pip install --pre -U -f https://mlc.ai/wheels mlc-chat-nightly-rocm56 mlc-ai-nightly-rocm56
conda activate your-environment
python3 -m pip install --pre -U -f https://mlc.ai/wheels mlc-chat-nightly-rocm57 mlc-ai-nightly-rocm57
Supported in all Linux packages.
Note
If encountering issues with GLIBC not found, please install the latest glibc in conda:
conda install -c conda-forge libgcc-ng
conda activate your-environment
python3 -m pip install --pre -U -f https://mlc.ai/wheels mlc-chat-nightly mlc-ai-nightly
Note
Always check if conda is installed properly in macOS using the command below:
conda info | grep platform
It should return “osx-64” for Mac with Intel chip, and “osx-arm64” for Mac with Apple chip.
conda activate your-environment
python3 -m pip install --pre -U -f https://mlc.ai/wheels mlc-chat-nightly mlc-ai-nightly
Note
If encountering the error below:
FileNotFoundError: Could not find module 'path\to\site-packages\tvm\tvm.dll' (or one of its dependencies). Try using the full path with constructor syntax.
It is likely zstd, a dependency to LLVM, was missing. Please use the command below to get it installed:
conda install zstd
Option 2. Build from Source¶
Upcoming.