MLC LLM: Universal LLM Deployment Engine With ML Compilation
Overview
MLC LLM is a machine learning compiler and high-performance deployment engine for large language models. The mission of this project is to enable everyone to develop, optimize, and deploy AI models natively on everyone’s platforms.
MLC LLM compiles and runs code on MLCEngine – a unified high-performance LLM inference engine across the above platforms. MLCEngine provides OpenAI-compatible API available through REST server, python, javascript, iOS, Android, all backed by the same engine and compiler that we keep improving with the community.
Get Started
Please visit our documentation to get started with MLC LLM.