Moonshot AI ships full open weights for Kimi K3, a 2.8-trillion-parameter model

Moonshot AI released the complete open weights for Kimi K3 on July 27, the largest open-weight model shipped to date. It's a sparse mixture-of-experts design with 896 experts and only 16 active per token, paired with Kimi Delta Attention, a hybrid linear-attention mechanism, and an addition the team calls Attention Residuals. That combination is what makes a genuinely usable 1-million-token context window possible: standard quadratic attention would make inference at that length prohibitively expensive, so the linear-attention approach is doing real engineering work, not just marketing. The model also has native visual understanding built in rather than bolted on. Weights are licensed under a Modified MIT license, and the full download is roughly 1.4TB, which means most teams will need serious multi-GPU or multi-node infrastructure (or a heavily quantized version) to run it themselves. The more interesting release for builders may be what shipped alongside the model: MoonEP, the expert-parallel training infrastructure, and AgentEnv, the environment tooling used to train Kimi's agentic behavior, were both open-sourced too. That means teams who can't realistically host a 2.8T-parameter model can still study or adapt the underlying techniques for their own smaller MoE or agent-training projects. For anyone choosing between closed frontier APIs and open weights, K3 narrows the gap: it's now plausible to get frontier-adjacent capability with full control over fine-tuning, hosting, and data handling instead of trailing a generation behind proprietary models. The long native context also cuts against the usual RAG/chunking workaround for large-document tasks.

Source

View on ShipDigest