Institute of Foundation Models (IFM) released K2 Horizon, 6 open AI models from 0.9B to 375B, covering tiny on-device models through large models for reasoning, coding and AI agents.

For every model, IFM is releasing weights, training code, configs, data or detailed data recipes, intermediate checkpoints, training logs and evaluations, covering the full process from pre-training through reasoning and agent training.

The 0.9B, 3.7B and 7B models set new state of the art in their size classes, while the sparse 36B-A4B gets close to the dense 32B while activating only about 4B parameters per token using IFM's new Mixture-of-Value Attention architecture.

Each model was trained on roughly 20T tokens, while released checkpoints and logs give researchers an unusually detailed record of how the models developed during training.

@IFM_AI is also releasing xLLM, the production training infrastructure used to build K2 Horizon, plus the full agentic post-training codebase including the RL pipeline. (Github link in the last post to this thread).