Open Mixture of Experts Models Use Divergent Routing Architectures
A comparison of ten open mixture of experts language models shows broad disagreement across five key routing configuration parameters.
FTMQ SI, written by our newsroom0 views
Open mixture of experts models share a foundational concept but diverge on almost every routing configuration detail, according to MarkTechPost. Expert counts across ten analyzed models range from eight to 384, while top-k parameters range from one to eight. Every evaluated model relies on token-choice routing, where individual tokens select their own experts. [1]
MarkTechPost reported that the models differ across five specific technical parameters. These knobs include the total expert count, experts assigned per token, the inclusion of shared experts, the gating scoring function, and the load-balancing method. Some architectures maintain an always-on shared expert, whereas other models omit it completely. [1]
Mixture of experts is a machine learning technique where multiple expert networks divide a problem space into homogeneous regions as a form of ensemble learning. Large language models use self-supervised learning on vast text datasets, with parameters scaling to support natural language processing tasks. [2][4]
In short
- Expert counts among ten open MoE models range from eight to 384.
- Top-k parameters in the analyzed open MoE models span from one to eight.
- All ten models compared by MarkTechPost utilize token-choice routing.
- MoE model routing implementations differ across five core design parameters.
Sources
Every paragraph above points to the numbered items it rests on. Read the originals here.
- [1]Which Routing Does Each MoE Model Use? DeepSeek, Qwen, Mixtral, Llama 4, gpt-oss ComparedMarkTechPost, 9d ago (the report this story comes from)
Background
- [2]Mixture of experts on Wikipedia
- [3]Mathematical model on Wikipedia
- [4]List of large language models on Wikipedia
Our newsroom writes these reports with the help of software, from the 4 sources listed and nothing else, and checks them against those sources. Facts can still be wrong or move on; the originals are the record. Spotted a mistake? Write to daniel@monsterkong.com.
Related from FTMQ SI
Earlier reports of ours on the same people and subjects.
- Alibaba releases Qwen4-Max model with native vision MoE architectureModels & Products, 1d ago
- Moonshot AI releases flagship Kimi K3.1 Pro modelModels & Products, 24d ago
- Open Chinese AI models approach closed frontier capabilities in network hacking testsTop Stories, 11d ago
- Hugging Face releases open Carbon-X model for DNA and natural languageModels & Products, 23d ago
More in Models & Products
- Microsoft unveils Copilot Workspaces and ROI tools at Ignite 20264h ago
- Meta deploys LLM test system to catch software bugs in production1d ago
- Alibaba releases Qwen4-Max model with native vision MoE architecture1d ago
- NiCE launches next generation Cognigy platform for agentic contact center automation2d ago
- AI scanning tool uncovers dozens of Linux kernel bugs2d ago
- Cmpnd releases dspy-wsp for weight space programming in DSPy2d ago
Get the day in one email
Reports like this one, the top news of the last 24 hours, every morning. Free, one email a day; readers can comment under every report.
By signing up you agree to our terms and privacy policy. Unsubscribe any time.

Comments
Loading
Join the conversation
Comments are open to readers of our daily email: the top news of the last 24 hours, every morning, free. Sign up and the comment box opens.
Already on the list? Enter the same address and we will send a sign-in link.
By signing up you agree to our terms and privacy policy. Unsubscribe any time.