The day in AI
No written reading for this day yet. Here is everything that was logged.
Everything from this day
Research
11 Jan 2024DeepSeekDeepSeekMoE DeepSeekMoE uses fine-grained expert segmentation plus always-on shared experts, the MoE design that V2, V3, R1 and V4 all inherit.11 Jan 2024DeepSeekDeepSeekMoE DeepSeekMoE uses fine-grained expert segmentation plus always-on shared experts, and a 16B MoE matches Llama 2 7B at about 40% of the compute.