DeepSeek

Qwen: Deepseek must have concluded that hybrid models are worse. Deepseek: Qwen must have cnocluded that hybrid models are better. More on reddit.com

Agreed. 3.1 feels “technically correct” but flat like it stripped all the flavor out of the RP. R1 wasn’t perfect but it had way more life and descriptive flair More on reddit.com

That proves that benchmarks are barely usefull now. More on reddit.com

OK, so here are my quick takes on DeepSeek V3.1. Improving agentic capability seems to be the focus of this update. More specifically: 29.8% on HLE with search and Python, compared to 24.8% for R1-0528, 35.2% for GPT-5 Thinking, 24.3% for o3, 38.6% for Grok 4, and 26.9% for Gemini Deep Research. Caveats apply: DeepSeek models are exclusively evaluated on text subset, although I believe this subset is not easier for SotA models. Grok 4 is (possibly) evaluated without a webpage filter so data contamination is possible. 66.0% on SWE-Bench Verified without Thinking, compared to 44.6% for R1-0528, 74.9% for GPT-5 Thinking, 69.1% for o3, 74.5% for Claude 4.1 Opus, and 65.8 for Kimi K2. Again, caveats apply: OpenAI models are evaluated on a subset of 477 problems, not the 500 full set. 31.3% on Terminal Bench with Terminus 1 framework, compared to 30.2% for o3, 30.0% for GPT-5, and 25.3% for Gemini 2.5 Pro. A slight bump on other coding and math capabilities (AIME, LiveCodeBench, Codeforces, Aider) but most users would not be able to tell the difference, as R1-0528 already destroys 98% of human programmers on competitive programming. A slight reduction on GPQA, HLE (offline, no tools), and maybe in your own use case. I do not find V3.1 Thinking to be better than R1-0528 as a Chat LLM, for example. A few concluding thoughts: Right now I am actually more worried about how the open-source ecosystem will be deploying DeepSeek V3.1 in an agentic environment more than anything else. For agentic LLMs, prompts and agent frameworks make a huge difference in user experience. Gemini, Anthropic, and OpenAI all have branded search and code agents (e.g. Deep Research, Claude Code), but DeepSeek has none. So it remains to be seen how well V3.1 can work with prompts and tools from Claude Code, for example. Maybe DeepSeek will open-source their internal search and coding framework in a future date to ensure the best user experience. I also noticed a lot of serverless LLM inference providers cheap out on their deployment. They may serve with lowered precision, pruned experts, or poor sampling parameters. So the provider you use will definitely impact your user experience. It also starts to make sense why they merged the R1 with V3 and made 128K context window the default on the API. Agentic coding usually does not benefit much from a long CoT but consume a ton of tokens. So a singular model is a good way to reduce deployment TCO. This is probably as far as they can push on the V3 base - you can already see some regression on things like GPQA, offline HLE. Hope to see V4 soon. More on reddit.com