1 month ago - For all our models, the maximum generation length is set to 32,768 tokens. For benchmarks requiring sampling, we use a temperature of $0.6$, a top-p value of $0.95$, and generate 64 responses per query to estimate pass@1.

May 29, 2025 - AIME 2024, surpassing Qwen3 8B by +10.0% and matching the performance of Qwen3-235B-thinking. We believe that the chain-of-thought from DeepSeek-R1-0528 will hold significant importance for both academic research on reasoning models and industrial ...

May 30, 2025 - Improvements are also dramatic in coding benchmarks. On the Codeforces programming challenge, R1–0528’s rating is about 1930, up from ~1530 before- a ~400-point leap that reflects far better code generation and problem-solving.

... Temperature: 0.6. Top-p value: 0.95. Pass@1 estimation: Generated 64 responses per query. ... DeepSeek R1 outperformed o1, Claude 3.5 Sonnet and other models in the majority of reasoning benchmarks

July 30, 2025 - Ahh thanks, that one is easy enough then even if TTFT is not so useful without knowledge of the prompt/context size. So I finally have a moment this morning after drinking some coffee to look at the fine print of that intel sglang benchmark results. To keep it simple let’s look only at the DeepSeek-R1-671B INT8 quant report: MODEL DATA TYPE SOCKETS llama.cpp TTFT (ms) llama.cpp TPOT (ms) SGLang TTFT (ms) SGLang TPOT (ms) Speedup TTFT Speedup TPOT DeepSeek-R1-671B INT8 2 24546.76 172.0...

October 13, 2025 - For engineering-related tasks, ... achieves outstanding results, significantly outperforming DeepSeek-V3 with scores of 90.8% on MMLU, 84.0% on MMLU-Pro, and 71.5% on GPQA Diamond....

June 4, 2025 - DeepSeek-R1 performs strongly with a score of 49.2%, slightly ahead of OpenAI o1-1217’s 48.9%. This result positions DeepSeek-R1 as a strong contender in specialized reasoning tasks like software verification.

Reasoning Tasks: Shows performance on par with OpenAI’s o1 model across complex reasoning benchmarks. Image source: DeepSeek R1 Research Paper (Modified)

Image Source: DeepSeek-R1 Supplementary Information · Performance-wise, R1 rivals or even surpasses OpenAI o1 (also a reasoning model, but does not fully disclose the thinking tokens as R1) in math, coding, and reasoning benchmarks.

April 24, 2025 - Set up and run the GPQA-Diamond benchmark on DeepSeek-R1's distilled models locally to evaluate its reasoning capabilities.

April 4, 2025 - Benchmark Excellence: R1 matches OpenAI o1 in key tasks, with some areas of clear outperformance. While DeepSeek R1 builds upon the collective work of open-source research, its efficiency and performance demonstrate how creativity and strategic ...

October 9, 2025 - The abrupt appearance of DeepSeek’s R1 advanced reasoning model at the start of the year was akin to the “shot heard ‘round the world” in AI circles. Major tech companies had spent recent years pouring billions into generative AI projects, products, and infrastructure. Meanwhile, Chinese startup DeepSeek created in just months a model as good as OpenAI’s then-most advanced product on industry-standard benchmarks...

1 week ago - These models have impressive benchmark numbers: Artificial Analysis rates V3.2 as the second best open model on their index, while V3.2 Speciale tops all models — open or closed — in the MathArena benchmark for final answer competitions.

Analysis of DeepSeek's DeepSeek R1 0528 (May '25) and comparison to other AI models across key metrics including quality, price, performance (tokens per second & time to first token), context window & more.

To support the research community, we have open-sourced DeepSeek-R1-Zero, DeepSeek-R1, and six dense models distilled from DeepSeek-R1 based on Llama and Qwen. DeepSeek-R1-Distill-Qwen-32B outperforms OpenAI-o1-mini across various benchmarks, ...

2 weeks ago - On the benchmark math and code problems, DeepSeek-R1-Zero performed better than the humans selected for the benchmark study, but the model still had issues. Being trained on both English and Chinese data, for example, led to outputs that mixed ...

January 26, 2025 - According to the same benchmark, ... and coding performance, it has a score of 96.3 in the Codeforce benchmark, 71.5 in the GPQA-diamond benchmark, and 97.3 in the MATH-500 benchmark....

January 26, 2025 - What’s jaw-dropping is that DeepSeek-R1 not only talks the talk with transparency, but it also walks the walk in terms of performance. DeepSeek-R1 surpasses OpenAI’s o1 in critical benchmarks—including the math-heavy AIME, the MATH-500 dataset, and coding challenges on Codeforces.

Emotional Intelligence Benchmarks for LLMs · Github | Paper | | Twitter | About