I Ran all three models for a coding task just to see how they behave when things aren’t clean or nicely phrased.

The goal was just to see who performs like a real dev.

here's my takeaway

Opus 4.5 handled real repo-issues the best. It fixed things without breaking unrelated parts and didn’t hallucinate new abstractions. Felt the most “engineering-minded

GPT-5.1 was close behind. It explained its reasoning step-by-step and sometimes added improvements I never asked for. Helpful when you want safety, annoying when you want precision

Gemini solved most tasks but tended to optimize or simplify decisions I explicitly constrained. Good output, but sometimes too “creative.”

On Refactoring and architecture-level tasks:
Opus delivered the most complete refactor with consistent naming, updated dependencies, and documentation.
GPT-5.1 took longer because it analyzed first, but the output was maintainable and defensive.
Gemini produced clean code but missed deeper security and design patterns.

Context windows (because it matters at repo scale):

• Opus 4.5: ~200K tokens usable, handles large repos better without losing track

• GPT-5.1: ~128K tokens but strong long-reasoning even near the limit

• Gemini 3 Pro: ~1M tokens which is huge, but performance becomes inconsistent as input gets massive

What's your experience been with these three? Used these frontier models Side by Side in my Multi Agent AI setup with Anannas LLM Provider & the results were interesting.

Have you run your own comparisons, and if so, what setup are you using?

I've been switching back and forth between Claude Sonnet 4.5 or Composer 1 and Gemini 3.0 and I’m trying to figure out which model actually performs better for real-world coding tasks inside Cursor AI. I'm not looking for a general comparison.

I want feedback specifically in the context of how these models behave inside the Cursor IDE.

Ran these three models through three real-world coding scenarios to see how they actually perform.

The tests:

Prompt adherence: Asked for a Python rate limiter with 10 specific requirements (exact class names, error messages, etc). Basically, testing if they follow instructions or treat them as "suggestions."

Code refactoring: Gave them a messy, legacy API with security holes and bad practices. Wanted to see if they'd catch the issues and fix the architecture, plus whether they'd add safeguards we didn't explicitly ask for.

System extension: Handed over a partial notification system and asked them to explain the architecture first, then add an email handler. Testing comprehension before implementation.

Results:

Test 1 (Prompt Adherence): Gemini followed instructions most literally. Opus stayed close to spec with cleaner docs. GPT-5.1 went defensive mode - added validation and safeguards that weren't requested.

Test 2 (TypeScript API): Opus delivered the most complete refactoring (all 10 requirements). GPT-5.1 hit 9/10, caught security issues like missing auth and unsafe DB ops. Gemini got 8/10 with cleaner, faster output but missed some architectural flaws.

Test 3 (System Extension): Opus gave the most complete solution with templates for every event type. GPT-5.1 went deep on the understanding phase (identified bugs, created diagrams) then built out rich features like CC/BCC and attachments. Gemini understood the basics but delivered a "bare minimum" version.

Takeaways:

Opus was fastest overall (7 min total) while producing the most thorough output. Stayed concise when the spec was rigid, wrote more when thoroughness mattered.

GPT-5.1 consistently wrote 1.5-1.8x more code than Gemini because of JSDoc comments, validation logic, error handling, and explicit type definitions.

Gemini is cheapest overall but actually cost more than GPT in the complex system task - seems like it "thinks" longer even when the output is shorter.

Opus is most expensive ($1.68 vs $1.10 for Gemini) but if you need complete implementations on the first try, that might be worth it.

Full methodology and detailed breakdown here: https://blog.kilo.ai/p/benchmarking-gpt-51-vs-gemini-30-vs-opus-45

What's your experience been with these three? Have you run your own comparisons, and if so, what setup are you using?