"how fast is asp.net core really?" More on reddit.com

Tl;dr?

More on reddit.com

The TechEmpower Benchmarks suffer from the fact that there's a lot of "cheating". Looking at the Go benchmarks (which the author didn't dive into quite enough), many are allocating a pool of structs and then just filling in a struct in order to avoid the garbage collector. In fact, I would say that the Go/atreugo fortunes benchmark violates the TechEmpower rules. The list data structure must be a dynamic-size or equivalent and should not be dimensioned using foreknowledge of the row-count of the database table. The Go/atreugo benchmark sizes the lists so that they won't need to be resized at runtime (so that they can avoid the copying and garbage collection). Go/atreugo is fast when you already know the size of the collection and size your lists appropriately and you effectively turn off the garbage collector by never releasing the memory that gets allocated. We know that the fortunes database has 12 elements in it and a 13th element is added at runtime. With the .NET tests, a new List() is created and the default capacity will be 4. When the 5th element is added, a capacity of 8 will be allocated, the 4 will be copied and the 5th added. When the 9th is added, a capacity of 16 will be allocated and the 8 copied... Should the .NET code be updated to be new List(16)? That's what the Go code is doing. Likewise, would the .NET implementation be faster if they didn't actually release the allocated lists and cause garbage collection? They could simply keep the lists and objects around and fill them rather than re-allocate. var fortuneList = fortuneListPool.Get() int i = 0; while (dataReader.Read()) { var fortune = fortunePool.Get() fortune.Id = dataReader.GetInt32(0); fortune.Message = dataReader.GetString(1); fortuneList[i++] = fortune; } https://github.com/TechEmpower/FrameworkBenchmarks/blob/e6eee12a57aa2c575db98e6bbe01a371bda25a7a/frameworks/CSharp/appmpower/src/RawDb.cs#L81 https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/frameworks/Go/atreugo/src/views/views.go#L99 That's what the Go code is doing to avoid memory allocation (which is expensive in Go since it's a non-compacting GC) and garbage collection - in addition to avoiding the list resizing. The problem is that the TechEmpower benchmarks aren't realistic - because the requirements don't evolve over time like any real-world app does. In the real world, you need to balance productivity with optimization. In the TechEmpower benchmarks, communities looking to win bragging rights can just over-fit the test. One of the things that becomes really obvious with the Go tests is that so much of your performance depends on how much you cheat. With all due respect to Sébastien Ros, with Go it doesn't depend on which framework you choose as much as he seems to imply in his comment. Even non-cheating things end up depending on library choices that have nothing to do with the framework. For example, Go/atreugo chooses to use QuickTemplate for templates which compiles the templates to Go code at compile-time. That makes their Fortunes implementation a lot faster than Gin which uses Go's built-in runtime-interpreted templating engine. https://github.com/SlinSo/goTemplateBenchmark Go's built-in templating takes 8.628 µs/op with 35 allocations per op. By comparison, QuickTemplate takes 0.181 µs/op with zero allocations. Is atreugo faster than Gin or did atreugo just choose a templating engine that's 48x faster and does zero memory allocation? I'm not saying that's cheating - I think that QuickTemplate arguably has better ergonomics than Go's built-in templating system. However, it means that we don't actually know whether atreugo is faster than Gin. We just know that the atreugo test choose a faster templating library (neither comes with templating in the framework). One of the issues with some of the Golang frameworks and benchmarks is that they use fasthttp which doesn't completely do http and a lot of the Go community thinks it should never be used. Go/atreugo uses fasthttp. The problem with benchmarks is that you can always get into arguments about what is legitimate or fair. I think QuickTemplate is very fair to use. I don't think it's fair to pre-allocate memory and effectively turn off the garbage collector. Others might disagree with me. If we leave implementations up to framework fanboys, we'll get over-fitted implementations meant to avoid all the problems in their framework/language. If we have a single implementer, the language/framework they're most comfortable with will have a distinct advantage. While the article doesn't delve into Python and PHP, I think some of the performance there shows "how much have we avoided having Python and PHP process things that can be done in C libraries?" PHP's standard library is mostly highly-optimized C functions. When you start having real business logic in your app, you end up doing a lot more in PHP which starts to dominate the time used. The more realistic the PHP app, the slower it'll become. The more micro-benchmark a PHP app, the more it can avoid doing any PHP. I don't want to sound too down on PHP here. In many ways, this strategy made PHP the major language it is today. Back in 2005, you could lean on the extremely well-optimized C functions and get performance out of really weak hardware. It could be embedded in the Apache web server with mod_php and because it included all the things you needed for a web program, you didn't need to include a lot of costly libraries. You could do things like mysql_fetch_assoc("MY SQL QUERY") and get an associative array (dictionary/hashmap) back and it'll be super fast. Then you'd just render that data. However, when you layer a lot of PHP code in (and do a lot of processing in PHP), things get really slow really fast. Laravel is dreadfully slow with the same performance as Ruby on Rails in these tests - even though other PHP tests will out-do Rails and Laravel by 20-40x. With the ASP.NET tests, we see the fastest outrunning the full ASP.NET/MVC/EntityFramework by 2.9x. Part of that is that we know the MVC routing is a bit slow. I believe Microsoft was hoping to close the gap between the minimal and MVC routing with .NET 7, but I haven't looked into it. Gin and Echo are the two Golang frameworks that are most used by the community. They both come out slower than ASP.NET in the TechEmpower benchmarks, but again I'd say that part of that is a bad implementation on Gin's part. Gin shouldn't be using the built-in Golang templates for their test. I think to really make a comparison, one needs to isolate what things are actually causing bottlenecks. Is it the template engine? Is it memory allocation for the objects/structs? Is it list-resizing? Is it GC? And then you have to have a realistic discussion about what can be avoided in idiomatic code where engineers are still productive and you aren't wasting resources just because it's a short-lived benchmark. Is it fair to say that you should never create a pre-allocated pool of structs? Maybe not. That rule would favor a language with a generational GC where short-lived objects are easily discarded (compared to Go's non-generational GC). But so much of this requires deep understanding of the languages and frameworks involved. I've already touched on things like different memory allocation, GC behavior, template engines, what C libraries might be available in certain ecosystems to avoid processing in the language itself, etc. But that's so hard because it requires a ton more work than people want to put into a fair comparison. More on reddit.com

Yes More on reddit.com