The test: https://github.com/curvednebula/perf-tests

So in the test we run 100'000 parallel tasks, in each task 10'000 small structs created, inserted into a map, and after that retrieved from the map by the key.

Go (goroutines):

• finished in 46.32s, one task avg 23.59s, min 0.02s, max 46.32s

• RAM: 1.5Gb - 4Gb

Rust (tokio tasks):

• finished in 67.85s, one task avg 33.237s, min 0.007s, max 67.854s

• RAM: 35Mb - 60Mb

[UPDATE]: After limiting number of goroutines running simultaneously to number of CPU threads, RAM usage decreased from 4Gb to 36Mb. Rust's tokio tasks handle the test gracefully out of the box - no optimization required - only mimalloc to reduce execution time was added.

First, I'm not an expert in those two languages. I'm evaluating them for my project. So my implementation is most likely not the most efficient one. While that's true for both Go and Rust, and I was impressed that Go could finish the task 33% faster. But the RAM usage...

I understand that golang's GC just can't keep up with 100'000 goroutines that keep allocating new structs. This explains huge memory usage compared to Rust.

Since I prefer Go's simplicity - I wanted to make it work. So I created another test in Go (func testWithPool(...)) - where instead of creating new structs every time, I'm using pool. So I return structs back to the pool when a goroutine finishes. Now goroutines could reuse structs from the pool instead of creating new ones. In this case GC doesn't need to do much at all. While this made things even worse and RAM usage went up to the max RAM available.

I'm wondering if Go's implementation could be improved so we could keep RAM usage under control.

-----------------

[UPDATE] After more testing and implementing some ideas from the comments, I came to the following conclusion:

Rust was 30% slower with the default malloc, but almost identical to Go with mimalloc. While the biggest difference was massive RAM usage by Go: 2-4Gb vs Rust only 30-60Mb. But why? Is that simply because GC can't keep up with so many goroutines allocating structs?

Notice that on average Rust finished a task in 0.006s (max in 0.053s), while Go's average task duration was 16s! A massive differrence! If both finished all tasks at roughtly the same time that could only mean that Go is execute thousands of tasks in parallel sharing limited amount of CPU threads available, but Rust is running only couple of them at once. This explains why Rust's average task duration is so short.

Since Go runs so many tasks in paralell it keeps thousands of hash maps filled with thousands of structs in the RAM. GC can't even free this memory because application is still using it. Rust on the other hand only creates couple of hash maps at once.

So to solve the problem I've created a simple utility: CPU workers. It limits number of parallel tasks executed to be not more than the number of CPU threads. With this optimization Go's memory usage dropped to 1000Mb at start and it drops down to 200Mb as test runs. This is at least 4 times better than before. And probably the initial burst is just the result of GC warming up.

[FINAL-UPDATE]: After limiting number of goroutines running simultaneously to number of CPU threads, RAM usage decreased from 4Gb to 36Mb. Rust's tokio tasks handle this test gracefully out of the box - no optimization required - only mimalloc to reduce execution time was added. But Go optimization was very simple, so I wouldn't call it a problem. Overall I'm impressed with Go's performance.

Hi!

I don't want to start another language war, so let me preface this by saying I love both Go and Rust like they are my own.

That out of the way: Do you have practical/real-world experience with the performance of Go, and the performance of Rust in the same setting?

How many people worked on the codebases? Which one is newer?

I'm asking because I've got a feeling that for smaller companies, Go would in practice (not in theory) outperform Rust because Rust's design encourages the use of inefficient patterns -- like cloning a bunch where you could theoretically pass a reference