not that I can think of; it is very straightforward. combining reports from shards is a little complicated but that's all covered in the docs. keeping the container version up to date with your playwright version is tedious I guess but it's not difficult. More on reddit.com

Great writeup. Really useful for my workflow!

More on reddit.com

TL;DR: really good. And at all auto frameworks not just Playwright. More on reddit.com

Flakiness is hard to tackle because it could be a test issue, app issue, or infra issue. You really have to go through the whole stack to find where the problem lies. Also sometimes the test may only fail while running the whole test-suite and not when the test is run alone. But there are ways to help reduce the net. To get rid of test issue possibility , run the tests locally on a beefy machine in parallel against itself. If you got bad test code or a race condition, it will show up here. For example If you run it serially 100 times and it passes a hundred times, but then run it with 2 workers in parallel 100 times and it fails horribly, then you know the culprit. If it fails sometimes then I keep digging and try to figure out if it’s a bug In the code. This one is more difficult and requires help from an expert in whatever domain the test is failing sometimes but it is worth digging in because these turn into bugs in production. If you find that increasing the parallel workers on a test or test suite increases the flakiness overall and you’re in the 10-20 worker area, chances are you’re hitting a infrastructure bottleneck That requires using tools like HTOP on the given machines to figure out what’s getting eaten up and starts another journey. We reduce flakiness by stress testing every newly added test and monitoring every tests flakiness on a dashboard. We also quarantine and tackle the test asap when it acts up because it’s possible it only fails in certain conditions If you have a good test culture you’ll notice flakiness remains at a minimal. If you find a test to continually be flaky no matter what you throw at it then it’s time to evaluate if the test is worth it or worth rewriting or testing in a diff way depending on how mission critical it is TLDR: Flaky tests are a bitch so if a best to prevent them in the first place and tackle them asap when they reach a flaky threshold. More on reddit.com