End to end and smoke tests give a really valuable angle on what the app is doing and can warn you about failures before they happen. However, because they’re working with a live app and a live database over a live network, they can introduce a lot of flakiness. Beyond just changes to the app, different data in the environment or other issues can cause a smoke test failure.
How do you handle the inherent flakiness of testing against a live app?
When do you run smokes? On every phoenix branch? Pre-prod? Prod only?
Who fixes the issues that the smokes find?
I find it interesting that the junior would fix these issues. In my team we established a fictive role which each developer takes in sprint rotation. The developer in this role would then handle these cases and also drive the investigation of incoming support requests / bug reports, playing third level support in a sense.
How do you handle support requests in your team/company? Is it the developers or do you have a dedicated team for that?