I have the same experience. I wrote a simple program with SDL2 to test a software renderer. All it does is create a window then go into an event loop and after each iteration it streams a framebuffer to a texture that gets displayed in the window. In the default mode (X11) my frame timings fluctuate a lot and for a while I tried to massage the code to get it stable because I was convinced that it was just my draw code. Then I eventually forced SDL2 to use Wayland and not only did the draw time per frame go down by 2ms but the fluctuations went away completely.