You don’t have to make it a part of the kernel, it could be an external module, like the firmware. I’ve done it before, it’s not as scary at it sounds. Yes, at a certain point, it will stop to work and you’d have to recompile with a new compiler (if that doesn’t work, code changes need to be implemented), but in most cases, you don’t have to change a thing, except download the new source for the driver and build it again.

It usually works for about a year or two, then you have to rebuild, so it’s not that big of an issue.