If you want to do research you pretty much need a PhD for any larger companies. Some startups might hire you with a msc but probably they would prefer PhDs as well. I think with a bsc it will be extremely hard to get remotely close to anything involving research. There are a lot of people doing ml right now and finding a job without the proper qualifications is very rough.

CPU bottlenecks can be easily found by monitoring the CPU usage during training. If all of your cores are constantly at 100% your cpu might be too slow. If both the cpu and gpu are idle from time to time your storage could be the bottleneck.

To increase data loading performance, you could try out Nvidias Dali or FFCV which are both libraries optimized for that purpose. They replace some of the inefficient python code with highly optimised code. FFCV is quite nice but it requires you to convert your dataset into a specific format.