I’ve dealt with similar issues in my own projects.
A couple of pointers :-
Use Image formats that are fast to decode for example BMP ( you can try converting all your images to BMP before you start training )
This will increase their size on disk but should reduce the CPU load.
If you are doing any complex preprocessing on large images in your dataset class, try preprocessing images first and storing them to disk and loading those directly
These are just some general suggestions. It’d be more helpful if we knew more about your task so that we can offer more directed suggestions :)
I’ve dealt with similar issues in my own projects.
A couple of pointers :-
Use Image formats that are fast to decode for example BMP ( you can try converting all your images to BMP before you start training ) This will increase their size on disk but should reduce the CPU load. If you are doing any complex preprocessing on large images in your dataset class, try preprocessing images first and storing them to disk and loading those directly
These are just some general suggestions. It’d be more helpful if we knew more about your task so that we can offer more directed suggestions :)