Is checkpointing inside of an epoch so that the dataloader is reinitialised correctly possible?

From looking around a bit it seems like the dataloader stuff isn’t saved so restoring from training in the middle of an epoch isn’t completely implemented right? Or did I miss something?

Hi ! It is if you use the ckpt_interval_minutes: argument in your yaml config file. We decided to do it time-wise as this is better than step-based (take into account the hardware speed).

I’m talking about the fact that if one stops training in the middle of an epoch, then speechbrain does not contain the functionality to save the state of the dataloader, so when you restart training you will start training from the start of an epoch instead of where one stopped!

I don’t see how setting ckpt_interval_minutes will help with that?