Pretraining,transfer learning

Hi,
I want to train a ASR Model on a very large ‘Hindi’ Language dataset and then use the same trained model to finetune on few other small datasets using transformer+ctc attention model and conformer models. is this possible using this toolkit?

Hey, yes wee these two tutorials:

Pretraining and Fine-tuning with HF

Speech Recognition From Scratch

2 Likes

But,pretraining and fine tuning is done using hugging face models. I want to fine-tune using the model I trained.

It is exactly the same, just replace the hugging face hub name with your path :slight_smile:

Hi,
I want to train xvector model for speaker identification with initial weights of pretrained model on CommonVoice dataset. What should the manifest format be like? and how do I use initial weights?
thanks.

Hi, start with this tutorial. The concept is really simple, put your xvector in the yaml, define the Pretrainer in the yaml as well (exemple: hyperparams.yaml · speechbrain/asr-crdnn-commonvoice-fr at main). And then calling

pretrain.collect_files()
pretrain.load_collected(device='cpu')

In y our recipe, and you are done !

thank you.
we follow this tutorial and train network without initial weights then we added this lines to train_speaker_embeddings.py file:

model = hparams["embedding_model"]
pretrain = Pretrainer(loadables={'model': speaker_brain}, paths={'model': 'speechbrain/spkrec-xvect-voxceleb/embedding_model.ckpt'})

pretrain.collect_files()
pretrain.load_collected(device='cpu')

I doubt the initial weights are loaded on the model. how can I test it? Is it true?

The logs should tell you if it is loaded or not :slight_smile: Otherwise you can just have a look at the weights directly by accessing them as for any PyTorch layer !