Thanks for your suggestion. I tried but this won’t resolve the issue, which makes sense I think. Let me share my thoughts:
Instantiating the pretrained model with the argument
freeze_params=False will only leave the model in
train mode and thus the parameters will have
But that’s not what I aim for. I’m rather happy that the model parameters are already frozen. Namely, I would like to backpropagate through the model w.r.t. to the input and not w.r.t. the parameters.
I initially fell short of additional motivation for my use case I guess. Namely, I am looking into generating adversarial examples. That’s the reason I’m interested in obtaining gradients w.r.t. to the original inputs and not w.r.t. to the parameters.
Following your suggestion, I continued to dig a bit deeper as well. As stated in my first post, the problem likely lies in the Filterbank, the feature extractor. It’s wrapped in a
torch.no_grad() context. Thus this will always break the computation graph.
Consider the following example:
Let’s assume we have set
requires_grad=True wherever necessary. The forward pass of a
Fbanks object calls the forward of a
Filterbank instance, see speechbrain/features.py at 5141b53a172385f08ea250774c9192e6114d647d · speechbrain/speechbrain · GitHub. In the latter forward call, we create a new tensor
f_central_mat derived from the attribute
self.f_central, which is a parameter and thus
f_central_mat.requires_grad=False. Again, this is expected due to the outer context
torch.no_grad() of the
Fbank forward pass.
I wonder, if it is possible to set the
torch.enable_grad() context when the
Fbank.requires_grad is set to
A similar issue holds for the
MFCC feature pipeline.
Looking forward to your thoughts.