On: Deep Learning in JavaScript

[GaggiX]

The nn.Block should probably be renamed to nn.DecoderBlock as it's not very clear if it's supposed to be an encoder or a decoder block, also an option to disable the mask from attention. That said, very cool project.

[eduardoleao052 → GaggiX]

You are absolutely right! Will solve that as soon as possible. Thanks for the feedback!

[jsight → GaggiX]

If you have the option to disable the mask, isn't it then a generic nn.Block?

[GaggiX → jsight]

The ability to disable the mask in nn.MultiHeadSelfAttention, then having nn.DecoderBlock and a nn.EncoderBlock.

[eduardoleao052 → GaggiX]

I think that’s it. I’ll probably add that soon

[eduardoleao052 → jsight]

It would be with some simple tweaks. For instance, the current block does not support Cross-Attention, just Self-Attention.