A walk through of the DeltaNet family of linear attention variants

This article provides a comprehensive walkthrough of the DeltaNet family of linear attention variants, covering theory, implementation details, and performance comparisons, ideal for those looking to understand efficient Transformers.

• Transformer
• linear attention
• DeltaNet