Why Attention: Seq2Seq the Bottleneck Problem

less than 1 minute read  ·  Updated:

A more general definition of attention:
Given a set of vector values, and a vector query, attention is a technique to compute a weighted sum of the values, dependent on the query.

alt text alt text alt text alt text

Reference

Leave a Comment