Comments by "clray123" (@clray123) on "Asianometry" channel.

  1.  @luciustarquiniuspriscus1408  The SSM/Mamba papers already address this. In fact you can train a GPT-3 like small model using Mamba right here and now, with excellent performance (both in terms of training speed and outputs). With "infinite attention" (well, limited by the capacity of the hidden state vector).
    1