non_flash branch

[nassim12sol]

Hi I have a question ,
Can i use the non_flash branch with the pretrained model , and would it provide the same results ?

[RJPenic]

Hi, you can use it! We haven't thoroughly tested it, but representation embeddings are roughly the same as with the Flash-Attention-version.