Frank Odom
Dec 23, 2021

--

Thanks, Zhiyuan. I believe the factors of two are correctly accounted for? See my response to James Liu, copied below:

"I believe the phase is correct, though, because it's applied to the positional encoding at index 2i (+ 1). That should offset dividing by two in the exponent."

Please let me know if you still see an error, and I'll be very happy to correct it!

--

--

No responses yet