Dec 23, 2021
Thanks, Zhiyuan. I believe the factors of two are correctly accounted for? See my response to James Liu, copied below:
"I believe the phase is correct, though, because it's applied to the positional encoding at index 2i (+ 1). That should offset dividing by two in the exponent."
Please let me know if you still see an error, and I'll be very happy to correct it!