why do I even need them in the first place?
->You don't. you might want them.
Why not just use local auth and check the users credentials every time or use a session?
-> You can (I don't know much about local auth but you can use sessions)
Shouldn't the lookup time to find the user with the token id be the same as looking up a username or session if ...