I am looking for a browser extension that can force the YouTube video to get entirely cached upon loading the page (or at worst when I start playing the video). If possible, for Chrome and free.
Well I am not sure if there is an extension for this, but you might be aware of Google Chrome's new HTML5 player which seems to do this and uses this new format on YouTube instead of Flash Player.
I find it really useful (Google Chrome's new format) and some of the benefits I noticed of this was that it pre-loads the YouTube video, even when you pause the YouTube video and also seems to load the entire video too when paused.
Sorry if I misunderstood your question here, but I don't think an extension may be needed for this though. But by default Chrome already uses this format on YouTube. See here too. :)