Is there any video chat app that works on Windows and Linux and has an always-on-top / pop out video window like Skype?
Bonus points if it's open-source and/or allows self-hosting.
You can try Ekiga. It's an open-source SoftPhone, video conferencing and instant messenger software. It runs on windows and linux. I believe you can host with it, but I've never done it. It should be explained in the Manual.
To make it always on top with windows, I use AutoHotkey. There are some good explanations in this HowToGeek article.
Jitsi meet. You can always make the browser window on top. It is open source.
As far as I remember, TrueConf allows self-hosting. It is not open-source. They have a separate server and clients for many platforms including Linux. I remember TrueConf company as an experts in video conferencing. I met their representatives on one exhibit.
Try Google Hangouts.
Download it as Application OR Use it as extension in the browser(Only in Chrome)