WinSCP matches all your requirements.
Graphical: Yes – There are two alternative interfaces:
Feels like a Windows explorer window: Yes – One of the interfaces is explicitly designed after Windows File Explorer.
Supports drag and drop: Yes – Both between file panels and between other applications.Uploading via ...
I have used the 3 following FTP clients during the past years so here is my feedback:
WinSCP: the one that I currently use. There is a portable version, just one .exe file (and the .ini file for configuration). Don't forget to turn on the "Explorer" interface:
View > Preferences > Environment > Interface > Explorer, which is (IMHO) handier.
Benchmark: it ...
WinSCP allows automatic updates to donors. So it's not "gratis". But it is possible and only $9 donation (or equivalent) is required.
FileZilla allows automatic updates to anyone (on Windows). See How to automatically update FileZilla?