Gufw Firewall looks like it could match your needs:

Gufw Firewall (source: ubuntu.com; click image for larger variant)
I have not tried it out myself, but according to some screenshots it seems to fulfill your requirements:
- Firewall for Ubuntu: Yes
- block/allow single applications: According to the screenshot: Yes (see the "Application" drop-down and, below that, the "Application filter". What makes me doubt this a little is reading between the lines: options for controlling firewall options for common programs and services could also be interpreted like "Selecting Postfix it would block ports 25, 465 and 587 (smtp/smtps/submission) globally for the device". This is not clarified on the page.
- have a GUI: Yes, as the screenshot proves.
There's a list of Linux firewall GUIs, but I'm afraid none of those has the "per application" feature.
Related questions/answers on our sister sites
As the AU article suggests, Douane seems to match your needs exactly:
When Douane is started, it will watch the ougoing network traffic and as soon as an unknown application tries to send some network packets, Douane will block it and ask you if you allow it or not:

What's left open in the description is whether you can define "exceptions" or make your choices permanent (imagine that popup appearing on each Firefox request) – which I assume can be done in the rules tab of the Douane Configurator:

Douane Configurator (source: Douane; click image to enlarge)
So let's check this one against your requirements:
- Firewall for Ubuntu: Yes (but currently needs to be installed manually, see below)
- block/allow single applications: Yes (see screenshots for prove, also the above linked AU article explicitly says so)
- have a GUI: Yes, as the screenshots prove.
Drawback: Packages are currently only available for ArchLinux, so you'd have to compile and install it manually. Note there's already an open request for Ubuntu packages.
Update: Meanwhile there's a Douane PPA available for Ubuntu.