First: generally speaking (which is to say, in a majority of cases), applications in OS X are just files with a .app extension*. You can move app files wherever you want, and double click on one to launch it. "Installing" an application usually consists of moving the .app file to the "Applications" folder that's preinstalled on any Mac, but you don't have to move it there. You can place the application wherever you want. I, for example, have all of my games in a special "Games" folder that's separate from "Applications". There is also nothing stopping me from moving an app to an external drive and running it from there.
The point I'm trying to make here is that just about any Mac application can be run from a USB stick. Just drag the .app file to the USB stick to "install" it, and delete it when you're done.
Edit: Rereading this answer two years later, there's one caveat I forgot to mention—most Mac applications will save user data to your home folder (and specifically your Library folder) regardless of where they are run. So while almost any Mac application (including Sublime Text) can be "run" from a USB stick, preferences and other user data will still be local to the machine.
*Which are, in turn, actually folders with more files inside of them, but OS X treats them as single files, so let's just pretend they are for simplicity's sake.