Yes, you should use git*.
Now let me explain why. Given the current (rather nebulous) set of criteria in your question the answer seems fairly obvious. If you knew any more, you wouldn't even be asking this question. You have already brought yourself to the edge of the cliff, now you just need coaxing to make the jump.
* Or Fossil, Mercurial, Darcs, Bazaar, or other DVCS depending on the front end tooling you prefer, to be explained.
The current scene and some history:
There are basically three kinds of version control systems: distributed, ham-fisted, and tapped-out. Allow me to expand on that technical terminology and how each came to be and would apply to your situation.
ham-fisted
Notable entries*: CVS, Subversion.
Before DVCS systems took the world by storm, there were VCS systems. These could be characterized by a central repository/server and a star pattern of user-workspaces/clients. These were an indispensably valuable tool for keeping a team of programmers on the same page and even adapted themselves to other uses. A single programmer could work from multiple systems and play around with branches and tags. They saved many a day. But they were inherently clumsy. They make some simple tasks harder. First there was the overhead of setting them up, the need for a server and specific protocols to connect them. Then there was the pain of dealing with those times you did something wrong. Suffice it to say these would get the job done for your use case but would introduce trade offs making life more complicated.
tapped-out
Notable entry: RCS.
For when a "full blown" system involving servers and clients and authentication and all that jazz was too much to swallow, it was possible to use a pared down system that lived in its own little bubble. RCS did this by eschewing the idea of a repository and just versioning one file at a time. You had file.txt and sitting right next to it a file.txt,v that had the version history. You could instantiate it easily on a per-file basis and use a handful of simple tools to work with diffs, roll back time, etc.
Now before you say, "Ah ha, that's just what I was looking for!", please read on because this is not the easiest or recommended way to do this any more. Easy entry came at the cost of a low operational ceiling that is pretty much guaranteed to cramp your style sooner or later.
distributed
At some point a bunch of smart programmers decided they had had enough of the pain and said, "We are going to eat cake and have it too." Amazingly, they succeed and distributed version control systems were born. These systems combine the best of both worlds—complete version history sitting on your local system right next to the original files plus the ability to share your history and interact with that of other remote repositories. It turns out there are no serious technical disadvantages to doing this.
The most significant barrier for some shops turned out to be the flexibility. Because the systems don't impose arbitrary restrictions on the way you work, it is sometimes painful to migrate from a system that forced you to have a certain workflow. Suddenly you have to think a little bit about how you want your system to work. Many things that used to be required (central node that always has everybody's latest stuff) became a matter convention to be used only when desired but you have to sit down and say, "This is how we are going to use this tool."
So lets sit down and say how you would use this tool.
For the purposes of this answer I am going to stick to git because it is one of the most widely adopted systems. It is easy to install on most any system (on the off chance it isn't installed already) and there is a wide range of documentation available covering almost any use case. It is also extensible and recognized by many third party systems, etc. That being said it is not necessarily inherently better that its nearest competition, Bazaar or Mercurial.
If you live in an Ubuntu ecosystem, you might want to give Bazaar a look because Canonical uses it for everything and it will integrate well with your environment. Their launchpad service is similar to Github, but tailored to Ubuntu software development. If you plan to play in that ecosystem, consider learning bzr instead of git so that one tool works for both your personal world and the ecosystem you participate in. If you don't work on projects coordinated on launchpad, I would suggest using git.
If any of your colleagues is big into Mercurial, you might want to look into using it. It's a very capable DVCS with some advantages over Git. It's frequently alleged to be faster for some operations due to a more streamlined data flow. The tooling is all wrapped into a single binary rather than being clobbered together from a bunch of separate (and sometimes redundant) tools like Git is. It it extensible using Python bindings and it's possible to built external systems that integrate very tightly with it. The paradigm is similar enough to Git that once you learn it you'll also be able to blunder your way around a git repository. In the end however git is the most popular player in the field right now and sticking with it will give you readier access to help when you need it.
* My apologies to all the VCS systems not named. CVSNT, CA, CC, Perforce, Plastic, PVCS, Star, SVK, Vault, Vesta, VSS and a litany of others. Your names will never be forgotten are already just a memory.
How git fits your use case:
You mentioned having used Github a little. That's great, but you need to keep in mind that Github is not git. It is a common miss-perception that what they are offering is a free way to get into the system by hosting your repositories for you. In reality what they offer is a layer built out on top of git that is part social-networking and part project-management. This is a great thing for the open-source community. Instead of trying to be an alternative system and fighting the market, they have brilliantly leveraged a good tool and carved out a market providing a value added service for corporations and serving the community at the same time. But Github is not git.
Git can, in fact, be used in a much simpler fashion.
Similar to RCS, git stores the version information locally right next to your content. The notable difference out of the gate is that it does this on a per directory basis rather than a per file basis.
RCS:
file1.txt
file1.txt,v
file2.txt
file2.txt,v
The ,v file for each file keeps a running list of the file history, storing the delta between each consecutive version.
git:
directory
+ file1.txt
+ file2.txt
+ .git
+ glob1
+ glob2
The stuff in the .git folder actually has funky names and is kind of complicated, but you don't need to know about it. Conceptually all it is doing is storing the differences between versions of stuff it your directory. Basically each glob is an image of what your directory looked like at the time of each commit. A lot of fancy math keeps the overhead down so that only the delta data is saved.
Now this may be sounding complex already, but you really don't need to know any of that. The tool-set keeps track of all the fancy stuff for you. The basic usage in very bit as easy as RCS but gives you room to grow down the road.
Getting started would go something like this:
# Change to the directory that has files you want to version.
cd ~/pathto/yourtextfiles
# Initialize git to keep track of that folder
git init
Done. No servers needed. Just you and your version controlled files. Except you don't have any files under surveillance yet so git isn't actually watching anything. Git is not greedy in the sense that it does not keep track of everything in a folder, only the specific things you tell it to. So the next step is to tell it what files you want to track.
# Add some files to the system, assuming these already exist the your dir
git add file1.txt file2.txt
# Commit the changes you just made
git commit -m "initial add"
Note that unlike most systems, this is a two step process. Before you commit things and stuff them in the repository as a new version, you have to 'stage' them. Not every change you make to your working directory is automatically assumed to be something you want to save to with each commit. Maybe you changed two files but only want to mark one.
# Edit a bunch of files
vim file1.txt
vim file2.txt
vim file3.txt
# Only mark one of them as going it your next commit
git add file2.txt
# Commit it to history
git commit -m "fixed typo"
Note that the changes to file1.txt are not yet saved to the repository and file3.txt is not tracked at all yet. You can see that a previously tracked file has unstaged changes by running git status and what those changes are by running git diff. It this point status would tell you that you have made changes to file1.txt and that there are there is an untracked file3.txt in your directory. Diff would show you the changes you made to file1.txt since the last time you staged and commited it.
One 'gotcha' that you should be warned about starting out so it doesn't bite you down the road is that – because git's concept of a repository is a whole directory and the changes to files are seen as an image of that directory at a point in time – you should consider having separate repositories for disparate projects. Rather than making a single repository out of "My Documents", you should make separate repositories out of directories that contain some meaningful subset of your documents, whether per-topic or per-format or per-project or whatever. This will make it easier down the road when you want to work with "all documents related to x" from another machine without having to have every document you've ever created on that machine. Git does not allow you to checkout a subtree of a repository*, it is all or nothing, so err on the side of making many granular repositories for closely related data sets, one per directory.
Really that's all there is to basic usage. From there, almost anything you can imagine to do is possible, but at that point you would be asking a usage question, not a software recommendation question.
* Subversion for example did allow this and I got used to it. This bit me early on when I assumed git would allow something similar. I had ALL my personal files in one large svn repo and naively assumed git would be a drop in replacement for that. Lesson learned, and my files are the better off for being categorized.
But command prompts are scary!
There are, of course, a multitude of front end GUIs available to keep you off the command line and visually represent what is going on with your files. Many of these have an IDE-ish flavor to them and might serve as your entry point into document management, using them to launch your favorite editor* or even use a built in one. Since you asked about the back-end about how your files should be stored, I have made a recommendation to use git as the version manger, but if you have in mind a way this should look and work from a front end perspective, you should ask that as a separate question.
* Of course gvim would be well suited for this use case ;-)
In conclusion:
Come on in, the water is fine!
One! Two! Three! Jump git init.
See that wasn't so hard.
version control (essential, obviously!)