Maybe the iMapping Tool is right for you.
It is a visual knowledge mapping tool, that lets you organize your text snippets, files and pictures visually on what could be described as an infinite pinboard.
Items can be nested and interlinked so you can add structure as your collection grows.
Comments to your requirements:
shallow learning curve, I want to start right away. Learning a bit markup is okay though
stores data locally
supports Tex/Mathjax like syntax for mathematical and chemical formulae
allows inclusion of video, pictures and pdf in some way
(√) pictures are displayed inline, local files and remote URLs (Videos) are referred to by clickable links (currently not clickable in free version but soon yes.)
high flexibility in organizing and re-organizing my data
√ That is what iMapping is made for.
You can also create duplicates (so-called equivalents) of whole parts of your maps, so you can rearrange one version while keeping the original. And these "eqivalent" items are still connected in the back-end, much like hard-links on linux.
What I don't like is the tree structure, every item can only have one parent. Much of the stuff I find intersting is interesting in more than one context
√ The mechanism of these equivalents lets you creade several instances of an item in different contexts. In the included semantic search tool, they are still one item.
Shouldnt make it needlessly hard to export the data and structure into another format when I change my mind about the system to use
Well, it is a little hard, but not needlassly ;) Data is stores locally in an open semantic format, which, however is not widely used.
Used to be but currently is not. You might get the sourcecode based on a bilateral agreement though.
runs on linux
√ While not officilly supported, it inofficially does.
The website is in German, while the tool itself is English. Just check out the video - you will get an impression.
Disclaimer: The iMapping tool is my own development I sell it commercially (although there is a free version availabe for download).