Last update: 2023-05-06
Having a similar need to the OP, I searched the whole web for current solutions. There's no current perfect solution, but there are a few ones that might be worth a try.
Note that I had additional requirements to realtime/inline previewing, such as: multidocuments tabbing, table of contents for quick navigation, lightweight, markup hiding/showing (sourcecode mode) with styling kept and finally autosave of documents and of session.
I will here describe two categories of editors that can fit OP's purpose depending on what exact kind of previewing is expected (inline preview vs inline styling). I will describe a few main features non exhaustively, you should consult the websites to get more information. I also chose to leave split-pane only editors (ie, without inline preview nor styling) out of the picture, as there are plenty and are explicitly excluded by OP.
Note that I have personally tested all the ones running on Windows.
Inline preview (aka realtime preview or WYSIWYG)
All the following editors allow you to type Markdown markup, and automagically display the resulting style instantly inside the editor (no split-pane for preview).
This is also sometimes called WYSIWYG, but this is different here as you do not necessarily have to use the toolbars to format the text: with inline preview you can just input markup code and it will display the result dynamically. In some sense, this is very similar to what LyX pioneered for LaTeX.
There are basically two types of inline preview: per block (refresh the styling after you leave a block, eg, by going to the next line/block) or per character (ie, as you type).
I am not demonstrating image insertion but they can almost all preview images, both local and remote.
Note that almost none of the app here supports multi-document tabs (yet?), except for the note-taking oriented apps.

Opensource
Cross-platform (Windows + MacOSX + Linux)
Inline preview
Mouse-friendly interface (easy to insert a markdown construct from menus, or by click at the beginning of the line, or by highlighting a text, options are then displayed)
Math support (LaTeX)
Sourcecode mode
Support for a lot of markdown constructs: tasks, code blocks, HTML block,
Export to HTML, PDF
Focus and typewriter modes
Table of content
Integrated search
Multi-tabs, multi-documents
Support for images, but not for embedded videos
Support for complex constructs, such as graphs, such as vega-lite, or mermaid
Basic autosave support (check File > Autosave). Currently working on a full session manager with more advanced autosave.
Basic support for attachments (either in a folder defined by user, or transparently uploaded to an online repository)
Note: this project is in very active development and already has a big community. It is a personal favorite, it managed to catch up with older editors in terms of features (or even surpass them).

Opensource
Cross-platform (Windows + MacOSX + Linux)
Inline preview (but mixing both rendered and some unrendered but dimmed down markdown formatting, eg, bold and italics)
No sourcecode mode (but there is a readability mode to highlight non-markup text content)
Full note manager, multi-tabs
Attachments support
Bibliography and citation support
Footnotes support
Tables support
Tasks support
Table of content
Mathematical equation support (LaTeX)
Lots of importing and exporting formats: HTML, PDF, DOCX, LaTeX (great for scientific collaborations!)
Code blocks with syntax highlighting
Search (and replace) function integrated
Statistics graphs
Autosave (any change is saved)
Notebook paradigm: you select a folder, and all your notes will be stored inside. It is thus easy to have multiple notebooks and switch between them.
Tags
Fully localized in lots of languages
Note: this software was initially made for scientists, hence the extensive support for attachments, bibliography and footnotes. The extensive import/export from/to LaTeX and DOCX is a killer feature for researchers which can ease collaborations. In the end, we could say it's the Markdown equivalent of Zim Desktop Wiki, they are very similar in their philosophy and feature set.

Commercial (but cheap: 19$)
Win + MacOSX
Inline preview + Presentation support too + Bibliography support + Footnotes
Math support
Can import LaTeX documents
No sourcecode mode (cannot show Markdown)
No autosave nor tabs.
No task list support.

Opensource (GPL)
Win + Mac + Linux
Inline preview + TOC navigation (Table of Content)
Math support
Only sourcecode mode (cannot hide markup, links cannot be beautified)
Actively in development (so the following issues may resolve fast)
Bug with loading external pictures under https
Cannot click on links
No [autosave](https://github.com/brrd/Abricotine/issues/17) nor [tabs](https://github.com/brrd/Abricotine/issues/10) (but issues are open, feel free to +1 or even contribute a patch if you know [Electron](http://electron.atom.io/)!)

Original announcement
Commercial, free during beta
Win + Mac (+ Linux in the future)
Inline preview + TOC navigation (called Outline)
Math support
Both sourcecode and no markup mode are available.
Autosave + documents recovery
Can insert an auto-updating TOC (in addition to TOC navigation)
No bibliography support (yet?) but can add "link reference" and footnotes.
Auto-reload notification when document changes.
Live preview as-you-type of mathematical equations.
No multi-documents tabs.

Opensource
Win + Mac + Linux
Zenware
Inline preview
Math support
No task list support

Commercial (free limited version available)
Mac
Inline preview (sourcecode mode only, cannot hide markup)
Automatic cumulative time computation from task description

OpenSource (AGPL v3)
Cross-platform (Windows, Mac, Linux, Android, Firefox, Chrome)
Inline preview
Documents manager
Tags
Note: this is a very interesting solution if you do not just want to write notes but also manage files.
Web-based apps with inline preview

Opensource (not released yet)
Win + Mac + Linux
Inline preview of Markdown + WYSIWYG
Video

Opensource (not released yet)
Win + Mac + Linux
Inline preview of Markdown
Note-taking oriented (multi-documents)
Same author as Marko Editor (and same codebase)
Mix between a note taking app and a mind-mapping software, with automatic graph reorganization algorithms
Everything is stored as plain text
Personal note: this app seems goddamn amazing, but it is not yet available unluckily! Please reach the author if you are interested too!

Zim sourcecode repository
Disclaimer: the following review is a bit biased as I am regularly using this software.
Opensource (GPL2)
Win + Linux + Mac + BSD (runs on Python + GTK2, should be adaptable to any platform supported by GTK2)
Inline previewing (can type wiki markup code and it's styled on-the-fly) + WYSIWYG
Note-taking oriented (multi-documents)
Notes syntax and stored in hybrid DocuWiki/Markdown "lightweight" markup
Can export/copy snippet, single or all notes to standard Markdown
TOC navigation (plugin)
Very mature project (started in 2007, 9 years of development) and still active development (as of 2016)
Multi-documents tabs (with memorization of the order you opened them)
Autosaving per character (aka realtime autosaving)
Tags, pictures, can attach files alongside notes, calendar, etc.
Can copy/paste pictures from Paint or similar directly into a note (and can afterward resize) - such pictures will be stored inside your Zim notes folder
Clickable links
Remember last cursor position for every notes
LaTeX math formulas support (can be later edited by right-click)
Reorderable tables (markdown syntax) support
Also, lots of plugins, included natively in the app (take a look, they are all amazing, can even interface with GNOME's Zeitgeist, Lilypond music sheet, draw ASCII diagrams, etc). Here are a few I advise:
- Support for multi-users editing or cloud storage (dropbox etc) with 3-way diff (automatic and manual) with this PR.
- Support for bibliography and citation with the zim-zotero plugin.
Cons:
No Markdown support for storing the files (can only import/export to Markdown, but otherwise the notes are stored as Zim markup language). UPDATE 2020: there is now a markdown-zim bidirectional converter: [markdown2zim][33]. This can be used to migrate to/from other markdown editors.
/UPDATE 2018: to get instant refresh for all markups (like bold, italic, etc.), you need to enable the option "Reformat wiki markup on the fly" in the Editing panel of Preferences.
- Personal note: I am using Zim + Typora until something like Abricotine matures enough to be used for serious multi note taking* - UPDATE: I now use Zim + MarkText, and I may migrate to Zettlr.

Collate (aka CollateNotes) is a note manager with an integrated Markdown editor mixed with YAML to store metadata. It has live rendering and editing abilities.
Everything is stored on local computer.
Made with crossplatform compatibility as a requirement (Windows, Mac, Linux).
Supports syntax highlighting for most languages.
Support for Github Flavored Markdown
Clear file format specification: https://github.com/Collateapp/CollateFileFormat
Tags
Commercial software, free trial for 14 days.
Inline formatting/styling
The following editors won't provide a full Markdown inline preview, but they provide a prototype formatting/styling of your document. This allows you to somewhat know what your document will look like, even if some elements may not be correctly styled (or previewed at all like images). They also all provide a split-pane or a separated window for a full preview.
Note: inline styling is NOT just a simple regex in a code editor (such as Notepad++, SublimeText, Atom, etc.), because they cannot differenciate between different markup depths (eg, all headers levels are shown with the same styling), hence, they are excluded here.

Freeware
Android
Stores each note as its own .md file directly in a Writer folder at the root of SD card (ease backups)
Easy to share notes by email (either directly as the email text content, or as attachment)
Probably a sequel to [Writer](https://play.google.com/store/apps/details?id=com.jamesmc.writer&hl=fr)
Note: I used this to replace ColorNotes, which stores notes in a database that is difficult to access and is prone to data loss. I use this one since years now instead and never lost a note since I back them up along with photos and other materials regularly.

Commercial
Cross-platform (Windows, Mac, Android, iOS)
Synchronization between devices
Note: it looks to me to be similar to Writer Plus, if I was using an iPhone or iPad, this is probably what I would be using.
Mou (archive.org)

Commercial
MacOSX

Opensource
Win + Linux
TOC navigation
Note: Seems like this software is not maintained anymore.

Opensource
Linux (+ Win in the future)
Note-taking oriented (multi-documents)
Preview when saving the document
Note: seems to be unmaintained since a few years now (the official website is dead, only the github remains).

Commercial
Mac
Sourcecode/preview switching + WYSIWYG

Opensource (GPL)
Win + Mac + Linux
Inline styling/formatting
Note-taking oriented (multi-documents)
Said to be an equivalent to Quiver
Multi-documents tabs
Support for encrypted notes
Make your ownCloud (+ web editor supports inline styling)
Support for versioning

Freeware (with ads) or commercial
Mac
Seems to support inline previewing
Note-taking oriented (multi-documents)
Web-based apps with inline styling/formatting
Video
Freeware
Win + Android
Purports to support Markdown, but I couldn't find how
Note-taking oriented (multi-documents)
Stores in HTML by default, not Markdown