My application most closely resembles that of a market trading application. It is a .NET 4.5 Console Application written in C# (could be converted to a Windows Service), using Entity Framework 6, and only interfaces with a database and a 3rd party API. The application received a continuous stream of data which it must: 1) Log to a data base, and 2) process and possibly respond to, logging all actions to the same database.
Goal is to implement fault-tolerance. I want to ensure that if one instance of my application fails (hangs / crashes), another instance of the application/service takes over. Bonus: Continuously streaming data received must be captured and logged to a database, even during down-time.
I'll try to keep this short.
If I were to re-invent the wheel, I would:
- have my application output a heartbeat to the database and write my IP address to a special table in the database stating that I am the master.
- Before EVERY write to the db, I would check to make sure I am still master.
- Second instance of app is alive and checking for the heartbeat (like a witness). If a heartbeat it missed, replace Master IP address property in db to my own (so if instance#1 wakes up, he'll know he is no longer master and should now go into "listening mode"), and write whatever data was missed into the database (since I have been listening to the stream silently as well until now)... the second application now takes over all functions and the first app is now the "listener" (or "witness").
This seems like I am re-inventing the wheel. There must be something already build I can use? A framework? A Service?
I will likely implement using AWS RDS so at least my database will be mirrored and all that is done for me.
Any pointers would be greatly appreciated. Goals: Simplicity. Reliability. Fault-tolerance.