I run a relatively small virtual environment out of my basement for basic services such as web hosting, file serving, and small media center. I try to keep close tabs on the performance of my environment through routine checks, but I also wanted a quick way of knowing whether or not all my services were actually activated.
What I'm using now
So a few months back, I developed a small bash script that I just called "status.sh". It's probably the ugliest piece of code ever to exist, but it works, so I kept using it.
You can find a run-down version here (I omitted a block that checked if my Tor hidden services were active because that's extraneous, along with some blocks that checked if my NAT port forwarding was configured properly). The program (if you can even call it that) is structured pretty rudimentarily: it's just a bunch of if statements that determine whether an IP responds to either a ping or a netcat TCP probe, and depending on the result, states that service is either "up" or "down." Naturally, if you try to run that in your own environment, most of the services will probably trigger as "down" because all the IP addresses are predefined from my own LAN setup.
A single variable, $(hostlist), contains a bunch of the if-statement triggers. I designed it that way because I also run a separate script that I've named "networkNotifier.sh," which sends my phone a notification if any of the servers are listed as "down." If I were to temporarily shut one of my servers down, I would be getting endless texts, so I made the $(hostlist) variable to be able to temporarily pull addresses of servers that I know are offline without deleting or commenting out whole blocks of code.
What I'm looking for
I skimmed that code so that all potentially sensitive material (external DNS records, Tor onion addresses, etc) was omitted and the script can run without them. But the real status.sh is about twice as big and twice as ugly. I'm looking for a simplistic server monitor program that only runs ICMP pings and TCP probes to check if a server is responding, and if not, triggers a script.
Please don't recommend anything that requires I install 3rd party software on every one of my servers. I'm looking for something that I can install/run on a dedicated monitoring server that probes the network. I don't need stats like CPU usage or disk usage of each server, just an up/down status.