I work for a large enterprise; we have a number of systems with interfaces to integrate data between them, some internal and some external websites, and some mobile applications which communicate with on site systems.
There are a number of useful tools available for monitoring various pieces of each of these; a few tools which can tackle a large number of pieces, but nothing which will handle everything (I suspect that's impossible outside of a bespoke system given the variety of things to be monitored; from infrastructure availability, resource usage and capacity planning through to user experience/application level.
A better alternative (in our view) would be to have a single top level system which could be fed by various monitors, and could "report" on the information coming from all of them. By report I mean both giving current status dashboard visibility, providing alerts (email / sms / other), and trend diagrams.
Is anyone aware of a high level (aggregate) monitoring solution?
- Receive data from a variety of third party monitors including such data as:
- Site Availability / Up time
- Server Resource Usage / Capacity Planning
- Message Tracking (where is a piece of data in the system-system interfaces)
- Event logs
- Network latency
- Database blocking
- (many more...)
- Allow an aggregated view of this data; e.g.
- By application (is an application healthy, performing poorly, unavailable)
- By location (e.g. is an application only unavailable in certain locations due to connectivity issues)
- When was an issue first & last detected (as opposed to seeing 200 individual notifications generated during the total outage period)
- Allow users & support engineers to customize what they view (e.g. which applications / functional areas / technologies are they interested in)
- Generate alerts
- Via email, but other technologies would be a bonus
- Allow users to customize what they receive (i.e. so they can decide what's useful and what's spam)
- Allow information to be linked
- i.e. support tickets associated with the issue
- Root cause analysis / communications to be attached (e.g. as with https://status.heroku.com/'s read more sections)
- Link to other related errors (e.g. to show that a network switch going offline is the cause of other apps becoming unavailable).
Currently we use a mix of the following tools for collecting information:
- What's Up Gold (CPU, Memory, Disk Capacity, Event Collation)
- Idera (SQL Server blocking, locking & resource monitoring)
- SCOM (Site Availability - we piggy back on another business unit's instance so haven't made great use of this resource)
- New Relic (free) (Site Availability, Throughput, Application Performance)
- Monitor.US (free) (Site Availability)
- Bespoke .Net, SQL, PowerShell & AutoIt scripts (service availability, data integrity, biztalk port & orchestration status, biztalk suspended messages, message tracking, system responsiveness monitoring)
- Various pieces included OOTB with our enterprise applications
The above is quite a rough collection of thoughts; hopefully it's enough to communicate the concept of what we're after though.
I'm currently investigating Data Dog (https://www.datadoghq.com/product/), but wanted to see what else may be out there for this.