2

We are searching for a tool that

  • can parse log files and checks if a certain pattern is matched, e.g. "ERROR" or "FATAL" is contained in a line of a log file.
  • can check if a logfile is present or has been modified at a certain point of time - this is to check if a routine has run, as these always generate log files
  • if these checks triggered, sends an email to us
  • (optional) gathers several events within a given period of time to send one mail instead of many

Non functional requirements

  • it must be on-premise - installed on our machine - as we are running inside an intranet and behind firewalls. Trying to get exception rules in order to consume internet based software as a service solutions is not an option
  • we are running on Red Hat Linux servers only, so Windows solutions are not an option
  • (optional) we are running several servers, so having a master that collects inputs from all these servers (agents) and coordinates alerts would be nice. But we are also willing to install and configure several installations from a tool on several machines
  • (optional) open source so that we may contribute, but closed source is also welcome, also if it costs money, we are willing to pay
  • (optional) installation consulting is welcome

What we had a look at so far

  • Nagios - from our point of view offers too much for what we want. To us it appears like a sledgehammer to crack that nut
  • The ELK Stack plus the Watcher plugin - appears to full fill our needs. All though it also provides too much functions we do not require. We do not need a history of the logs, so the Kibana part is not of interest for us and a whole elasticsearch server is also a bit much.
Improve this question
4
  • If you can solve the "file exists" separately (e.g. using a cron-job with a shell script), would Fail2ban be an option? Instead of the "Ban" part, it also can use notifications or other actions (you can even define your own actions, as you can define your filters – so it could be setup to report directly into a database, for example). If that would meet your needs, let me know, and I write a corresponding answer with some additional details.
    – Izzy
    Commented Oct 15, 2015 at 12:22
  • Thanks for your input @Izzy but we are looking for a one-stop solution.
    – cheffe
    Commented Oct 16, 2015 at 9:14
  • Thought so – which was why I've put it as a comment, not an answer. I still leave it there – you might change your mind in case no real/suitable one-stop-solution turns up. I'm not aware that Fail2Ban also can notify on missing logs – but then I had no need for that, and simply might have missed it. It detects "log rotation", so it might do even more. I'm far from using its full potential I guess ;)
    – Izzy
    Commented Oct 16, 2015 at 9:19
  • Does this have to be a company-grade software? I know of someone who has created a fully customisable Nagios alternative for on premise monitoring and performs almost everything you're after. It's in early stages of being deployed but we could contact you once done.
    – DankyNanky
    Commented Oct 18, 2015 at 4:05

1 Answer 1

Reset to default
1

I am searching a tool like you do for some time.

This one looks good:

https://hekad.readthedocs.org

Here are the major features (from homepage):

Heka is an open source stream processing software system developed by Mozilla. Heka is a “Swiss Army Knife” type tool for data processing, useful for a wide variety of different tasks, such as:

  • Loading and parsing log files from a file system.
  • Accepting statsd type metrics data for aggregation and forwarding to upstream time series data stores such as graphite or InfluxDB.
  • Launching external processes to gather operational data from the local system.
  • Performing real time analysis, graphing, and anomaly detection on any data flowing through the Heka pipeline.
  • Shipping data from one location to another via the use of an external transport (such as AMQP) or directly (via TCP).
  • Delivering processed data to one or more persistent data stores.
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.