Logserver with web interface, user login, custom filters

Question

I am working at a company where we have a few hundred programs running scientific calculations on several servers. Each program produces logs. At the moment, important log messages are sent via email back to the programmers.

Now instead, I am looking for some server software where everyone could access these log messages via some web interface. It should also have an login with ldap and ideally I should be able to restrict access for some people to specific messages.

It should be possible for our programmers to set filters like program=foo,host=bar,message =*baz and to save these filters for reuse. It only needs to show the messages in a (filterable) list. Diagrams are not needed.

I already looked into some log servers, but most of the time they are targeted at server administration. Also I have not yet seen one where I can set filters like this.

Answer 1

There are several -- more than several, actually, but I'll list the few I've worked with -- tools available for this. They are not for novices, though. i.e. The expertise expected to use these tools is "intermediate to advanced system administrator with several years of experience"

Sumologic

Sumo is a SaaS provider that allows extremely flexible querying and reporting of aggregated system logs.

It's pretty expensive, but can do what you are asking, and much more.

Splunk

Splunk is an on-premise server log aggregator that behaves extremely similarly to Sumologic. They also have a SaaS offering.

Also very expensive, but does what you want and more. Is more mature and possibly better built than Sumologic. Has been around longer than Sumo.

Sumo and Splunk are extremely similar in their features and the way they work.

ELK Stack

Elasticsearch Logstash Kibana

Open Source suite that is basically an open-source version of Splunk or Sumologic. But not exactly.

Will do what you are describing, but may require a lot of work up front to do exactly what you want.

Free/OSS version is available. I'm sure there's a paid support version, too.

I've never used ELK stack, I'm just aware of it.