4

I'm working on a project where it is assumed that a few thousand distributed custom devices need to be monitored. So there are pretty simple questions which should be answered like:

  • What's the load/memory/disk space of device X now?
  • What was the disk usage lately?
  • What was the load at the first time the device was online, maybe years back in time?

And also more complicated questions like: Which devices did run a custom task, and how long did it take? This one would be used for accounting purposes.

One option to solve this is to build a custom monitoring and reporting tool. But this won't be a simple one day hack, so I wanted to ask if anyone knows some open source software which is able to

  • add new metrices in a very flexible way like Ganglia's gmetric tool
  • keep the data forever in some kind of storage to query for simple data points
  • use a JSON API to query datapoints and to be able to generate complex queries via some kind of API
  • build graphs like everyone already knows from Ganglia or such

I've tried Ganglia already, but gmetric seems to dump the collected data as soon as the device is offline. But the flexiblity of gmetric is one of the key features since we don't know what might be relevant to monitor in the future.

1

I suggest taking a look at GRR Rapid Response:

  • Clients are Python based so cross platform
  • Detailed monitoring of client CPU, memory, IO usage and self-imposed limits.
  • Powerful search and download capabilities for files and the Windows registry.
  • AngularJS Web UI and RESTful JSON API.
  • Fully scriptable IPython console access. which means you can have any graphs that you need
  • You can gather almost any data you need via Flows & Hunts you can even have hunts for specific Artifacts.
  • Datastore can be configured as MySQL Advanced or HTTPDatastore + SQLite.
  • The FAQ reports some installations with 32k clients
  • Can be cloud deployed on GCE or EC2
  • Analysis offline with BigQuery, via the iPython console or via export to CSV and then process with Pandas or otherwise.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.