Logging to files works since the unix epoch 1970.
I think it is time for a change.
- rotating files is a pain: Server needs to get reloaded after rotating ...
- logging structured data means this workflow:
- nice high level data structures gets serialized to a byte string
- later the byte string needs to be constructed to a nice high level data structures again. Can break if two processes log to the same file.
- Yes, grep is nice for most cases. But complex queries (spanning several lines) are hard.
But what to choose?
- Linux severs
- Mostly Python software
- Satellite systems:
- Our software runs in intranets of customers.
- The satellite runs on its own, but gets managed from our central office
- Network connection between satellite and central office is sometimes slow or down.
- Network outage between satellite and central office can happen daily. Some sort of buffering is needed
- grep-replacement: We need simple way to "grep" the logs from the shell (on satellite and central office)
- structure data support: We need to be able to log structured data (json or yaml)
- Support for Python virtualenvs: We run several virtualenvs on one linux server. Logs need to be kept separated from each virtualenv