Graylog is a fully integrated platform for collecting, indexing, and analyzing both structured and unstructured data from almost any source. Non Graylog-authored components Include MongoDB for metadata and Elasticsearch for log file storage and text search.
Logs are a mess. Too many formats, no common timestamps, and too many amazing one-liner scripts to process them. Ugh! There's hope! Logstash and other tools are here to help. This talk will cover common logging practices (good and bad), some tools you might want to learn about later, as well as an overview of logstash and how you can use it to achieve victory in the battle against log abuse!
Centralize all your log messages
- Collect terabytes of log messages
- Process in real-time
- Search and analyze in seconds
"logging: logstash and other things" by Jordan Sissel, czar of logging at DreamHost. From PuppetConf '12. Learn more about Puppet: http://bit.ly/OUeBLk
Kibana is an open source (Apache Licensed), browser based analytics and search dashboard for ElasticSearch. Kibana is a snap to setup and start using. Written entirely in HTML and Javascript it requires only a plain webserver, Kibana requires no fancy server side components. Kibana strives to be easy to get started with, while also being flexible and powerful, just like Elasticsearch.
logstash is a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (like, for searching). Speaking of searching, logstash comes with a web interface for searching and drilling into all of your logs.
It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.
logstash is now a part of the Elasticsearch family! This allows us to build better software much faster as well as offering production support
By combining the massively popular Elasticsearch, Logstash and Kibana we have created an end-to-end stack that delivers actionable insights in real-time from almost any type of structured and unstructured data source. Built and supported by the engineers behind each of these open source products, the Elasticsearch ELK stack makes searching and analyzing data easier than ever before.