Kafka - Bringing reliable stream processing to a confusing, dark world

Kafka is a new distributed publish-subscribe system recently open sourced by LinkedIn designed to provide extremely high-throughput, reliable message passing. Kafka is focused on processing high-volume, site-critical stream activity and provides integration for loading data into Hadoop for further processing. Kafka is under heavy development and in active production use at LinkedIn.