NSQ is a realtime distributed messaging platform, built entirely in Go, that promotes distributed and decentralized topologies without single points of failure, enabling fault tolerance and high availability coupled with a reliable message delivery guarantee.
It was built to support bitly's data engineering systems and has continued to grow in adoption to power infrastructure at Path, Hailo, Life360, Trendrr, Simplereach, and others. It is a vibrant member of the Go open source community.
Building easy-to-operate, large scale, high volume distributed systems for production tends to present unique and interesting problems that stretch ones assumptions.
This talk is the product of our experience developing NSQ and operating Go services in production.
It covers the evolution of bitly's infrastructure to the design and implementation of NSQ, a match made in heaven for Go. This includes the challenge of optimizing a garbage collected language, managing those pesky goroutines, and a healthy dose of distributed systems.