Thanks to utility computing providers such as Amazon EC2, the basic infrastructure for processing Big Data is available to everyone. Whereas earlier it was critical to design the software so that it would not hit limits of your infrastructure, now the software needs to avoid hitting your credit limit.
In this talk, we will tell how and why we have used Erlang to build Bitdeli, a new backend service for data-intensive web applications. A key design goal was to make the system extremely cost-efficient without sacrificing scalability or latency. This required a language that can cope with a rapidly changing environment, which made Erlang a natural choice for the job.