Building search app for public mail lists in 15 minutes with ElasticSearch
There are several online services that provide search for public mail lists. But many of them keep important details secret. In this talk I will build and discuss alternative search service for everyone without any secrets.
We will start by downloading public mail list archive, then setup ElasticSearch server, index mails into it and search them via AJAX based web app (we will use a few facet query types and other non trivial search queries). There are some interesting challenges in this like mail thread detection, parsing and mapping complex mail content into search engine/Lucene documents and tuning relevancy. The result will be a running app on a local system that can be used even without internet connection (all content is served directly from ElasticSearch) and thus provide interesting alternative to online search services. Part of the talk will be a demo of some tools that I use for ElasticSearch cluster monitoring (mostly AJAX based web app built on top of its REST API).