Case Study Highlights

Harness the power of social media

  • Track 1.7 billion messages
  • Search through 4 million new social messages per day
  • Easily scale to handle massive growth

Enable rapid response

  • Process 50 messages per second
  • Deliver results in real time
  • Provide advanced search functionality

Tracking 1.7 billion social media messages

Engagor provides a platform for social media management, including monitoring, analytics and engagement. Engagor tracks down all relevant conversations about the brands, products, competitors and industry – 1.7 billion items of content across all major social networks – for 320 customers including McDonalds, Audi, Ikea, T-Mobile, Pepsico and the European Parliament.

Search is a key to Engagor’s system, enabling users to find specific questions, complaints or remarks in a deluge of social media messages, and respond rapidly to provide excellent customer service, as well as manage the conversation.

“Before Elasticsearch, we used Solr ,” recounts Jurriaan Persyn, CTO of Engagor. “We had a lot of trouble scaling Solr, configuring it and making it work. We would outgrow one Solr instance very quickly. It was hard for us to manage that growth with Solr.”

“Solr 3.1 didn’t have easy scaling built-in, so we had to build a lot of that functionality ourselves – clustering, master detection, node failover – which would have taken up valuable development time.”

Processing 4 million messages per day in Elasticsearch

Engagor switched to Elasticsearch, which now serves as the main database and search component of the Engagor product.

“Engagor is a database of social messages – tweets, Facebook comments, blog posts – and all that data is in Elasticsearch,” Lien Brusselmans, Marketing Manager of Engagor, states.

The Engagor service is made up of two parts. First Engagor provides an inbox displaying all the latest messages collected from different social platforms (social media, news sites, forums, blogs and more) and delivered via Elasticsearch. Engagor consumes 50 messages per second in Elasticsearch – 4 million new social messages per day.


The second part of Engagor’s service is analytics, which are aggregated by Elasticsearch. Engagor users can set up search queries based on specific parameters to analyze trends. For example, the user could search for messages on Twitter from people with at least 500 followers, with a negative sentiment, in a certain language or coming from a certain country. The possibilities to combine filters are numerous.

“With Elasticsearch, we can give our customers the flexibility to set up their own keyword searches and their own ways of drilling down into their own data,” says Brusselmans. “The fact that they can set it up themselves and start digging into their own social data makes Engagor very powerful for them. We often see clients using search or data in ways that we did not think of beforehand.”

“Thanks to Elasticsearch’s powerful keyword searches and filters, our customers can identify all mentions about the brand, and reply to everything, so their customers receive the best possible customer service,” Persyn adds.

Real-time search for real world social media

Engagor is delivered as Software-as-a-Service (SaaS), and all customers access data through the Engagor web interface. The data is processed in real time through Elasticsearch.

“The moment we receive a tweet or Facebook post, the content shows up in our customer’s Inbox within 3 to 4 seconds,” explains Brusselmans. “All our analytics are computed in real time thanks to Elasticsearch.”

In social media, real-time responsiveness is an important part of crisis management. One single negative tweet or blog post can have huge consequences for a brand’s reputation. So the ability for Engagor’s customers to quickly identify and respond to messages is essential. Whenever a customer asks a question or makes a comment, they expect a swift reply, and this is only possible if Engagor shows the message in real time.

“Real-time is especially important in social media because everything goes really fast and customers expect a response from brands within the hour or even faster,” Brusselmans elaborates. “If we do not process all the messages in real-time, our customers would not be able to offer good customer service on social media. That is why Elasticsearch is critical.”

Delivering out-of-the-box scalability

Scalability is a continuous challenge for Engagor, another reason why Elasticsearch is such an important part of the system. The number of Engagor’s clients continues to grow, the social media industry continues to mature, and the number of consumers adopting social media for communication with brands is increasing, driving corresponding growth in the number of messages that must be tracked by Engagor. Elasticsearch delivers an easily and economically scalable solution to meet Engagor’s growth expectations.

“Elasticsearch works out-of-the-box,” Persyn confirms. “Adding extra servers is very easy. There’s not much configuration needed, and query performance is very good. Features that Elasticsearch offers regarding sharding and clustering are very powerful and much easier for us to configure than our previous search solutions.”

Persyn also points out that Elasticsearch is easy to use for the company’s development team, saving time and effort as Engagor’s service is scaled and upgraded.

“Half of our team right now are juniors, coming right from school or with one or two years experience in the field, but it has been really easy for them to work with the Elasticsearch platform,” he concludes. “Even without an extensive background in data or search, it is easy for new developers to start to work with Elasticsearch because of its simple API.”

Engagor’s benefits using Elasticsearch

  • Real-time responsiveness
    When a social media message is posted, it is available to Engagor customers in seconds via Elasticsearch, enabling major brands to respond to comments or complaints in real-time and resolve issues quickly.
  • Advanced search and analytics
    Engagor customers take advantage of Elasticsearch’s advanced search capabilities to dive deep into their data and gain actionable insight.
  • Streamlined scalability
    The social media management business is growing significantly, and Elasticsearch provides Engagor with easy scalability to handle search across billions of social media messages.
  • Development productivity
    Elasticsearch is easy to learn and use, even for Engagor’s newest junior developers, enabling the company’s development team to be more productive.