Working with ElasticSearch using Kibana
1. Working with ElasticSearch using Kibana
Head over to the Elasticsearch dashboard, select our cluster, and click on the Kibana link. When we open Kibana for the first time, it will ask us to define our first index pattern. This needs to match what we told Firehose for Elasticsearch to know that we are dealing with that "table of data". After we define the pattern, it will show us all the fields that are coming in through Firehose! If we head over to discover, we can see the raw data coming in, under the source field. Let's add the created_at date, the language field, the sentiment field, and the text of the tweet. Now, let's save the view and call it sd_tweets_view. A view is a subset of an index pattern. We will use this view later as we define visualizations and dashboards. Next, let's head to visualizations and create a new one. Under Buckets, select Split Series, then terms, then sentiment.keyword Also, for X Axis, add date:Histogram. This will give us the count of tweets with positive, negative and neutral sentiment. Click update. Save it as tweet sentiment. Let's add one more visualization that's a simple count. Now, let's head to dashboards, create new and add the Tweet Counts and Tweet Sentiment visualizations. Finally, let's create an alert. Head over to alerts, destinations, and we will create a destination - who receives the alert. We will use SNS for alerting, but you can also use slack or another custom webhook. I will use the role ARN and topic from before. Next, let's go to Monitors and create a monitor. This is what will watch the data seeing if an alert needs to be triggered. Let's define it using a visual graph against our sd_tweets_tr index. Let's adjust the query to be for the past 15 minutes, and search only for negative sentiment tweets. Since this is a monitor, it will only search for tweets. In the monitor schedule, we'll specify the monitor frequency. By saying 3 minutes, we ask the Monitor to check the last 15 minutes of tweets every 3 minutes. Click update, and now we create a trigger. A trigger is the threshold that will initiate the alert. We can use the severity level to create tiered alerts. But for now, we'll leave it at 1, and specify that the trigger condition is above 3. When there are more than 3 negative tweets in the past 15 minutes, we will fire an alert. Create an action to send an alert to the alert receivers, and click save. Now we have a system that will check the last 15 minutes of data every 3 minutes, and if there are more than negative tweets in that time range, it will create an alert.2. Let's practice!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.