Elastic Stack Essentials
April 30th, 2018
BigData Training Architect II in Content
This course is for those new to the Elastic Stack to get an introductory overview of its core services (Elasticsearch, Logstash, Kibana, Beats), features, terms, and basic administration. This course will follow a real-world use case of setting up a log aggregation pipeline for web access logs and analyzing said logs with Kibana via search, visualizations, and dashboards.
If you are looking for "ELK Stack" material, this is the place! Now that Elastic Stack is more than ElasticSearch, LogStash and Kibana, the "ELK" naming convention has been retired!
In this video, we will introduce the course by covering who this course is for, what prerequisites you should have, and what topics this course includes.
About the Author
Get to know a little bit about me, the author!
Nomenclature: ELK vs Elastic Stack
You may hear references to ELK and Elastic Stack and not know the difference. Well, lets clear that up with this short video describing the nomenclature of Elastic's product suite.
What is the Elastic Stack?
In this video, we will introduce the Elastic Stack Beat clients to better understand the use cases and capabilities of each one. This will allow you to determine the right Beat for each data shipping use case.
This video will give a brief conceptual overview of the Logstash service and what its particular role is in the Elastic Stack. This will give you a sense of what Logstash is capable of and how you might use it to parse and enrich your data in real-time.
In this video, we will talk about Elasticsearch. We will go over how it is used with and without the rest of the Elastic Stack, important terms, node types, and best practices. This information will enable you to identify how Elasticsearch applies to your potential use case(s) and how to get started with sizing your cluster.
In this video, we talk about Kibana. We will cover the various plugins within Kibana and what each is used for. This will allow you to quickly identify which plugins you want to use to answer questions about your data or to help manage your Elastic Stack itself.
Premium Services and Platforms
In this video, we talk about the premium plugin pack for the Elastic Stack known as X-Pack. We talk about each of the plugins in this pack so that you can gain an understanding of the premium features and capabilities Elastic offers with a subscription.
Elastic Cloud and Elastic Cloud Enterprise
Here we talk about some of the hosting options you have when building your Elastic Stack. This video will help you decide which hosting option is right for you so that you can identify how to get started providing Elastic Stack services to your company.
Elastic Stack Features and Services
Web Access Log Aggregation
Elasticsearch: Create a Multi-Node Cluster
In this video, we go hands-on by setting up a multi-node Elasticsearch cluster. We will walk you through how to install and configure Elasticsearch master and data nodes on Linux Academy's cloud servers. So feel free to follow along.
Logstash: Install and Configure a Pipeline
In this video, we show how to install Logstash with a data pipeline to parse apache access events. Since this is an introductory course for the Elastic Stack, we aren't going to get too deep into writing the code for the pipeline but we will install a pre-created pipeline and explain what it is doing and how.
Filebeat: Install and Ship Log Events
In this video, we will demonstrate how to install Filebeat and use the built-in modules to collect and ship web access log sample data to Logstash. You will also learn how to create your own log prospectors and ship multiline logs.
Kibana: Install and Visualize
In this video, we show how to install Kibana and connect it to your Elasticsearch cluster to explore the data in your indexes. We will demonstrate using the Discover plugin to search and filter your raw logs, the Visualize plugin to create meaningful visual representations of your data, and the Dashboard plugin to bring our saved searches and visualizations into a single-pane-of-glass view. This should provide the necessary inspiration to show how Kibana can be used to visualize and explore your Elasticsearch data.
Elastic Stack Installation and Configuration
Let's recap what we learned in this course and talk about some next steps to build upon everything we learned.
Learn how to showcase your success in completing this course with this video.