Define Elasticsearch Indexes

Hands-On Lab

 

Photo of Myles Young

Myles Young

BigData Training Architect II in Content

Length

01:30:00

Difficulty

Advanced

In Elasticsearch, the data we index is stored in an index. The word index here is used as both a verb and a noun. Essentially, we perform an index operation in order to store data in an index. Before we can do this, however, we first need to understand the structure of an index and how to define one to meet specific requirements. In this hands-on lab, you will define Elasticsearch indexes by completing the following tasks: Configure the number of primary and replica shards Allocate the shards for one index to hot nodes Allocate the shards for another index to warm nodes Associate indexes with aliases

What are Hands-On Labs?

Hands-On Labs are scenario-based learning environments where learners can practice without consequences. Don't compromise a system or waste money on expensive downloads. Practice real-world skills without the real-world risk, no assembly required.

Define Elasticsearch Indexes

Introduction

In Elasticsearch, the data we index is stored in an index. The word index here is used as both a verb and a noun. Essentially, we perform an index operation in order to store data in an index. Before we can do this, however, we first need to understand the structure of an index and how to define one to meet specific requirements. In this hands-on lab, you will define Elasticsearch indexes by completing the following tasks:

  • Configure the number of primary and replica shards
  • Allocate the shards for one index to hot nodes
  • Allocate the shards for another index to warm nodes
  • Associate indexes with aliases

OPTIONAL: Set up a remote tunnel to Kibana on the master node

NOTE: This task only needs to be performed if you're using the Kibana UI to interact with Elasticsearch.

Set Up a Remote Tunnel

  1. Open a new terminal window and use ssh to log in to the master node as cloud_user with port forwarding.

    ssh cloud_user@PUBLIC_IP_ADDRESS -L 5601:localhost:5601

Open the Kibana Console Tool

  1. In your local web browser, go to http://localhost:5601.
  2. In Kibana, navigate to Dev Tools in the side navigation bar.
  3. Select the Console tool (it should be the default tool that loads).

Create the logs-01 index

Option 1: Kibana Console Tool

Create logs-01

  1. To create the logs-01 index, use the Console to send the following request to Elasticsearch:

    PUT /logs-01
    {
      "aliases": {
        "this_week": {}
      },
      "settings": {
        "number_of_shards": 3,
        "number_of_replicas": 1,
        "index.routing.allocation.require.temp": "hot"
      }
    }

Option 2: Command line curl

Create logs-01

  1. To create the logs-01 index, execute the following from the command line of one of the nodes:

    curl -XPUT "http://localhost:9200/logs-01" -H 'Content-Type: application/json' -d'
    {
      "aliases": {
        "this_week": {}
      },
      "settings": {
        "number_of_shards": 3,
        "number_of_replicas": 1,
        "index.routing.allocation.require.temp": "hot"
      }
    }'

Create the logs-02 index

Option 1: Kibana Console Tool

Create logs-02

  1. To create the logs-02 index, use the Console to send the following request to Elasticsearch:

    PUT /logs-02
    {
      "aliases": {
        "last_week": {}
      },
      "settings": {
        "number_of_shards": 3,
        "number_of_replicas": 1,
        "index.routing.allocation.require.temp": "warm"
      }
    }

Option 2: Command line curl

Create logs-02

  1. To create the logs-02 index, execute the following from the command line of one of the nodes:

    curl -XPUT "http://localhost:9200/logs-02" -H 'Content-Type: application/json' -d'
    {
      "aliases": {
        "last_week": {}
      },
      "settings": {
        "number_of_shards": 3,
        "number_of_replicas": 1,
        "index.routing.allocation.require.temp": "warm"
      }
    }'

Conclusion

Congratulations — you've completed this hands-on lab!