Monitoring Infrastructure and Containers with Prometheus

Hands-On Lab

 

Photo of Elle Krout

Elle Krout

Content Team Lead in Content

Length

00:30:00

Difficulty

Intermediate

Prometheus is able to monitor our infrastructure and applications at multiple levels: on the host itself, on any containers, and on the application. In this hands-on lab, we're going to address the first two options for monitoring: our virtual machine host and our containers. We'll first set up monitoring for our virtual machine by using Prometheus's Node Exporter, and then we'll set up container monitoring for the provided container using Google's cAdvisor. By the time we're done, we'll be able to view metrics across two levels of our system in Prometheus to track changes and view trends on our systems.

What are Hands-On Labs?

Hands-On Labs are scenario-based learning environments where learners can practice without consequences. Don't compromise a system or waste money on expensive downloads. Practice real-world skills without the real-world risk, no assembly required.

Monitoring Infrastructure and Containers with Prometheus

In this hands-on lab, we'll first set up monitoring for our virtual machine by using Prometheus's Node Exporter, and then we'll set up container monitoring for the provided container using Google's cAdvisor.

Start a new terminal session and log in to the Application server using the provided credentials. Open a second terminal and log in to the Monitoring server, too, using the provided credentials.

Set Up the Node Exporter

  1. In the Application server terminal, create a system user:

    sudo useradd --no-create-home --shell /bin/false node_exporter
  2. Change directory:

    cd /tmp/
  3. Download the Node Exporter from Prometheus's download page:

    wget https://github.com/prometheus/node_exporter
    /releases/download/v0.17.0/node_exporter-0.17.0.linux-amd64.tar.gz
  4. Extract its contents:

    tar -xvf node_exporter-0.17.0.linux-amd64.tar.gz
  5. Move into the newly created directory:

    cd node_exporter-0.17.0.linux-amd64/
  6. Move the provided binary:

    sudo mv node_exporter /usr/local/bin/
  7. Set the ownership:

    sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter
  8. Drop back to your main directory:

    cd
  9. Create a systemd service file:

    sudo $EDITOR /etc/systemd/system/node_exporter.service
  10. Add the following to the file:

    [Unit]
    Description=Node Exporter
    After=network.target
    
    [Service]
    User=node_exporter
    Group=node_exporter
    Type=simple
    ExecStart=/usr/local/bin/node_exporter
    
    [Install]
    WantedBy=multi-user.target

    Save and exit when done.

  11. Start the Node Exporter:

    sudo systemctl daemon-reload
    sudo systemctl start node_exporter

Set Up cAdvisor

  1. Launch cAdvisor:

    sudo docker run 
    --volume=/:/rootfs:ro 
    --volume=/var/run:/var/run:ro 
    --volume=/sys:/sys:ro 
    --volume=/var/lib/docker/:/var/lib/docker:ro 
    --volume=/dev/disk/:/dev/disk:ro 
    --publish=8000:8080 
    --detach=true 
    --name=cadvisor 
    google/cadvisor:latest
  2. List available containers to confirm it's working:

    docker ps

Add Endpoints

  1. In the Monitoring server terminal, open the Prometheus configuration file:

    sudo $EDITOR /etc/prometheus/prometheus.yml
  2. Add the Node Exporter and cAdvisor endpoints:

    - job_name: 'node_exporter'
      static_configs:
      - targets: ['APPLICATIONHOST:9100']
    - job_name: 'cadvisor'
      static_configs:
      - targets: ['APPLICATIONHOST:8000']

    Save and exit.

  3. Restart Prometheus:

    sudo systemctl restart Prometheus
  4. Back in the browser, copy the public IP address of the Monitoring server and paste it into a new tab, adding :9090 at the end.

  5. On the Prometheus page, navigate to Status > Targets to ensure the endpoints are working.

Conclusion

Congratulations on completing this hands-on lab!