Troubleshooting file system issues

Hands-On Lab

 

Photo of Michael Christian

Michael Christian

Course Development Director in Content

Length

04:00:00

Difficulty

Advanced

Warning: This lab represents a significantly broken environment and goes well beyond simple troubleshooting. In this lab, you must resolve any issues preventing the content mounted at /storage from being accessed.

What are Hands-On Labs?

Hands-On Labs are scenario-based learning environments where learners can practice without consequences. Don't compromise a system or waste money on expensive downloads. Practice real-world skills without the real-world risk, no assembly required.

Troubleshooting file system issues

Introduction

Warning: This lab represents a significantly broken environment and goes well beyond simple troubleshooting.

In this lab, you must resolve any issues preventing the content mounted at /storage from being accessed.

Resolve any and all issues preventing access to the contents mounted at /storage.

  • 10.0.1.10 is an iSCSI target for the volume mounted at /storage

  • 10.0.1.11 is the iSCSI initiator

  • A backup of the LUKS header is available at /root/vg_1-lv_1.header

Do not format any volumes, do not modify /etc/fstab or /etc/crypttab

The integrity of the existing encryption, filesystems, and files located on the iSCSI volume must be preserved.

Solution

Start by logging in to the lab servers using the credentials provided on the hands-on lab page:

ssh cloud_user@PUBLIC_IP_ADDRESS

Become the root user:

sudo su -

Be sure to log in to both Server1 and Server2 in separate tabs or windows.

Verify the iSCSI target configuration and status

On Server1 (10.0.1.10)

  1. Verify the target service is running:

    systemctl status target
  2. Start and enable the target service:

    systemctl start target && systemctl enable target
  3. Verify the process is listening on port 3260:

    ss -ltnp | grep 3260
  4. Check if the firewall is permitting traffic over 3260/tcp:

    firewall-cmd --list-all
  5. Permit incoming traffic over port 3260/tcp:

    firewall-cmd --permanent --add-port=3260/tcp
  6. Reload the firewall:

    firewall-cmd --reload
  7. View the iSCSI target configuration:

    targetcli
    /> ls

Verify the iSCSI initiator configuration and status

On Server2 (10.0.1.11)

  1. View any existing node entires:

    iscsiadm -m node
  2. Verify the initiator name matches the ACL from the target:

    cat /etc/iscsi/initiatorname.iscsi
  3. Edit the initiatorname.iscsi file:

    vim /etc/iscsi/initiatorname.iscsi
  4. Change the name to match the target:

    InitiatorName=iqn.1994-05.com.redhat:c1cd6e78d22
  5. Save and close the file:

    :wq
  6. Verify security settings match those of target:

    less /etc/iscsi/iscsid.conf
    #node.session.auth.authmethod = CHAP
    ...
    #node.session.auth.username = username
    #node.session.auth.password = password
  7. Restart the iscsid service to pick up the change:

    systemctl restart iscsid
  8. Discover iSCSI targets from 10.0.1.10:

    iscsiadm -m discovery -t sendtargets -p 10.0.1.10
  9. Log in to target:

    iscsiadm -m node -T iqn.2003-01.org.linux-iscsi.ip-10-0-1-10.x8664:sn.a3776832068c -l
  10. Enable the iscsid service to permit connection upon reboot:

    systemctl enable iscsid

Resolve disk encryption

  1. Review the contents of /etc/crypttab and /etc/fstab:

    cat /etc/crypttab
    cat /etc/fstab
  2. Attempt to manually open the volume using the key referenced in /etc/crypttab:

    cryptsetup luksOpen /dev/mapper/vg_1-lv_1 luks-vg_1-lv_1 --key-file /root/passphrase.key
  3. View the volume key slots:

    cryptsetup luksDump /dev/mapper/vg_1-lv_1
  4. Restore the LUKS header:

    cryptsetup luksHeaderRestore /dev/mapper/vg_1-lv_1 --header-backup-file /root/vg_1-lv_1.header
  5. Enter capital YES.

  6. Open the volume manually:

    cryptsetup luksOpen /dev/mapper/vg_1-lv_1 luks-vg_1-lv_1 --key-file /root/passphrase.key
  7. View the entries in /dev/mapper:

    ll /dev/mapper
  8. Attempt to mount /storage:

    mount /storage
  9. View the file system label:

    blkid
  10. The volume appears to be a swap format, however the entry in /etc/fstab suggests it's an XFS file system. Repair the file system:

    xfs_repair /dev/mapper/luks-vg_1-lv_1
  11. Mount /storage:

    mount /storage
  12. View the contents of /storage:

    ll /storage

Conclusion

Congratulations, you've completed this hands-on lab!