So you’re getting reports that people are unable to make instances in your small, five node OpenStack cluster that your company uses for their development environment. You log into your controller and source your admin credentials and check your Compute services:

. admin-openrc
openstack compute service list

Now, looking at the output, you notice that the services on your controller are being shown as down, so you do a ps:

ps -ef | grep nova

You can see that your services are up and running. The next step is to check the logs in /var/log/nova:

Grepping the logs for ERROR does bring back some information:

Look at the scheduler log for a starting point:

You’ll notice that the log entries are dated 2013-09-25 and that the ERROR seems to be related to a timeout. So, the next step is to check the server date command:

There we can see the server time has somehow been set to a date in the past. When we check chrony’s status, we see it is attempting to correct the issue.

So, we can help chrony fix the time with:
chronyc -a "burst 3/5"
chronyc -a makestep

And you can see the server is back on the correct time.

If you wait, the services may come back on their own, or you may need to restart the nova services on the controller:

Each of the OpenStack Services has their own documentation. If you are experiencing an issue, it is recommended to search base on the service and OpenStack Release. So for this issue, I would utilize OpenStack’s Troubleshoot Compute Guide to help assist you further.

Fun fact: Today is OpenStack’s 8th Birthday!
See how the Linux Academy team joined in on the celebration, and find out more about our new OpenStack training courses!

 

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Get actionable training and tech advice

We'll email you our latest articles up to once per week.