Managed IT Security

Solve Ramdisk Full Error Issue Post Upgrading to ESXi 6.5

Indicators-

After upgrading ESXi on Dell/HP ProLiant server to ver.6.5, you may run into a Ramdisk error.
Below are the indicators.

  • Can’t connect to ESXi server from vCenter Server.
  • Below error message displayed when creating a virtual machine on an affected ESXi host:
  • A general system error occurred: failed to create journal file provider:failed to open “/var/log/vmware/journal/1349795091.4″ for write:no such file”
  • In vCenter Server and vmkernel logs, you see the message:
    The Ramdisk ‘root’ is full.

 

Reason-

This issue occurs if the root Ramdisk is filled with logs. IBM/Dell/HP custom ESXi points the scratch partition to a folder, which resides on the root Ramdisk.

 

To verify-

Login to ESXi by using SSH and run the vdf -h command.
If you have the root uses as 100%, they confirm that you are having a Ramdisk utilization issue.

ESXi by using SSH

 

Solution-

Rebooting the host can temporarily resolve the issue, but it will reoccur in a few days.
However, moving the Scratch partition to a different local data store will fix this issue permanently.
*Do not use ISCSI or fibre datastore for Scratch partition.

Configuring a persistent scratch location by using the vSphere Client

To configure a persistent scratch space for ESXi by using the vSphere Client:
1. Connect to vCenter Server or the ESXi host by using the vSphere Client.
2. Click the ESXi host in the inventory.
3. Click the Configuration tab > Storage.
4. Right-click a datastore and click Browse.
5. Create a uniquely-named directory for this ESXi host.
For example – .locker-ESXHostname
6. Close the Datastore Browser.
7. Click Software > Advanced Settings.
8. Select the ScratchConfig section.
9. Change the ScratchConfig.ConfiguredScratchLocation configuration option, specifying the full path to the directory.
For example:/vmfs/volumes/DatastoreUUID/.locker-ESXHostname

 

 

 

To Determine the Datastore UUID:

a.) Navigate to Configuration > Storage.
b.) Select the datastore.
c.) View Location under Datastore Details.

The Datastore UUID appears similar to /vmfs/volumes/51dda02d-fade5016-8a08-005056171889 where 51dda02d-fade5016-8a08-005056171889 is the UUID.
10. Click OK.
11. Put the ESXi host into maintenance mode and reboot for the configuration change to take effect.

 

Validating the Fix:

After reboot check ramdisk utilization, it should be below 10%.
The ScratchConfig.CurrentScratchLocation should not be set to /scratch.

In case of validation failure move to additional fixes.

check ramdisk utilization

 

Additional Fixes-

This is common when you are using the Dell/HP/IBM Customized image of ESXi 6.5.
This happens due to the elxiscsi driver and the associated shared library elx-esx-libelxima.

To check if an emulex driver is present
localcli software vib list |gerp elx-esx-libelxima.so

Login to the esxi host and check if the emulex driver vib is not in use, following which the same can be removed safely.
esxcfg-scsidevs -a
esxcfg-nics -l

The solution lies in uninstalling the elxiscsi driver and the associated shared library elx-esx-libelxima, if Emulex is not a part of the configuration. To uninstall the elxiscsi driver, run the following commands:

1. /etc/init.d/hostd stop
2. Localcli software vib remove -n elxiscsi -n elx-esx-libelxima.so
3. Reboot

emulex driver

21 thoughts on “Solve Ramdisk Full Error Issue Post Upgrading to ESXi 6.5”

  1. I’ve been browsing online more than 4 hours today, yet I never
    found any interesting article like yours. It is pretty worth enough for me.
    In my opinion, if all web owners and bloggers made good content as you did, the internet
    will be a lot more useful than ever before.

  2. I absolutely enjoyed every little bit of it. I have got you book marked to check out new stuff of your.

  3. I am sure this article has touched all the internet visitors, its really really fastidious piece of writing.

  4. Really helpful and informative content which will help us to solve issue after updating ESXi 6.5 .

  5. Just desire to say your article is as amazing. The clarity in your post is simply nice and i can assume youre an expert on this subject. Well with your permission allow me to grab your RSS feed to keep updated with forthcoming post. Thanks a million and please continue the gratifying work.

  6. Youre so right. Im there with you. Your weblog is certainly worth a read if anyone comes throughout it. Im lucky I did because now Ive received a whole new view of this. I didnt realise that this issue was so important and so universal. You surely put it in perspective for me.

  7. Pretty insightful submit. Never thought that it was this simple after all. I had spent a beneficial deal of my time looking for someone to explain this topic clearly and you’re the only one that ever did that. Kudos to you! Keep it up

  8. You’ve done a brilliant job making sure that people understand where you’re approaching from. And let me tell you, I get it. huge stuff. What you’ve got to say is important and needs to be read.

  9. I’m thinking you should reboot this host to reset the root ramdisk as it may not be releasing space properly. Also, it’s probably worth applying patch 11 that came out in mid-September as it fixes a number of issues.

  10. Very useful information about how to solve randisk full error issue post upgrading to ESXi 6.5. Thank you for the blog.

  11. Thanks for taking the time to discuss this, I feel strongly about it and like finding out additional on this subject. If feasible, as you acquire expertise, would you mind updating your weblog with additional details? Its very helpful for me.

  12. I just couldnt leave your website before telling you that we really enjoyed the quality information you offer to your visitors Will be back often to check up on new posts

Leave a Reply

Your email address will not be published. Required fields are marked *