Indicators-

After upgrading ESXi on Dell/HP ProLiant server to ver.6.5, you may run into a Ramdisk error.
Below are the indicators.

  • Can’t connect to ESXi server from vCenter Server.
  • Below error message displayed when creating a virtual machine on an affected ESXi host:
  • A general system error occurred: failed to create journal file provider:failed to open “/var/log/vmware/journal/1349795091.4″ for write:no such file”
  • In vCenter Server and vmkernel logs, you see the message:
    The Ramdisk ‘root’ is full.

 

Reason-

This issue occurs if the root Ramdisk is filled with logs. IBM/Dell/HP custom ESXi points the scratch partition to a folder, which resides on the root Ramdisk.

 

To verify-

Login to ESXi by using SSH and run the vdf -h command.
If you have the root uses as 100%, they confirm that you are having a Ramdisk utilization issue.

ESXi by using SSH

 

Solution-

Rebooting the host can temporarily resolve the issue, but it will reoccur in a few days.
However, moving the Scratch partition to a different local data store will fix this issue permanently.
*Do not use ISCSI or fibre datastore for Scratch partition.

Configuring a persistent scratch location by using the vSphere Client

To configure a persistent scratch space for ESXi by using the vSphere Client:
1. Connect to vCenter Server or the ESXi host by using the vSphere Client.
2. Click the ESXi host in the inventory.
3. Click the Configuration tab > Storage.
4. Right-click a datastore and click Browse.
5. Create a uniquely-named directory for this ESXi host.
For example – .locker-ESXHostname
6. Close the Datastore Browser.
7. Click Software > Advanced Settings.
8. Select the ScratchConfig section.
9. Change the ScratchConfig.ConfiguredScratchLocation configuration option, specifying the full path to the directory.
For example:/vmfs/volumes/DatastoreUUID/.locker-ESXHostname

 

 

 

To Determine the Datastore UUID:

a.) Navigate to Configuration > Storage.
b.) Select the datastore.
c.) View Location under Datastore Details.

The Datastore UUID appears similar to /vmfs/volumes/51dda02d-fade5016-8a08-005056171889 where 51dda02d-fade5016-8a08-005056171889 is the UUID.
10. Click OK.
11. Put the ESXi host into maintenance mode and reboot for the configuration change to take effect.

 

Validating the Fix:

After reboot check ramdisk utilization, it should be below 10%.
The ScratchConfig.CurrentScratchLocation should not be set to /scratch.

In case of validation failure move to additional fixes.

check ramdisk utilization

 

Additional Fixes-

This is common when you are using the Dell/HP/IBM Customized image of ESXi 6.5.
This happens due to the elxiscsi driver and the associated shared library elx-esx-libelxima.

To check if an emulex driver is present
localcli software vib list |gerp elx-esx-libelxima.so

Login to the esxi host and check if the emulex driver vib is not in use, following which the same can be removed safely.
esxcfg-scsidevs -a
esxcfg-nics -l

The solution lies in uninstalling the elxiscsi driver and the associated shared library elx-esx-libelxima, if Emulex is not a part of the configuration. To uninstall the elxiscsi driver, run the following commands:

1. /etc/init.d/hostd stop
2. Localcli software vib remove -n elxiscsi -n elx-esx-libelxima.so
3. Reboot

emulex driver