VMware PSOD happening on VMware host server
-
Dear Team,
My customer facing Purple screen of death issue on VMware Esxi 5.1 hypversior. I tired to analysis the issue. But i didnt get any Root cause of the issue. kindly provide your support.
For your information, please see below logs information,017-10-31T18:10:27.827Z cpu2:9350)[45m[33;1mVMware ESXi 5.1.0 [Releasebuild-799733 x86_64][0m
PCPU 6 locked up. Failed to ack TLB invalidate (total of 2 locked up, PCPU(s): 6,9).
2017-10-31T18:10:27.828Z cpu2:9350)cr0=0x80010031 cr2=0x62d7c000 cr3=0x135b18000 cr4=0x2768
2017-10-31T18:10:27.828Z cpu2:9350)pcpu:0 world:17228 name:"vmm1:SPTEST" (V)
2017-10-31T18:10:27.829Z cpu2:9350)pcpu:1 world:17230 name:"vmm3:SPTEST" (V)
2017-10-31T18:10:27.829Z cpu2:9350)pcpu:2 world:9350 name:"hostd-worker" (U)
2017-10-31T18:10:27.829Z cpu2:9350)pcpu:3 world:17229 name:"vmm2:SPTEST" (V)
2017-10-31T18:10:27.829Z cpu2:9350)pcpu:4 world:17226 name:"vmm0:SPTEST" (V)
2017-10-31T18:10:27.829Z cpu2:9350)pcpu:5 world:17923 name:"vmm1:NTAnalytics" (V)
2017-10-31T18:10:27.829Z cpu2:9350)pcpu:6 world:2080462 name:"get_imm_schema" (U)
2017-10-31T18:10:27.829Z cpu2:9350)pcpu:7 world:17202 name:"vmm4:OCR2" (V)
2017-10-31T18:10:27.829Z cpu2:9350)pcpu:8 world:17927 name:"vmm5:NTAnalytics" (V)
2017-10-31T18:10:27.829Z cpu2:9350)pcpu:9 world:1905832 name:"sfcb-vmware_raw" (U)
2017-10-31T18:10:27.829Z cpu2:9350)pcpu:10 world:8679 name:"vmklinux_9:ipmi_thread" (S)
2017-10-31T18:10:27.829Z cpu2:9350)pcpu:11 world:2028316 name:"vmm0:Vcenter1" (V)
2017-10-31T18:10:27.829Z cpu2:9350)@BlueScreen: PCPU 6 locked up. Failed to ack TLB invalidate (total of 2 locked up, PCPU(s): 6,9).
2017-10-31T18:10:27.829Z cpu2:9350)Code start: 0x418018c00000 VMK uptime: 17:13:00:41.057
2017-10-31T18:10:27.830Z cpu2:9350)Saved backtrace from: pcpu 6 TLB NMI
2017-10-31T18:10:27.831Z cpu2:9350)0x41227b39b830:[0x41801918889a][email protected]#9.2+0x1 stack: 0x0
2017-10-31T18:10:27.832Z cpu2:9350)0x41227b39b870:[0x4180192271e5]sender@<None>#<None>+0x254 stack: 0x41227b39b900
2017-10-31T18:10:27.834Z cpu2:9350)0x41227b39b970:[0x41801921e103][email protected]#9.2.1.0+0x53a stack: 0x0
2017-10-31T18:10:27.835Z cpu2:9350)0x41227b39ba00:[0x41801921edc9][email protected]#9.2.1.0+0xb0 stack: 0x4122fffff
2017-10-31T18:10:27.837Z cpu2:9350)0x41227b39baa0:[0x41801922e42b]handle_send_req@<None>#<None>+0x12e stack: 0x4100017cc000
2017-10-31T18:10:27.838Z cpu2:9350)0x41227b39bba0:[0x41801922f113]compat_ipmi_ioctl@<None>#<None>+0x172 stack: 0x41227b39bc40
2017-10-31T18:10:27.839Z cpu2:9350)0x41227b39bc10:[0x41801918c8ad][email protected]#9.2+0x160 stack: 0xffee2ed8
2017-10-31T18:10:27.841Z cpu2:9350)0x41227b39bc80:[0x418018cb34e8]VMKAPICharDevDevfsWrapIoctl@vmkernel#nover+0x8f stack: 0x41227b39bcf
2017-10-31T18:10:27.842Z cpu2:9350)0x41227b39bd30:[0x418018e4e3c1]DevFSIoctl@vmkernel#nover+0x46c stack: 0x41227b39be94
2017-10-31T18:10:27.844Z cpu2:9350)0x41227b39bd90:[0x418018e2fc39]FSS2_IoctlByFH@vmkernel#nover+0x37c stack: 0xbcd00000000
2017-10-31T18:10:27.845Z cpu2:9350)0x41227b39bde0:[0x418018e2fd09]FSS_IoctlByFH@vmkernel#nover+0x98 stack: 0xffee2ed88014690d
2017-10-31T18:10:27.847Z cpu2:9350)0x41227b39be00:[0x41801908fb3f]UserFile_PassthroughIoctl@<None>#<None>+0x3e stack: 0x41227b39be40
2017-10-31T18:10:27.848Z cpu2:9350)0x41227b39be70:[0x4180190c96e7]UserVmfs_Ioctl@<None>#<None>+0x66 stack: 0x412200000048
2017-10-31T18:10:27.849Z cpu2:9350)0x41227b39beb0:[0x4180190955f9]LinuxFileDesc_Ioctl@<None>#<None>+0x68 stack: 0x418018db84d9
2017-10-31T18:10:27.851Z cpu2:9350)0x41227b39bef0:[0x4180190766fa]User_LinuxSyscallHandler@<None>#<None>+0xe5 stack: 0x0
2017-10-31T18:10:27.852Z cpu2:9350)0x41227b39bf10:[0x418018ca82be]User_LinuxSyscallHandler@vmkernel#nover+0x19 stack: 0xffee3048
2017-10-31T18:10:27.853Z cpu2:9350)0x41227b39bf20:[0x418018d10064]gate_entry@vmkernel#nover+0x63 stack: 0x0
2017-10-31T18:10:27.856Z cpu2:9350)base fs=0x0 gs=0x418040800000 Kgs=0x0
2017-10-31T18:10:27.618Z cpu6:2080462)NMI: 579: NMI IPI recvd. We Halt. eip(base):ebp:cs [0x58889a(0x418018c00000):0x41227b39b830:0x4010](Src0x2, CPU6)
2017-10-31T18:10:27.618Z cpu6:2080462)NMI: 579: NMI IPI recvd. We Halt. eip(base):ebp:cs [0x58889a(0x418018c00000):0x41227b39b830:0x4010](Src0x2, CPU6)
2017-10-31T18:10:27.859Z cpu2:9350)Backtrace for current CPU #2, worldID=9350, ebp=0x41221219ba10
2017-10-31T18:10:27.860Z cpu2:9350)0x41221219ba10:[0x418018c7abff]PanicvPanicInt@vmkernel#nover+0x56 stack: 0x3000000010, 0x41221219bb
2017-10-31T18:10:27.860Z cpu2:9350)0x41221219baf0:[0x418018c7b345]Panic_WithBacktrace@vmkernel#nover+0xa8 stack: 0x101219bc00, 0x0, 0x
2017-10-31T18:10:27.861Z cpu2:9350)0x41221219bc20:[0x418018ca77fd]TLBDoInvalidate@vmkernel#nover+0x434 stack: 0xbe3, 0x0, 0x0, 0x0, 0x
2017-10-31T18:10:27.861Z cpu2:9350)0x41221219bc70:[0x418019089e17]UserMem_CartelFlush@<None>#<None>+0xce stack: 0x41221219bca0, 0x4180
2017-10-31T18:10:27.862Z cpu2:9350)0x41221219bd80:[0x41801908ab91]UserMemUnmapStateCleanup@<None>#<None>+0x58 stack: 0x41221219be30, 0
2017-10-31T18:10:27.862Z cpu2:9350)0x41221219be60:[0x41801908b97d]UserMemUnmap@<None>#<None>+0x104 stack: 0x4122121a7000, 0x0, 0x41221
2017-10-31T18:10:27.862Z cpu2:9350)0x41221219bea0:[0x41801908bf20]UserMem_Unmap@<None>#<None>+0xe3 stack: 0x41221219bef0, 0x4180190766
2017-10-31T18:10:27.863Z cpu2:9350)0x41221219beb0:[0x41801909aa7f]LinuxMem_Munmap@<None>#<None>+0x5a stack: 0x418018db84d9, 0x41221219
2017-10-31T18:10:27.863Z cpu2:9350)0x41221219bef0:[0x4180190766fa]User_LinuxSyscallHandler@<None>#<None>+0xe5 stack: 0x0, 0x62d73000,
2017-10-31T18:10:27.864Z cpu2:9350)0x41221219bf10:[0x418018ca82be]User_LinuxSyscallHandler@vmkernel#nover+0x19 stack: 0x62f9feb0, 0x1e
2017-10-31T18:10:27.864Z cpu2:9350)0x41221219bf20:[0x418018d10064]gate_entry@vmkernel#nover+0x63 stack: 0x0, 0x13b, 0x0, 0x5b, 0x20000
2017-10-31T18:10:27.864Z cpu2:9350)vmkernel 0x0 .data 0x0 .bss 0x0
2017-10-31T18:10:27.864Z cpu2:9350)chardevs 0x41801906f000 .data 0x417fc0000000 .bss 0x417fc00008a0
2017-10-31T18:10:27.864Z cpu2:9350)user 0x418019074000 .data 0x417fc0400000 .bss 0x417fc0413240
2017-10-31T18:10:27.864Z cpu2:9350)vprobe 0x4180190fa000 .data 0x417fc0800000 .bss 0x417fc080cb40
2017-10-31T18:10:27.864Z cpu2:9350)vmkapi_socket 0x418019131000 .data 0x417fc0c00000 .bss 0x417fc0c00700
2017-10-31T18:10:27.864Z cpu2:9350)vmkapei 0x418019134000 .data 0x417fc1000000 .bss 0x417fc1000160
2017-10-31T18:10:27.864Z cpu2:9350)vmkapi_v2_0_0_0_vmkernel_shim 0x418019139000 .data 0x417fc1400000 .bss 0x417fc140b218
2017-10-31T18:10:27.864Z cpu2:9350)procfs 0x41801913a000 .data 0x417fc1800000 .bss 0x417fc1800220
2017-10-31T18:10:27.864Z cpu2:9350)procMisc 0x41801913d000 .data 0x417fc1c00000 .bss 0x417fc1c00000 -
5.1 isn’t a supported release (Its ancient). Update the host to something Modern (even 5.5 has less than a year left).
-
-
As @StorageNinja said, you should upgrade to something newer. The problem was described at https://kb.vmware.com/s/article/1020214?language=en_US
-
@ghani said in VMware PSOD happening on VMware host server:
Dear Team,
customer dont have VMware SnS support for upgrading VMware latest.
Then I guess you’re stuck. Either purchase the support or use a different hypervisor.
-
@ghani said in VMware PSOD happening on VMware host server:
Dear Team,
customer dont have VMware SnS support for upgrading VMware latest.
In any case where you are running VMware ESXi without support, you always either get support or move them to a platform that they can afford. Really no exception to that process.
-
@triple9 said in VMware PSOD happening on VMware host server:
As @StorageNinja said, you should upgrade to something newer. The problem was described at https://kb.vmware.com/s/article/1020214?language=en_US
Yup, this is a known problem that the customer is choosing at this point. Once could say that things are working as intended.
-
@ghani said in VMware PSOD happening on VMware host server:
Dear Team,
customer dont have VMware SnS support for upgrading VMware latest.
You don't need active SnS to upgrade to 5.5 which is still in support for another 300 days or so...
