systemVM stuck in starting state

[DaanHoogland]

When a host running system VM's ( SSVM/ Console Proxy / VR ) undergoes a planned or un-planned reboot, after the reboot, the system VM's get stuck in "Starting" state and do not start up properly causing new VM deployment issues.
The behaviour was tested on version 4.13.

ISSUE TYPE

• Bug Report

COMPONENT NAME

HA systemVMs

CLOUDSTACK VERSION

4.13.0

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

[andrijapanicsb]

@DaanHoogland is this related to local storage, or in general?

[DaanHoogland]

@andrijapanicsb I'd have to test, i don't know.

[Spaceman1984]

Tested with NFS, no error - Will test with local storage.

[yadvr]

Not seen recently.

[Spaceman1984]

Tested with local storage, unable to reproduce.

[andrijapanicsb]

Did you try/test this by powering off (in parent lab) the hypervisor where the SSVM/CPVM are running (KVM, local storage)?

[Spaceman1984]

I tested with restarting my Centos 7 KVM Hypervisor, 2 Different zones, 1 with local storage, one with NFS.

[yadvr]

@DaanHoogland are you able to reproduce it still on master?

[DaanHoogland]

tried on latest master (4.15 pre-release) a 2 kvm nested test env with centos7 hosts:

the host and systemVM show as up and connected but the console is not reachable and on the host virsh list does not show any VMs.

[DaanHoogland]

retsting again as it finds no traction, will close and open a new one if it costs any effort to reproduce.