Also i was finding that when i closed Virt-Manager on the Windows machine, then came back later the connection was lost and i had to re-add the connection and delete the default one that keeps coming back.
This was solved by running sudo apt-get install dbus-x11
on ubuntu
Now when i reboot my Windows machine and re-open virt-manager the default is gone and my KVM connects automatically