CentOS 7 VM on Hyper-V losing DHCP assigned address
-
I have many CentOS 7 VMs on Hyper-v and randomly (and rarely) I have this problem.
The system will be unreachable. I log in to the console and i see it has no IP address. a quick ifdown/ifup and it is back. I assume it is failing to renew?
Where can I look to try and locate the cause?
-
In theory, /var/log/messages should hold some clues.
-
@scottalanmiller said in CentOS 7 VM on Hyper-V losing DHCP assigned address:
In theory, /var/log/messages should hold some clues.
but what do i look for is the better question I guess.
-
Your DHCP server's logs may also hold some clues if you can narrow down the time frame when the IP address vanishes.
-
WTF.. the date went south...
[root@owncloud ~]# grep DHCP /var/log/messages-20160516 May 11 08:49:13 owncloud dhclient[1134]: DHCPREQUEST on eth0 to 10.201.1.7 port 67 (xid=0x2ca58a7d) May 11 08:49:13 owncloud dhclient[1134]: DHCPACK from 10.201.1.7 (xid=0x2ca58a7d) May 11 08:49:13 owncloud NetworkManager[804]: <info> (eth0): DHCPv4 state changed bound -> bound May 11 18:15:47 owncloud dhclient[1134]: DHCPREQUEST on eth0 to 10.201.1.7 port 67 (xid=0x2ca58a7d) May 11 18:15:47 owncloud dhclient[1134]: DHCPACK from 10.201.1.7 (xid=0x2ca58a7d) May 11 18:15:47 owncloud NetworkManager[804]: <info> (eth0): DHCPv4 state changed bound -> bound May 12 06:11:42 owncloud dhclient[1134]: DHCPREQUEST on eth0 to 10.201.1.7 port 67 (xid=0x2ca58a7d) May 12 06:11:42 owncloud dhclient[1134]: DHCPACK from 10.201.1.7 (xid=0x2ca58a7d) May 12 06:11:42 owncloud NetworkManager[804]: <info> (eth0): DHCPv4 state changed bound -> bound May 12 17:00:03 owncloud dhclient[1134]: DHCPREQUEST on eth0 to 10.201.1.7 port 67 (xid=0x2ca58a7d) May 12 17:00:03 owncloud dhclient[1134]: DHCPACK from 10.201.1.7 (xid=0x2ca58a7d) May 12 17:00:03 owncloud NetworkManager[804]: <info> (eth0): DHCPv4 state changed bound -> bound May 13 02:14:18 owncloud dhclient[1134]: DHCPREQUEST on eth0 to 10.201.1.7 port 67 (xid=0x2ca58a7d) May 13 02:14:18 owncloud dhclient[1134]: DHCPACK from 10.201.1.7 (xid=0x2ca58a7d) May 13 02:14:18 owncloud NetworkManager[804]: <info> (eth0): DHCPv4 state changed bound -> bound May 22 23:30:53 owncloud NetworkManager[817]: <info> Activation (eth0) Beginning DHCPv4 transaction (timeout in 45 seconds) May 22 23:30:54 owncloud dhclient[1146]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 3 (xid=0x18ffd4fe) May 22 23:30:54 owncloud dhclient[1146]: DHCPREQUEST on eth0 to 255.255.255.255 port 67 (xid=0x18ffd4fe) May 22 23:30:54 owncloud dhclient[1146]: DHCPOFFER from 10.201.1.7 May 22 23:30:54 owncloud dhclient[1146]: DHCPACK from 10.201.1.7 (xid=0x18ffd4fe) May 22 23:30:54 owncloud NetworkManager[817]: <info> (eth0): DHCPv4 state changed unknown -> bound May 16 16:36:07 owncloud NetworkManager[803]: <info> Activation (eth0) Beginning DHCPv4 transaction (timeout in 45 seconds) May 16 16:36:07 owncloud dhclient[1133]: DHCPREQUEST on eth0 to 255.255.255.255 port 67 (xid=0x27078c0b) May 16 16:36:07 owncloud dhclient[1133]: DHCPACK from 10.201.1.7 (xid=0x27078c0b) May 16 16:36:07 owncloud NetworkManager[803]: <info> (eth0): DHCPv4 state changed unknown -> bound [root@owncloud ~]#
-
So the
ntpd
service is not present on this system. but I know I enabled an NTP server during the CentOS 7 setup.Anyone know where that setting is? I'm not finding anything with my Google Fu this morning except answers relating to
ntpd
. -
It doesn't use ntpd anymore. I think it uses chrony.
Make sure you have it set so that the time isn't coming from the hardware, I had that issue with Linux VMs on both VMWare and Hyper-V in the past.
-
Yeah, chrony is the new replacement.
-
@coliver said in CentOS 7 VM on Hyper-V losing DHCP assigned address:
It doesn't use ntpd anymore. I think it uses chrony.
Make sure you have it set so that the time isn't coming from the hardware, I had that issue with Linux VMs on both VMWare and Hyper-V in the past.
Thanks, Once i knew the right name I had no problem finding the config file.
It seems i left a couple public servers in there. But hardware, should only be possibly hit on boot up right?
/me goes off to double check Hyper-V settigns.
# These servers were defined in the installation: server 10.201.1.7 iburst server 10.201.1.1 iburst server 0.centos.pool.ntp.org iburst server 3.centos.pool.ntp.org iburst # Use public servers from the pool.ntp.org project. # Please consider joining the pool (http://www.pool.ntp.org/join.html). # Ignore stratum in source selection. stratumweight 0 # Record the rate at which the system clock gains/losses time. driftfile /var/lib/chrony/drift # Enable kernel RTC synchronization. rtcsync # In first three updates step the system clock instead of slew # if the adjustment is larger than 10 seconds. makestep 10 3 # Allow NTP client access from local network. #allow 192.168/16 # Listen for commands only on localhost. bindcmdaddress 127.0.0.1 bindcmdaddress ::1 # Serve time even if not synchronized to any NTP server. #local stratum 10 keyfile /etc/chrony.keys # Specify the key used as password for chronyc. commandkey 1 # Generate command key if missing. generatecommandkey # Disable logging of client accesses. noclientlog # Send a message to syslog if a clock adjustment is larger than 0.5 seconds. logchange 0.5 logdir /var/log/chrony #log measurements statistics tracking
-
@JaredBusch said in CentOS 7 VM on Hyper-V losing DHCP assigned address:
@coliver said in CentOS 7 VM on Hyper-V losing DHCP assigned address:
It doesn't use ntpd anymore. I think it uses chrony.
Make sure you have it set so that the time isn't coming from the hardware, I had that issue with Linux VMs on both VMWare and Hyper-V in the past.
Thanks, Once i knew the right name I had no problem finding the config file.
It seems i left a couple public servers in there. But hardware, should only be possibly hit on boot up right?
/me goes off to double check Hyper-V settigns.
# These servers were defined in the installation: server 10.201.1.7 iburst server 10.201.1.1 iburst server 0.centos.pool.ntp.org iburst server 3.centos.pool.ntp.org iburst # Use public servers from the pool.ntp.org project. # Please consider joining the pool (http://www.pool.ntp.org/join.html). # Ignore stratum in source selection. stratumweight 0 # Record the rate at which the system clock gains/losses time. driftfile /var/lib/chrony/drift # Enable kernel RTC synchronization. rtcsync # In first three updates step the system clock instead of slew # if the adjustment is larger than 10 seconds. makestep 10 3 # Allow NTP client access from local network. #allow 192.168/16 # Listen for commands only on localhost. bindcmdaddress 127.0.0.1 bindcmdaddress ::1 # Serve time even if not synchronized to any NTP server. #local stratum 10 keyfile /etc/chrony.keys # Specify the key used as password for chronyc. commandkey 1 # Generate command key if missing. generatecommandkey # Disable logging of client accesses. noclientlog # Send a message to syslog if a clock adjustment is larger than 0.5 seconds. logchange 0.5 logdir /var/log/chrony #log measurements statistics tracking
I'm not sure about that. I know I had to disable it per VM on Hyper-V.
-
@coliver said in CentOS 7 VM on Hyper-V losing DHCP assigned address:
I'm not sure about that. I know I had to disable it per VM on Hyper-V.
Right, I know that, but i thought it always checks hardware on boot regardless of setting... It is not checked.
-
@JaredBusch said in CentOS 7 VM on Hyper-V losing DHCP assigned address:
@coliver said in CentOS 7 VM on Hyper-V losing DHCP assigned address:
I'm not sure about that. I know I had to disable it per VM on Hyper-V.
Right, I know that, but i thought it always checks hardware on boot regardless of setting... It is not checked.
Good, then maybe that public server was the issue.
-
@coliver said in CentOS 7 VM on Hyper-V losing DHCP assigned address:
@JaredBusch said in CentOS 7 VM on Hyper-V losing DHCP assigned address:
@coliver said in CentOS 7 VM on Hyper-V losing DHCP assigned address:
I'm not sure about that. I know I had to disable it per VM on Hyper-V.
Right, I know that, but i thought it always checks hardware on boot regardless of setting... It is not checked.
Good, then maybe that public server was the issue.
I hope so. highly annoying
-
Same issue different server.
Why is the time so f'd up.
I checked the Hyper-V server, it has the correct time.
I checked chrony on the CentOS box, it had public NTP servers. Boom problem.
Changed the ntp servers to internal sources and magic.[root@owncloud ~]# grep chrony /var/log/messages Sep 19 14:14:44 owncloud chronyd[765]: Can't synchronise: no selectable sources Sep 21 17:02:28 owncloud chronyd[758]: chronyd version 2.1.1 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +DEBUG +ASYNCDNS +IPV6 +SECHASH) Sep 21 17:02:28 owncloud chronyd[758]: Frequency -25.319 +/- 0.011 ppm read from /var/lib/chrony/drift Sep 20 14:12:20 owncloud chronyd[758]: Selected source 209.208.79.69 Sep 20 14:12:20 owncloud chronyd[758]: System clock wrong by -96617.583787 seconds, adjustment started Sep 20 14:12:20 owncloud chronyd[758]: System clock was stepped by -96617.583787 seconds Sep 20 14:12:21 owncloud chronyd[758]: Selected source 104.238.179.130 Sep 20 19:02:11 owncloud chronyd[758]: Selected source 209.208.79.69 Sep 22 08:35:19 owncloud chronyd[758]: chronyd version 2.1.1 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +DEBUG +ASYNCDNS +IPV6 +SECHASH) Sep 22 08:35:19 owncloud chronyd[758]: Frequency -25.359 +/- 0.039 ppm read from /var/lib/chrony/drift Sep 21 11:26:54 owncloud chronyd[758]: Selected source 10.202.1.1 Sep 21 11:26:54 owncloud chronyd[758]: System clock wrong by -76114.165928 seconds, adjustment started Sep 21 11:26:54 owncloud chronyd[758]: System clock was stepped by -76114.165928 seconds
[root@owncloud ~]# date Wed Sep 21 11:38:02 CDT 2016 [root@owncloud ~]#
-
So the reason I posted to this again.. If you cannot trust ntp.org anymore how are we supposed to handle this.
-
@JaredBusch said in CentOS 7 VM on Hyper-V losing DHCP assigned address:
So the reason I posted to this again.. If you cannot trust ntp.org anymore how are we supposed to handle this.
What did I miss? Why can't we trust NTP.org?
-
@scottalanmiller said in CentOS 7 VM on Hyper-V losing DHCP assigned address:
@JaredBusch said in CentOS 7 VM on Hyper-V losing DHCP assigned address:
So the reason I posted to this again.. If you cannot trust ntp.org anymore how are we supposed to handle this.
What did I miss? Why can't we trust NTP.org?
It pulled the wrong date for the server, which then f'd up the DHCP renew.
-
@scottalanmiller said in CentOS 7 VM on Hyper-V losing DHCP assigned address:
@JaredBusch said in CentOS 7 VM on Hyper-V losing DHCP assigned address:
So the reason I posted to this again.. If you cannot trust ntp.org anymore how are we supposed to handle this.
What did I miss? Why can't we trust NTP.org?
Crazy time skew with ntp.org and chrony. Never had the issue with the ntpd system. I've had it on some of my new CentOS servers as well. We have a atomic clock on site here but if using ntp,org we get some serious skew.
-
Are we sure that ntp.org was the issue? Is this repeatable?
-
@scottalanmiller said in CentOS 7 VM on Hyper-V losing DHCP assigned address:
Are we sure that ntp.org was the issue? Is this repeatable?
Second time it has caught me. Completely different server. Completely different client.