CentOS 7 networking issues

JaredBusch

For pretty much ever CentOS 7 system I have installed I have networking issues with DHCP, seeming caused by some time sync issues with the Hyper-V integration services hardware clock.

Today, I was looking to switch a system to a static address for now because I have not had time to troubleshoot the actual time sync issue.

This should be simple right? Nope.

First to determine what I need to work with.

[root@nginix ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:15:5d:01:03:09 brd ff:ff:ff:ff:ff:ff
    inet 10.201.1.18/24 brd 10.201.1.255 scope global dynamic eth0
       valid_lft 74467sec preferred_lft 74467sec
    inet6 fe80::fb6c:e1ec:b116:5220/64 scope link
       valid_lft forever preferred_lft forever

Looks like eth0 with MAC: 02:15:5d:01:03:09, great.

Let's go change it. Wait WTF MAC is that?
nano /etc/sysconfig/network-scripts/ifcfg-eth0


HWADDR="00:15:5D:01:03:09"
TYPE="Ethernet"
BOOTPROTO="dhcp"
DEFROUTE="yes"
PEERDNS="yes"
PEERROUTES="yes"
IPV4_FAILURE_FATAL="no"
IPV6INIT="yes"
IPV6_AUTOCONF="yes"
IPV6_DEFROUTE="yes"
IPV6_PEERDNS="yes"
IPV6_PEERROUTES="yes"
IPV6_FAILURE_FATAL="no"
NAME="eth0"
UUID="54dcedcf-528a-4556-9082-67c42b8a9ff7"
ONBOOT="yes"

To the googles! Looks like Network Manager was introduced.
Let's see what network manager has to say.

[root@nginix ~]# nmcli
eth0: connected to Wired connection 1
        ethernet (hv_netvsc), 02:15:5D:01:03:09, hw, mtu 1500
        ip4 default
        inet4 10.201.1.18/24
        inet6 fe80::fb6c:e1ec:b116:5220/64

lo: unmanaged
        loopback (unknown), 00:00:00:00:00:00, sw, mtu 65536

Use "nmcli device show" to get complete information about known devices and
"nmcli connection show" to get an overview on active connection profiles.

Consult nmcli(1) and nmcli-examples(5) manual pages for complete usage details.

Well there is the right MAC address at least.. Let's check the details as recommended.

[root@nginix ~]# nmcli device show
GENERAL.DEVICE:                         eth0
GENERAL.TYPE:                           ethernet
GENERAL.HWADDR:                         02:15:5D:01:03:09
GENERAL.MTU:                            1500
GENERAL.STATE:                          100 (connected)
GENERAL.CONNECTION:                     Wired connection 1
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/ActiveConnection/0
WIRED-PROPERTIES.CARRIER:               on
IP4.ADDRESS[1]:                         10.201.1.18/24
IP4.GATEWAY:                            10.201.1.1
IP4.DNS[1]:                             10.201.1.7
IP4.DOMAIN[1]:                          domain.local
IP6.ADDRESS[1]:                         fe80::fb6c:e1ec:b116:5220/64
IP6.GATEWAY:

GENERAL.DEVICE:                         lo
GENERAL.TYPE:                           loopback
GENERAL.HWADDR:                         00:00:00:00:00:00
GENERAL.MTU:                            65536
GENERAL.STATE:                          10 (unmanaged)
GENERAL.CONNECTION:                     --
GENERAL.CON-PATH:                       --
IP4.ADDRESS[1]:                         127.0.0.1/8
IP4.GATEWAY:
IP6.ADDRESS[1]:                         ::1/128
IP6.GATEWAY:

Oh, a config file! Nope.

[root@nginix ~]# ls /org/freedesktop/NetworkManager/ActiveConnection/0
ls: cannot access /org/freedesktop/NetworkManager/ActiveConnection/0: No such file or directory

The googles say to use the Network manager TUI. Ok.
nmtui

0_1488407884960_upload-7a0fd0ed-f7a3-4ace-9c5d-c122506666fa

Great, let's edit. Wait WTF?

0_1488407911605_upload-4e5daffc-c77a-483f-b532-646c641af811
0_1488407956866_upload-ab482be4-9764-4e33-8dd7-7bdce3f1ded0
0_1488407981394_upload-23748854-249f-4563-9ec6-91f97f5279c2

WTF is this? Thinking back I know where it came from. When this VM was initially made, I forgot to set a static MAC address in the Hyper-V console. At some point after the install completed, I realized my mistake and shutdown the VM and edited the MAC to be static in Hyper-V.

But why is this all here? Why is there a file in the legacy config script location as well as this funky network manager section?

Which one is supposed to be used?

JaredBusch

So I go and delete eth0 from nmtui and then systemctl restart network

Everything still works, but the old file is gone.

[root@nginix ~]# ls /etc/sysconfig/network-scripts/ifcfg-eth0
ls: cannot access /etc/sysconfig/network-scripts/ifcfg-eth0: No such file or directory

JaredBusch

I then edited "Wired connection 1" to show this.
0_1488408575453_upload-f6b7cd4f-26c1-4408-bf77-ffdeaeb06a78
0_1488408598656_upload-20e5aea3-4ee3-4798-b693-8bf5cbaf342e

It all works.

[root@nginix ~]# nmtui
[root@nginix ~]# systemctl restart network
[root@nginix ~]# ip a sh
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 02:15:5d:01:03:09 brd ff:ff:ff:ff:ff:ff
    inet 10.201.1.18/24 brd 10.201.1.255 scope global dynamic eth0
       valid_lft 86396sec preferred_lft 86396sec
    inet6 fe80::fb6c:e1ec:b116:5220/64 scope link
       valid_lft forever preferred_lft forever

But now there is a legacy file again.

[root@nginix ~]# ls /etc/sysconfig/network-scripts/ifcfg-eth0
/etc/sysconfig/network-scripts/ifcfg-eth0
[root@nginix ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
HWADDR=02:15:5D:01:03:09
TYPE=Ethernet
BOOTPROTO=dhcp
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=eth0
UUID=a7cf382b-8b9f-3abe-a3fe-2d4e8490a666
ONBOOT=yes
AUTOCONNECT_PRIORITY=-999

JaredBusch

and nmcli device show is different.

[root@nginix ~]# nmcli device show
GENERAL.DEVICE:                         eth0
GENERAL.TYPE:                           ethernet
GENERAL.HWADDR:                         02:15:5D:01:03:09
GENERAL.MTU:                            1500
GENERAL.STATE:                          100 (connected)
GENERAL.CONNECTION:                     eth0
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/ActiveConnection/1
WIRED-PROPERTIES.CARRIER:               on
IP4.ADDRESS[1]:                         10.201.1.18/24
IP4.GATEWAY:                            10.201.1.1
IP4.DNS[1]:                             10.201.1.7
IP4.DOMAIN[1]:                          domain.local
IP6.ADDRESS[1]:                         fe80::fb6c:e1ec:b116:5220/64
IP6.GATEWAY:

GENERAL.DEVICE:                         lo
GENERAL.TYPE:                           loopback
GENERAL.HWADDR:                         00:00:00:00:00:00
GENERAL.MTU:                            65536
GENERAL.STATE:                          10 (unmanaged)
GENERAL.CONNECTION:                     --
GENERAL.CON-PATH:                       --
IP4.ADDRESS[1]:                         127.0.0.1/8
IP4.GATEWAY:
IP6.ADDRESS[1]:                         ::1/128
IP6.GATEWAY:

Obsolesce

First thing I do on CentOS is disable Network Manager. I hate it.

JaredBusch

@Tim_G said in CentOS 7 networking issues:

First thing I do on CentOS is disable Network Manager. I hate it.

Wrong answer, sorry. Network manager is the new default. You should get used to it and use it appropriately.

JaredBusch

So I changed it to static like this.

0_1488409027886_upload-6278cf40-5144-4ad9-a5ed-a3593f25ef3e

Obsolesce

Part of my CentOS routine:

Change run level: nano /etc/inittab (use id:3:initdefault:)
cd /etc/rc.d/rc3.d
ls -lh
mv S23NetworkManager K23NetworkManager
Reboot
ls -l /etc/sysconfig/network-scripts (to see what's there)
nano /etc/sysconfig/network-scripts/ifcfg-eth0
(if dhcp)
DEVICE=eth0
BOOTPROTO=dhcp
ONBOOT=yes
(save it)

ifup eth0
chkconfig network on

travisdh1

Network Manager, still throwing admins for a loop even after all these years

stacksofplates

NetworkManager still creates the ifcfg files under network-scripts. So those configs will still be there, but should have NM_CONTROLLED=yes

Also, for some reason CentOS ships with both network and NetworkManager on at the same time. So that's most likely why you saw the DHCP for NetworkManager and the static file that you created. I always disable and mask network to keep people from turning it back on when they aren't paying attention.

stacksofplates

@Tim_G said in CentOS 7 networking issues:

Part of my CentOS routine:

Change run level: nano /etc/inittab (use id:3:initdefault:)

cd /etc/rc.d/rc3.d

ls -lh

mv S23NetworkManager K23NetworkManager

Reboot

ls -l /etc/sysconfig/network-scripts (to see what's there)

nano /etc/sysconfig/network-scripts/ifcfg-eth0
(if dhcp)
DEVICE=eth0
BOOTPROTO=dhcp
ONBOOT=yes
(save it)

ifup eth0
chkconfig network on

Targets (run levels) are handled through systemd now so you should use systemctl set-default multi-user.target.

And as @JaredBusch said, NetworkManager is the new default, and handles things like teaming much better.

Obsolesce

@stacksofplates Didn't realize he was being forced to use defaults. Since that's the case, I guess he has no other option than to use Network Manager?

JaredBusch

@Tim_G said in CentOS 7 networking issues:

@stacksofplates Didn't realize he was being forced to use defaults. Since that's the case, I guess he has no other option than to use Network Manager?

No one is forcing me to use anything. But use of defaults is a best practice. without a really good reason, you do not stray from them.

Obsolesce

@JaredBusch said in CentOS 7 networking issues:

@Tim_G said in CentOS 7 networking issues:

@stacksofplates Didn't realize he was being forced to use defaults. Since that's the case, I guess he has no other option than to use Network Manager?

No one is forcing me to use anything. But use of defaults is a best practice. without a really good reason, you do not stray from them.

Losing VM functionality due to time issues because of a default isn't a good enough reason? Isn't it a production VM? Does it being down not lose the company money on top of paying you to mess with it? I'd rather use a non-default that works just as well for a particular function or task, rather than having to waste so much of a clients money to fix something that can arguably be replaced with something else non-default. Maybe Network Manager handles teaming better, but are you using teaming? Are you using teaming at the VM level or at the Hypervisor level? Just do what's best for the client. Don't use a default because it does something better with something you aren't even using. Know what I mean?

Maybe for your case it's worth it to get Network Manager working properly due to other things I don't know about. I'm not giving a blanket statement, just throwing out a point is all.

Sometimes best practice is a square, and what you are dealing with is a circle. It might not fit.

stacksofplates

@Tim_G said in CentOS 7 networking issues:

Losing VM functionality due to time issues because of a default isn't a good enough reason

That's not the reason. The downtime was due to both services running. I have literally never had a network issue with just NetworkManager running. It also takes about 10 seconds to set up an interface through NetworkManager, it's much simpler.

Also network is being phased out, and not receiving feature updates. So while you can use it currently, it's most likely not going to be there in 8.

And yes, I am using teaming, but on the host.

stacksofplates

@stacksofplates said in CentOS 7 networking issues:

NetworkManager still creates the ifcfg files under network-scripts. So those configs will still be there, but should have NM_CONTROLLED=yes

Also, for some reason CentOS ships with both network and NetworkManager on at the same time. So that's most likely why you saw the DHCP for NetworkManager and the static file that you created. I always disable and mask network to keep people from turning it back on when they aren't paying attention.

So it looks like they took out the NM_CONTROLLED section. I must have been thinking of RHEL 6. But NM still creates the ifcfg files in the same directory.