Traffic not flowing for hosts behind NAT - Edge Router Lite
-
This weekend I failed to deploy my ERLs, because of a traffic issue occurring with one of the two.
ERL1 = local office ERL. I was able to test the configuration and found it to function with no problems. Traffic passed fine to the devices with 1-to-1 NAT
ERL2 = data center ERL. Due to [reasons] it was not possible to test this prior to the deployment attempt; thus, I mimicked the configuration of the known, working ERL1 as much as I could, obviously changing IP addresses as necessary. Traffic for hosts that weren't involved with 1-to-1 NAT worked fine. Traffic for hosts that had 1-to-1 NAT failed, which makes me think the issue is with how I've configured NAT.
Configuration caveats:
- Security through obscurity: I've altered the IP addresses of the hosts and interfaces, but they should be valid for the /24 and /28 networks involved.
- The old Cisco ASA had two interfaces in the outside-VLAN with one address assigned to them.
eth1
will eventually be that second interface. - I was so focused on just getting Internet connectivity to all of the hosts, I did not test the success of the site-to-site VPN.
- All configurations (except the obscuring of the IP addresses) were done using the GUI.
This is the configuration of the ERL2 as it was Saturday night.
Here's the current question: Do you folks see a problem with the NAT configuration? There should be 1-to-1 NAT for the IIS server and PostFix server. All other hosts (SQL server, etc.) use NAT-overload as its destination NAT.
firewall { all-ping enable broadcast-ping disable ipv6-receive-redirects disable ipv6-src-route disable ip-src-route disable log-martians enable name WAN_IN { default-action drop description "WAN to internal" rule 10 { action drop description "Deny traffic from 74.6.168.184" log disable protocol all source { address 74.6.168.184 } } rule 20 { action drop description "Deny traffic from 23.253.40.182" log disable protocol all source { address 23.253.40.182 } } rule 30 { action drop description "Deny traffic from 72.30.14.0/24" log disable protocol all source { address 72.30.14.0/24 } } rule 40 { action accept description "Allow HTTP to IIS" destination { address 172.16.254.12 port 80 } log disable protocol tcp } rule 50 { action accept description "Allow HTTPS to IIS" destination { address 172.16.254.12 port 443 } log disable protocol tcp } rule 60 { action accept description "Allow FTP to IIS" destination { address 172.16.254.12 port 21 } log disable protocol tcp } rule 70 { action accept description "Allow SMTP to PostFix" destination { address 172.16.254.14 port 25 } log disable protocol tcp } rule 80 { action accept description "Allow SMTP-SSL to PostFix" destination { address 172.16.254.14 port 465 } log disable protocol tcp } rule 90 { action accept description "Allow SMTP-MSA to PostFix" destination { address 172.16.254.14 port 587 } log disable protocol tcp } rule 100 { action accept description "Allow established/related" state { established enable related enable } } rule 110 { action drop description "Drop invalid state" state { invalid enable } } } name WAN_LOCAL { default-action drop description "WAN to router" rule 10 { action accept description "Allow established/related" state { established enable related enable } } rule 20 { action drop description "Drop invalid state" state { invalid enable } } } receive-redirects disable send-redirects enable source-validation disable syn-cookies enable } interfaces { ethernet eth0 { address 65.254.219.164/28 address 65.254.219.165/28 address 65.254.219.166/28 description Internet duplex auto firewall { in { name WAN_IN } local { name WAN_LOCAL } } speed auto } ethernet eth1 { address 172.16.100.1/24 description "Local 2" duplex auto speed auto } ethernet eth2 { address 172.16.254.1/24 description Local duplex auto speed auto } loopback lo { } } service { dns { forwarding { cache-size 150 listen-on eth1 listen-on eth2 } } gui { http-port 80 https-port 443 older-ciphers enable } nat { rule 1 { description "Outside to IIS" destination { address 65.254.219.165 } inbound-interface eth0 inside-address { address 172.16.254.12 } log disable protocol all type destination } rule 2 { description "Outside to PostFix" destination { address 65.254.219.166 } inbound-interface eth0 inside-address { address 172.16.254.14 } log disable type destination } rule 5000 { description "IIS to Outside" log disable outbound-interface eth0 outside-address { address 65.254.219.165 } protocol all source { address 172.16.254.12 } type source } rule 5001 { description "PostFix to Outside" log disable outbound-interface eth0 outside-address { address 65.254.219.166 } protocol all source { address 172.16.254.14 } type source } rule 5002 { description "masquerade for WAN" outbound-interface eth0 type masquerade } } ssh { port 22 protocol-version v2 } unms { disable } } system { gateway-address 65.254.219.161 host-name erl-peak10 login { user SomeUser { authentication { encrypted-password GIBBERISH. } level admin } } name-server 8.8.8.8 ntp { server 0.ubnt.pool.ntp.org { } server 1.ubnt.pool.ntp.org { } server 2.ubnt.pool.ntp.org { } server 3.ubnt.pool.ntp.org { } } syslog { global { facility all { level notice } facility protocols { level debug } } } time-zone America/New_York traffic-analysis { dpi enable export enable } } vpn { ipsec { auto-firewall-nat-exclude enable esp-group FOO0 { compression disable lifetime 3600 mode tunnel pfs enable proposal 1 { encryption aes128 hash sha1 } } ike-group FOO0 { ikev2-reauth no key-exchange ikev1 lifetime 28800 proposal 1 { dh-group 14 encryption aes128 hash sha1 } } site-to-site { peer 50.241.10.49 { authentication { mode pre-shared-secret pre-shared-secret GIBBERISH } connection-type initiate description Office ike-group FOO0 ikev2-reauth inherit local-address 172.16.254.1 tunnel 1 { allow-nat-networks disable allow-public-networks disable esp-group FOO0 local { prefix 172.16.254.0/24 } remote { prefix 172.16.253.0/24 } } } } } } /* Warning: Do not remove the following line. */ /* === vyatta-config-version: "config-management@1:conntrack@1:cron@1:dhcp-relay@1:dhcp-server@4:firewall@5:ipsec@5:nat@3:qos@1:quagga@2:system@4:ubnt-pptp@1:ubnt-unms@1:ubnt-util@1:vrrp@1:webgui@1:webproxy@1:zone-policy@1" === */ /* Release version: v1.9.7+hotfix.4.5024004.171005.0403 */
-
This means you messed up the source and/or destination NAT rules.
I am on my phone and cannot look at this carefully.
I can also post examples of how to configure this from a working site.
-
@jaredbusch said in Traffic not flowing for hosts behind NAT - Edge Router Lite:
This means you messed up the source and/or destination NAT rules.
I am on my phone and cannot look at this carefully.
I can also post examples of how to configure this from a working site.
I agree. I’ll look at it again after teaching tonight. I’m sure it’s staring back at me and for some reason I’m blind.
-
To check my understanding of NAT.
IIS Server
- Public: 65.254.219.165
- Private: 172.16.254.12
PostFix Server
- Public: 65.254.219.166
- Private: 172.16.254.14
Static 1:1 NAT is what's needed, and I'd have two rules for each host.
Rule 1 would be destination NAT. A packet enters the eth0 interface bound for the public IP address for the intended server. The ERL translates the destination address of this packet to the private IP address of the intended server.
Rule 2 would be a source NAT. A pack exits the eth0 interface bound for the Internet from a server with the source address as the server's private IP address. The ERL translates the source address of the pack to the public IP address of the server.
So the above example for the IIS server would look like this.
Rule 1. If a packet's destination is 65.254.219.165 is ingress to eth0, translate that packet's destination to 172.16.254.12, then send packet on to have firewall rules checked.
Rule 2. If a packet's source is 172.16.254.12 is to egress from eth0, translate that packet's source to 65.254.219.165, then send the packet on its way.
-
going to my office in a few minutes. will be able to help.
-
Delayed by kids. Putting them to bed now though
-
@jaredbusch said in Traffic not flowing for hosts behind NAT - Edge Router Lite:
Delayed by kids. Putting them to bed now though
Worry not. I’m turning early myself. As always, I appreciate the wisdom whenever it comes.
-
@eddiejennings said in Traffic not flowing for hosts behind NAT - Edge Router Lite:
@jaredbusch said in Traffic not flowing for hosts behind NAT - Edge Router Lite:
Delayed by kids. Putting them to bed now though
Worry not. I’m turning early myself. As always, I appreciate the wisdom whenever it comes.
They are going to bed. Not me.
anyway. post deluge incoming.
-
Dashboard, view showing eth0 (WAN port)
-
The Port forwarding rules only work against the first IP listed on the specified port.
In this case the
.42
oneth0
FYI:.1.14
is the internal Exchange server and.0.20
is the HAProxy server for Exchange. I am only proxying SSL.
-
I have a few firewall groups setup
-
The only LAN firewall rules are to block SMTP.
-
For anything you are going to NAT in to something other than the first IP address on the default masquerade, you have to have both firewall rules and NAT destination rules.
For anything that you want to go out something other than the default masquerade, you have to have a NAT source rule.
This is the WAN_IN and is likely part of your problem.
This is the NAT (source and destination) rules that are the other part of your problem.
-
For the screenshots above, the default NAT traffic will show a "What is my IP" as
12.XXX.XXX.42
as that is the first IP listed in the config of theeth0
port.All traffic goes out through the default NAT masquerade (shown as order 4 in the Source NAT section) unless there is a Source NAT rule to override it.
As I have more than one thing happening here, we will just focus on the Nginx ones as the stuff not going through the default masquerade.
A
Source NAT
is used for outbound traffic. You are the source of the traffic.
ADestination NAT
is used for inbound traffic. You are the destination of the traffic.So to send traffic from the server with an internal IP address of
10.202.1.16
out the external IP address of12.XXX.XXX.43
instead of the default masquerade external IP of12.XXX.XXX.42
we need to make aSource NAT
rule.Make a rule and fill it out like this. Obviously, there are a lot more options here than I am going over. If you have multiple machines that you need to use this IP outbound, then create a firewall group and select the
Src Address Group
drop down instead of specifying the IP address as I did here.
Now if you check your
What is my IP
from the specified server, it should return the12.XX.XXX.43
-
The inbound is harder because you have to setup both a
Destination NAT
rule as well as allow it on the firewall'sWAN_IN
rule.First make the
Destination NAT
rule as you are already on this screen from making theSource NAT
rule.As you can see you simply reverse what you did in the
Source NAT
rule. This time the local IP goes in the Translation and the WAN IP goes in theDest Address
field. Anything coming in on this destination IP will be translated to this local IP.As you can also see, I further restricted this translation to only be TCP/UDP and only ports included in a firewall port group.
In case you are curious, here are the ports in that firewall port group.
-
Now make a firewall rule in the policy assigned to the
IN
direction of yourWAN
interface.The wizards name this rule
WAN_IN
by default.You want to make the settings match when it comes to the protocol and port settings. But the destination is now the internal IP address as the translation has already happened by the NAT rules before the firewall rules see it.
-
Now you should have traffic properly flowing to and from your alternate IP addresses.
And yes, I noticed..
-
Well done Jared.
-
When I tried this last night, my new NAT rules were all below the default masquerade one. I tried moving (click and drag) above but it wouldn't actually move.
I then added a third rule (just some fake crap), then upon having three rules I was able to move my desired rule above the default one.
I'm on firmware v1.9.7-hotfix.4
-
@jaredbusch said in Traffic not flowing for hosts behind NAT - Edge Router Lite:
Now you should have traffic properly flowing to and from your alternate IP addresses.
Thanks for the above. I'm comparing that to my configuration now.
And yes, I noticed..
Ah, then you know the commercials.