Traffic not flowing for hosts behind NAT - Edge Router Lite

Mike Davis

Well done Jared.

Dashrender

When I tried this last night, my new NAT rules were all below the default masquerade one. I tried moving (click and drag) above but it wouldn't actually move.

I then added a third rule (just some fake crap), then upon having three rules I was able to move my desired rule above the default one.

I'm on firmware v1.9.7-hotfix.4

EddieJennings

@jaredbusch said in Traffic not flowing for hosts behind NAT - Edge Router Lite:

Now you should have traffic properly flowing to and from your alternate IP addresses.

Thanks for the above. I'm comparing that to my configuration now.

And yes, I noticed..

Ah, then you know the commercials.

EddieJennings

@dashrender said in Traffic not flowing for hosts behind NAT - Edge Router Lite:

When I tried this last night, my new NAT rules were all below the default masquerade one. I tried moving (click and drag) above but it wouldn't actually move.

I then added a third rule (just some fake crap), then upon having three rules I was able to move my desired rule above the default one.

I'm on firmware v1.9.7-hotfix.4

Even though it didn't move, did the rule order number change?

Dashrender

@eddiejennings said in Traffic not flowing for hosts behind NAT - Edge Router Lite:

@dashrender said in Traffic not flowing for hosts behind NAT - Edge Router Lite:

When I tried this last night, my new NAT rules were all below the default masquerade one. I tried moving (click and drag) above but it wouldn't actually move.

I then added a third rule (just some fake crap), then upon having three rules I was able to move my desired rule above the default one.

I'm on firmware v1.9.7-hotfix.4

Even though it didn't move, did the rule order number change?

good question - I don't recall. I did see the issue where when making firewall Ruleset changes, when I would drag and drop them, the order on screen would change to some jumble, but the actual numerical value would be the desired change. Saving the rule order would fix the display to display them in numerical order.

Dashrender

I just tried it again now
https://i.imgur.com/ZmoxUun.png

This is what is normally looks like
https://i.imgur.com/yi1wL5G.png

As you can see in the top image, I can't even see the other line item to move it above or below. I've zoomed the page in and out, no option there allows me to see where I'm placing it.
Additionally, after dropping it somewhere, the numerical order does not change.

As mentioned above, creating a third entry allowed me to work around this.

EddieJennings

Below is the GUI for the ERL. I'm going to some firewall groups, as that seems to be a cleaner way to do that.

There are the differences I see in Jared's configuration and mine.

Jared's NAT rules include port matching, rather than just matching all traffic
Jared's WAN_IN firewall rules have the "Accept Established / Related" and "Drop invalid" at the the top
Jared's firewall rule example explicitly allows the New state

Perhaps I'm being thick, but I'm failing to see the smoking gun as to why my configuration failed.

Dashboard

NAT Rules

IIS Source NAT rule details

IIS Destination NAT rule details

Firewall Rules

IIS HTTPS Rule detail (all other rules follow this pattern)

JaredBusch

@EddieJennings
0_1513092680278_26f1a2f9-ce4f-4d8f-9b4b-d800ebe96c30-image.png

JaredBusch

@eddiejennings said in Traffic not flowing for hosts behind NAT - Edge Router Lite:

Below is the GUI for the ERL. I'm going to some firewall groups, as that seems to be a cleaner way to do that.
There are the differences I see in Jared's configuration and mine.

Jared's NAT rules include port matching, rather than just matching all traffic
Jared's WAN_IN firewall rules have the "Accept Established / Related" and "Drop invalid" at the the top
Jared's firewall rule example explicitly allows the New state

Perhaps I'm being thick, but I'm failing to see the smoking gun as to why my configuration failed.

You always want the most hit firewall rules to be first.

Always. This is not an Ubiquiti thing, this is an always thing.

Firewall rules are processed sequentially and processing stops once a match is made.

Thus you always want the thing that is gong to match the most to be checked first.

In all cases, for standard NAT traffic hitting the inbound firewall, the most hit rule will always be the Established/Related.

Next, you drop in valid because well, it is invalid. This comes second, because most traffic is still Established/Related.

Then you add in your rules.

EddieJennings

@jaredbusch That make sense, as if I specify nothing, then nothing would match.

It's curious though that the exact same rules (with state unspecified) worked flawlessly with the other ERL.

JaredBusch

@eddiejennings said in Traffic not flowing for hosts behind NAT - Edge Router Lite:

@jaredbusch That make sense, as if I specify nothing, then nothing would match.

It's curious though that the exact same rules (with state unspecified) worked flawlessly with the other ERL.

No they don't. Something would be different.

EddieJennings

@jaredbusch I agree. The question is finding what's different.

Toying around, if I were to add a new rule, by default, there is no state specified.

JaredBusch

@eddiejennings said in Traffic not flowing for hosts behind NAT - Edge Router Lite:

@jaredbusch I agree. The question is finding what's different.

Toying around, if I were to add a new rule, by default, there is no state specified.

Yes, because they don't know what you are trying to allow.

JaredBusch

@eddiejennings said in Traffic not flowing for hosts behind NAT - Edge Router Lite:

@jaredbusch I agree. The question is finding what's different.

Toying around, if I were to add a new rule, by default, there is no state specified.

Simplicity itself.

From both routers.

show configuration commands

Then compare them with a line comparison tool.

JaredBusch

Looks like this.
0_1513094671891_9a98c9b4-3d24-4e16-9ff4-86943e431490-image.png

EddieJennings

I have another opportunity to test the ERL tomorrow morning. Going line-by-line the only significant differences between my ERLs were IP addresses (obviously), the fact that one had configuration for remote-access VPN, and a DHCP server. Structure of the config for NAT and firewall rules were the same.

Dashrender

I wonder if your switch wasn't updating the MAC table to send the traffic to the ERL. Not a common issue, normally solved by flushing the MAC table or rebooting the switch after replacing the ASA.

EddieJennings

So. With the limited time I had this morning, here's what went down.

Swapped ASA for ERL. Non NAT'd stuff = fine. NAT'd stuff = not so much.
Put the ASA back in to make sure stuff would work again (because I'm a fool).
Put a switch between my network and the data center network drop, and so I can have both ASA and ERL in service (have enough public IPs to do this).
Get a call from the data center making sure everything was ok because I apparently caused a STP problem for them by having a flapping connection (see the end of #2).
Just now had time to work with my new setup (data center > switch 1> ASA, 2> ERL).

The result:

I setup an apache server, changed IPs in the ERL's configuration to reflect the new public IP, updated the NAT rules to reflect the new IPs, updated the rule to allow port 80 traffic to reflect the new internal IP. No other configuration changes other than changing the IPs. Traffic passed for my NAT'd device (the apache server) with zero problems.

So after my embarrassment this morning and various menial tasks during the day, there is some light. My little test environment had traffic flowing.

Here's the curious thing from this morning. You can see at the top source NAT was doing translations. The packet capture shows traffic hitting the eth0 interface bound for the public IP of the IIS server, but you see on the bottom right, no destination NAT translations happening. In the test environment described above, I do have destination NAT translations.

Dashrender

@eddiejennings said in Traffic not flowing for hosts behind NAT - Edge Router Lite:

I setup an apache server, changed IPs in the ERL's configuration to reflect the new public IP, updated the NAT rules to reflect the new IPs, updated the rule to allow port 80 traffic to reflect the new internal IP. No other configuration changes other than changing the IPs. Traffic passed for my NAT'd device (the apache server) with zero problems.

Sadly - I've seen this before. The output telling you that settings were right, but something in the black box is just stuck, and only wiping and reloading fixed it. So you changing the IPs fixed whatever was stuck.

Dashrender

@eddiejennings said in Traffic not flowing for hosts behind NAT - Edge Router Lite:

So. With the limited time I had this morning, here's what went down.

Swapped ASA for ERL. Non NAT'd stuff = fine. NAT'd stuff = not so much.

Put the ASA back in to make sure stuff would work again (because I'm a fool).

Put a switch between my network and the data center network drop, and so I can have both ASA and ERL in service (have enough public IPs to do this).

Get a call from the data center making sure everything was ok because I apparently caused a STP problem for them by having a flapping connection (see the end of #2).

Just now had time to work with my new setup (data center > switch 1> ASA, 2> ERL).

I'm a bit surprised you didn't have issues before you even left the DC assuming the ASA and ERL were both on the same IPs. At least it was an easy fix. Did you have to return to the DC to fix it?