Website down, but only for organization Network
-
Scratching my head on this one.
During website edits for our website, when making changes I get ERR_TIMED_OUT (Chrome) when trying to publish edits. I might get a few edits published, but then it inevitably loses connection.
The website goes down, or appears to for about 5 minutes. Hosting Provider assures me there is no PHP issues and everything looks to be in order.
Tricky thing is, if I take my phone off of our organizations WiFi, the site is still operational. In fact, the site never really goes down to the rest of the world.
I ran tracert from my workstation and everything appears to be fine with DNS.
It appears to affect all devices on domain.
My next step is going to be contacting ISP, but I figured anyone who's ever experienced this would remember what the issue was.
Does this ring any bells for anyone?
-
@Mr-Jones said in Website down, but only for organization Network:
Scratching my head on this one.
During website edits for our website, when making changes I get ERR_TIMED_OUT (Chrome) when trying to publish edits. I might get a few edits published, but then it inevitably loses connection.
The website goes down, or appears to for about 5 minutes. Hosting Provider assures me there is no PHP issues and everything looks to be in order.
Tricky thing is, if I take my phone off of our organizations WiFi, the site is still operational. In fact, the site never really goes down to the rest of the world.
I ran tracert from my workstation and everything appears to be fine with DNS.
It appears to affect all devices on domain.
My next step is going to be contacting ISP, but I figured anyone who's ever experienced this would remember what the issue was.
Does this ring any bells for anyone?
Does your network have any kind of security stuff on the workstations or firewalls that monitors that kind of traffic? I've seen some rare instances where the Firewall or AV software would start blocking after a minute or two, and then it would crash and restart and then everything would be happy for another few minutes.
-
Well, ISP wiped their hands with this one.
Next step is resetting Modem, Router when I can find an appropriate window.
-
@dafyre said in Website down, but only for organization Network:
Does your network have any kind of security stuff on the workstations or firewalls that monitors that kind of traffic? I've seen some rare instances where the Firewall or AV software would start blocking after a minute or two, and then it would crash and restart and then everything would be happy for another few minutes.
I've disabled local firewall, and put Web Filter in Audit Mode with no affect.
If resetting the Modem, Router, switches doesn't work, I'll move to the Network Firewall as I agree there might be some security DDOS protection or otherwise that's at play here.
-
@Mr-Jones said in Website down, but only for organization Network:
My next step is going to be contacting ISP, but I figured anyone who's ever experienced this would remember what the issue was.
Safe to assume that this can't be the issue. Unless all websites are down, it's not your ISP looking up your domain and blocking it.
-
@Mr-Jones said in Website down, but only for organization Network:
Well, ISP wiped their hands with this one.
Next step is resetting Modem, Router when I can find an appropriate window.
It can't be them (it can, but not in any plausible way.) It's not reasonably your networking equipment either. This isn't a networking issue.
-
@Mr-Jones said in Website down, but only for organization Network:
If resetting the Modem, Router, switches doesn't work, I'll move to the Network Firewall as I agree there might be some security DDOS protection or otherwise that's at play here.
Possible. What kind of firewall is it?
-
@dafyre said in Website down, but only for organization Network:
The website goes down, or appears to for about 5 minutes. Hosting Provider assures me there is no PHP issues and everything looks to be in order.
Tricky thing is, if I take my phone off of our organizations WiFi, the site is still operational. In fact, the site never really goes down to the rest of the world.Putting these two things together, I'm going to say it's almost certainly Fail2Ban. I bet something on your network is sending a bad password automatically and causing an auto-ban for a few minutes,.
-
@scottalanmiller said in Website down, but only for organization Network:
@Mr-Jones said in Website down, but only for organization Network:
If resetting the Modem, Router, switches doesn't work, I'll move to the Network Firewall as I agree there might be some security DDOS protection or otherwise that's at play here.
Possible. What kind of firewall is it?
Barracuda.
-
@Mr-Jones said in Website down, but only for organization Network:
@scottalanmiller said in Website down, but only for organization Network:
@Mr-Jones said in Website down, but only for organization Network:
If resetting the Modem, Router, switches doesn't work, I'll move to the Network Firewall as I agree there might be some security DDOS protection or otherwise that's at play here.
Possible. What kind of firewall is it?
Barracuda.
That might have some sort of limiting. But the expectation would be that it would be a firewall on the other side (host) if it was a firewall issue. If the Barracuda is limiting OUTBOUND requests, you've got a serious design issue (not that Barracuda has much of a reputation outside of being a security gap as they have open backdoors).
I can't imagine that it is the Barracuda. Plausible, yes, likely... definitely not.
-
So I get the Network Firewall folks on the phone, and now everything wants to work as smooth as ever. Ugh.
I've spent maybe 30 minutes trying to replicate the issue that's plagued me repeatedly since 6:15am yesterday, and I cannot now.
Great news that it's all working now, embarrasing that I can't replicate it when asked.
I still feel like I need to know what happened.
Would a PHP error cause this? That's the only thing I can think of, as I was editing some conditional logic on the website yesterday morning, but I'm failing to see the correlation given the context of the issue. I feel like if the site had funky PHP, that would take the site down for everyone.
-
@scottalanmiller said in Website down, but only for organization Network:
@dafyre said in Website down, but only for organization Network:
The website goes down, or appears to for about 5 minutes. Hosting Provider assures me there is no PHP issues and everything looks to be in order.
Tricky thing is, if I take my phone off of our organizations WiFi, the site is still operational. In fact, the site never really goes down to the rest of the world.Putting these two things together, I'm going to say it's almost certainly Fail2Ban. I bet something on your network is sending a bad password automatically and causing an auto-ban for a few minutes,.
Could be a password, could just be some of the actions actions that your editor / editing is performing that are triggering rules, depending on what the setup is like on the other end. See if they can either whitelist your corp IP(s) or tune out the false-positive rule.
-
@Mr-Jones said in Website down, but only for organization Network:
Would a PHP error cause this? That's the only thing I can think of, as I was editing some conditional logic on the website yesterday morning, but I'm failing to see the correlation given the context of the issue. I feel like if the site had funky PHP, that would take the site down for everyone.
The right kind of PHP error could definitely cause this.
-
@scottalanmiller said in Website down, but only for organization Network:
I feel like if the site had funky PHP, that would take the site down for everyone.
If it is making a bad calls that are similar to logins, that would do it. Fail2Ban (or similar) look for errors from an IP address. So a single IP gets affected.
