All Yealink Phones Down at One Site
-
@fuznutz04 said in All Yealink Phones Down at One Site:
Also, are all the Yealinks connected to a common switch? Has that been power cycled?
Yes, but so is everything else. One switch for the whole facility.
-
I see similar behavior sometimes here. Come in and all the phones will not be working, but everythign else is. Usually this is because the firewall/IPS/IDS has been triggered due to lots of SIP attemtps at once, like if many of the phones want to try an reregister at once. I just go and look for blocked sites in the firewall/utm/whatever, and usually it is our voip/sip provider's ip listed there. Remove it and the phones start working almost instantly.
-
@momurda said in All Yealink Phones Down at One Site:
I see similar behavior sometimes here. Come in and all the phones will not be working, but everythign else is. Usually this is because the firewall/IPS/IDS has been triggered due to lots of SIP attemtps at once, like if many of the phones want to try an reregister at once. I just go and look for blocked sites in the firewall/utm/whatever, and usually it is our voip/sip provider's ip listed there. Remove it and the phones start working almost instantly.
We control all those points and have it whitelisted.
-
@scottalanmiller said in All Yealink Phones Down at One Site:
@momurda said in All Yealink Phones Down at One Site:
I see similar behavior sometimes here. Come in and all the phones will not be working, but everythign else is. Usually this is because the firewall/IPS/IDS has been triggered due to lots of SIP attemtps at once, like if many of the phones want to try an reregister at once. I just go and look for blocked sites in the firewall/utm/whatever, and usually it is our voip/sip provider's ip listed there. Remove it and the phones start working almost instantly.
We control all those points and have it whitelisted.
Have you actually temporarily disabled the Responsive FW?
-
Factory reset a phone and try manually programming? How about a packet capture at the site firewall, do you see attempts from that IP phone to where? I am assuming that the PBX is hosted elsewhere.
-
@momurda said in All Yealink Phones Down at One Site:
sometimes here. Come in and all the phones will not be working, but everythign else is. Usually this is because the firewall/IPS/IDS has been triggered due to lots of SIP attemtps at once, like if many of the phones want to try an reregister at once. I just go and look for blocked sites in the firewall/utm/whatever, and usually it is our voip/sip provider's ip listed there. Remove i
@scottalanmiller Have you rebooted the PBX yet?
-
Did fail2ban block that entire subnet? That wouldn't explain why a single polycom device was working correctly though.
-
I had this issue with a small office network but Polycom phones. I upgraded all firmware on the switches and router, rebooted everything to no prevail. It only started working when I factory re-set the main router and re-deployed from scratch! so strange!
-
@coliver said in All Yealink Phones Down at One Site:
Did fail2ban block that entire subnet? That wouldn't explain why a single polycom device was working correctly though.
No, that's the IDS and we turned it off to test.
-
@fuznutz04 said in All Yealink Phones Down at One Site:
@momurda said in All Yealink Phones Down at One Site:
sometimes here. Come in and all the phones will not be working, but everythign else is. Usually this is because the firewall/IPS/IDS has been triggered due to lots of SIP attemtps at once, like if many of the phones want to try an reregister at once. I just go and look for blocked sites in the firewall/utm/whatever, and usually it is our voip/sip provider's ip listed there. Remove i
@scottalanmiller Have you rebooted the PBX yet?
Yes, we did early on. No change.
-
@i3 said in All Yealink Phones Down at One Site:
Factory reset a phone and try manually programming? How about a packet capture at the site firewall, do you see attempts from that IP phone to where? I am assuming that the PBX is hosted elsewhere.
This is basically where we went. We did this series of events and it appears to be fixed...
- Rebooted the firewall.
- Removed SIP-ALG, but this appears to have had no effect.
- Updated the firmware on each phone.
- Went to static DNS rather than DHCP set.
- DNS to CloudFlare instead of AD.
- Ensures STUN was on and set correctly.
- Added the PBX as a Proxy, manually enabled STUN for the proxy.
After those seven steps, all phones are now registered. Couldn't find any step there that individually made a difference.
-
@scottalanmiller said in All Yealink Phones Down at One Site:
@i3 said in All Yealink Phones Down at One Site:
Factory reset a phone and try manually programming? How about a packet capture at the site firewall, do you see attempts from that IP phone to where? I am assuming that the PBX is hosted elsewhere.
This is basically where we went. We did this series of events and it appears to be fixed...
- Rebooted the firewall.
- Removed SIP-ALG, but this appears to have had no effect.
- Updated the firmware on each phone.
- Went to static DNS rather than DHCP set.
- DNS to CloudFlare instead of AD.
- Ensures STUN was on and set correctly.
- Added the PBX as a Proxy, manually enabled STUN for the proxy.
After those seven steps, all phones are now registered. Couldn't find any step there that individually made a difference.
Ah yes, I love it when the solution is unknown. Been there plenty of times.
-
At least the fix was repeatable.
-
@scottalanmiller said in All Yealink Phones Down at One Site:
At least the fix was repeatable.
I think that is silly.
Was a packet capture not possible?
-
@scottalanmiller said in All Yealink Phones Down at One Site:
@i3 said in All Yealink Phones Down at One Site:
Factory reset a phone and try manually programming? How about a packet capture at the site firewall, do you see attempts from that IP phone to where? I am assuming that the PBX is hosted elsewhere.
This is basically where we went. We did this series of events and it appears to be fixed...
- Rebooted the firewall.
- Removed SIP-ALG, but this appears to have had no effect.
- Updated the firmware on each phone.
- Went to static DNS rather than DHCP set.
- DNS to CloudFlare instead of AD.
- Ensures STUN was on and set correctly.
- Added the PBX as a Proxy, manually enabled STUN for the proxy.
After those seven steps, all phones are now registered. Couldn't find any step there that individually made a difference.
To me looks like the DNS was caching something for the devices to get to. That's my theory.
@scottalanmiller said in All Yealink Phones Down at One Site:
- Updated the firmware on each phone.
- Went to static DNS rather than DHCP set.
- DNS to CloudFlare instead of AD.
After those seven steps, all phones are now registered. Couldn't find any step there that individually made a difference.
-
@dbeato said in All Yealink Phones Down at One Site:
@scottalanmiller said in All Yealink Phones Down at One Site:
@i3 said in All Yealink Phones Down at One Site:
Factory reset a phone and try manually programming? How about a packet capture at the site firewall, do you see attempts from that IP phone to where? I am assuming that the PBX is hosted elsewhere.
This is basically where we went. We did this series of events and it appears to be fixed...
- Rebooted the firewall.
- Removed SIP-ALG, but this appears to have had no effect.
- Updated the firmware on each phone.
- Went to static DNS rather than DHCP set.
- DNS to CloudFlare instead of AD.
- Ensures STUN was on and set correctly.
- Added the PBX as a Proxy, manually enabled STUN for the proxy.
After those seven steps, all phones are now registered. Couldn't find any step there that individually made a difference.
To me looks like the DNS was caching something for the devices to get to. That's my theory.
If so, why did changing the DNS alone not fix it?
-
@scottalanmiller said in All Yealink Phones Down at One Site:
@dbeato said in All Yealink Phones Down at One Site:
@scottalanmiller said in All Yealink Phones Down at One Site:
@i3 said in All Yealink Phones Down at One Site:
Factory reset a phone and try manually programming? How about a packet capture at the site firewall, do you see attempts from that IP phone to where? I am assuming that the PBX is hosted elsewhere.
This is basically where we went. We did this series of events and it appears to be fixed...
- Rebooted the firewall.
- Removed SIP-ALG, but this appears to have had no effect.
- Updated the firmware on each phone.
- Went to static DNS rather than DHCP set.
- DNS to CloudFlare instead of AD.
- Ensures STUN was on and set correctly.
- Added the PBX as a Proxy, manually enabled STUN for the proxy.
After those seven steps, all phones are now registered. Couldn't find any step there that individually made a difference.
To me looks like the DNS was caching something for the devices to get to. That's my theory.
If so, why did changing the DNS alone not fix it?
Not sure, I am just going by what you said. Basically very strange.
