VoIP One-way Audio and Voice drops
-
Just an update to what I worked on last night.
- PBX is now on its own dedicated host, no other VMs are running
- PBX is running on an SSD array with a dedicated ethernet port on the host
- PBX has a dedicated line to the firewall
- Desk phone is wired directly to the firewall
My testing this morning has shown that we still have intermittent audio problem on both SIP Trunks during any call. Either if the SIP trunk is registered directly to the phone or if it is registered to the PBX.
Just from the last 10 minutes of real-time logging it looks like every switch port is seeing under 1% utilization. I've got a few more to check out still though.
-
@coliver said:
@scottalanmiller said:
Might be worth looking into that. There are some free options for that. Ubiquiti and Meraki both have some built in options that are better than nothing. But you can use free tools to collect total traffic from them (at least from the Ubiquiti) that will provide you some historical numbers which should help a lot for correlating that. I would start by tracking when the phones are good and bad in a manual "log".
My guess is that Solarwinds has something free and easy to use for this scale.
Thanks, I grabbed a SolarWinds Real-Time monitor (under their free section) lets see if that will help.
That's the one that I was thinking of.
-
@coliver said:
@scottalanmiller said:
Might be worth looking into that. There are some free options for that. Ubiquiti and Meraki both have some built in options that are better than nothing. But you can use free tools to collect total traffic from them (at least from the Ubiquiti) that will provide you some historical numbers which should help a lot for correlating that. I would start by tracking when the phones are good and bad in a manual "log".
My guess is that Solarwinds has something free and easy to use for this scale.
The problem is that they are always bad. Seems to be every 5-10 seconds that they cut out.
Wait, they drop from time to time or it drops after five seconds and never comes back?
-
@coliver said:
Just from the last 10 minutes of real-time logging it looks like every switch port is seeing under 1% utilization. I've got a few more to check out still though.
Don't really care about switch ports. It's the WAN link that matters.
-
@scottalanmiller said:
@coliver said:
@scottalanmiller said:
Might be worth looking into that. There are some free options for that. Ubiquiti and Meraki both have some built in options that are better than nothing. But you can use free tools to collect total traffic from them (at least from the Ubiquiti) that will provide you some historical numbers which should help a lot for correlating that. I would start by tracking when the phones are good and bad in a manual "log".
My guess is that Solarwinds has something free and easy to use for this scale.
The problem is that they are always bad. Seems to be every 5-10 seconds that they cut out.
Wait, they drop from time to time or it drops after five seconds and never comes back?
Drops for a few seconds every 10-15 seconds then picks back up again.
-
@scottalanmiller said:
@coliver said:
Just from the last 10 minutes of real-time logging it looks like every switch port is seeing under 1% utilization. I've got a few more to check out still though.
Don't really care about switch ports. It's the WAN link that matters.
That one isn't as easy as the switch ports... Meraki doesn't really support SNMP, it says it does but I've never really found anything that can correctly read it.
-
@coliver said:
@scottalanmiller said:
@coliver said:
@scottalanmiller said:
Might be worth looking into that. There are some free options for that. Ubiquiti and Meraki both have some built in options that are better than nothing. But you can use free tools to collect total traffic from them (at least from the Ubiquiti) that will provide you some historical numbers which should help a lot for correlating that. I would start by tracking when the phones are good and bad in a manual "log".
My guess is that Solarwinds has something free and easy to use for this scale.
The problem is that they are always bad. Seems to be every 5-10 seconds that they cut out.
Wait, they drop from time to time or it drops after five seconds and never comes back?
Drops for a few seconds every 10-15 seconds then picks back up again.
OH! That is very different from what we've been thinking. Or at least what I've been thinking. That's a dropping issue, not one way audio. One way audio, or what is often called that, is that just one way gets audio. This is one way has audio cutting out. Not the same. Not sure how to term them, but I was thinking you were referring to a set up issue. This is definitely unrelated to STUN or NAT or anything like that, those don't "come back".
This is almost certainly a WAN saturation issue and or packet loss issue. You are losing RTP packets or they are so late that they are thrown away. Pretty much this is your WAN or your SIP trunk provider. Nothing that you can fix yourself.
-
@coliver said:
That one isn't as easy as the switch ports... Meraki doesn't really support SNMP, it says it does but I've never really found anything that can correctly read it.
Yeah, go to Ubiquiti for better testing. Meraki falls above the home use category, but below the enterprise category. It's pretty low end SMB in a lot of its capabilities and features and performance (and support.)
I have a tough time with Meraki. In some ways it's a solid SMB piece of gear. In other ways, their failure to keep pace with the industry because of products like Ubiquiti makes it, in many ways, fall solidly below the home line and actually become a level of gear that, even if it was dirt cheap, I wouldn't use anymore. Meraki started strong, but these days I'd list it pretty much as consumer gear. It's not "bad", it just isn't "good enough" to meet a minimum standard.
-
@scottalanmiller said:
@coliver said:
@scottalanmiller said:
@coliver said:
@scottalanmiller said:
Might be worth looking into that. There are some free options for that. Ubiquiti and Meraki both have some built in options that are better than nothing. But you can use free tools to collect total traffic from them (at least from the Ubiquiti) that will provide you some historical numbers which should help a lot for correlating that. I would start by tracking when the phones are good and bad in a manual "log".
My guess is that Solarwinds has something free and easy to use for this scale.
The problem is that they are always bad. Seems to be every 5-10 seconds that they cut out.
Wait, they drop from time to time or it drops after five seconds and never comes back?
Drops for a few seconds every 10-15 seconds then picks back up again.
OH! That is very different from what we've been thinking. Or at least what I've been thinking. That's a dropping issue, not one way audio. One way audio, or what is often called that, is that just one way gets audio. This is one way has audio cutting out. Not the same. Not sure how to term them, but I was thinking you were referring to a set up issue. This is definitely unrelated to STUN or NAT or anything like that, those don't "come back".
This is almost certainly a WAN saturation issue and or packet loss issue. You are losing RTP packets or they are so late that they are thrown away. Pretty much this is your WAN or your SIP trunk provider. Nothing that you can fix yourself.
I'm getting packet loss on the PBX which didn't exist on Friday when these issues started, it was just insane latency at that point, now I am getting 10-30% (depending on the ping) packet loss on the PBX. Oddly I don't get any of that on my desktop. Both are plugged directly into the firewall, the phone I used to test the 3rd party SIP trunk was also directly attached to the firewall.
PBX
Desktop
Both had been running for about the same amount of time and both had been started at the same time.
-
Wow, so now to track down the packet loss. If the desktops don't see it.... where is it coming from?
What is the packet loss when trying to hit your firewall? What about hitting the router on the other end of the WAN?
-
Is the PBX still running in Hyper-V?
-
@scottalanmiller said:
Wow, so now to track down the packet loss. If the desktops don't see it.... where is it coming from?
What is the packet loss when trying to hit your firewall? What about hitting the router on the other end of the WAN?
PBX
Desktop
-
@thecreativeone91 said:
Is the PBX still running in Hyper-V?
It is, I don't have another option right now.
-
So no issues hitting the local firewall. Now the other wise of the WAN but still "local"?
-
You don't have a Broadcom NIC, by chance?
-
-
I really think it's related to the Hyper-V Nic Driver & Linux. Make sure the Hyper-V host has all windows updates installed. You could also Disable VMQ and Disable Large Send Offload Version 2
-
@thecreativeone91 said:
I really think it's related to the Hyper-V Nic Driver & Linux. Make sure the Hyper-V host has all windows updates installed. You could also Disable VMQ and Disable Large Send Offload Version 2
I moved to a different host last night. I disabled VMQ, not Large Send Offload though, I will look into that.
-
@scottalanmiller said:
So no issues hitting the local firewall. Now the other wise of the WAN but still "local"?
This is the WAN access IP address.
PBX
Desktop
-
Tested that ping and did not see any errors.
