CloudatCost Issues
-
I wonder if there's a DDoS in progress. Maybe I pissed of the folks in HK when I blocked all of their SSH attempts in iptables?
-
@thanksajdotcom There was 166 updates, but it did this same thing yesterday, and the updates took less then a minute, both Dev1 boxes. Something is up,
-
@Aaron-Studer said:
Yum -y update took 15 minutes
That's slow but not crazy. Which OS are you using? It is a big operation with a lot of stuff to download, then to setup in the database, then to roll out one package at a time. Updates really can take a bit of time. Fifteen minutes seems long, but if there is any IO issues, then it would slow down a lot.
-
@Aaron-Studer said:
@thanksajdotcom There was 166 updates, but it did this same thing yesterday, and the updates took less then a minute, both Dev1 boxes. Something is up,
Could be the IO issues that they list on their site. But it could be more network problems with Rogers, too.
-
@Danp said:
I'm also having trouble with my C@C Dev1 box. I'm unable to connect via SSH, Yum install via web console throws errors, etc.
Did SSH work before? Or is this a new build?
-
Oh you know what, is everyone having login problems using Ubuntu instead of CentOS? I've seen that issue too.
-
I got some huge delays just typing to the console. Like 30 seconds for a line that I was typing to continue.
-
I'm checking the Ubuntu instances and am seeing issues around passwords failures in the auth.log.
-
Looks like you just need to run pwconv after creating a user on Ubuntu 14.04.02. That's all. Works fine once you do that.
-
As I suspected, it's IOWait problems. Always check SAR first for performance problems.
Linux 3.13.0-32-generic (ubuntu) 03/09/2015 _x86_64_ (1 CPU) 02:05:11 AM CPU %user %nice %system %iowait %steal %idle 02:15:01 AM all 0.10 0.00 0.31 7.35 0.00 92.24 02:25:01 AM all 0.02 0.00 0.21 5.97 0.00 93.81 02:35:01 AM all 0.00 0.00 0.17 2.01 0.00 97.82 02:45:01 AM all 0.19 0.00 0.28 20.81 0.00 78.73 02:55:39 AM all 1.63 0.00 0.46 28.88 0.00 69.03 03:05:01 AM all 0.22 0.00 0.14 9.61 0.00 90.02 03:15:01 AM all 0.00 0.00 0.12 1.73 0.00 98.15 03:25:01 AM all 0.00 0.00 0.14 7.41 0.00 92.45 Average: all 0.28 0.00 0.23 10.63 0.00 88.86 -
I'm seeing rather large IO delays here too.
-
So far I've only seen the IO issues on the Dev1 instances.
-
-
@Aaron-Studer said:
@scottalanmiller said:
So far I've only seen the IO issues on the Dev1 instances.
I wonder why?
I'm sure the over-commit rate on those is much higher. And the IO needs are higher as there are so many OSes doing things, even when idle, in the same IO space.
-
What is a "good" average? On my Dev1 box, I'm seeing 8.41%. On my Dev3 box, I'm seeing 4.45%.
-
@Danp said:
What is a "good" average? On my Dev1 box, I'm seeing 8.41%. On my Dev3 box, I'm seeing 4.45%.
Average "should" approach zero. Definitely way under 1%.
-
For example, the MangoLassi database, which you can imagine is relatively busy, produces only .03% IOWait state average. It almost never spikes above .08% in any ten minute period.
That's point ZERO three. So 8.41 is 280 times higher!!
-
@scottalanmiller said:
For example, the MangoLassi database, which you can imagine is relatively busy, produces only .03% IOWait state average. It almost never spikes above .08% in any ten minute period.
That's point ZERO three. So 8.41 is 280 times higher!!
Does this than say that the servers C@C are providing are over provisioned? I understand that they are growing capacity and are working to resolve this.
-
@scottalanmiller said:
@Danp said:
What is a "good" average? On my Dev1 box, I'm seeing 8.41%. On my Dev3 box, I'm seeing 4.45%.
Average "should" approach zero. Definitely way under 1%.
Assuming this is a VM, and it's working correctly, even if he Host machine is super busy, would you expect to see these numbers be outside their normal range if the host is saturated?
-
@Dashrender said:
Does this than say that the servers C@C are providing are over provisioned? I understand that they are growing capacity and are working to resolve this.
It states this right on their status page. They know that they overprovisioned IO and are adding capacity.
