ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    ScreenConnect High CPU Usage

    IT Discussion
    screenconnect digitalocean iaas centos 7 centos linux rhel rhel 7
    6
    46
    8.5k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      Dashrender
      last edited by

      OIC. What I don't know is - is my SC session running on that same VM, or it's own? It's not suppose to be part of the main NTG group of SC users.

      S 2 Replies Last reply Reply Quote 0
      • S
        scottalanmiller @Dashrender
        last edited by

        @Dashrender said in ScreenConnect High CPU Usage:

        OIC. What I don't know is - is my SC session running on that same VM, or it's own? It's not suppose to be part of the main NTG group of SC users.

        You have your own SC system, this is ours that I'm looking at. This isnt a thread about you 😉

        D 1 Reply Last reply Reply Quote 0
        • S
          scottalanmiller @Dashrender
          last edited by

          @Dashrender said in ScreenConnect High CPU Usage:

          It's not suppose to be part of the main NTG group of SC users.

          I don't understand. Are you seeing NTG users on your system?

          D 1 Reply Last reply Reply Quote 0
          • S
            scottalanmiller
            last edited by

            Okay, the CPU has dropped on its own. Maybe this was an update process that was running.

            11:40:01 AM     all     55.06      0.00      1.17      0.23      1.41     42.13
            11:50:01 AM     all      9.44      0.00      0.26      0.10      0.47     89.73
            

            Keeping an eye on it now to make sure that it does not spike back up on its own.

            1 Reply Last reply Reply Quote 0
            • S
              scottalanmiller
              last edited by

              There is a bit of writing going on from the SC process. Something major is going on. Maybe it is some sort of database update process?

              VIRT   RES  CPU%  MEM%   PID USER        NI S    TIME+ IOR/s IOW/s NAME
              1.2G  501M  12.6  50.5  9663 root         0 S  8:44.64     0  121K mono
              

              121K Write IOPS isn't trivial.

              1 Reply Last reply Reply Quote 1
              • D
                Dashrender @scottalanmiller
                last edited by

                @scottalanmiller said in ScreenConnect High CPU Usage:

                @Dashrender said in ScreenConnect High CPU Usage:

                OIC. What I don't know is - is my SC session running on that same VM, or it's own? It's not suppose to be part of the main NTG group of SC users.

                You have your own SC system, this is ours that I'm looking at. This isnt a thread about you 😉

                Good to have that confirmed, but it does seem like a semi related problem - they are both having performance issues.

                S 1 Reply Last reply Reply Quote 0
                • D
                  Dashrender @scottalanmiller
                  last edited by

                  @scottalanmiller said in ScreenConnect High CPU Usage:

                  @Dashrender said in ScreenConnect High CPU Usage:

                  It's not suppose to be part of the main NTG group of SC users.

                  I don't understand. Are you seeing NTG users on your system?

                  Well yes frankly, I know Gene has an account, and I'm fine with that. 🙂 But that wasn't my point. I was under an impression, but hadn't had it confirmed that my SC was in fact on it's won VM - are you confirming that now?

                  S 1 Reply Last reply Reply Quote 0
                  • S
                    scottalanmiller @Dashrender
                    last edited by

                    @Dashrender said in ScreenConnect High CPU Usage:

                    @scottalanmiller said in ScreenConnect High CPU Usage:

                    @Dashrender said in ScreenConnect High CPU Usage:

                    OIC. What I don't know is - is my SC session running on that same VM, or it's own? It's not suppose to be part of the main NTG group of SC users.

                    You have your own SC system, this is ours that I'm looking at. This isnt a thread about you 😉

                    Good to have that confirmed, but it does seem like a semi related problem - they are both having performance issues.

                    I don't believe that yours was updated, though. This appears to be an issue with the update.

                    Everything on the Internet is having performance problems right now, so that the two share that during a national Internet outage isn't too telling.

                    D 1 Reply Last reply Reply Quote 0
                    • S
                      scottalanmiller @Dashrender
                      last edited by

                      @Dashrender said in ScreenConnect High CPU Usage:

                      Well yes frankly, I know Gene has an account, and I'm fine with that. 🙂 But that wasn't my point. I was under an impression, but hadn't had it confirmed that my SC was in fact on it's won VM - are you confirming that now?

                      Yes, you have a totally unique, independent SC system.

                      D 1 Reply Last reply Reply Quote 0
                      • D
                        Dashrender @scottalanmiller
                        last edited by

                        @scottalanmiller said in ScreenConnect High CPU Usage:

                        @Dashrender said in ScreenConnect High CPU Usage:

                        Well yes frankly, I know Gene has an account, and I'm fine with that. 🙂 But that wasn't my point. I was under an impression, but hadn't had it confirmed that my SC was in fact on it's won VM - are you confirming that now?

                        Yes, you have a totally unique, independent SC system.

                        Thanks.

                        1 Reply Last reply Reply Quote 0
                        • S
                          scottalanmiller
                          last edited by

                          I'm an idiot, should have grabbed the block device report straight away. Here is the disk activity:

                          09:46:39 AM       LINUX RESTART
                          
                          09:50:02 AM       tps      rtps      wtps   bread/s   bwrtn/s
                          10:00:01 AM     10.39      0.05     10.34      0.73    297.16
                          10:10:07 AM     95.03     85.88      9.15  16396.11    276.51
                          10:20:01 AM    155.68    146.07      9.61  28524.76    260.43
                          10:30:02 AM      2.29      0.07      2.22      1.03     50.41
                          10:40:01 AM      5.55      0.00      5.55      0.00    162.94
                          10:50:01 AM      2.00      0.04      1.96      7.22     42.68
                          11:00:01 AM      5.77      0.00      5.77      0.00    158.67
                          11:10:01 AM      3.17      1.45      1.72     19.73     41.24
                          11:20:01 AM      6.14      0.03      6.11      0.38    167.57
                          11:30:02 AM     25.66     24.06      1.61   1020.60     48.93
                          11:40:01 AM     16.00      5.80     10.20    350.94    265.18
                          11:50:01 AM      1.43      0.07      1.36     20.54     31.69
                          12:00:01 PM      6.53      1.02      5.51     18.00    150.42
                          Average:        24.83     19.44      5.39   3387.89    147.87
                          

                          That's some crazy load even for a RAID 10 SSD array. No wonder it is slowing down. Something major is going to disk.

                          D 1 Reply Last reply Reply Quote 0
                          • D
                            Dashrender @scottalanmiller
                            last edited by

                            @scottalanmiller said in ScreenConnect High CPU Usage:

                            @Dashrender said in ScreenConnect High CPU Usage:

                            @scottalanmiller said in ScreenConnect High CPU Usage:

                            @Dashrender said in ScreenConnect High CPU Usage:

                            OIC. What I don't know is - is my SC session running on that same VM, or it's own? It's not suppose to be part of the main NTG group of SC users.

                            You have your own SC system, this is ours that I'm looking at. This isnt a thread about you 😉

                            Good to have that confirmed, but it does seem like a semi related problem - they are both having performance issues.

                            I don't believe that yours was updated, though. This appears to be an issue with the update.

                            Everything on the Internet is having performance problems right now, so that the two share that during a national Internet outage isn't too telling.

                            Is the slowness coming from latency introduced by the attacks? Is that why you say they could be related?

                            S 1 Reply Last reply Reply Quote 0
                            • S
                              scottalanmiller @Dashrender
                              last edited by

                              @Dashrender said in ScreenConnect High CPU Usage:

                              Is the slowness coming from latency introduced by the attacks? Is that why you say they could be related?

                              That's our guess. No other changes, nothing visible on the system.

                              The system here had updates run, twice, and an obvious and immediate system impact after the updates. And looking at the reports, the updates were immediately followed by massive disk activity. So the guess is that the system is running a database compression process or something and that that is using some massive amount of disk IO.

                              1 Reply Last reply Reply Quote 0
                              • S
                                scottalanmiller
                                last edited by

                                And loads of disk IO leads to IOWaits.

                                1 Reply Last reply Reply Quote 1
                                • S
                                  scottalanmiller
                                  last edited by

                                  Since @Dashrender brought it up, for comparison this is his ScreenConnect instance during the same window:

                                  07:20:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
                                  07:30:01 AM     all      0.68      0.00      0.16      0.03      0.08     99.06
                                  07:40:01 AM     all      0.36      0.00      0.14      0.03      0.06     99.41
                                  07:50:01 AM     all      0.60      0.00      0.20      0.04      0.09     99.08
                                  08:00:01 AM     all      0.44      0.00      0.18      0.03      0.07     99.29
                                  08:10:01 AM     all      0.80      0.00      0.23      0.03      0.09     98.85
                                  08:20:01 AM     all      0.47      0.00      0.15      0.04      0.07     99.27
                                  08:30:01 AM     all      0.74      0.00      0.22      0.04      0.10     98.90
                                  08:40:01 AM     all      1.21      0.00      0.21      0.04      0.11     98.44
                                  08:50:01 AM     all      2.02      0.00      0.30      0.05      0.23     97.40
                                  09:00:01 AM     all      0.70      0.00      0.17      0.04      0.10     98.99
                                  09:10:01 AM     all      1.31      0.00      0.34      0.06      0.12     98.18
                                  09:20:01 AM     all      1.30      0.00      0.21      0.04      0.11     98.33
                                  09:30:02 AM     all      2.37      0.00      0.38      0.06      0.27     96.92
                                  09:40:01 AM     all      1.16      0.00      0.24      0.03      0.15     98.41
                                  09:50:01 AM     all      1.11      0.00      0.22      0.04      0.12     98.51
                                  10:00:01 AM     all      0.67      0.00      0.19      0.03      0.08     99.04
                                  10:10:02 AM     all      1.29      0.00      0.30      0.05      0.08     98.29
                                  10:20:01 AM     all      0.66      0.00      0.17      0.03      0.06     99.08
                                  10:30:01 AM     all      1.57      0.00      0.57      0.05      0.13     97.68
                                  10:40:01 AM     all      1.12      0.00      0.57      0.05      0.13     98.13
                                  10:50:01 AM     all      1.48      0.00      0.58      0.07      0.16     97.72
                                  11:00:01 AM     all      1.00      0.00      0.34      0.04      0.11     98.51
                                  11:10:01 AM     all      1.25      0.00      0.30      0.05      0.10     98.31
                                  11:20:01 AM     all      0.88      0.00      0.20      0.04      0.08     98.80
                                  11:30:01 AM     all      1.10      0.00      0.19      0.04      0.10     98.57
                                  11:40:01 AM     all      0.70      0.00      0.17      0.04      0.10     99.00
                                  11:50:01 AM     all      1.04      0.00      0.24      0.06      0.11     98.55
                                  12:00:01 PM     all      0.70      0.00      0.20      0.04      0.09     98.98
                                  Average:        all      0.68      0.01      0.21      0.03      0.07     99.00
                                  

                                  And here are the disks:

                                  07:20:01 AM       tps      rtps      wtps   bread/s   bwrtn/s
                                  07:30:01 AM      1.01      0.00      1.01      0.00     22.10
                                  07:40:01 AM      0.75      0.00      0.75      0.00     13.15
                                  07:50:01 AM      1.06      0.00      1.06      0.00     22.80
                                  08:00:01 AM      0.84      0.00      0.84      0.00     15.65
                                  08:10:01 AM      1.32      0.00      1.32      0.00     25.53
                                  08:20:01 AM      0.86      0.00      0.86      0.00     16.36
                                  08:30:01 AM      1.11      0.00      1.11      0.00     25.73
                                  08:40:01 AM      1.09      0.01      1.08      0.08     22.73
                                  08:50:01 AM      1.18      0.00      1.18      0.00     28.18
                                  09:00:01 AM      1.14      0.00      1.14      0.03     20.68
                                  09:10:01 AM      1.66      0.31      1.35     13.68     31.35
                                  09:20:01 AM      0.99      0.01      0.98      0.05     20.73
                                  09:30:02 AM      1.22      0.01      1.21      0.55     28.42
                                  09:40:01 AM      0.93      0.00      0.93      0.03     19.95
                                  09:50:01 AM      1.43      0.00      1.43      0.00     31.65
                                  10:00:01 AM      0.96      0.00      0.96      0.00     20.60
                                  10:10:02 AM      1.42      0.16      1.26      6.77     30.39
                                  10:20:01 AM      1.25      0.09      1.16      4.04     22.91
                                  10:30:01 AM      1.80      0.00      1.79      0.09     47.44
                                  10:40:01 AM      1.64      0.00      1.64      0.00     39.45
                                  10:50:01 AM      1.77      0.00      1.77      0.00     46.79
                                  11:00:01 AM      1.38      0.00      1.38      0.00     30.81
                                  11:10:01 AM      1.32      0.06      1.26     10.12     29.75
                                  11:20:01 AM      1.05      0.00      1.05      0.00     22.53
                                  11:30:01 AM      1.17      0.00      1.17      0.00     28.66
                                  11:40:01 AM      1.13      0.00      1.13      0.00     22.02
                                  11:50:01 AM      1.19      0.00      1.19      0.00     28.63
                                  12:00:01 PM      1.00      0.00      1.00      0.00     20.82
                                  Average:         1.05      0.06      0.99      5.29     20.66
                                  

                                  As you can see, totally different performance. But same OS, host, VM configuration, etc.

                                  1 Reply Last reply Reply Quote 0
                                  • S
                                    scottalanmiller
                                    last edited by

                                    Both systems are on a similar system update and reboot schedule, only hours off from each other. Both are at identical patches right now:

                                    # cat /etc/redhat-release 
                                    CentOS Linux release 7.2.1511 (Core)
                                    
                                    1 Reply Last reply Reply Quote 0
                                    • D
                                      Dashrender @scottalanmiller
                                      last edited by

                                      @scottalanmiller said in ScreenConnect High CPU Usage:

                                      I'm an idiot, should have grabbed the block device report straight away. Here is the disk activity:

                                      09:46:39 AM       LINUX RESTART
                                      
                                      09:50:02 AM       tps      rtps      wtps   bread/s   bwrtn/s
                                      10:00:01 AM     10.39      0.05     10.34      0.73    297.16
                                      10:10:07 AM     95.03     85.88      9.15  16396.11    276.51
                                      10:20:01 AM    155.68    146.07      9.61  28524.76    260.43
                                      10:30:02 AM      2.29      0.07      2.22      1.03     50.41
                                      10:40:01 AM      5.55      0.00      5.55      0.00    162.94
                                      10:50:01 AM      2.00      0.04      1.96      7.22     42.68
                                      11:00:01 AM      5.77      0.00      5.77      0.00    158.67
                                      11:10:01 AM      3.17      1.45      1.72     19.73     41.24
                                      11:20:01 AM      6.14      0.03      6.11      0.38    167.57
                                      11:30:02 AM     25.66     24.06      1.61   1020.60     48.93
                                      11:40:01 AM     16.00      5.80     10.20    350.94    265.18
                                      11:50:01 AM      1.43      0.07      1.36     20.54     31.69
                                      12:00:01 PM      6.53      1.02      5.51     18.00    150.42
                                      Average:        24.83     19.44      5.39   3387.89    147.87
                                      

                                      That's some crazy load even for a RAID 10 SSD array. No wonder it is slowing down. Something major is going to disk.

                                      what command is this? or is this from the logs?

                                      S 1 Reply Last reply Reply Quote 0
                                      • S
                                        scottalanmiller @Dashrender
                                        last edited by

                                        @Dashrender said in ScreenConnect High CPU Usage:

                                        @scottalanmiller said in ScreenConnect High CPU Usage:

                                        I'm an idiot, should have grabbed the block device report straight away. Here is the disk activity:

                                        09:46:39 AM       LINUX RESTART
                                        
                                        09:50:02 AM       tps      rtps      wtps   bread/s   bwrtn/s
                                        10:00:01 AM     10.39      0.05     10.34      0.73    297.16
                                        10:10:07 AM     95.03     85.88      9.15  16396.11    276.51
                                        10:20:01 AM    155.68    146.07      9.61  28524.76    260.43
                                        10:30:02 AM      2.29      0.07      2.22      1.03     50.41
                                        10:40:01 AM      5.55      0.00      5.55      0.00    162.94
                                        10:50:01 AM      2.00      0.04      1.96      7.22     42.68
                                        11:00:01 AM      5.77      0.00      5.77      0.00    158.67
                                        11:10:01 AM      3.17      1.45      1.72     19.73     41.24
                                        11:20:01 AM      6.14      0.03      6.11      0.38    167.57
                                        11:30:02 AM     25.66     24.06      1.61   1020.60     48.93
                                        11:40:01 AM     16.00      5.80     10.20    350.94    265.18
                                        11:50:01 AM      1.43      0.07      1.36     20.54     31.69
                                        12:00:01 PM      6.53      1.02      5.51     18.00    150.42
                                        Average:        24.83     19.44      5.39   3387.89    147.87
                                        

                                        That's some crazy load even for a RAID 10 SSD array. No wonder it is slowing down. Something major is going to disk.

                                        what command is this? or is this from the logs?

                                        sar -b

                                        1 Reply Last reply Reply Quote 0
                                        • gjacobseG
                                          gjacobse
                                          last edited by

                                          Unless specifically requested, I have no reason to sign into your (or any other) SC system.

                                          I have been aware of an update, but have held off on it. I updated NTG SC to attempt to correct an issue unrelated to @Dashrender previously stated issue.

                                          S D 2 Replies Last reply Reply Quote 0
                                          • S
                                            scottalanmiller @gjacobse
                                            last edited by

                                            @gjacobse said in ScreenConnect High CPU Usage:

                                            Unless specifically requested, I have no reason to sign into your (or any other) SC system.

                                            I have been aware of an update, but have held off on it. I updated NTG SC to attempt to correct an issue unrelated to @Dashrender previously stated issue.

                                            Looks like the latest update has some major baggage with it. Needs a good six hour window in which to deploy due to the IOPS that it needs. So far, the system is looking pretty good now that the process has cooled down.

                                            1 Reply Last reply Reply Quote 0
                                            • 1
                                            • 2
                                            • 3
                                            • 1 / 3
                                            • First post
                                              Last post