What to do when you don't agree with the opinion of an IT consultant
-
The IT consultant is going to take all the servers and put them in a new box via virtualization and because they will be adding a SAN this will make the LOB program run faster than it does now.
So let me tell you about a project I have been working on since 2009. Pomis is a medical practice management program which my client (billing office) uses. This program is uses flat files instead of a database. Also the program stores PDF s of claims, payments and other important information. A Pomis directory can be as small as 500MB without any PDF files but can grow to 91GB.
Each medical practice that the billing company takes care of uses their own Pomis Directory. Basically I create a folder named for the doctors practice. This folder will have all the Pomis files in it to run the program. Each practice may have up to 4 users accessing data from the billing company and the doctors practice could have 1-8 users accessing data.
AD/File Server (odds and ends)
Dell PE2850: Xeon 3GHz (2 procs) 8GB RAM Server 08r2 RAID 1 2x 146GB USCSI RAID 5 3x 146SCSI (only minor usage as a file server on the RAID 5 (more like a storage space for users to share internal office information)LOB Servers (amount of directories are related to the size of storage) run OS and Remote Desktop Services and LOB software.
Dell RE1950: Xeon 5160 @ 3GHz (2 procs) 4GB RAM Server 08r2 Big RAID 10 4 146GB SAS 10k drives
Dell PE2950: Xeon 5160 @ 3GHz (2 procs) 8GB RAM Server 08r2 Big RAID 10 4 750GB SATA 7.2k drives
Gateway E-9525R: Xeon 5130 @ 2GHz (1 proc) 6GB RAM Server 2012 Big RAID 10 6 x 750 SATA 7.2k drivesActual data size for all servers is 656GB.
The owners procured the older units for a song which is why I get to use them. I add servers as the disk space and the hard drive usage goes up. Each time I have moved practices off of one server to another, the server with less data improved performance. I pretty much have the data equal distributed among the servers.
When complaints of slowness or temporary white screens (pre-cursor to a not responding incident) I would watch the Resource monitor and noticed that the CPU and Memory was almost always 50-60% usage while the disk was a solid block of green. Although doctors and billing staff (day and night shift) access the servers 24/7, there are low load times when i see the CPU is at 4% and Memory is at 30% usage and I can actually see the disk graph. This led me to believe that I was having I/O issues and that by adding more drives via servers I was addressing the I/O issues.
I recently spoke to the IT consultant who will be suggesting the hardware which will be in data center. He asked a few questions about the servers and confirmed no Exchange and that there was not a SQL database and was like ok, thanks for the information. I asked if he wanted to know about the program and tried to give him some information. After I explained the pain points I was having. He said, first you need a new faster server with more processors and RAM and we will visualize all the servers and use a SAN and that will work.
Enough of what I think, although I believe working on this since 2009 I would have some insight, perhaps I am mistaken in my diagnosis of the problems. What say you peeps (Easter reference again - BAM!)?
-
Has the consultant run performance gathering metrics? NTG (or others) could do an analysis of your network (I think the software has to run for at least a week, two weeks to a month is better) that will tell you how much of what you are using and where you are currently bottle necking. I don't know the cost of this service.
What is your role to this customer? are you also a consultant? employee?
-
Profile and work out where the bottleneck is. SAN doesn't fix a bottlneck if the bottleneck is CPU or RAM, and if the bottleneck is disk it could be as simple as adding an SSD or two.
-
Was that a consultant, or a Dell rep? Tossing a SAN at something's typically something I see from resellers. From what you've explained, it sounds like you're short on IOPS. With the modern technologies available for localized and distributed tiered storage, SAN wouldn't be the way to go. What's your RTO for these systems?
-
Definitely sounds like a salesman. Not a consultant. Is he the one selling the SAN?
-
@Dashrender - my business has provided all their IT services since 2009. No one is doing anything but pretending to know what to do. Personally I don't think they would pay for performance gathering metrics but I would be interested in the cost.
@Nara - "consultants" I believe the one at the datacenter is a consulting or IT person
@scottalanmiller - SAN was a possible add from one consultant, in 2010 CDW sent me someone at IBM/Lenovo who wanted to sell me a SAN (this was before the larger data sets)
@hutchingsp - I don't know how to do that.I know I can just sit back and wait for the bomb to go off when they do this without me, but I would really rather have the saving play put together and have delivered it to the client before they spend money and it makes it worse. If they decline, at least they will know I tried to get them the best information.
-
CDW is a reseller and IBM is a vendor. Neither provides consultants. They only provide sales people. Best to never interact with CDW in that way. It only has negative or neutral outcomes, not good ones.
-
@scottalanmiller said:
CDW is a reseller and IBM is a vendor. Neither provides consultants. They only provide sales people. Best to never interact with CDW in that way. It only has negative or neutral outcomes, not good ones.
What's worse about CDW is that they sell themselves as consultants. I've listened to more speals from them than I can remember. Then I always come back and ask why they went a certain way - the answer always has to do with some hardware/software they are pimping even though they say they aren't.
It's really to bad - I'd love to use them as a resource for knowledge sake, but they just seem to vendor specific.
-
Every salesman sells themselves as a consultant. They tell people want they want to hear. There is no deception, no one thinks that CDW doesn't make money selling you stuff.
-
@technobabble said:
@Dashrender - my business has provided all their IT services since 2009. No one is doing anything but pretending to know what to do. Personally I don't think they would pay for performance gathering metrics but I would be interested in the cost.
If you have provided them all of the IT services since 2009, why are they even talking to someone in a Datacenter? Do they no longer trust you? Did a vendor send them a wine basket and now they feel intrigued by them?
Seems like a bad situation, best of luck with it. Situations like this definitely test your metal. If your company wants to keep them, you'll have to go the extra mile to show them why the other solutions being suggested are actually worse for them than your current ones, as well as what other solutions would be better. -
@Nara said:
Was that a consultant, or a Dell rep? Tossing a SAN at something's typically something I see from resellers. From what you've explained, it sounds like you're short on IOPS. With the modern technologies available for localized and distributed tiered storage, SAN wouldn't be the way to go. What's your RTO for these systems?
I smell an opportunity for 3 ESXi hosts and VMWare vSAN, or 2 hosts with flash storage and Veeam cross-host replication.
-
@Dashrender said:
@technobabble said:
@Dashrender - my business has provided all their IT services since 2009. No one is doing anything but pretending to know what to do. Personally I don't think they would pay for performance gathering metrics but I would be interested in the cost.
If you have provided them all of the IT services since 2009, why are they even talking to someone in a Datacenter? Do they no longer trust you? Did a vendor send them a wine basket and now they feel intrigued by them?
Seems like a bad situation, best of luck with it. Situations like this definitely test your metal. If your company wants to keep them, you'll have to go the extra mile to show them why the other solutions being suggested are actually worse for them than your current ones, as well as what other solutions would be better.The silent partner is becoming involved. This has been an on-going discussion for the last 2 years. First it was just move the servers from the office in FL to the data center in Maine. Then it was host in Tampa or near by. Questions of why is is so expensive to buy new equipment. Can't one server rule them all for $2k? So between the pressure of the NOT so silent partner they have reached out the IT guy from the bank. Trust is not the issue, its the price. Always has been for everything.
-
@alexntg said:
@Nara said:
Was that a consultant, or a Dell rep? Tossing a SAN at something's typically something I see from resellers. From what you've explained, it sounds like you're short on IOPS. With the modern technologies available for localized and distributed tiered storage, SAN wouldn't be the way to go. What's your RTO for these systems?
I smell an opportunity for 3 ESXi hosts and VMWare vSAN, or 2 hosts with flash storage and Veeam cross-host replication.
What? By the way what is the cost of NTG running performance gathering metrics or is this something I can do?
-
I had a thought earlier today about the lag and stress on the hard drives.
They are constantly scanning PDF's (usually 3 people from 8-5) into the directories while users are trying to access the drives for their data. I wonder if we should be looking to the programmers to code the program to allow our "PDF" files to be on a server that doesn't have the files to run the program. That would free up some hard drive access.
-
@technobabble said:
@alexntg said:
@Nara said:
Was that a consultant, or a Dell rep? Tossing a SAN at something's typically something I see from resellers. From what you've explained, it sounds like you're short on IOPS. With the modern technologies available for localized and distributed tiered storage, SAN wouldn't be the way to go. What's your RTO for these systems?
I smell an opportunity for 3 ESXi hosts and VMWare vSAN, or 2 hosts with flash storage and Veeam cross-host replication.
What? By the way what is the cost of NTG running performance gathering metrics or is this something I can do?
If you can run a Dell DPACK for a 24-hour period over two random days, and get the analysis from Dell, I can take a look at your storage needs and formulate an effective strategy. That'll take about an hour for me to analyze. In the interim, I have a few questions for you:
What's the RPO and RTO for these systems?
What are you using for backup?
What's the projected workload growth percentage on these servers over the next 3 years? -
From the pricing end, that's up to @Minion-Queen
-
RTO and RPO are points I bring up constantly, of course I call it business continuity/disaster planning, which we have none.
I have AM and PM daily backups, which overwrite every few days, using fBackup program.
300%
-
@technobabble said:
RTO and RPO are points I bring up constantly, of course I call it business continuity/disaster planning, which we have none.
I have AM and PM daily backups, which overwrite every few days, using fBackup program.
300%
Until the RPO and RTO can be determined, it's impossible to determine the level of redundancy and backup needed, as well if high availability should come into play. The next step should be to find these things out.
-
I understand. If anything goes down, the company and doctors can't work. However it hasn't happened yet and therefore it is considered back burner stuff. rolling eyes. They provide the program over the internet and yet they still haven't allowed a secondary/backup ISP. We have a dual Wan router ready and waiting for the backup internet connection. Of course if they move to the data center, then it will have the ISP redundancy.
-
Can't work means that you are losing money. But how much is what you have to figure out. A lot, a little. Makes a big difference. And what if a SAN dies?
