Engineering vs Administration - That's what makes Windows and FreeNAS so risky
-
So here is a thought, and I think that this will make a great video, but we were talking in another thread about why engineers are the juniors to administrators in a proper environment. In the real world, engineers have the vastly simpler job - working in green fields and just following online guides most of the time. They can test, look things up, have someone look over their shoulders, take extra time, double check, document, get sign offs, etc. and at the end of the day, their stuff rarely even has to work because the admins will fix it later.
It is this that actually creates one of the biggest problems that we see in IT day to day - shoddy engineering work that screws administration later. We all deal with it almost every day. But why does this happen? Simple, because all kinds of tools are created to make engineering trivially easy. From the Windows GUI to silly products like FreeNAS, they are there to make the job of setting something up (engineering) really easy to get to a point of "working". Sure, really good, solid, proper engineering would involve loads of research and careful testing. But in the real world, so much is done to allow engineers to simply skip all of the hard work and just install something using a simple GUI, follow some simple online guides and no one is the wiser. The company that hired you sees a product up and running quickly and all of the potential problems that this brings is left to someone else (the admin).
Problem come in all sorts of forms. Maybe it's spinning disks on RAID 5 that won't likely cause hard issues for many years, long after no one remembers who configured them. Maybe it's a bloated Windows install that has loads of tools installed for a one time task. Maybe it's an OS where all the defaults are just taken and nothing more is done. Maybe it's storage installed with a NAS OS with a cute GUI that makes creating shares a breeze and does nothing to address dealing with hardware failure or disk corruption.
The IT product work is almost entirely focused on making the engineers (the ones who typically influence a purchase) have little to no work to do and almost nothing is done to make it easier to be an administrator. Admins have little option to fix a misconfigured RAID array, yet have to deal with its consequences. Admins have to deal with the attack surface or drive usage of a server once bloat is already there. Admins have to drop to the command line and know the intricate commands for the operating system, logical volume manager, RAID controller and more to work with the NAS that some engineer who can't even figure out how to log in deployed using a GUI.
We talk often about how it's way, way too easy to deploy systems today. But we never really step back and analyze what's happening. It's that engineering has become so trivially easy in many cases that there is nothing needed at all. in fact, in dealing with SMB clients, it's extremely common for non-IT staff like a secretary, owner's nephew, some random person with no idea what they are doing, to buy and install systems and get them "up and running." Then, when things go wrong, actual experts in IT are called in once they are in over their head.
This happens because it literally requires no technical knowledge to engineer most common solutions. None. In fact, you can order a server from almost any vendor, and the vendor will do a half-assed demo OS install for you with every aspect of it wrong or poor, and it will work right out of the box. Put in a username and password and tell your business that you've "engineered a solution." You might as well be plugging in a lamp. The gap in requirements to be a functional engineer and a function admin are so wide that we often don't even think of those roles in that way and simply think of the engineering tasks as "non-technical" or marginally so, and the admin tasks as "real IT".
"Oh, your system set up by some high school kid who once played a video game is hosed now? Time to call the real IT people."
We just used "kid who once played a video game" to mean engineer. And "real IT people" to mean admins.
-
@scottalanmiller I'm going to let Microsoft Edge read aloud to me.
:astonished_face: -
@scottalanmiller said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
So here is a thought, and I think that this will make a great video, but we were talking in another thread about why engineers are the juniors to administrators in a proper environment. In the real world, engineers have the vastly simpler job - working in green fields and just following online guides most of the time. They can test, look things up, have someone look over their shoulders, take extra time, double check, document, get sign offs, etc. and at the end of the day, their stuff rarely even has to work because the admins will fix it later.
It is this that actually creates one of the biggest problems that we see in IT day to day - shoddy engineering work that screws administration later. We all deal with it almost every day. But why does this happen? Simple, because all kinds of tools are created to make engineering trivially easy. From the Windows GUI to silly products like FreeNAS, they are there to make the job of setting something up (engineering) really easy to get to a point of "working". Sure, really good, solid, proper engineering would involve loads of research and careful testing. But in the real world, so much is done to allow engineers to simply skip all of the hard work and just install something using a simple GUI, follow some simple online guides and no one is the wiser. The company that hired you sees a product up and running quickly and all of the potential problems that this brings is left to someone else (the admin).
Problem come in all sorts of forms. Maybe it's spinning disks on RAID 5 that won't likely cause hard issues for many years, long after no one remembers who configured them. Maybe it's a bloated Windows install that has loads of tools installed for a one time task. Maybe it's an OS where all the defaults are just taken and nothing more is done. Maybe it's storage installed with a NAS OS with a cute GUI that makes creating shares a breeze and does nothing to address dealing with hardware failure or disk corruption.
The IT product work is almost entirely focused on making the engineers (the ones who typically influence a purchase) have little to no work to do and almost nothing is done to make it easier to be an administrator. Admins have little option to fix a misconfigured RAID array, yet have to deal with its consequences. Admins have to deal with the attack surface or drive usage of a server once bloat is already there. Admins have to drop to the command line and know the intricate commands for the operating system, logical volume manager, RAID controller and more to work with the NAS that some engineer who can't even figure out how to log in deployed using a GUI.
We talk often about how it's way, way too easy to deploy systems today. But we never really step back and analyze what's happening. It's that engineering has become so trivially easy in many cases that there is nothing needed at all. in fact, in dealing with SMB clients, it's extremely common for non-IT staff like a secretary, owner's nephew, some random person with no idea what they are doing, to buy and install systems and get them "up and running." Then, when things go wrong, actual experts in IT are called in once they are in over their head.
This happens because it literally requires no technical knowledge to engineer most common solutions. None. In fact, you can order a server from almost any vendor, and the vendor will do a half-assed demo OS install for you with every aspect of it wrong or poor, and it will work right out of the box. Put in a username and password and tell your business that you've "engineered a solution." You might as well be plugging in a lamp. The gap in requirements to be a functional engineer and a function admin are so wide that we often don't even think of those roles in that way and simply think of the engineering tasks as "non-technical" or marginally so, and the admin tasks as "real IT".
"Oh, your system set up by some high school kid who once played a video game is hosed now? Time to call the real IT people."
We just used "kid who once played a video game" to mean engineer. And "real IT people" to mean admins.
So, what's the flip-side? I feel you only told half the story here.
-
It's not unlike the front wheel drive vs. rear wheel drive problem on snow. Front wheel drive is less likely to get stuck in snow, but will also get you up to dangerous speeds really easily. No one actually struggles with getting a car to move, it's driving it well and not hitting things that is hard.
-
@Obsolesce said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
So, what's the flip-side? I feel you only told half the story here.
The flip side is simply that good engineering is needed and people need to put in effort there. But at the end of the day, the needs and responsibilities of engineering is less than that of administration. It's not nothing, good engineering matters and we need people to take it seriously - and we need organizations to actually hold engineering accountable. Since probably effectively never arise until products are in production, it feels natural to hold admins accountable for the mistakes of engineers. Engineers toss the match over their shoulders and get to walk away. Organizations need to spend more time validating in real time what engineers do, not allow them to take the easy was out, and not leave them out of disaster discussions later.
-
This thinking kind of excludes roles like SRE where the engineers and admins are the same. The companies where engineers are paid heavily is where that type of thinking exists.
-
I think alot of the stuff you are saying is true, but that will always be the case with more senior roles. The contractors building and designing houses dont do the majority of the work. They are responsible, however, for knowing about each task is necessary to build a house. It's also very unlikely that these contractors havent done these jobs themselves before.
It's the same way in IT. Most people will not start in engineering roles or design roles. We never see these positions as entry level with zero experience.
-
@scottalanmiller said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
Since probably effectively never arise until products are in production, it feels natural to hold admins accountable for the mistakes of engineers. Engineers toss the match over their shoulders and get to walk away. Organizations need to spend more time validating in real time what engineers do, not allow them to take the easy was out, and not leave them out of disaster discussions later.
Maybe in SMB it's like that. But in the larger places I've seen lately the world revolves around an Agile approach and constant feedback between stakeholders, admins, and engineers. There's thorough testing, test groups, pilots, and business pilots. Nobody is playing the blame game on admins that I've seen, not even a hint. The validation of what engineers are doing is huge from what I've seen. Though, definitely not in the SMB.
Maybe in the SMB you have 2 IT dudes, Engineer 1 and Admin 1.... Engineer 1 implements FreeNAS for the company's new storage solution and it fucks up because nobody knows what they are doing. But I can tell you that shit wouldn't wouldn't even pass as a concept in the Enterprises I've seen and talked to.
-
Admin roles are also dying with immutable infrastructure and HA. Designing a system that is immutable and highly available isn't expensive or time consuming on the cloud anymore.
The cloud done right is always cheaper than on premise. The key being "done right" hence the shift towards engineers vs admins.
Admins are generally the ones fighting the cloud, because they could be out of the job.
Troubleshooting severs is completely different these days and done after the fact if you even care. Spin up a new VM and move the snapshot of the old one so you can figure out what happened if you care (most of the time you don't).
-
@IRJ said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
It's the same way in IT. Most people will not start in engineering roles or design roles. We never see these positions as entry level with zero experience.
Same with administration, though. Admins require even more experience and knowledge to not be dangerous.
I'm actually saying that we see zero experience engineers everywhere, every day. It's the most common thing to have engineers have zero idea what's going on so much of the time.
-
@Obsolesce said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
@scottalanmiller said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
Since probably effectively never arise until products are in production, it feels natural to hold admins accountable for the mistakes of engineers. Engineers toss the match over their shoulders and get to walk away. Organizations need to spend more time validating in real time what engineers do, not allow them to take the easy was out, and not leave them out of disaster discussions later.
Maybe in SMB it's like that. But in the larger places I've seen lately the world revolves around an Agile approach and constant feedback between stakeholders, admins, and engineers. There's thorough testing, test groups, pilots, and business pilots. Nobody is playing the blame game on admins that I've seen, not even a hint. The validation of what engineers are doing is huge from what I've seen. Though, definitely not in the SMB.
Maybe in the SMB you have 2 IT dudes, Engineer 1 and Admin 1.... Engineer 1 implements FreeNAS for the company's new storage solution and it fucks up because nobody knows what they are doing. But I can tell you that shit wouldn't wouldn't even pass as a concept in the Enterprises I've seen and talked to.
SMB "always" makes engineering an afterthought role of an admin. They hire admins, and have them do engineering for the itty bitty bit that is needed. Even a 300 person company might need under one week of engineering time a year. And any remotely qualified admin can do the engineering in their sleep most of the time. So that's what the SMB does.
-
@scottalanmiller said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
@IRJ said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
It's the same way in IT. Most people will not start in engineering roles or design roles. We never see these positions as entry level with zero experience.
Same with administration, though. Admins require even more experience and knowledge to not be dangerous.
I'm actually saying that we see zero experience engineers everywhere, every day. It's the most common thing to have engineers have zero idea what's going on so much of the time.
I have never seen a true engineering position that says zero experience required before. I frequent LinkedIn, glassdoor, and other job boards.
Have any you care to share?
-
@IRJ said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
Admin roles are also dying with immutable infrastructure and HA. Designing a system that is immutable and highly available isn't expensive or time consuming on the cloud anymore.
That's not really a thing yet. People talk about it, but it's not out there impacting people in practice. And when you make things too immutable, you just shift the roles around and it all starts over again. Cloud really isn't here or there, it just shifts the names of things. But someone is still designing the initial system and someone (maybe the same person) is managing it.
-
@IRJ said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
@scottalanmiller said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
@IRJ said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
It's the same way in IT. Most people will not start in engineering roles or design roles. We never see these positions as entry level with zero experience.
Same with administration, though. Admins require even more experience and knowledge to not be dangerous.
I'm actually saying that we see zero experience engineers everywhere, every day. It's the most common thing to have engineers have zero idea what's going on so much of the time.
I have never seen a true engineering position that says zero experience required before. I frequent LinkedIn, glassdoor, and other job boards.
Have any you care to share?
Sure you have. You see them every day. No one CALLS it engineering, but I already gave examples. For example... literally every FreeNAS ever deployed.
What would job boards tell you about this? You are using companies "titles" as a reference to their "roles". The two are rarely related.
-
@scottalanmiller said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
@IRJ said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
Admin roles are also dying with immutable infrastructure and HA. Designing a system that is immutable and highly available isn't expensive or time consuming on the cloud anymore.But someone is still designing the initial system and someone (maybe the same person) is managing it.
Yeah so you don't have an admin here as you admit. You have an engineer designing the system and replacing the system if there is issues. It's all design and no maintenance. Maintenance is automated during build.
-
Every system that exists has been engineered. Every one. Most companies don't have a single engineering titled person, yet every one has someone that did their engineering. Almost all of those use either a zero experience engineer and/or they have nearly full time admins doing engineering on the side as an afterthought. Those two approaches account for almost the entire market.
The market for full time, professionally trained, skilled system engineers represents probably only 1-5% of the entire engineering in IT effort that exists. If your company has an engineering role, then it's already an outlier statistically within the field.
-
@IRJ said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
@scottalanmiller said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
@IRJ said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
Admin roles are also dying with immutable infrastructure and HA. Designing a system that is immutable and highly available isn't expensive or time consuming on the cloud anymore.But someone is still designing the initial system and someone (maybe the same person) is managing it.
Yeah so you don't have an admin here as you admit. You have an engineer designing the system and replacing the system if there is issues. It's all design and no maintenance. Maintenance is automated during build.
Not in the real world. That's a nice theory, but applies to effectively no one anywhere. In the real world, engineering almost always is a trivial effort that involves almost no time, skill or planning, and all the effort goes into years of administration that deals with that haphazard system.
-
@scottalanmiller said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
@Obsolesce said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
@scottalanmiller said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
Since probably effectively never arise until products are in production, it feels natural to hold admins accountable for the mistakes of engineers. Engineers toss the match over their shoulders and get to walk away. Organizations need to spend more time validating in real time what engineers do, not allow them to take the easy was out, and not leave them out of disaster discussions later.
Maybe in SMB it's like that. But in the larger places I've seen lately the world revolves around an Agile approach and constant feedback between stakeholders, admins, and engineers. There's thorough testing, test groups, pilots, and business pilots. Nobody is playing the blame game on admins that I've seen, not even a hint. The validation of what engineers are doing is huge from what I've seen. Though, definitely not in the SMB.
Maybe in the SMB you have 2 IT dudes, Engineer 1 and Admin 1.... Engineer 1 implements FreeNAS for the company's new storage solution and it fucks up because nobody knows what they are doing. But I can tell you that shit wouldn't wouldn't even pass as a concept in the Enterprises I've seen and talked to.
SMB "always" makes engineering an afterthought role of an admin. They hire admins, and have them do engineering for the itty bitty bit that is needed. Even a 300 person company might need under one week of engineering time a year. And any remotely qualified admin can do the engineering in their sleep most of the time. So that's what the SMB does.
OKay, so the SMB only has "admins", who are designing their systems, administering them, and then taking the heat? Well yeah, can't blame the engineer if there isn't one, when it was the Admin who did set it all up.
-
@scottalanmiller said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
@IRJ said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
@scottalanmiller said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
@IRJ said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
Admin roles are also dying with immutable infrastructure and HA. Designing a system that is immutable and highly available isn't expensive or time consuming on the cloud anymore.But someone is still designing the initial system and someone (maybe the same person) is managing it.
Yeah so you don't have an admin here as you admit. You have an engineer designing the system and replacing the system if there is issues. It's all design and no maintenance. Maintenance is automated during build.
Not in the real world. That's a nice theory, but applies to effectively no one anywhere. In the real world, engineering almost always is a trivial effort that involves almost no time, skill or planning, and all the effort goes into years of administration that deals with that haphazard system.
That's completely false. Engineering is almost always a trivial effort......
-
@Obsolesce said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
@scottalanmiller said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
@Obsolesce said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
@scottalanmiller said in Engineering vs Administration - That's what makes Windows and FreeNAS so risky:
Since probably effectively never arise until products are in production, it feels natural to hold admins accountable for the mistakes of engineers. Engineers toss the match over their shoulders and get to walk away. Organizations need to spend more time validating in real time what engineers do, not allow them to take the easy was out, and not leave them out of disaster discussions later.
Maybe in SMB it's like that. But in the larger places I've seen lately the world revolves around an Agile approach and constant feedback between stakeholders, admins, and engineers. There's thorough testing, test groups, pilots, and business pilots. Nobody is playing the blame game on admins that I've seen, not even a hint. The validation of what engineers are doing is huge from what I've seen. Though, definitely not in the SMB.
Maybe in the SMB you have 2 IT dudes, Engineer 1 and Admin 1.... Engineer 1 implements FreeNAS for the company's new storage solution and it fucks up because nobody knows what they are doing. But I can tell you that shit wouldn't wouldn't even pass as a concept in the Enterprises I've seen and talked to.
SMB "always" makes engineering an afterthought role of an admin. They hire admins, and have them do engineering for the itty bitty bit that is needed. Even a 300 person company might need under one week of engineering time a year. And any remotely qualified admin can do the engineering in their sleep most of the time. So that's what the SMB does.
OKay, so the SMB only has "admins", who are designing their systems, administering them, and then taking the heat? Well yeah, can't blame the engineer if there isn't one, when it was the Admin who did set it all up.
No one was about blaming them. The point being - it's not being treated as a discipline. It's treated as "well he's an admin, so he can just engineer, too" because generally everyone sees it as a trivial "tack on" to administration. They don't think about the fact that it's the engineering role, they just think of engineering overall as something not worth thinking about. That's the point - it's overlooked because people generally think of it as trivial and unimportant and easy.
