This was a June 28th-thing...
-
We recently decided, okay, I am the a$$ who decided to test in sandbox whether or not to implement ManageEngines MDM Free Hosted Software. While my IT staff was setting this up on the server, and made the proper port allowances for the ME MDM, something strange happened.
Side story, I am spending time this week with my daughter, whose 12 and don't see much (she's in NJ and I am in Tx). So, I left early for her. When I noticed I am getting calls from my COO, and clients I deal with directly, that cannot access their web pages. So, I try to submit a ticket, cannot get to the ticket page, I remote into my office computer to use the internal link, that works.
So, then I have a discussion with my IT manager, and he's telling me everything he has done, I even ask him about the codes in directly in the DNS. I return to work with my daughter, promising we won't be long. So, the issue is:
- DNS is working
- We tried to stop/start the DNS
- We even rebooted the server (not fun)
- All internal links were working
- All requests that need be forwarded from the DNS work
- The only DNS failing was the items stored on the web server
So, there is me with 20 years in IT, my IT Manager and his staff with a wide range of certs and education looking in to different scenarios that can be causing this. Meanwhile, my daughter says, "Could it be something with a firewall?" Just as my IT Mgr was getting ready to say that he checked necessary ports, like he told me...I said, "Did you make the adjustment to the hardware FW, or a subordinate?" He implied a subordinate. I said, "Show me the firewall."
Soon as he brought up the port listings there it was, the webserver address accidentally got changed and was pointing to the wrong server for internet calls. We immediately fixed the problem, broke the fingers of the subordinate, and I instrumented a new policy of allowing "little girls in the war room". Side note, my daughter received a consulting fee of a Red Lobster dinner for that.
-
That's awesome! And now she can put this on her resume too
-
Very nice!
Tasty consulting fee! -
Great story. Hope that lobster was good.
And your daughter now can tell about her very first consultancy job during interviews
-
@acs77043 your daughter should be an elite member of ML now!
-
Yeah, why doesn't your daughter have an ML account? We need her answering questions!
-
@scottalanmiller said in This was a June 28th-thing...:
Yeah, why doesn't your daughter have an ML account? We need her answering questions!
She can be an MLNews member too!
-
FYI, @acs77043 was someone who used to come in to my retail store and we'd talk shop. One of the few customers who I could talk real shop with, and we always had a good time.
-
@acs77043 , I still remember your story about the PC you piecemealed together and it was overheating, and then you stuck a box fan up against it to cool it and all was good. Still makes me chuckle to think about! LOL
-
@thanksajdotcom said in This was a June 28th-thing...:
FYI, @acs77043 was someone who used to come in to my retail store and we'd talk shop. One of the few customers who I could talk real shop with, and we always had a good time.
Oh neat. Did you bring him in over here, or was it coincidence?
-
@scottalanmiller said in This was a June 28th-thing...:
@thanksajdotcom said in This was a June 28th-thing...:
FYI, @acs77043 was someone who used to come in to my retail store and we'd talk shop. One of the few customers who I could talk real shop with, and we always had a good time.
Oh neat. Did you bring him in over here, or was it coincidence?
Take a wild guess
-
@thanksajdotcom said in This was a June 28th-thing...:
@scottalanmiller said in This was a June 28th-thing...:
@thanksajdotcom said in This was a June 28th-thing...:
FYI, @acs77043 was someone who used to come in to my retail store and we'd talk shop. One of the few customers who I could talk real shop with, and we always had a good time.
Oh neat. Did you bring him in over here, or was it coincidence?
Take a wild guess
Does it involve Tequila, a water buffalo, fridge magnets and Twister?
Ā
EDIT: ^that's my wild guess. Not sure how that all fits together but I'm hoping that you can explain. -
That's a pretty cool story, I say definitely nurture her technical side. I've noticed that in the west, especially America, there's a subcurrent of almost discouragement for girls to be interested in technology. It's vague, it's subtle, but it's certainly there. You just need to counteract any potential crap she might get, primarily from people in TV marketing. it sounds like you're doing a great job already so I don't really need to say any more on that.
It does remind me though how on Spiceworks and even in my own IT company, that when there's a service disruption similar to yours or a machine cannot talk to the world, they don't go through the OSI model and check everything one by one. Doing that saves a lot of time, because I think IT people often simply assume it's always either bad cable or software misconfiguration/failure on the machine itself, not potentially each layer in between. A similar situation happened a few months ago, where a network switch went bad at one of our offices so the connections to a few servers suddenly vanished.
One of our novice employees decided to hard reboot all of the VM hosts he couldn't contact, which was 4 out of about 12 total. Everything presumably came back up and started as it's supposed to, but it still didn't work, and he and someone else were having a hell of a time trying to figure this out. They called the office manager who was out and he suggested checking the switch, and they did, and it still didn't work. The other guy was pretty busy as well, otherwise the rest of this story probably wouldn't have happened.
So this young guy hard resets the machines again, and still nothing, so finally he calls me. As a general rule, you're never supposed to call me unless it's a major problem and nobody can figure it out.
So, he tells me all that happened, and the first thing I said to him, verbatim was:
Don't you think if the network connections to multiple physical machines stopped working, it wouldn't be the machines, but probably something else?
He said:
Oh, yeah, I guess that makes sense, I just assumed it was the hosts.
So, I told him to go through the OSI model, use it as a general guide, and check everything and told another guy there to make sure he did it right and understood checking doesn't mean looking to see if the light is on. During the first stage, they figure out it was is the switch that had gone bad. It still lights up, it seems like it works, but it simply doesn't. They changed it out and everything came back up.
Except... email, the hard rebooting really messed with Exchange, and even thought it was spanned across three servers with DAG, it flipped out. That wasn't hard to sort out, obviously and things were back to normal in no time.
Checking each thing first and knowing whether or not it works is a good way to go about it, and I think a lot of young people especially forget that. In a sense, if it's not something I can fix right through this terminal and it's still plugged in, then it's completely broken.
You may be wondering also why there wasn't teaming with multiple switches. Actually, there was, however they were in the process of moving the machines to the other server room across the building, so it was a situation of the bare minimum being there just for that day until after EOD. It was a hell of a time for the switch to go out, but it taught this guy a powerful lesson.
I didn't fire him, though he thought I was going to. I told him "that's why you're paid what you are, it's entry level, any of the other guys made the same mistake they'd probably be fired, primarily for the idiotic hard rebooting thing. Next time think about the problem, and go down the OSI model list, and never, ever hard reboot anything unless you know for certain the machine is hung up."
I guess that was kind of long, but I also see people on Spiceworks running into this thing a lot too. Sometimes I jump the gun myself and then when my initial suspicions are wrong, I remember to go back to the list.
-
@tonyshowoff Great story, and it's basically the same I'm telling trainees on day one: observe, follow the path (OSI etc.), act later.
Thanks for sharing.
-
@tonyshowoff said in This was a June 28th-thing...:
That's a pretty cool story, I say definitely nurture her technical side. I've noticed that in the west, especially America, there's a subcurrent of almost discouragement for girls to be interested in technology. It's vague, it's subtle, but it's certainly there.
I'd say it is more huge and in your face than vague or subtle. Girls are often outright faced (often from other women more than men) with "that's not a girl's job", or talk that girls won't be good at that kind of thing or, more often, just told that girls don't enjoy that kind of work.
The subtle stuff is mostly in the weird "get girls into STEM" stuff where they act like they are doing special STEM encouragement projects but really it is just a "treat girls differently and do less with them" process. My nieces suffer from this. Their schools offer all of this extra-curricular activities around robotics and programming and stuff... but none of it is even slightly technical or challenging and causes them to get even less exposure to computers and tech jobs than you would hope that you would get in a normal classroom. It's so bad that even with a school focus on engineering and programming by soon to graduate niece has never actually seen programming!
-
@tonyshowoff said in This was a June 28th-thing...:
I guess that was kind of long, but I also see people on Spiceworks running into this thing a lot too. Sometimes I jump the gun myself and then when my initial suspicions are wrong, I remember to go back to the list.
I think that part of being a senior technician is knowing when to go with gut feelings rather than long processes. I've been known to eyeball latency in threads and pick out what was the problem. It's not some magic sauce, it's just loads of exposure with a decent empathy for the machine. That's a lot of what makes seniors faster, we know how things behave under different scenarios, know how the parts interact and can often guess what is wrong before proving it. A junior needs processes to prove it or else they are hit and miss all over the place. Seniors will often guess it the first time to save time and effort.
-
@thanksajdotcom for a while that was the jewel in the collection. She was an ugly bitch (lol), but Windows 98 SE, 233-K6 processor, 54x CD-Rom/Burner, no modem, just an Ethernet port, 20GB HDD, oh and the best part, 512-MB SD Ram. I used to love to watch the beginning memtest, and the system count all of it.
-
@scottalanmiller said in This was a June 28th-thing...:
@tonyshowoff said in This was a June 28th-thing...:
That's a pretty cool story, I say definitely nurture her technical side. I've noticed that in the west, especially America, there's a subcurrent of almost discouragement for girls to be interested in technology. It's vague, it's subtle, but it's certainly there.
I'd say it is more huge and in your face than vague or subtle. Girls are often outright faced (often from other women more than men) with "that's not a girl's job", or talk that girls won't be good at that kind of thing or, more often, just told that girls don't enjoy that kind of work.
Well I was just trying to be polite for all the Americans here. I've pointed out on Spiceworks many times in those types of threads that if women inherently dislike that kind of thing, why does Russia have more female engineers and doctors than male ones? Most of the programmers I've met here were women as well. It's not completely inverted in every case, but at the bear minimum is at least 50/50 on the low end. So if there's some sort of genetic predisposition to women not like technology, then for some reason Slavs don't seem to carry this. Of course the typical response is either silence or "well you'll never convince me otherwise [because I'm a lonely white guy in America who thinks women shouldn't like technology so I have a huge confirmation bias, my mom isn't a programmer, therefore women don't want to be.]"
-
@tonyshowoff said in This was a June 28th-thing...:
@scottalanmiller said in This was a June 28th-thing...:
@tonyshowoff said in This was a June 28th-thing...:
That's a pretty cool story, I say definitely nurture her technical side. I've noticed that in the west, especially America, there's a subcurrent of almost discouragement for girls to be interested in technology. It's vague, it's subtle, but it's certainly there.
I'd say it is more huge and in your face than vague or subtle. Girls are often outright faced (often from other women more than men) with "that's not a girl's job", or talk that girls won't be good at that kind of thing or, more often, just told that girls don't enjoy that kind of work.
Well I was just trying to be polite for all the Americans here. I've pointed out on Spiceworks many times in those types of threads that if women inherently dislike that kind of thing, why does Russia have more female engineers and doctors than male ones? Most of the programmers I've met here were women as well. It's not completely inverted in every case, but at the bear minimum is at least 50/50 on the low end. So if there's some sort of genetic predisposition to women not like technology, then for some reason Slavs don't seem to carry this. Of course the typical response is either silence or "well you'll never convince me otherwise [because I'm a lonely white guy in America who thinks women shouldn't like technology so I have a huge confirmation bias, my mom isn't a programmer, therefore women don't want to be.]"
Ironically 3 of my last 4 jobs the IT Directors were women, lol. And they were GOOD at IT, not just meh.
-
@dafyre said in This was a June 28th-thing...:
Ironically 3 of my last 4 jobs the IT Directors were women, lol. And they were GOOD at IT, not just meh.
In the US it is considered acceptable for women to be managers, but not technicians. Women in IT often go into management very quickly. Or start in management without coming through IT.
