Posts made by RamblingBiped

@stacksofplates said in Fedora 26 No Space Left on Device with Plenty of Space Available:

@scottalanmiller said in Fedora 26 No Space Left on Device with Plenty of Space Available:

@travisdh1 said in Fedora 26 No Space Left on Device with Plenty of Space Available:

@jaredbusch said in Fedora 26 No Space Left on Device with Plenty of Space Available:

@wirestyle22 said in Fedora 26 No Space Left on Device with Plenty of Space Available:

@jaredbusch said in Fedora 26 No Space Left on Device with Plenty of Space Available:

@travisdh1 said in Fedora 26 No Space Left on Device with Plenty of Space Available:

@scottalanmiller said in Fedora 26 No Space Left on Device with Plenty of Space Available:

@wirestyle22 said in Fedora 26 No Space Left on Device with Plenty of Space Available:

@scottalanmiller I could've sworn there was a post here related to a reboot solving it

Someone mentioned rebooting to try to solve it. But it did not (had already done that.) It was real files causing the issue, nothing orphaned. Literally, we were making 30K files an hour or so.

Tiny files to, right?

That is not relevant except for that fact that large files would have filled drive space and likely been noticed.

How is that not relevant? more files = more inodes being used

Size of the files is not relevant.

It actually makes sense that @scottalanmiller said it was mostly directories. Files of any size will almost always run out of drive space before inodes run out in 99.9999% of situations. This is the first time I've actually heard of this happening, ever.

Only not relevant for those who actually know about inodes already. The only reason I even know about them is they got mentioned in SGI's IRIX Sysadmin courses.

The one major exception is marker files. "touch thishappened" as a file automatically and never clean up and you are using inodes without using any space. That's who you can easily learn about inode depletion. But who does that?

People writing bad scripts that use lock files and forget to remove them.

That wouldn't create enough to do this though.

If you start using configuration management tools to manage infrastructure with code you get the chance to see some of these one-off oddities in the wild a little more frequently than you'd expect. Like having Java developers not use Java's log facilities to manage log rotation, and then having a generic log rotation configuration completely bork things by delete application logs that are still being accessed by the Java application.

I got to see this issue a few dozen times a few months ago before another of our Engineers disassociated the Java applications from our generic log rotation recipe.

Rebooting was the quick fix for us prior to fixing the actual problem.

I've seen this behavior a fair amount on systems where log rotation for java applications is misconfigured. Log files are removed while the java application is still writing to them so it shows the file being deleted even though the connection to the inode is still held open.

A reboot is usually the quickest way to release the inodes.

lsof +L1

FTW...

You'll not really benefit from containerized workloads unless you are deploying applications that are designed to be ephemeral and stateless. Trying to deploy an application using a container the same way you deploy an application to a virtual machine, is not the way containers were intended to be used.

Containers should not be thought about in relation to virtual machines, they are two different things. Containers provide an encapsulation and abstraction of a service from the operating system. Containers are not a miniature virtual machine that you deploy your software on or to.

Check out the 12-factor App Methodology for a better explanation:

https://12factor.net/

Lights Out - Ted Koppel

https://www.goodreads.com/book/show/25733923-lights-out

Just glancing at the resume, I feel like you're giving away too much information. Highlight specific projects/accomplishments, but don't go too deep or technical. I think you can reduce the wordiness of your bullet points considerably and still communicate what you are trying to get across. You want to highlight your achievements without giving away specifics. The specifics of what you did are what you want to discuss in an interview. Bait the hook, don't give them a free lunch.

I also agree with adding in your consulting business and your experience with T3 support.

Look at Bash, Python, Ruby, and JavaScript.

And Powershell if you're one of those people...

@travisdh1 said in Quitting Vi:

@ramblingbiped said in Quitting Vi:

@dustinb3403 said in Quitting Vi:

@scottalanmiller said in Quitting Vi:

@coliver said in Quitting Vi:

I prefer vi... I've been burned by not having nano in the past and forced myself to learn vi because of that.

Same here. I've seen Fortune 100 companies production brought down because no one knew vi and nothing else was available in a crash.

These scenarios are so out past the moon for most of us that they would rarely occur. Like buying 4 servers to protect from 2 of them dying.

Now this doesn't mean don't learn Vi (I try and learn myself) I just don't like the interface.

We have over 1000 Linux servers and Vi is the only option on the greater majority (75%+). None of our QA, Stage, or Production systems come with anything else.

I like and use Vim, and go with Vi if that's all I've got. Here's a great book for anyone looking to become a Vim power user: Practical Vim (https://pragprog.com/book/dnvim2/practical-vim-second-edition)

At that point, why do you even need to touch any individual server rather than using your management platform of choice (salt, tower, etc.)

We don't normally unless we have to investigate any issues that might arise. And even then we don't normally need a text editor.

@black3dynamite said in MangoLassi TV:

@jaredbusch said in MangoLassi TV:

I don't watch videos.

So all your recent little video responses are lost content.

Would you preferred a MangoLassi Podcast instead?

I think I would prefer an animated series.

@marcinozga said in I need some ideas how to use spare server:

I have a Supermicro server with 2x Xeon E5530 and 24GB Ram, with 1TB storage. It currently runs free ESXi with some VMs that are going to be shut down in a few days. I also have aging Nagios/Opsview server that needs modern replacement. I need a few ideas what to put on that server. I plan on running all Linux VMs, and only free, open source apps. I also cannot add any local storage, it has 8x 15k SAS drives, replacing it would be fairly expensive.

This is spare server at work, so things like Plex or Minecraft server are off the table.

Suggestions?

Install KVM and work on using configuration tools like Chef, Saltstack, Puppet, and Ansible for configuration management of Webservers (Apache2, Nginx), Databases(MySQL, MariaDB, Postgres), Load Balancers(HAProxy) ,Container Hosts(Docker Host, CoreOS), and other services(ELK stack, Nagios, BIND, DHCP, etc...).

When you've finished send me your resume; especially if you get good with Chef. We're looking for a couple/few good DevOps Engineers.

Also, installing Vim and typing 'vimtutor' at any terminal will launch an interactive beginner's tutorial on Vim.

@dustinb3403 said in Quitting Vi:

@scottalanmiller said in Quitting Vi:

@coliver said in Quitting Vi:

I prefer vi... I've been burned by not having nano in the past and forced myself to learn vi because of that.

Same here. I've seen Fortune 100 companies production brought down because no one knew vi and nothing else was available in a crash.

These scenarios are so out past the moon for most of us that they would rarely occur. Like buying 4 servers to protect from 2 of them dying.

Now this doesn't mean don't learn Vi (I try and learn myself) I just don't like the interface.

We have over 1000 Linux servers and Vi is the only option on the greater majority (75%+). None of our QA, Stage, or Production systems come with anything else.

I like and use Vim, and go with Vi if that's all I've got. Here's a great book for anyone looking to become a Vim power user: Practical Vim (https://pragprog.com/book/dnvim2/practical-vim-second-edition)

December 18, 2009

https://community.spiceworks.com/topic/84516-xfps-3-or-4-cheating-or-not

I haven't done it myself, but you should be able to script nmap to populate a list of hosts in csv format and import that into excel for review.

@PenguinWrangler said in Well, that really, really sucks.:

@RamblingBiped said in Well, that really, really sucks.:

Also, if you're just staying in Illinois because of the booze, Michigan has WAY better beer and is just across the Lake. You could literally stop in damn near any small town throughout the state and find a decent brewery.

Even if you only consider Three Floyd's brewery in Munster, Indiana has way better beer than Illinois. If you also throw in Upland's wheat they win hands down.

And, well, Wisconsin has always been superior to Illinois in booze and alcoholics.

I can't think of a good reason to move to Iowa, Misery, or any other state to the South...

Screw Chicago, I am from Southern Illinois, damn arrogant bastards up north. If you really want my opinion.......

Anything South of 80 is Southern Illinois, right?

Do all of the White House press briefings count? I hear they know all about the cybers.

Also, if you're just staying in Illinois because of the booze, Michigan has WAY better beer and is just across the Lake. You could literally stop in damn near any small town throughout the state and find a decent brewery.

Even if you only consider Three Floyd's brewery in Munster, Indiana has way better beer than Illinois. If you also throw in Upland's wheat they win hands down.

And, well, Wisconsin has always been superior to Illinois in booze and alcoholics.

I can't think of a good reason to move to Iowa, Misery, or any other state to the South...

@PenguinWrangler said in Well, that really, really sucks.:

@JaredBusch said in Well, that really, really sucks.:

Illinois has plenty of fucked up problems, but buying alcohol is not one of them. screw all you people I guess I will stay here.

I agree with you Jared. I am going to stay in my Illinois(remember kids there is no noise in Illinois, the s is silent). Our roads might crumble (all construction on roads is coming to a stop if no budget is passed by June 30 which there won't be http://www.bnd.com/news/local/article85091162.html). Before that announcement IDOT was out on TV and radio stations warning that with the upcoming summer heat that roads might buckle and drivers need to be aware of this, I told my wife that is IDOTs way of saying we ain't fixing S#!T drive at your own risk. We are seriously close to bankruptcy (http://www.cbsnews.com/news/could-illinois-be-the-first-state-to-file-for-bankruptcy/). However we can get all the liquor we need or want, right now we need a lot.

I left Illinois in 2013, and it was probably one of the best decisions I've made. Initially people were migrating away from the smaller cities and toward larger metropolitan centers. Now they are even leaving Chicago in droves.

I can't imagine the corrupt legislation's answer to the budget crisis (if and when that EVER happens) isn't going to end up being more taxes... Eff that noise.

http://www.chicagotribune.com/news/local/breaking/ct-chicago-census-population-loss-met-20170322-story.html

Easiest way would be give the job to the Summer Interns and tell them to figure it out. If they do it manually... Well, it gets done. If they figure out a quicker automated way of doing it have them document it and give them a nice pat on the back.

Either way, less of your time is wasted.

@BRRABill Yes, they do. However, I make my own beer and I got a significant enough raise I don't care. And you can still buy all the booze, you just have to drive to a State run store to do so (they are everywhere).

And Colorado is just right over there if you're a heathen.

Sinner.