BRRABill's Field Report With Linux

scottalanmiller

@Dashrender said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@DustinB3403 said in BRRABill's Field Report With Linux:

@BRRABill Adding a second drive to a VM is literally nothing though.

It would be better practice to add a drive, than to try and extend the existing one.

But in theory, that 15G partition is part of the 19.5GB VHD the GrayLog appliance sets up.

You're losing the 15G, right?

I know 15G isn't much, but I was just thinking for future reference, if it was more than 15G.

Losing 15GB? Not if you are thin provisioned.

Well, as of right now, this is how things rolled...

Imported the GrayLog OVA appliance to XS.

It creates a 19.5GB virtual disk where it does its magic.

Part of that magic is this 15GB partition that is now full.

So, even thin provisioned, isn't that space already taken? (AKA once the data fills it, it still uses it even if the data is deleted, correct?)

Sure it is, but after you copy that data to the new drive, you'll delete it from the old drive making it empty... Assuming XS can reclaim now empty space, you'll gain that 15 GB back.

Yes, in reality this is all that you do. Make a new one, remove the old.

scottalanmiller

@BRRABill said in BRRABill's Field Report With Linux:

@Dashrender said

Sure it is, but after you copy that data to the new drive, you'll delete it from the old drive making it empty... Assuming XS can reclaim now empty space, you'll gain that 15 GB back.

That's not how I understood thin provisioning to work.

You can take the VHD offline and resize it in WIndows, but I don't think you can in XS.

Don't resize.... remove.

BRRABill

@scottalanmiller said in BRRABill's Field Report With Linux:

@Dashrender said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@DustinB3403 said in BRRABill's Field Report With Linux:

@BRRABill Adding a second drive to a VM is literally nothing though.

It would be better practice to add a drive, than to try and extend the existing one.

But in theory, that 15G partition is part of the 19.5GB VHD the GrayLog appliance sets up.

You're losing the 15G, right?

I know 15G isn't much, but I was just thinking for future reference, if it was more than 15G.

Losing 15GB? Not if you are thin provisioned.

Well, as of right now, this is how things rolled...

Imported the GrayLog OVA appliance to XS.

It creates a 19.5GB virtual disk where it does its magic.

Part of that magic is this 15GB partition that is now full.

So, even thin provisioned, isn't that space already taken? (AKA once the data fills it, it still uses it even if the data is deleted, correct?)

Sure it is, but after you copy that data to the new drive, you'll delete it from the old drive making it empty... Assuming XS can reclaim now empty space, you'll gain that 15 GB back.

Yes, in reality this is all that you do. Make a new one, remove the old.

But in this particular appliance scenario, that is not possible, correct?

scottalanmiller

@BRRABill said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@Dashrender said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@DustinB3403 said in BRRABill's Field Report With Linux:

@BRRABill Adding a second drive to a VM is literally nothing though.

It would be better practice to add a drive, than to try and extend the existing one.

But in theory, that 15G partition is part of the 19.5GB VHD the GrayLog appliance sets up.

You're losing the 15G, right?

I know 15G isn't much, but I was just thinking for future reference, if it was more than 15G.

Losing 15GB? Not if you are thin provisioned.

Well, as of right now, this is how things rolled...

Imported the GrayLog OVA appliance to XS.

It creates a 19.5GB virtual disk where it does its magic.

Part of that magic is this 15GB partition that is now full.

So, even thin provisioned, isn't that space already taken? (AKA once the data fills it, it still uses it even if the data is deleted, correct?)

Sure it is, but after you copy that data to the new drive, you'll delete it from the old drive making it empty... Assuming XS can reclaim now empty space, you'll gain that 15 GB back.

Yes, in reality this is all that you do. Make a new one, remove the old.

But in this particular appliance scenario, that is not possible, correct?

I don't have the appliance in front of me, are there not separate disks for these things?

DustinB3403

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@Dashrender said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@DustinB3403 said in BRRABill's Field Report With Linux:

@BRRABill Adding a second drive to a VM is literally nothing though.

It would be better practice to add a drive, than to try and extend the existing one.

But in theory, that 15G partition is part of the 19.5GB VHD the GrayLog appliance sets up.

You're losing the 15G, right?

I know 15G isn't much, but I was just thinking for future reference, if it was more than 15G.

Losing 15GB? Not if you are thin provisioned.

Well, as of right now, this is how things rolled...

Imported the GrayLog OVA appliance to XS.

It creates a 19.5GB virtual disk where it does its magic.

Part of that magic is this 15GB partition that is now full.

So, even thin provisioned, isn't that space already taken? (AKA once the data fills it, it still uses it even if the data is deleted, correct?)

Sure it is, but after you copy that data to the new drive, you'll delete it from the old drive making it empty... Assuming XS can reclaim now empty space, you'll gain that 15 GB back.

Yes, in reality this is all that you do. Make a new one, remove the old.

But in this particular appliance scenario, that is not possible, correct?

I don't have the appliance in front of me, are there not separate disks for these things?

No, the OVA imports a single disk with 2 LV's.

BRRABill

@DustinB3403 said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@Dashrender said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@DustinB3403 said in BRRABill's Field Report With Linux:

@BRRABill Adding a second drive to a VM is literally nothing though.

It would be better practice to add a drive, than to try and extend the existing one.

But in theory, that 15G partition is part of the 19.5GB VHD the GrayLog appliance sets up.

You're losing the 15G, right?

I know 15G isn't much, but I was just thinking for future reference, if it was more than 15G.

Losing 15GB? Not if you are thin provisioned.

Well, as of right now, this is how things rolled...

Imported the GrayLog OVA appliance to XS.

It creates a 19.5GB virtual disk where it does its magic.

Part of that magic is this 15GB partition that is now full.

So, even thin provisioned, isn't that space already taken? (AKA once the data fills it, it still uses it even if the data is deleted, correct?)

Sure it is, but after you copy that data to the new drive, you'll delete it from the old drive making it empty... Assuming XS can reclaim now empty space, you'll gain that 15 GB back.

Yes, in reality this is all that you do. Make a new one, remove the old.

But in this particular appliance scenario, that is not possible, correct?

I don't have the appliance in front of me, are there not separate disks for these things?

No, the OVA imports a single disk with 2 LV's.

Correct.

  LV     VG         Attr      LSize  Pool Origin Data%  Move Log Copy%  Convert
  root   graylog-vg -wi-ao--- 15.29g
  swap_1 graylog-vg -wi-ao---  4.00g

scottalanmiller

@DustinB3403 said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@Dashrender said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@DustinB3403 said in BRRABill's Field Report With Linux:

@BRRABill Adding a second drive to a VM is literally nothing though.

It would be better practice to add a drive, than to try and extend the existing one.

But in theory, that 15G partition is part of the 19.5GB VHD the GrayLog appliance sets up.

You're losing the 15G, right?

I know 15G isn't much, but I was just thinking for future reference, if it was more than 15G.

Losing 15GB? Not if you are thin provisioned.

Well, as of right now, this is how things rolled...

Imported the GrayLog OVA appliance to XS.

It creates a 19.5GB virtual disk where it does its magic.

Part of that magic is this 15GB partition that is now full.

So, even thin provisioned, isn't that space already taken? (AKA once the data fills it, it still uses it even if the data is deleted, correct?)

Sure it is, but after you copy that data to the new drive, you'll delete it from the old drive making it empty... Assuming XS can reclaim now empty space, you'll gain that 15 GB back.

Yes, in reality this is all that you do. Make a new one, remove the old.

But in this particular appliance scenario, that is not possible, correct?

I don't have the appliance in front of me, are there not separate disks for these things?

No, the OVA imports a single disk with 2 LV's.

That's bad design.

travisdh1

@scottalanmiller said in BRRABill's Field Report With Linux:

@DustinB3403 said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@Dashrender said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@DustinB3403 said in BRRABill's Field Report With Linux:

@BRRABill Adding a second drive to a VM is literally nothing though.

It would be better practice to add a drive, than to try and extend the existing one.

But in theory, that 15G partition is part of the 19.5GB VHD the GrayLog appliance sets up.

You're losing the 15G, right?

I know 15G isn't much, but I was just thinking for future reference, if it was more than 15G.

Losing 15GB? Not if you are thin provisioned.

Well, as of right now, this is how things rolled...

Imported the GrayLog OVA appliance to XS.

It creates a 19.5GB virtual disk where it does its magic.

Part of that magic is this 15GB partition that is now full.

So, even thin provisioned, isn't that space already taken? (AKA once the data fills it, it still uses it even if the data is deleted, correct?)

Sure it is, but after you copy that data to the new drive, you'll delete it from the old drive making it empty... Assuming XS can reclaim now empty space, you'll gain that 15 GB back.

Yes, in reality this is all that you do. Make a new one, remove the old.

But in this particular appliance scenario, that is not possible, correct?

I don't have the appliance in front of me, are there not separate disks for these things?

No, the OVA imports a single disk with 2 LV's.

That's bad design.

How XS works storage has made me question what in the world they're doing often times.

My particular problem here is that I probably messed up when importing the local storage.

DustinB3403

No there is no way to have messed up importing the "appliance".

They provide it as a single OVA file, the boots directly to a login prompt.

momurda

Doesn't ova use 1 disk automatically? If you export/import a vm as ova that contains multiple disks (c,d,e) they all are aggregated as 1 ova file during the export/import. I thin it says so on XS and VMWare when setting up these operations. ovf will give you different disks for you

DustinB3403

@momurda even if it does, this would be GrayLog's issue for building an OVA file, rather than an OVF.

But I'm not certain of that, maybe.

momurda

Yes, I agree. You could just increase the ova size then extend the logging lvm to give yourself more free space.

stacksofplates

@DustinB3403 said in BRRABill's Field Report With Linux:

@momurda even if it does, this would be GrayLog's issue for building an OVA file, rather than an OVF.

But I'm not certain of that, maybe.

The OVF is inside of the OVA. If you want multiple disks, you can just edit the OVF.

stacksofplates

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@DustinB3403 said in BRRABill's Field Report With Linux:

@BRRABill Adding a second drive to a VM is literally nothing though.

It would be better practice to add a drive, than to try and extend the existing one.

But in theory, that 15G partition is part of the 19.5GB VHD the GrayLog appliance sets up.

You're losing the 15G, right?

I know 15G isn't much, but I was just thinking for future reference, if it was more than 15G.

Losing 15GB? Not if you are thin provisioned.

Well, as of right now, this is how things rolled...

Imported the GrayLog OVA appliance to XS.

It creates a 19.5GB virtual disk where it does its magic.

Part of that magic is this 15GB partition that is now full.

So, even thin provisioned, isn't that space already taken? (AKA once the data fills it, it still uses it even if the data is deleted, correct?)

Oh sorry, yes. Don't use appliances, build your own with proper specs

It handles quite a bit for what it is. I used it specifically to test what one server would handle. I changed it to 8 GB RAM and 2x6 CPUs. We're hammering it with around 60-70 million messages per day and it doesn't even blink. I did have to up the journal size, but other than that it's pretty amazing what it's doing.

At some point I'm going to build a cluster because searching a string over everything takes around 10 seconds, but it's going strong.

Dashrender

@stacksofplates said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@DustinB3403 said in BRRABill's Field Report With Linux:

@BRRABill Adding a second drive to a VM is literally nothing though.

It would be better practice to add a drive, than to try and extend the existing one.

But in theory, that 15G partition is part of the 19.5GB VHD the GrayLog appliance sets up.

You're losing the 15G, right?

I know 15G isn't much, but I was just thinking for future reference, if it was more than 15G.

Losing 15GB? Not if you are thin provisioned.

Well, as of right now, this is how things rolled...

Imported the GrayLog OVA appliance to XS.

It creates a 19.5GB virtual disk where it does its magic.

Part of that magic is this 15GB partition that is now full.

So, even thin provisioned, isn't that space already taken? (AKA once the data fills it, it still uses it even if the data is deleted, correct?)

Oh sorry, yes. Don't use appliances, build your own with proper specs

It handles quite a bit for what it is. I used it specifically to test what one server would handle. I changed it to 8 GB RAM and 2x6 CPUs. We're hammering it with around 60-70 million messages per day and it doesn't even blink. I did have to up the journal size, but other than that it's pretty amazing what it's doing.

At some point I'm going to build a cluster because searching a string over everything takes around 10 seconds, but it's going strong.

LOL - 10 seconds. what's the business case for putting more money into making the log searches faster? I'm sure there is one, I'm just curious.

stacksofplates

@Dashrender said in BRRABill's Field Report With Linux:

@stacksofplates said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@DustinB3403 said in BRRABill's Field Report With Linux:

@BRRABill Adding a second drive to a VM is literally nothing though.

It would be better practice to add a drive, than to try and extend the existing one.

But in theory, that 15G partition is part of the 19.5GB VHD the GrayLog appliance sets up.

You're losing the 15G, right?

I know 15G isn't much, but I was just thinking for future reference, if it was more than 15G.

Losing 15GB? Not if you are thin provisioned.

Well, as of right now, this is how things rolled...

Imported the GrayLog OVA appliance to XS.

It creates a 19.5GB virtual disk where it does its magic.

Part of that magic is this 15GB partition that is now full.

So, even thin provisioned, isn't that space already taken? (AKA once the data fills it, it still uses it even if the data is deleted, correct?)

Oh sorry, yes. Don't use appliances, build your own with proper specs

It handles quite a bit for what it is. I used it specifically to test what one server would handle. I changed it to 8 GB RAM and 2x6 CPUs. We're hammering it with around 60-70 million messages per day and it doesn't even blink. I did have to up the journal size, but other than that it's pretty amazing what it's doing.

At some point I'm going to build a cluster because searching a string over everything takes around 10 seconds, but it's going strong.

LOL - 10 seconds. what's the business case for putting more money into making the log searches faster? I'm sure there is one, I'm just curious.

It's going to get much bigger. That was only over around 200 million messages. We have to keep a years worth, so unless I close old indices and manually open them again, it's going to end up taking a while.

Closing them may be the way to go, but their interface only has an option to do one action after a period of time. I might have to set up a cron job with an API call for that.

scottalanmiller

@Dashrender said in BRRABill's Field Report With Linux:

@stacksofplates said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@scottalanmiller said in BRRABill's Field Report With Linux:

@BRRABill said in BRRABill's Field Report With Linux:

@DustinB3403 said in BRRABill's Field Report With Linux:

@BRRABill Adding a second drive to a VM is literally nothing though.

It would be better practice to add a drive, than to try and extend the existing one.

But in theory, that 15G partition is part of the 19.5GB VHD the GrayLog appliance sets up.

You're losing the 15G, right?

I know 15G isn't much, but I was just thinking for future reference, if it was more than 15G.

Losing 15GB? Not if you are thin provisioned.

Well, as of right now, this is how things rolled...

Imported the GrayLog OVA appliance to XS.

It creates a 19.5GB virtual disk where it does its magic.

Part of that magic is this 15GB partition that is now full.

So, even thin provisioned, isn't that space already taken? (AKA once the data fills it, it still uses it even if the data is deleted, correct?)

Oh sorry, yes. Don't use appliances, build your own with proper specs

It handles quite a bit for what it is. I used it specifically to test what one server would handle. I changed it to 8 GB RAM and 2x6 CPUs. We're hammering it with around 60-70 million messages per day and it doesn't even blink. I did have to up the journal size, but other than that it's pretty amazing what it's doing.

At some point I'm going to build a cluster because searching a string over everything takes around 10 seconds, but it's going strong.

LOL - 10 seconds. what's the business case for putting more money into making the log searches faster? I'm sure there is one, I'm just curious.

Same as anywhere else. If you are waiting around for ten seconds for every little log view and you do that with any regularity that is tons of time wasted. And if you need those logs for triage, that might equate to downtime.

Consider if you do 100 log searches a day (not necessarily from one person) that's 1,000 seconds. That's 17 minutes of people just sitting around waiting each day. But it's far worse than that. Ten seconds starts to disrupt your thinking. A ten second wait on a log might turn into distraction. It might be 30 minutes of lost productivity.

If your team is $50K each, that's about $15 lost per day or $3,000 annually. Magnify that if you are more distracted, earn over $50K, have lost productivity from the wait, have an impact to triage or do over 100 log lookups per day.

DustinB3403

Speaking of distractions I'm running a dell diag on my host1 before setting it backup for production use.

It was acting funky, xByte and Dell Support were wonderful with getting things squared away, but this was one thing I wanted to get completed and didn't.

Doing it now via iDrac connection. Out of band management is freaking awesome!

DustinB3403

And I've got an error, now to investigate.

DustinB3403

And it's a warning more than an error stating the logs haven't been checked.