Getting started with automated provisioning?

scottalanmiller

@biggen said in Getting started with automated provisioning?:

Just trying to wrap my head around the first steps, I understand how to install a Salt Master/Minion. I don’t understand how to get a VM provisioned BEFORE that portion in a somewhat automated fashion.

That's the trick, you don't. You just need to have the thing that holds the VMs also be managed from Salt / Ansible or whatever. In this case, the Xen host.

biggen

@scottalanmiller said in Getting started with automated provisioning?:

@biggen said in Getting started with automated provisioning?:

Just trying to wrap my head around the first steps, I understand how to install a Salt Master/Minion. I don’t understand how to get a VM provisioned BEFORE that portion in a somewhat automated fashion.

That's the trick, you don't. You just need to have the thing that holds the VMs also be managed from Salt / Ansible or whatever. In this case, the Xen host.

Ahh ok. That makes sense. So does the Xen host need some type of API running under the hood? Does something like this exist for xcp-ng?

scottalanmiller

@biggen said in Getting started with automated provisioning?:

@scottalanmiller said in Getting started with automated provisioning?:

@biggen said in Getting started with automated provisioning?:

Just trying to wrap my head around the first steps, I understand how to install a Salt Master/Minion. I don’t understand how to get a VM provisioned BEFORE that portion in a somewhat automated fashion.

That's the trick, you don't. You just need to have the thing that holds the VMs also be managed from Salt / Ansible or whatever. In this case, the Xen host.

Ahh ok. That makes sense. So does the Xen host need some type of API running under the hood? Does something like this exist for xcp-ng?

It needs an API if you are going to connect from the outside, rather than having an agent on it. Ansible pushes an agent over SSH when needed, Salt has an agent always installed (by default.)

scottalanmiller

Terraform is very popular for infrastructure level provisioning, too.

https://www.terraform.io/

scottalanmiller

@biggen said in Getting started with automated provisioning?:

So does the Xen host need some type of API running under the hood?

XCP-NG was designed from the beginning for this.

https://xcp-ng.org/forum/topic/1150/terraform-provider-for-xen-orchestra

biggen

@scottalanmiller Excellent! Thanks Scott!

It may be more than I really need to mess with since I don't really create many new VMs but I wanted to take some time to learn it at any rate. But I keep hearing all these "buzz words" like cloud-init, Salt, Packer, Anisble, etc... and I fell like I'm being left behind with not learning these new tools.

biggen

It looks like the Terraform plugin for xcp-ng is experimental is not well/at all supported so that might be a bust unfortunately.

I'll do some more looking around.

scottalanmiller

@biggen said in Getting started with automated provisioning?:

It looks like the Terraform plugin for xcp-ng is experimental is not well/at all supported so that might be a bust unfortunately.

I'll do some more looking around.

Might be an overall problem with XCP-NG. Small user base.

biggen

@scottalanmiller So I found out that xcp-ng does actually have an API for configuration/management of VMs via Xen Orchestra. Its called xo-cli.

What would be your preferred method of wanting to automate VM configuration Scott whilst utilizing the XO API? I mean, do I simply just put the commands into a Bash script?

scottalanmiller

@biggen said in Getting started with automated provisioning?:

What would be your preferred method of wanting to automate VM configuration Scott whilst utilizing the XO API? I mean, do I simply just put the commands into a Bash script?

Certainly can be that simple, yes.

1337

Go with xe instead. That is the native command line tool for xenserver/xcp-ng hosts and you run it on your host (or remotely).

Easiest is probably to create new VMs from a template.

If you want to automate debian install from scratch you need a preseed file.

biggen

@Pete-S Thank you sir. Yeah, I've been playing around with xo-cli but documentation is really lacking. I'd imagine there are tons of examples using the built in xe tool I could find.

1337

I'm also looking to create VMs automagically and have the same setup (xcp-ng hosts, debian VMs).
Unfortunately I haven't had time yet so right now I just clone a base install and go from there.

BTW, I picked up the ebook Ansible for DevOps on leanpub for the Ansible part.

1337

@biggen said in Getting started with automated provisioning?:

I'd imagine there are tons of examples using the built in xe tool I could find.

Yes, just look at Citrix documentation for xenserver. It's xe vm-install.

PS.

The pdf called Xenserver Virtual Machine Users Guide has the info you need in one place.
Look at Chapter 5. Creating Linux VMs
https://docs.citrix.com/en-us/legacy-archive/downloads/xs-vm-users-guide-7-5.pdf

For how to do thing on the host itself, it's the Administrators Guide pdf you want.
https://docs.citrix.com/en-us/xenserver/7-1/downloads/administrators-guide.pdf

biggen

@Pete-S I may decide to work backwards on this. Learn the Ansible/Salt/Puppet/whatever part first. Get that down and then learn VM provisioning.

Although, cloning a base install sure sounds easier. How do you deal with hostname/MAC address conflicts with the clones?

I’ll check out that book. Thanks for the help!

1337

@biggen said in Getting started with automated provisioning?:

@Pete-S I may decide to work backwards on this. Learn the Ansible/Salt/Puppet/whatever part first. Get that down and then learn VM provisioning.

Although, cloning a base install sure sounds easier. How do you deal with hostname/MAC address conflicts with the clones?

I’ll check out that book. Thanks for the help!

I just clone the VM in xencenter (Copy VM) and it takes care of the VM name conflict and the MAC address conflict automatically.

The hostname inside the VM and then installing whatever additional stuff you need, has to be taken care of inside the VM after it has been booted up.

1337

I just checked and you have two options to make a new VM with xe.

vm-clone (which makes a fast clone without actually copying the entire VM)
vm-copy (which makes a new VM by copying the old)

It takes care of the mac addr conflict automatically.

To try it run this on the host:

xe vm-clone new-name-label="clone" vm="base-install"

base-install would be the name of the VM you want to clone.
clone is the name of the new VM.

It's as easy as that.

If you want to make a full copy instead, just use vm-copy instead of vm-clone with the same options.

To start the VM after you created it:

xe vm-start vm="clone"

To shutdown the VM gracefully:

xe vm-shutdown vm="clone"

To remove the VM completely:

 xe vm-uninstall vm="clone" force=true

biggen

@Pete-S This is so helpful! Thank you very much.

I’m not sure the difference between a clone and a copy. I’ll look that up.

This seems much easier than having to create a vm from scratch using xo-cli. I guess using the xe commands means I’m running these directly on the host preferable from a Bash script when I get them working how I like?

1337

@biggen said in Getting started with automated provisioning?:

@Pete-S This is so helpful! Thank you very much.

I’m not sure the difference between a clone and a copy. I’ll look that up.

This seems much easier than having to create a vm from scratch using xo-cli. I guess using the xe commands means I’m running these directly on the host preferable from a Bash script when I get them working how I like?

Hey, you're welcome. Just happy to help out a colleague. Post your progress and we can all share!

You can run the command on the host over ssh or directly on it physically or inside xencenter aka xcp-ng center under the Console tab.

You can also run it on another machine if you have xe installed locally.
Using xe -s <hostname_or_ip> -u <username> -pw <password> whatever_commands_you_want_to_run
I don't know how to install xe on a generic server though. But you can use this command if you have a several independent xcp-ng hosts.

1337

@biggen said in Getting started with automated provisioning?:

I’m not sure the difference between a clone and a copy. I’ll look that up.

It's just how the storage is handled.

Say that you have a debian install that uses 10GB. If you make a clone, the clone will also use the same 10GB of storage and only whatever blocks that it changes are stored separately. So if you make 9 clones maybe the entire 10 VMs will use a total of 11GB or something.

If you make a copy, the storage from the original VM is copied into a new but mostly identical 10GB. And the second another 10GB and so on. With a total of 10 VMs the total storage used are 100GB.

Clones are more frugal on storage but has more overhead so might be a little slower.
Clones use the same storage mechanism as snapshots while copied VMs are just a copy.

Practically speaking it's faster to work with clones but if you're cloning small VMs like minimal installs and have SSD disks the difference in time is small.