Getting started with automated provisioning?

biggen

I’d like to get started learning some automated processes when it comes to spinning up a VM. I have a single host xcp-ng environment. Right now, I create VMs the hard way: Sit through the entire OS install menu (usually use Debian) clicking away.

I only have a very high level abstract understanding of Salt/Ansible. I understand that you can create playbooks to have the target system install/configure services you define in the playbook. However, how do you actually create the VM first?

What would be the first steps I need to learn to actually provision the VM? I mean, I guess I could simply clone the base/virgin OS VM after an initial install. But I’d have to change hostnames, ips, etc... for the new VM I cloned.

Just trying to wrap my head around the first steps, I understand how to install a Salt Master/Minion. I don’t understand how to get a VM provisioned BEFORE that portion in a somewhat automated fashion.

scottalanmiller

@biggen said in Getting started with automated provisioning?:

However, how do you actually create the VM first?

By automating the underlying platform.

Example:

I want to install Apache. To do so I automate the underlying platform: Debian.

I want to install Debian. To do so I automate the underlying platform: XCP-NG.

scottalanmiller

@biggen said in Getting started with automated provisioning?:

Just trying to wrap my head around the first steps, I understand how to install a Salt Master/Minion. I don’t understand how to get a VM provisioned BEFORE that portion in a somewhat automated fashion.

That's the trick, you don't. You just need to have the thing that holds the VMs also be managed from Salt / Ansible or whatever. In this case, the Xen host.

biggen

@scottalanmiller said in Getting started with automated provisioning?:

@biggen said in Getting started with automated provisioning?:

Just trying to wrap my head around the first steps, I understand how to install a Salt Master/Minion. I don’t understand how to get a VM provisioned BEFORE that portion in a somewhat automated fashion.

That's the trick, you don't. You just need to have the thing that holds the VMs also be managed from Salt / Ansible or whatever. In this case, the Xen host.

Ahh ok. That makes sense. So does the Xen host need some type of API running under the hood? Does something like this exist for xcp-ng?

scottalanmiller

@biggen said in Getting started with automated provisioning?:

@scottalanmiller said in Getting started with automated provisioning?:

@biggen said in Getting started with automated provisioning?:

Just trying to wrap my head around the first steps, I understand how to install a Salt Master/Minion. I don’t understand how to get a VM provisioned BEFORE that portion in a somewhat automated fashion.

That's the trick, you don't. You just need to have the thing that holds the VMs also be managed from Salt / Ansible or whatever. In this case, the Xen host.

Ahh ok. That makes sense. So does the Xen host need some type of API running under the hood? Does something like this exist for xcp-ng?

It needs an API if you are going to connect from the outside, rather than having an agent on it. Ansible pushes an agent over SSH when needed, Salt has an agent always installed (by default.)

scottalanmiller

Terraform is very popular for infrastructure level provisioning, too.

https://www.terraform.io/

scottalanmiller

@biggen said in Getting started with automated provisioning?:

So does the Xen host need some type of API running under the hood?

XCP-NG was designed from the beginning for this.

https://xcp-ng.org/forum/topic/1150/terraform-provider-for-xen-orchestra

biggen

@scottalanmiller Excellent! Thanks Scott!

It may be more than I really need to mess with since I don't really create many new VMs but I wanted to take some time to learn it at any rate. But I keep hearing all these "buzz words" like cloud-init, Salt, Packer, Anisble, etc... and I fell like I'm being left behind with not learning these new tools.

biggen

It looks like the Terraform plugin for xcp-ng is experimental is not well/at all supported so that might be a bust unfortunately.

I'll do some more looking around.

scottalanmiller

@biggen said in Getting started with automated provisioning?:

It looks like the Terraform plugin for xcp-ng is experimental is not well/at all supported so that might be a bust unfortunately.

I'll do some more looking around.

Might be an overall problem with XCP-NG. Small user base.

biggen

@scottalanmiller So I found out that xcp-ng does actually have an API for configuration/management of VMs via Xen Orchestra. Its called xo-cli.

What would be your preferred method of wanting to automate VM configuration Scott whilst utilizing the XO API? I mean, do I simply just put the commands into a Bash script?

scottalanmiller

@biggen said in Getting started with automated provisioning?:

What would be your preferred method of wanting to automate VM configuration Scott whilst utilizing the XO API? I mean, do I simply just put the commands into a Bash script?

Certainly can be that simple, yes.

1337

Go with xe instead. That is the native command line tool for xenserver/xcp-ng hosts and you run it on your host (or remotely).

Easiest is probably to create new VMs from a template.

If you want to automate debian install from scratch you need a preseed file.

biggen

@Pete-S Thank you sir. Yeah, I've been playing around with xo-cli but documentation is really lacking. I'd imagine there are tons of examples using the built in xe tool I could find.

1337

I'm also looking to create VMs automagically and have the same setup (xcp-ng hosts, debian VMs).
Unfortunately I haven't had time yet so right now I just clone a base install and go from there.

BTW, I picked up the ebook Ansible for DevOps on leanpub for the Ansible part.

1337

@biggen said in Getting started with automated provisioning?:

I'd imagine there are tons of examples using the built in xe tool I could find.

Yes, just look at Citrix documentation for xenserver. It's xe vm-install.

PS.

The pdf called Xenserver Virtual Machine Users Guide has the info you need in one place.
Look at Chapter 5. Creating Linux VMs
https://docs.citrix.com/en-us/legacy-archive/downloads/xs-vm-users-guide-7-5.pdf

For how to do thing on the host itself, it's the Administrators Guide pdf you want.
https://docs.citrix.com/en-us/xenserver/7-1/downloads/administrators-guide.pdf

biggen

@Pete-S I may decide to work backwards on this. Learn the Ansible/Salt/Puppet/whatever part first. Get that down and then learn VM provisioning.

Although, cloning a base install sure sounds easier. How do you deal with hostname/MAC address conflicts with the clones?

I’ll check out that book. Thanks for the help!

1337

@biggen said in Getting started with automated provisioning?:

@Pete-S I may decide to work backwards on this. Learn the Ansible/Salt/Puppet/whatever part first. Get that down and then learn VM provisioning.

Although, cloning a base install sure sounds easier. How do you deal with hostname/MAC address conflicts with the clones?

I’ll check out that book. Thanks for the help!

I just clone the VM in xencenter (Copy VM) and it takes care of the VM name conflict and the MAC address conflict automatically.

The hostname inside the VM and then installing whatever additional stuff you need, has to be taken care of inside the VM after it has been booted up.

1337

I just checked and you have two options to make a new VM with xe.

vm-clone (which makes a fast clone without actually copying the entire VM)
vm-copy (which makes a new VM by copying the old)

It takes care of the mac addr conflict automatically.

To try it run this on the host:

xe vm-clone new-name-label="clone" vm="base-install"

base-install would be the name of the VM you want to clone.
clone is the name of the new VM.

It's as easy as that.

If you want to make a full copy instead, just use vm-copy instead of vm-clone with the same options.

To start the VM after you created it:

xe vm-start vm="clone"

To shutdown the VM gracefully:

xe vm-shutdown vm="clone"

To remove the VM completely:

 xe vm-uninstall vm="clone" force=true

biggen

@Pete-S This is so helpful! Thank you very much.

I’m not sure the difference between a clone and a copy. I’ll look that up.

This seems much easier than having to create a vm from scratch using xo-cli. I guess using the xe commands means I’m running these directly on the host preferable from a Bash script when I get them working how I like?