# Using libvirt¶

I (nwf) highly suggest against it if at all possible. What a pain in the rump. You’ll probably be spending a lot of time on over here learning how to write domain files. Here’s a collection of site-specific notes. The only reason it’s here is that this turns out to be the best easy-and-mostly-standard way of wrangling QEMUs into running at boot for the world’s benefit, which unfortunately we must do in order for afs to be reasonably manageable. Use virsh autostart ${DOMAIN} once you’ve got the domain created, to make that auto-run happen. ## Debugging XML¶ Egads this is painful. You can get a good/no-good bit result out of the tool virt-xml-validate, but you should ignore the error messages it prints, most of the time, as it is a remarkably poor producer. I suggest the usual divide-and-conquer approach of commenting out half the file and narrowing from there. You can also compare a round-trip through libvirt: use virsh define${YOUR_XML_DOCUMENT} and virsh dumpxml ${YOUR_DOMAIN_NAME}. ## Using Ceph¶ http://ceph.com/docs/master/rbd/libvirt/ will tell you what’s up, but for ease of administration, here are the steps: • You’ll want a ceph principal for your host; may I suggest client.$(HOSTNAME).libvirt. Go use Creating a New Ceph User to make one and extract its key. For caps, you’ll probably want something like:

"osd" "allow class-read object_prefix rbd_children, \
allow rwx pool=volumes_2, allow rw pool=images_2,
allow rwx pool=scratch"
"mon" "allow r"


If you’re standing up an AFS server using Ceph partitions, you probably need to augment those pool lists.

• You need to register this secret with libvirt. Create an XML file (of course):

cat >virsh-secret.xml <<HERE
<secret ephemeral='no' private='yes'>
<usage type='ceph'>
<name>client.$(HOSTNAME).libvirt secret</name> </usage> </secret> HERE  Then, virsh secret-define --file virsh-secret.xml which will print out a UUID for you, which you can also retrieve with virsh secret-list. Then virsh secret-set-value${UUID} --base64 ${EXTRACTED_KEY}. Then rm${EXTRACTED_KEY} virsh-secret.xml.

• Now you need to actually use that key with your RBD stanzas in domain XML. Try something like:

<disk type='network' device='disk'>
<driver name='qemu' type='raw' cache='writeback'/>
<secret type='ceph' usage='client.magellan.libvirt secret'/>
</auth>
<source protocol='rbd' name='...'/>
<target dev='vd...' bus='virtio'/>
</disk>


It appears that the virtio drivers are more robustly tested with Ceph than, for example, the IDE drivers. We have had I/O timeouts and guest kernel panics with the latter but have not seen them with the former. Modern QEMU BIOSes are up to snuff enough to boot off of virtio devices.

## Serial Console¶

<serial type="pty"><target port="0"/></serial>
<console type="pty"><target type="serial" port="0"/></console>


And on the guest, follow the instructions in Linux Serial Console. Then you can use virsh console \${DOMAIN} once it’s up and running to attach to the console from the host.

Note

It’d be better if we could run -curses mode inside screen or something, but this is not to be, so here we are. This probably means that you should do the install of a libvirt guest separately from the libvirt side of things, which necessitates learning not only how to write a domain file but how to write QEMU arguments directly.

## Libvirt Events¶

Your domain file should probably contain these hooks:

<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<on_lockfailure>poweroff</on_lockfailure>