I’ve been working on a project that lets you quickly move systems between your private Xen implementation and Amazon’s EC2 service. There are a lot of hurdles to get this to work, and most of them are surrounding how Amazon doesn’t let you download a Kernel or Ramdisk image out of S3 unless you’re the owner. You can download someone elses image if you’ve saved it as your own but you still can’t download the kernel and ramdisk. Also, EC2 has specific requirements for how the image is built. Here’s how you can get your image out of Amazon and run it locally on your own Xen hypervisor. I will assume you are already using Amazon Web Services and have created an account. If you haven’t then sign up.

Amazon calls their instance images Amazon Machine Images or AMI’s. If you want to be able to grab one of the many images from Amazon you can use download the Amazon AMI tools and AWS tools then do the following. You can download the tools here

Find and Download the AMI


$: ec2-describe-images
IMAGE   ami-cc6386a5    ubuntu-hardy-ruby/image.manifest.xml    848278689040    available       private         i386    machine
IMAGE   ami-386c8951    ubuntu-ruby-lapack/image.manifest.xml   848278689040    available       private         i386    machine
$:

Fields 3 and 4 contain important information. For this example I’m listing the images that I own. Optionally you can provide a switch that will list all Amazon images by including ‘-a’ to the end of the ec2-describe-images command.

Field 3 is the unique identifier for the AMI, and field 4 is the bucket and AMI “manifest” - or a file that describes the AMI. Because users can specify the name of the manifest, you should pay attention to this value when trying to run the next set of commands.

The AMI tools from Amazon include a utility called ‘ec2-download-bundle’. This will download the manifest file from the bucket, parse through to see what other files it needs to download, then it will reassemble the AMI image locally, and check its signature. The AMI’s are encrypted in small (usually 10 meg) chunks. The signatures for those chunks are also included in the manifest.

To download the first AMI listed above, run the following commands


mkdir 'image-to-download'
cd 'image-to-download'
ec2-download-bundle --bucket ubuntu-ruby-lapack -m image.manifest.xml --access-key $AWS_ACCESS_KEY --secret-key $AWS_SECRET --privatekey $EC2_PRIVATE_KEY

That will start downloading the bundle to your local system.

Rebuild the AMI

Now we have to unbundle the files

ec2-unbundle -m image.manifest.xml -k $EC2_PRIVATE_KEY

This will decrypt and reassemble the image from all the individual components in the list

Now you have an image named ‘image’ in your directory. You can take a look at this file by mounting it

mkdir /mnt/image
mount -t ext3 -o loop image /mnt/image
cd /mnt/image

If you’re lucky there will be copies of the kernel and perhaps the ramdisk in the /boot partition. Otherwise you’ve got to do something really tricky : You have to guess as to what kernel will work the best. Thankfully we have a good understanding of what’s required to boot one of these images.

If you’ve created an image for Xen already then chances are your kernel will work just fine, but your ramdisk might need some adjusting. A trick you can use is to chroot to the /mnt/image folder, specify which modules you want loaded and rebuild the ramdisk - then exit the chroot, copy the kernel and ramdisk out of /mnt/image and you’ll have all the components you’ll need.

I know what you’re thinking: That’s a lot of work / guessing

You’re in luck. While there are a couple sites for sharing pre-built Xen images, the community is nowhere near as large as the Parallels or VMware ‘appliance’ sites. Jailtime.org has a hanful of images but they don’t follow any sort of standard, and the disk layouts / configurations aren’t compatible with Amazon’s EC2.

LayerBoom has a Xen image that is completely compatible with Amazons AMI format, and it can run in your own environment. This means you can copy a system into Amazon from your test environment without any hassle. It also works with the Eucalyptus project, and can be booted in xVM server as well (Instructions are coming)

Download the Xen package

url: http://layerboom.com/files/xen/images/centos52-20080930.tar.gz
md5: d54a83fc22f1ec052db6ebe3c258ee45

u/l :root/password