This is my setup of running LXC Linux Containers on my workstation Ubuntu 10.04 x86_64 and wanting to run test servers running CentOS 5 x86_64. Runs very well in this configuration because of the container base setup instead of full virtualization like VMware, Virtualbox and KVM.

Here are the steps I took to get it setup and some of the problems I ran into.

Create control group filesystem as the root user.
# mkdir -p /cgroup
# mount none -t cgroup /cgroup

Add to fstab so it will be there after a reboot.
# vi /etc/fstab and add line below
none /cgroup cgroup defaults 0 0

Applications to install.
# apt-get install libvirt-bin debootstrap

Download and install
http://lxc.sourceforge.net/ and download current LXC, I used lxc_0.7.2-1_amd64.deb
# dpkg -i lxc_0.7.2-1_amd64.deb

Reason I installed this version is because I was having small problems with the version that came with Ubuntu 10.04 which this version fixed up.

Also I tried bridge network which didn't work in my configuration. Everytime I loaded the container my host network would stop working but the container network had no problems, I think it is something to do with the broadcast address not being setup properly. Libvirt-bin virbr0 NAT networking is working well so I worked with that.

Working directory for container client files is /var/lib/lxc, create if needed.

Now I created a working directory in my home directory and change into it.
# exit if you are still logged in as root
$ mkdir $HOME/LXC
$ cd LXC

You need a Container image Operating System and I choose CentOS 5 x64_64
$ wget http://download.openvz.org/template/precreated/centos-5-x86_64.tar.gz

Extract out the files.
$ tar -zxvf centos-5-x86_64.tar.gz

Change into the directory.
$ cd Centos5-x86_64/

Now there is some work to get this client system ready for LXC.

Clean up some errors when I run shutdown -h now in the container.
$ vi etc/rc.d/rc0.d/S01halt Notice I am working in the directory so I am missing the first forward slash.

Search for swapoff and then rem out all stuff about swapoff like below.
# Turn off swap, then unmount file systems.
#[ -f /proc/swaps ] && SWAPS=`awk '! /^Filename/ { print $1 }' /proc/swaps`
#if [ -n "$SWAPS" ]; then
#    action $"Turning off swap: " swapoff $SWAPS
#    for dst in $SWAPS; do
#       if [[ "$dst" =~ "^/dev/mapper"]]
#           && [ "$(dmsetup status "$dst" | cut -d ' ' -f 3)" = crypt ]; then
#           backdev=$(/sbin/cryptsetup status "$dst"
#               | awk '$1 == "device:" { print $2 }')
#           /sbin/cryptsetup remove "$dst"
#       fi
#   done
#fi

Search for Syncing hardware
#[ -x /sbin/hwclock ] && action $"Syncing hardware clock to system time" /sbin/hwclock $CLOCKFLAGS

Search for quota
#[ -x /sbin/quotaoff ] && action $"Turning off quotas: " /sbin/quotaoff -aug

Save changes.

Next lets fix the timezone.
$ rm -f etc/localtime
$ cp usr/share/zoneinfo/YourTime/Zone etc/localtime

SSH Setup for faster connection etc.
$ vi etc/ssh/sshd_config remove everything and then add below.
Port 22
Protocol 2
HostKey /etc/ssh/ssh_host_key
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key
SyslogFacility AUTHPRIV
LogLevel INFO
LoginGraceTime 120
PermitRootLogin yes
StrictModes yes
UseDNS no
IgnoreRhosts yes
IgnoreUserKnownHosts yes
PermitEmptyPasswords no
X11Forwarding no
PrintMotd yes
UsePrivilegeSeparation yes
Subsystem sftp /usr/libexec/openssh/sftp-server

Add service you want to load at boot up and network configuration, I had problems with the documentation about this, broadcast address was all 0's or 192.168.122.0 instead of what it should be, so here is how I fixed it.

If you run a ifconfig you will see a virbr0 interface, this is going to be your gateway and I just picked a ip address to use.

$ vi etc/rc.local
ifconfig eth0 192.168.122.51 netmask 255.255.255.0 broadcast 192.168.122.255 up
route add default gw 192.168.122.1 eth0


Disable IPV6
$ vi etc/modprobe.d/blacklist-compat and add below at the bottom of file.
blacklist ipv6
blacklist net-pf-10

Disable some iptables stuff
$ vi etc/sysconfig/iptables-config
IPTABLES_MODULES=""
IPTABLES_MODULES_UNLOAD="no"

Edit inittab and rem out of remove everything and then add below.
$ vi etc/inittab
id:3:initdefault:
si::bootwait:/etc/init.d/boot
l0:0:wait:/etc/init.d/rc 0
l1:1:wait:/etc/init.d/rc 1
l2:2:wait:/etc/init.d/rc 2
l3:3:wait:/etc/init.d/rc 3
l6:6:wait:/etc/init.d/rc 6
ls:S:wait:/etc/init.d/rc S
~~:S:respawn:/sbin/sulogin
p6::ctrlaltdel:/sbin/init 6
p0::powerfail:/sbin/init 0
cons:2345:respawn:/sbin/mingetty --noclear console screen

Create a boot file
$ vi etc/init.d/boot and add information below.
#! /bin/bash
rm -f /etc/mtab /etc/init.d/boot.d/*.udev* /etc/init.d/boot.udev* /var/run/*.{pid,lock}
touch /etc/mtab /fastboot
exit 0

Add a name server
$ vi etc/resolv.conf and add below
nameserver 192.168.1.1 or whatever your DNS servers are.

Remove a repo for OpenVZ
$ rm etc/yum.repo/vz.repo

OK, now we are at the point to create your first Container configuration files.

$ sudo su -
# vi /etc/lxc/lxc-centos.conf and add below
lxc.utsname = centos
lxc.tty = 4
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = virbr0
lxc.network.name = eth0
lxc.network.mtu = 1500
# lxc.network.hwaddr = after first time ran take mac add and put here, see example below.
lxc.network.hwaddr = 4a:49:43:49:79:bd
lxc.network.ipv4 = 0.0.0.0
lxc.rootfs = /var/lxc/centos
lxc.mount = /var/lxc/centos.fstab

# vi /etc/lxc/lxc-centos.fstab and add information below
none /var/lib/lxc/centos/rootfs/dev/pts devpts defaults 0 0
none /var/lib/lxc/centos/rootfs/proc proc defaults 0 0
none /var/lib/lxc/centos/rootfs/sys sysfs defaults 0 0


# Create a container
# lxc-create -f /path/to/{CONTAINER_NAME}/main/config/file -n {CONTAINER_NAME}
# lxc-create -f /etc/lxc/lxc-centos.conf -n centos

Create a rootfs directory
# mkdir /var/lib/lxc/centos/rootfs

Copy files you worked on earlier.
# cp -r /home/yourhomeidirectory/LXC/Centos5-x86_64/* /var/lib/lxc/{CONTAINER_NAME}/rootfs
# cp -r /home/yourhomeidirectory/LXC/Centos5-x86_64/* /var/lib/lxc/centos/rootfs

Create this file for creating and setup default devices for your containers.
# vi  /usr/local/bin/lxc-config and add information below, also only needs to be run once for every new container you create.
Go to bottom of page for the file and download Support File, which I have already created or go to this website.
http://blog.bodhizazen.net/linux/lxc-configure-fedora-containers/

Change into rootfs of container and run lxc-config
# cd /var/lib/lxc/CONTAINER_NAME}/rootfs/
# cd /var/lib/lxc/centos/rootfs/
# /usr/local/bin/lxc-config # fix /dev

Chrooting into the rootfs file system for centos
# chroot /var/lib/lxc/centos/rootfs/

Setup root password for your container.
# passwd root

Start or Remove some services from running
# chkconfig sshd on
# chkconfig httpd on
# chkconfig xinetd off
# chkconfig sendmail off

When finish type exit to finish chroot session.

Destroy container and files
# lxc-destroy -n {CONTAINER_NAME}
# lxc-destroy -n centos

Start container
# lxc-start -n {CONTAINER_NAME} init
# lxc-start -n centos init

Stop container
# lxc-stop -n {CONTAINER_NAME}
# lxc-stop -n centos

Connect to tty console
# lxc-console -n centos -t 1 doesn't seem to work for me.

Check your configuration
# lxc-checkconfig

List containers
# lxc-ls
If you receive this error lxc-ls warning, got bogus unix line, reinstall uml-utiilities
# sudo apt-get --reinstall install uml-utilities

Is it RUNNING or STOPPED
# lxc-info -n {CONTAINER_NAME}
# lxc-info -n centos

Ran lxc-tab key for more commands


All the websites I used to set this up.
http://lxc.sourceforge.net/
http://jkyamog.blogspot.com/2010/07/lxc-on-ubuntu-1004-lucid-lynx.html
http://blog.bodhizazen.net/linux/lxc-configure-fedora-containers/
http://wiki.openvz.org/Download/template/precreated
http://blogs.techrepublic.com.com/opensource/?p=1299
http://lxc.teegra.net/#_linux_containers