Free Mind Systems

I’m sure many people think the cloud is the future, and more or less have migration plannings. Also, some of these people point to Amazon as supplier for their price, stability and capabilities. This first post of EC2 are things you need to know from the standpoint of a sysadmin.

For a System Administrator, Amazon EC2 is a big challenge,  mostly because their ephemeral nature:

Storage:

• When you starts and instance (from and image or AMI) EC2 attach a disk in /mnt, the size depends of the instance type (150 GB for the small instance type)

adminsis@ec2instance:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             9.9G  2.6G  6.9G  27% /
tmpfs                 854M     0  854M   0% /lib/init/rw
varrun                854M   64K  854M   1% /var/run
varlock               854M     0  854M   0% /var/lock
udev                  854M   80K  854M   1% /dev
tmpfs                 854M     0  854M   0% /dev/shm
/dev/sda2             147G  188M  140G   1% /mnt

• All the default storage is volatil, as the overall instance. you can reboot the instance but if you shutdown, all the modifications will be lost.
• Use EBS (Elastic Block Store) to create persistent storage for your Instances (also you will gain with snapshoots of the file system)

## Create Volume new EBS volume of 5GB in Europe)
#ec2-create-volume [--size <code>size | --snapshot snapshot] --availability-zone zone
ec2-create-volume -s 5 -z eu-west-1b
## Attach it on a device (/dev/sdf) on your instance:
ec2-attach-volume vol-aaaaaa -i i-bbbbbb -d /dev/sdf
## Create the file system
mkfs.ext3 /dev/sdf
## And mount
mkdir /srv
mount -t ext3 /dev/sdf /srv

• Now you will have and persistent storage on /srv that will survive to instance live. If your instances goes down, launch other, associate the volume to this instance and mount it.

adminsis@ec2instance:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             9.9G  2.6G  6.9G  27% /
tmpfs                 854M     0  854M   0% /lib/init/rw
varrun                854M   64K  854M   1% /var/run
varlock               854M     0  854M   0% /var/lock
udev                  854M   80K  854M   1% /dev
tmpfs                 854M     0  854M   0% /dev/shm
/dev/sda2             147G  188M  140G   1% /mnt
/dev/sdf              5.0G  1.2G  3.6G  24% /srv

Networking

• Each instance have only a network interface (eth0) configured by dhcp in the Amazon “LAN”.
• This ethernet is NAT’ed to an public dhcp address (dns-based)
• This NAT is firewalled with Security Groups, by default all traffic is closed and you open the necessary ports (tcp, udp, icmp) in groups of rules that can be reutilized.
• The Amazon DNS assings a public DNS name to each intance, and:
  • Called outside the EC2 infrastructure will return the public ip
  • Called inside the EC2 infrastructure will return the private ip
• Elastic ip’s are public fixed ip addresses that can be assigned to instances. The current IPv4 limitations
  make amazon decided to limit the number of Elastic ip’s to 5.

If your application needs a fixed configuration, ex. webservers database servers names or dhcp with a non-controlled dns server is not for you, you can install a dns server and autoregister the instances on boot.

#!/bin/bash
cat << EOF | /usr/bin/nsupdate -k Kyourdomain.com.+157+46088.private -v
server 127.0.0.1
zone yourdomain.com
update delete $1 A
update add $1 60 A $2
show
send
EOF

Parametrized Launches

When you creates an instance, you can pass parameters that you can programmatically retrieve on instance startup to make bootstraping, configuration, deploy, … task.

In this article from amazon you can follow this great step by step tutorial, by example you can:

Instance launch with an example parameter

ec2-run-instances $AMI_ID --region eu-west-1 -n 1 -g $SEC_GROUP -t $INSTANCE_TYPE -k $KEY -z eu-west-1b -d webserver

Note the -d parameter “webserver”, when the instance has started you can call the Amazon EC2 webservice as follows:

#curl http://169.254.169.254/1.0/user-data
--- webserver

You can think that this is good but not “impressive”, but without this the instance autoconfiguration is a lot harder to implement, think as the begin of the history:

• Instance Launch (parameters, ex. instance role on your system, config files to execute, …)
• Have an EC2 AMI that on startup (init.d/rc.local) read the parameters and starts the autoconfiguration
• Automate everything

Also you allways can get the instance metadata from the EC2 webservices, from the instance you can by example:

LOCAL_IP=`curl http://169.254.169.254/latest/meta-data/local-ipv4`
echo "The local ip of this instance is "$LOCAL_IP
PUBLIC_IP=`curl http://169.254.169.254/latest/meta-data/public-ipv4`
echo "The public ip of this instance is "$PUBLIC_IP
echo "This is the list of metas"
curl http://169.254.169.254/latest/meta-data/
ami-id
ami-launch-index
ami-manifest-path
ancestor-ami-ids
block-device-mapping/
hostname
instance-action
instance-id
instance-type
kernel-id
local-hostname
local-ipv4
placement/
public-hostname
public-ipv4
public-keys/
ramdisk-id
reservation-id
security-groups

Soon I will write some posts on the automation arena and EC2.