Skip to content

WIP Creating clusterware AMI images

Jump to bottom Edit New page
Vaughan Jones edited this page Jan 6, 2016 · 26 revisions

Create a basic CentOS 6/7 AMI with Alces Clusterware incorporated

##Provision a CentOS 6 AMI

Perform the following steps to create a simple cluster-capable AMI incorporating Alces Clusterware:

  • Create a blank instance using the "CentOS 6 (with updates) HVM" AMI
  • During the instance configuration wizard, create a secondary EBS volume and attach to /dev/xvdb

Run the following commands on the newly created CentOS 6 machine:

yum -y groupinstall "Development Tools"
yum -y update
yum -y install vim
export alces_OS=el6
curl -sL http://git.io/clusterware-installer | /bin/bash
PATH=/opt/clusterware/bin:$PATH
alces handler enable clusterable
alces handler enable clusterable-aws-compat
alces handler enable cluster-gridware
alces handler enable session-firewall
alces handler enable cluster-nfs
alces handler enable cluster-sge
alces handler enable taskable
alces session enable gnome
rm -rf /var/lib/cloud/
mkdir /var/lib/cloud
rm -fv /etc/ssh/ssh_host_*
rm -rf /home/centos/*
rm -rf /home/centos/.ssh
rm -rf /root/*
rm -rf /root/.ssh
cat << 'EOF' > /etc/cloud/templates/hosts.rhel.tmpl
# The following lines are desirable for IPv4 capable hosts
127.0.0.1 ${fqdn} ${hostname}
127.0.0.1 localhost.localdomain localhost
127.0.0.1 localhost4.localdomain4 localhost4

# The following lines are desirable for IPv6 capable hosts
::1 ${fqdn} ${hostname}
::1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
EOF
echo 'manage_etc_hosts: true' >> /etc/cloud/cloud.cfg
sed -e 's/name:\ centos/name:\ alces/g' \
    -e 's/gecos:\ Cloud\ User/gecos:\ Alces\ Administrator/g' \
    -i /etc/cloud/cloud.cfg
sed -e 's/^SELINUX=enforcing.*/SELINUX=disabled/g' -i /etc/selinux/config
sed -e 's/root\ alces/root/g' -i /opt/clusterware/etc/setup-sshkey.rc
chkconfig NetworkManager off
yum clean all
# prepare as publicly available AMI ( https://bugs.centos.org/view.php?id=6228#c20524 )
# check root partition is mounted on /dev/xvda and EBS vol /dev/xvdb
mkfs.ext4 /dev/xvdb 
dd bs=65536 if=/dev/xvda of=/dev/xvdb

This script is available as a gist which can be executed with the one-shot command:

curl https://gist.githubusercontent.com/mjtko/46da10e544e1611d296f/raw/provision.sh | bash

Finally, clear up all history:

history -c
logout

Once the instance has been shut down - perform the following actions from the EC2 console:

  • Detach both the root volume and EBS volume from the instance
  • Reattach the EBS volume and attach as /dev/sda1
  • From the Instances tab, select Create Image from your instance
  • From the AMIs tab, once creation has finished - modify the permissions to allow public access

##Provision a CentOS 7 AMI Perform the following steps to create a simple cluster-capable AMI incorporating Alces ClusterWare:

  • Create a blank instance using the "CentOS 7 (with updates) HVM" AMI
  • During the instance configuration wizard, create a secondary EBS volume and attach to /dev/xvdb

Run the following commands on the newly created CentOS 7 machine (as root):

yum -y groupinstall "Development Tools"
yum -y update
yum -y install vim
export alces_OS=el7
curl -sL http://git.io/clusterware-installer | /bin/bash
PATH=/opt/clusterware/bin:$PATH
alces handler enable clusterable
alces handler enable clusterable-aws-compat
alces handler enable cluster-gridware
alces handler enable session-firewall
alces handler enable cluster-nfs
alces handler enable cluster-sge
alces handler enable taskable
alces session enable gnome
rm -rf /var/lib/cloud/
mkdir /var/lib/cloud
rm -fv /etc/ssh/ssh_host_*
rm -rf /home/centos/*
rm -rf /home/centos/.ssh
rm -rf /root/*
rm -rf /root/.ssh
cat << 'EOF' > /etc/cloud/templates/hosts.rhel.tmpl
# The following lines are desirable for IPv4 capable hosts
127.0.0.1 ${fqdn} ${hostname}
127.0.0.1 localhost.localdomain localhost
127.0.0.1 localhost4.localdomain4 localhost4

# The following lines are desirable for IPv6 capable hosts
::1 ${fqdn} ${hostname}
::1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
EOF
echo 'manage_etc_hosts: true' >> /etc/cloud/cloud.cfg
sed -e 's/name:\ centos/name:\ alces/g' \
    -e 's/gecos:\ Cloud\ User/gecos:\ Alces\ Administrator/g' \
    -i /etc/cloud/cloud.cfg
sed -e 's/^SELINUX=enforcing.*/SELINUX=disabled/g' -i /etc/selinux/config
sed -e 's/root\ alces/root/g' -i /opt/clusterware/etc/setup-sshkey.rc
chkconfig NetworkManager off
yum clean all
# prepare as publicly available AMI ( https://bugs.centos.org/view.php?id=6228#c20524 )
# check root partition is mounted on /dev/xvda and EBS vol /dev/xvdb
mkfs -t xfs /dev/xvdb
dd bs=65536 if=/dev/xvda of=/dev/xvdb

Shut the instance down - then perform the following actions from the EC2 console:

  • Detach both the root volume and EBS volume from the instance
  • Reattach the EBS volume and attach as /dev/sda1
  • From the Instances tab, select Create Image from your instance
  • From the AMIs tab, once creation has finished - modify the permissions to allow public access if required

Example user data for launching instances

userdata.master

#cloud-config
hostname: login1
fqdn: login1.mycluster.alces.network
write_files:
- content: |
    cluster:
      uuid: '7b4249e8-637a-11e5-a343-7831c1c0e63c'
      token: '8R0c6lyMG1pJGP/wzg8dHA=='
      name: 'mycluster'
      role: 'master'
      tags:
        scheduler_roles: ':master:compute:'
      quorum: 3
      service_url: https://flight.alces-software.com
      gridware:
        depots:
        - name: benchmark
          url: https://s3-eu-west-1.amazonaws.com/packages.alces-software.com/depots/benchmark
        - name: chem
          url: https://s3-eu-west-1.amazonaws.com/packages.alces-software.com/depots/chem
          disabled: true
      aws_access_key: AKIAI43FYI4AL2PFHJ3A
      aws_secret_key: <SECRET>
      aws_zone_name: cluster
    instance:
      users:
      - username: steve
        uid: 509
        group: steve
        gid: 509
        groups:
          - gridware
          - admins:388
        ssh_public_key: |
          ssh-rsa XXX
      - username: markt
        groups:
          - gridware
          - admins:388
        ssh_public_key: |
          ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+0G7pJYXeKfcIFYmpzAQYWNI5kSiVQItfbnChcgb3IOF0qtJpm0gxsNvU1tu8LA0toswV11c/XQ76yREeBi7W50fOtpxgElKbfQO5YxMGteH/JSU/I/vilJC5IlGLYk1YCauMAfMNaizShF/taVpAtThL5z/FzhO48wlGvE34gexyLck1+Uc8gjpEEHSFnoeyAzALJw6OHM6sOroEZ1pUeJfgKD6DbLS9cxj5BLSWwShS7f37m9X8HYFuxt+ywV+f0KtXaaTYKbcbU9uzJ+wmxV35EKgWi6yhIXyIU85pk3hpJQ5F9RSU2B/flgbkLDnycpLNi8H0l4M4bnfEZMYP markt
  owner: root:root
  path: /opt/clusterware/etc/config.yml
  permissions: '0640'

userdata.slave

#cloud-config
hostname: node01
fqdn: node01.mycluster.alces.network
write_files:
- content: |
    cluster:
      uuid: '7b4249e8-637a-11e5-a343-7831c1c0e63c'
      token: '8R0c6lyMG1pJGP/wzg8dHA=='
      name: 'mycluster'
      role: 'slave'
      tags:
        scheduler_roles: ':compute:'
      quorum: 3
      service_url: https://flight.alces-software.com
      aws_access_key: AKIAI43FYI4AL2PFHJ3A
      aws_secret_key: <SECRET>
      aws_zone_name: cluster
    instance:
      users:
      - username: steve
        uid: 509
        group: steve
        gid: 509
        groups:
          - gridware
          - admins:388
        ssh_public_key: |
          ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAxOnS0K1/YsHERk1y3I2SKxT2fHM3IJzYtoTpxbRWz2zKP8Zi1P1jLBYoj86EI8GV1KpQN/gi/OHcPiJ1t9vXHlvNsJjX+JOe5/vGWAPk6xG+Pa/jHP6vUQ8FJB1X3S0UcKaPUzkILXsX+Pqs7eIjOGstVcRZ/vLYKzgvnDqSEpwjn5orpViSHFdZlozHX81iMsc5InMbsP2vRyUnSdyRhRV/5fQr/pUV2I4DRIjs+aocyfBi0EkhMP2h6OoXt0rBAJg++QVtjM9B6VoVsV4zRacrAJuWZ3x8OZa56SS7Xe04mtSizeWGAeptECfDkAdJAa8hF1p9yvw8cJfEzze7Uw== steve
      - username: markt
        groups:
          - gridware
          - admins:388
        ssh_public_key: |
          ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+0G7pJYXeKfcIFYmpzAQYWNI5kSiVQItfbnChcgb3IOF0qtJpm0gxsNvU1tu8LA0toswV11c/XQ76yREeBi7W50fOtpxgElKbfQO5YxMGteH/JSU/I/vilJC5IlGLYk1YCauMAfMNaizShF/taVpAtThL5z/FzhO48wlGvE34gexyLck1+Uc8gjpEEHSFnoeyAzALJw6OHM6sOroEZ1pUeJfgKD6DbLS9cxj5BLSWwShS7f37m9X8HYFuxt+ywV+f0KtXaaTYKbcbU9uzJ+wmxV35EKgWi6yhIXyIU85pk3hpJQ5F9RSU2B/flgbkLDnycpLNi8H0l4M4bnfEZMYP markt
  owner: root:root
  path: /opt/clusterware/etc/config.yml
  permissions: '0640'

Copyright 2015 Alces Software Ltd

Clone this wiki locally