Rigel Group

They shoot Yaks, don't they?

DevOps for People Who Hate DevOps

DevOps started out as scrappy developers who were just trying to Get Shit Done. They needed a couple of machines set up, and thought, “Hey, I am programmer. Why don’t I program the machine to set itself up!”. Viola! Genius, I tell you! But then, sadly, DevOps grew to become a Movement, and was co-opted by System Administrators and large commercial enterprises looking for something complicated to sell. The tools (Puppet, Chef, Im looking at you) became Rube Goldberg contraptions orders of magnitude more complicated than the little old LAMP stack we were trying to setup in the first place. Now you need DevOps infrastructure to manage your infrastructure. DevOps took the place of SysAdmins, and the average developers were left behind.

Then, like a breath of fresh air, Ansible came on the scene. Billed as a dead-simple DevOps system that relies on nothing more than SSH, it was love at first sight. Unfortunately, Ansible, it seems, has succummed to the siren song of the Enterprise, and if you were to look at the AnsibleWorks web site today, you would weep at the amount of marketing techospeak that has been strewn about. But fear not! The core of Ansible has not changed, and if you can wade past the BS you will find a jewel that may become the sharpest tool in your belt.

So, if we break down Ansible to it’s core, it is essentially a way to script an SSH session to a server. There are no prerequisites for the server. As long as you can reach it via SSH, and it has Python installed, Ansible can manage it. The magic sauce that Ansible brings to the party, is that it’s playbooks are YAML files that declaritively specify how the machine should look, and Ansible will do whatever needs to be done to make the machine look like that. So running Ansible is idempotent, you can run the same playbook against a machine multiple times, and if everything has already been done, it won’t do it again.

Let’s get started, for OSX follow along:

Make sure you have homebrew installed, and then

1
$ brew install python

This gives you a nice local version of python we will install Ansible into.

1
2
3
4
5
$ pip install jinja2
$ pip install PyYAML
$ pip install paramiko
$ pip install boto
$ pip install ansible

will have an ansible command to use. We can test it out by doing this (on OSX, make sure you have Remote Login checked in System Preferences –> Sharing so that you can SSH into localhost):

1
2
3
4
5
6
7
$ echo "[local]\n127.0.0.1 ansible_python_interpreter=/usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python" >> ~/ansible_hosts
$ export ANSIBLE_HOSTS=~/ansible_hosts
$ ansible all -m ping --ask-pass     #this will fail if you dont have an SSH server running on localhost.
127.0.0.1 | success >> {
    "changed": false,
    "ping": "pong"
}

launch_instance —ami=ami-bfd3a3d6 —type=m1.small —key=ansible-ec2-us-east —dns —groups=Web —region=us-east-1 ~/.boto

Now, we need a herd of boxen to manage, so for that grab your Amazon EC2 keys, make sure you have the EC2 command line tools installed and configured, and saddle up.

First, we are going to create a playbook, which is a YAML file that describes what we want done. This first playbook is a bit of a mindbender, because we are going to script Ansible to log into our local machine, and then run EC2 scripts to provision and boot EC2 instances. Once we have those, we can then use Ansible to manage our newborn servers.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
---
# Ansible Playbook to create and manage EC2 servers

- name: Provision servers
  hosts: local
  connection: local
  user: john
  gather_facts: false

  tags:
      - provision

  vars:
      keypair: ~/ansible_ec2.pem
      instance_type: t1.micro
      security_group: Web
      # bitnami-cloud-us-west-2/lampstack/bitnami-lampstack-5.4.11-1-linux-ubuntu-12.04.1-x86_64-s3.manifest.xml
      image: ami-0021ab30
      instance_count: 2

  # Provision 2 servers...

  tasks:
    - name: Launch server
      local_action: ec2 keypair=${keypair} group=${security_group} instance_type=${instance_type} image=${image} wait=yes count=${instance_count}
      register: ec2

    # Use with_items to add each instances public IP to a new hostgroup for use in the next play.

    - name: Add new servers to host group
      local_action: add_host name=${item.public_dns_name} groups=deploy
      with_items: ${ec2.instances}

    - name: Wait for SSH to be available
      local_action: wait_for host=${item.public_dns_name} port=22
      with_items: ${ec2.instances}

    - name: Wait for full boot
      pause: seconds=15

# Now, configure our new servers

- name: Configure servers
  hosts: deploy
  user: ubuntu
  sudo: yes
  gather_facts: true

  tags:
    - config
    - configure

  # Install Java, install Elasticsearch, replace settings....

  tasks:

    - name: Install JRE
      apt: pkg=openjdk-6-jre-headless state=latest install_recommends=no update_cache=yes

    - name: Download ElasticSearch package
      get_url: url=http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.0.deb dest=~/elasticsearch-0.90.0.deb

    - name: Install ES .deb file
      shell: dpkg -i ~/elasticsearch-0.90.0.deb
      notify: restart elasticsearch

    - name: Install cloud-aws plugin
      shell: /usr/share/elasticsearch/bin/plugin -install elasticsearch/elasticsearch-cloud-aws/1.11.0
      notify: restart elasticsearch

    - name: Make elasticsearch config dir
      file: path=/etc/elasticsearch/ state=directory

    - name: Copy over Elasticsearch settings
      copy: src=./elasticsearch/elasticsearch.yml dest=/etc/elasticsearch/elasticsearch.yml
      notify: restart elasticsearch

  handlers:
    - name: restart elasticsearch
      action: service name=elasticsearch state=restarted