Hi, this blog is no longer maintained, my new blog is here

Ruby On Rails and a Conning Israeli entrepreneur

Rails and Amazon EC2 - Beginners guide

First of all, Amazon AWS rocks. it's a great, stable and not so expensive way to get your application up and running, but also ready for any disaster to come (Someone said Digg effect?).
Getting your application on to Amazon EC2 is not as complicated as you think, managing and controlling your instances may require a professional system administrator, but i would recommend that anyway (except if you go and host your application on EngineYard than you are worry-free, but you'll pay.. ohh you'll pay for that sense of security).

Get your application Ready

There is no real need to setup your EC2 instances in day 1, you can wait until the application is mature enough to be deployed onto a production infrastructure. Amazon EC2 costs per usage, so, it will cost you to have your application up and running on EC2, keep that in mind (although the prices are a joke).

Setting things up
Once your application is ready to be deployed to EC2, you'll need an image.
Image, in EC2 terms, is a reference to an OS + all required installations and configuration needed to run your application.
The best all-round image to use with rails is Pawl Dowman's Rails on EC2 bundle (or here). EC2onRails is great. Unfortunately, if you’ve never used EC2 before, you probably won’t be able to “Deploy a Ruby on Rails app on EC2 in Five Minutes” as the documentation claims, so this document will try to fill in the gaps for someone who has never worked with EC2.

note: it's never too early to sit down and read Amazon's documentations, they are not that bad


Local machine
First thing is first, your development machine, the one from which you will be deploying to EC2 should undoubtably hold:

  • Ruby on Rails (do yourself a favor, at least version 2.1)
  • MySQL (Well, we are going to work with RailsonEC2, which includes it, if you need another db.. you'll need another image).
  • Java Development Kit 1.5 or later installed.
    Mac users should be ready-to-go (Hah! Hah!), Windows/Linux users if you don’t have it, download it from Sun's website.
    Make sure to download a version labeled “JDK”.
    Java is required for the tools that Amazon provides to manage EC2 instances. (rumor says that there are ruby management tools also, but i prefer the ones that Amazon gives).
Signing up for the services
Once you are set with these issues, you'll need to sign up for the Amazon EC2 service, do that right here.
Now, sign up for the S3 service.
Why? becaous althougn your image (later: 'instance') includes storage, images/instances can be brought down, up, dropped and even deleted, that will ultimatly, kill your data.

S3 is Amazon’s “Simple Storage Service”. S3 is a super-inexpensive service to store files into “buckets in the cloud”. S3 will be used for database backups of your Rails Application.
Sign up for S3 here.

What you'll need to keep somewhere
After signing up, you will need to collect four pieces of information from your AWS account by visiting the Access Identifiers page.
  • Your account number. The account number can be found at the upper right of the Access Idenfiers page and should be three four-digit numbers separated by dashes.
  • Your Access Key ID. This is a 20 or so character key found in a beige box a little below your account number.
  • Your Secret Access Key. This is a 40 or so character key found in another beige box just below your Access Key ID. If this is your first time on this page, you may have to generate your key. Click the “Show” button to display your 40 character key.
  • Your X.509 Certificate Create and download the X.509 certificate below the Secret Access Key section. Place the public and private keys into a folder called “.ec2″ in your home directory.
double check it! it's very very very very important.

Download and Install the EC2 Command Line Tools

The EC2 command-line tools are a Java-based set of tools that allow you to create and manage machine instances on EC2.

Download the Command Line Tools
Download here Amazon EC2 Command-Line Tools and extract the zip file, remember where you put it, i always keep it in my home folder as a hidden folder named .ec2 (/Users/eizesus/.ec2/ on OSX).



Set Appropriate Environment Variables
export EC2_PRIVATE_KEY=/Users/eizesus/.ec2/pk-5xxxxxxxx7.pem
export EC2_CERT=/Users/eizesus/.ec2/cert-5xxxxxxxxxxxxx7.pem
export EC2_HOME=/Applications/java/ec2-api-tools-1.3-24159
export PATH=$EC2_HOME/bin:$PATH
Remember to replace my library with yours.

I suggest adding these lines to your startup profile, whatever it maybe on your OS. (OSX, it's in /etc/profile).
After you do that, reload your session (close the console/command line and open it again).

Create a Key Pair the Deployment machine


ec2-add-keypair my-secret-code
The string my-secret-code can and should be anything that you like, try to pick something that makes sense for your setup (remember that i will keep on writing my-secret-code, use your phrase instead).

Save the output from that into a file named id_rsa-my-secert-code and paste everything between (and including) the “—–BEGIN RSA PRIVATE KEY—–” and “—–END RSA PRIVATE KEY—–” lines into it. Confirm that the file contents looks exactly like this, then save the file into the file:
~/.ssh/id_rsa-my-secret-code

Set permissions on the new key file:
chmod 600 ~/.ssh/id_rsa-my-secret-code

Start Up an Amazon EC2 Instance

An Amazon Machine Image (AMI) is a named configuration of an EC2 image.
The current AMI id’s for EC2onRails are:
  • ami-c9bc58a0 (32-bit)
  • ami-cbbc58a2 (64-bit)
Start up an instance of the 32-bit EC2onRails image:

ec2-run-instances ami-c9bc58a0 -k my-secret-code
The second line of the results returned will look like:
INSTANCE        i-XXXXXXXX      ami-c9bc58a0                    pending my-secret-code       0
The pending my-secret-code means that the image is pending.
To check the status of the instance build type:

ec2-describe-instances i-XXXXXXXX
Replace the i-XXXXXXXX above with the string that comes after INSTANCE in the second line of the results from the ec2-run-instances command.
Run it again until it says running my-secret-code, than you have an Amazon EC2 instance running! (yippie!).
Take note of the string that looks like
ec2-xx-xxx-xxx-xx.compute-1.amazonaws.com
that is your machine’s address.

Authorize SSH and HTTP

ec2-authorize default -p 22
should result in:
PERMISSION default ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0
ec2-authorize default -p 80
This should return something like:
PERMISSION default ALLOWS tcp 80 80 FROM CIDR 0.0.0.0/0
you are up and ready to go!
You should be able to ssh into the your new machine, replace
ec2-xx-xxx-xxx-xx.compute-1.amazonaws.com
with your own machine’s address:
ssh -i ~/.ssh/id_rsa-ec2-rails-keypair root@ec2-xx-xxx-xxx-xx.compute-1.amazonaws.com
You may have to type “yes” to accept the authenticity of the host.

Prepare Your Application for EC2

EC2 for Rails requires Capistrano:
sudo gem install capistrano -v 2.4.3
And then install the EC2 for Rails gem:
sudo gem install ec2onrails
Okay, you need to add three configuration files to your Rails application.
  1. Capfile - save this file at the root of your application.
  2. deploy.rb - save this file in the /config folder.
  3. s3.yml - also save this file in the /config folder.

Customize the EC2 Configurations
capfile can an be left as-downloaded.
config/s3.yml - This configuration is pretty simple, under the production section, put your AWS information that you noted above.

  aws_access_key: (you got me when you signed up)
  aws_secret_access_key: (me too)

  bucket_base_name: production.yourname.com
config/deploy.rb - There are a lot of definitions to be made, read all the comments and setup the deploy process as your applicaiton requires.

config/database.yml - For the production section, basically add any good password that you like and add hostname: db_primary. It should look something like:
production:
  adapter: mysql
  encoding: utf8
  database: appname_production
  username: user (not ROOT!)
  password: password (no, no empty passwords)
  hostname: db_primary

Run the EC2 on Rails Capistrano Tasks

cap ec2onrails:get_public_key_from_server cap ec2onrails:server:set_roles

Configure the db and stuff:

cap ec2onrails:setup
And launch your app:
cap deploy:cold
And wooooohoooo, check your application url.

That's it basically, you are up.

14 comments:

  J. Aaron Farr

January 19, 2009 at 10:32 PM

I just put together an EC2 image for OpenSolaris 2008.11 using Passenger (patched) and Rails 2.2. I've been thinking of putting together a similar shared AMI and related tools for those that would prefer something other than Ubuntu and Mongrel.

  Elad Meidar

January 19, 2009 at 11:21 PM

Well great,
when you're done, let me know and i will add a link to your image up here.

  Unknown

January 20, 2009 at 7:27 AM

And there is a meetup about AWS today in Tel Aviv,


http://www.facebook.com/event.php?eid=43977104468

  Elad Meidar

January 20, 2009 at 9:46 AM

Well, i wish i could be there... i'd be happy to know how it went later.

  Shmueli

January 21, 2009 at 6:36 PM

Hey Guys,

AWS Conf in Tel Aviv was great.
Martin Buht laid out the whole concept really nicely and cleary. Some Israeli companies which are already using EC2 for their startups answered questions from the crowd.
Very interesting, can't wait to get some AMIs up and running.

Thanks Elad for a great post.

  david

January 30, 2009 at 2:25 PM

Elad, great work with this awesome easy-to-follow tutorial! got as far as the last step, and then run into something I don't know how to deal with :(

when executing 'cap deploy:cold' I'll get the following error:
/usr/local/lib/ruby/gems/1.8/gems/capistrano-2.5.3/lib/capistrano/recipes/deploy/scm/subversion.rb:58:in `query_revision': tried to run `svn info http://svn.foo.com/svn/ocproject/trunk -rHEAD' and got unexpected result "" (RuntimeError)

I understand the 'cold' task means it should deploy the code in my box, not from any subversion repository... is he trying to reach a repository I need to set up?

I'd appreciate any help or tip :)

thanks again for your great work!

  Elad Meidar

January 30, 2009 at 2:36 PM

Hi David,
Thank you very much, i am always happy to help.
Regarding your issue, i believe it has something to do with your Subversion version (yeah, funny), as explained here http://www.ruby-forum.com/topic/130169,
Try to update your subversion (on the deployed-to machine of course).

the "cold" deploy is the first deployment you are doing, setting up your initial environment on the destination machine, therefore "cold".

  david

January 30, 2009 at 3:17 PM

subversion seemed to be up to date, but I reinstalled it just in case, and keep seeing the same error.

all I needed to do is run 'apt-get install subversion' from the ssh console of my amazon machine, right?

yeah, I'm a newbie... I'll keep googling and see if I figure out what to do.

thanks for the help tho,
david.

  Elad Meidar

January 30, 2009 at 3:25 PM

Well, i really can't tell without seeing the full error stack, it might be a superseded error like here:
On Ruby-forum.
try to update your local client copy of subversion as well, and if you don't succeed than, contact me on #rubyonrails at irc.freenode.net, the nick is eladmeidar.

  david

January 30, 2009 at 3:29 PM

thanks Elad, I actually realized the problem is that seems like I do need a repository, as the capistrano deploy.rb file points at it (set :repository, "http://svn.foo.com/svn/#{application}/trunk") and obviously it dosn't find it because it does not exist. I'll see if I can point it to my local git repository... if not having a repository it won't be a bad idea at all :)

hope our thread helps someone else in the future!

  Elad Meidar

January 30, 2009 at 7:07 PM

Yeah well, i guess you must have a repository if you want the deployment relay on checking out the source code from one. :)
Good luck mate.

  Matt Conway

February 21, 2009 at 8:49 AM

Great writeup. I've automated some of these common steps in my RoR/ec2 plugin (rubber). You should check it out if you have the chance - it has some functionality in common with Paul's setup, but more easily lets you expand your ec2 layout to accommodate multiple instances.

  Elad Meidar

February 21, 2009 at 9:07 AM

Hi Matt, strange coincidence.. i was just yesterday reading about your Plugin and wanted to have another follow up on this post with it.

Keep around, i will post it soon.

  Matt Conway

February 21, 2009 at 10:00 AM

Sounds good. Let me know if you run into any problems - I use rubber for my job's production setup (20+ instances), but its been a while since I bootstrapped from bare metal, so always interested to hear if things have stopped working due to changes in external dependencies. Prefer queries on mailing list if you can.

For David: by default rubber lets you deploy from a local filesystem, so you don't need a svn repo to test it (and ec2) out.


The Web Ask eizesus.com

Subscribe

    follow me on Twitter

    Twiters Around

    About Me

    My photo
    I am a web developer for more than 9 years, managed, cried, coded, designed and made money in this industry. now trying to do it again.

    Blog Archive

    Labels