First of all, Amazon AWS rocks. it's a great, stable and not so expensive way to get your application up and running, but also ready for any disaster to come (Someone said Digg effect?).
Getting your application on to Amazon EC2 is not as complicated as you think, managing and controlling your instances may require a professional system administrator, but i would recommend that anyway (except if you go and host your application on EngineYard than you are worry-free, but you'll pay.. ohh you'll pay for that sense of security).
Get your application Ready
There is no real need to setup your EC2 instances in day 1, you can wait until the application is mature enough to be deployed onto a production infrastructure. Amazon EC2 costs per usage, so, it will cost you to have your application up and running on EC2, keep that in mind (although the prices are a joke).
Setting things up
Once your application is ready to be deployed to EC2, you'll need an image.
Image, in EC2 terms, is a reference to an OS + all required installations and configuration needed to run your application.
The best all-round image to use with rails is Pawl Dowman's Rails on EC2 bundle (or here). EC2onRails is great. Unfortunately, if you’ve never used EC2 before, you probably won’t be able to “Deploy a Ruby on Rails app on EC2 in Five Minutes” as the documentation claims, so this document will try to fill in the gaps for someone who has never worked with EC2.
note: it's never too early to sit down and read Amazon's documentations, they are not that bad
Local machine
First thing is first, your development machine, the one from which you will be deploying to EC2 should undoubtably hold:
- Ruby on Rails (do yourself a favor, at least version 2.1)
- MySQL (Well, we are going to work with RailsonEC2, which includes it, if you need another db.. you'll need another image).
- Java Development Kit 1.5 or later installed.
Mac users should be ready-to-go (Hah! Hah!), Windows/Linux users if you don’t have it, download it from Sun's website.
Make sure to download a version labeled “JDK”.
Java is required for the tools that Amazon provides to manage EC2 instances. (rumor says that there are ruby management tools also, but i prefer the ones that Amazon gives).
Once you are set with these issues, you'll need to sign up for the Amazon EC2 service, do that right here.
Now, sign up for the S3 service.
Why? becaous althougn your image (later: 'instance') includes storage, images/instances can be brought down, up, dropped and even deleted, that will ultimatly, kill your data.
S3 is Amazon’s “Simple Storage Service”. S3 is a super-inexpensive service to store files into “buckets in the cloud”. S3 will be used for database backups of your Rails Application.
Sign up for S3 here.
What you'll need to keep somewhere
After signing up, you will need to collect four pieces of information from your AWS account by visiting the Access Identifiers page.
- Your account number. The account number can be found at the upper right of the Access Idenfiers page and should be three four-digit numbers separated by dashes.
- Your Access Key ID. This is a 20 or so character key found in a beige box a little below your account number.
- Your Secret Access Key. This is a 40 or so character key found in another beige box just below your Access Key ID. If this is your first time on this page, you may have to generate your key. Click the “Show” button to display your 40 character key.
- Your X.509 Certificate Create and download the X.509 certificate below the Secret Access Key section. Place the public and private keys into a folder called “.ec2″ in your home directory.
Download and Install the EC2 Command Line Tools
The EC2 command-line tools are a Java-based set of tools that allow you to create and manage machine instances on EC2.
Download the Command Line ToolsDownload here Amazon EC2 Command-Line Tools and extract the zip file, remember where you put it, i always keep it in my home folder as a hidden folder named .ec2 (/Users/eizesus/.ec2/ on OSX).
Set Appropriate Environment Variables
export EC2_PRIVATE_KEY=/Users/eizesus/.ec2/pk-5xxxxxxxx7.pem export EC2_CERT=/Users/eizesus/.ec2/cert-5xxxxxxxxxxxxx7.pem export EC2_HOME=/Applications/java/ec2-api-tools-1.3-24159 export PATH=$EC2_HOME/bin:$PATHRemember to replace my library with yours.
I suggest adding these lines to your startup profile, whatever it maybe on your OS. (OSX, it's in /etc/profile).
After you do that, reload your session (close the console/command line and open it again).
Create a Key Pair the Deployment machine
ec2-add-keypair my-secret-codeThe string my-secret-code can and should be anything that you like, try to pick something that makes sense for your setup (remember that i will keep on writing my-secret-code, use your phrase instead).
Save the output from that into a file named id_rsa-my-secert-code and paste everything between (and including) the “—–BEGIN RSA PRIVATE KEY—–” and “—–END RSA PRIVATE KEY—–” lines into it. Confirm that the file contents looks exactly like this, then save the file into the file:
~/.ssh/id_rsa-my-secret-code
Set permissions on the new key file:
chmod 600 ~/.ssh/id_rsa-my-secret-code
Start Up an Amazon EC2 Instance
An Amazon Machine Image (AMI) is a named configuration of an EC2 image.The current AMI id’s for EC2onRails are:
- ami-c9bc58a0 (32-bit)
- ami-cbbc58a2 (64-bit)
ec2-run-instances ami-c9bc58a0 -k my-secret-codeThe second line of the results returned will look like:
INSTANCE i-XXXXXXXX ami-c9bc58a0 pending my-secret-code 0The pending my-secret-code means that the image is pending.
To check the status of the instance build type:
ec2-describe-instances i-XXXXXXXXReplace the i-XXXXXXXX above with the string that comes after INSTANCE in the second line of the results from the ec2-run-instances command.
Run it again until it says running my-secret-code, than you have an Amazon EC2 instance running! (yippie!).
Take note of the string that looks like
ec2-xx-xxx-xxx-xx.compute-1.amazonaws.comthat is your machine’s address.
Authorize SSH and HTTP
ec2-authorize default -p 22should result in:
PERMISSION default ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0
ec2-authorize default -p 80This should return something like:
PERMISSION default ALLOWS tcp 80 80 FROM CIDR 0.0.0.0/0
you are up and ready to go!
You should be able to ssh into the your new machine, replace
ec2-xx-xxx-xxx-xx.compute-1.amazonaws.com
with your own machine’s address:
ssh -i ~/.ssh/id_rsa-ec2-rails-keypair root@ec2-xx-xxx-xxx-xx.compute-1.amazonaws.comYou may have to type “yes” to accept the authenticity of the host.
Prepare Your Application for EC2
EC2 for Rails requires Capistrano:sudo gem install capistrano -v 2.4.3And then install the EC2 for Rails gem:
sudo gem install ec2onrailsOkay, you need to add three configuration files to your Rails application.
- Capfile - save this file at the root of your application.
- deploy.rb - save this file in the /config folder.
- s3.yml - also save this file in the /config folder.
Customize the EC2 Configurations
capfile can an be left as-downloaded.
config/s3.yml
- This configuration is pretty simple, under the production section, put your AWS information that you noted above. aws_access_key: (you got me when you signed up) aws_secret_access_key: (me too) bucket_base_name: production.yourname.comconfig/deploy.rb - There are a lot of definitions to be made, read all the comments and setup the deploy process as your applicaiton requires.
config/database.yml - For the production section, basically add any good password that you like and add hostname: db_primary. It should look something like:
production: adapter: mysql encoding: utf8 database: appname_production username: user (not ROOT!) password: password (no, no empty passwords) hostname: db_primary
Run the EC2 on Rails Capistrano Tasks
cap ec2onrails:get_public_key_from_server cap ec2onrails:server:set_roles
Configure the db and stuff:
cap ec2onrails:setupAnd launch your app:
cap deploy:coldAnd wooooohoooo, check your application url.
That's it basically, you are up.
14 comments:
January 19, 2009 at 10:32 PM
I just put together an EC2 image for OpenSolaris 2008.11 using Passenger (patched) and Rails 2.2. I've been thinking of putting together a similar shared AMI and related tools for those that would prefer something other than Ubuntu and Mongrel.
January 19, 2009 at 11:21 PM
Well great,
when you're done, let me know and i will add a link to your image up here.
January 20, 2009 at 7:27 AM
And there is a meetup about AWS today in Tel Aviv,
http://www.facebook.com/event.php?eid=43977104468
January 20, 2009 at 9:46 AM
Well, i wish i could be there... i'd be happy to know how it went later.
January 21, 2009 at 6:36 PM
Hey Guys,
AWS Conf in Tel Aviv was great.
Martin Buht laid out the whole concept really nicely and cleary. Some Israeli companies which are already using EC2 for their startups answered questions from the crowd.
Very interesting, can't wait to get some AMIs up and running.
Thanks Elad for a great post.
January 30, 2009 at 2:25 PM
Elad, great work with this awesome easy-to-follow tutorial! got as far as the last step, and then run into something I don't know how to deal with :(
when executing 'cap deploy:cold' I'll get the following error:
/usr/local/lib/ruby/gems/1.8/gems/capistrano-2.5.3/lib/capistrano/recipes/deploy/scm/subversion.rb:58:in `query_revision': tried to run `svn info http://svn.foo.com/svn/ocproject/trunk -rHEAD' and got unexpected result "" (RuntimeError)
I understand the 'cold' task means it should deploy the code in my box, not from any subversion repository... is he trying to reach a repository I need to set up?
I'd appreciate any help or tip :)
thanks again for your great work!
January 30, 2009 at 2:36 PM
Hi David,
Thank you very much, i am always happy to help.
Regarding your issue, i believe it has something to do with your Subversion version (yeah, funny), as explained here http://www.ruby-forum.com/topic/130169,
Try to update your subversion (on the deployed-to machine of course).
the "cold" deploy is the first deployment you are doing, setting up your initial environment on the destination machine, therefore "cold".
January 30, 2009 at 3:17 PM
subversion seemed to be up to date, but I reinstalled it just in case, and keep seeing the same error.
all I needed to do is run 'apt-get install subversion' from the ssh console of my amazon machine, right?
yeah, I'm a newbie... I'll keep googling and see if I figure out what to do.
thanks for the help tho,
david.
January 30, 2009 at 3:25 PM
Well, i really can't tell without seeing the full error stack, it might be a superseded error like here:
On Ruby-forum.
try to update your local client copy of subversion as well, and if you don't succeed than, contact me on #rubyonrails at irc.freenode.net, the nick is eladmeidar.
January 30, 2009 at 3:29 PM
thanks Elad, I actually realized the problem is that seems like I do need a repository, as the capistrano deploy.rb file points at it (set :repository, "http://svn.foo.com/svn/#{application}/trunk") and obviously it dosn't find it because it does not exist. I'll see if I can point it to my local git repository... if not having a repository it won't be a bad idea at all :)
hope our thread helps someone else in the future!
January 30, 2009 at 7:07 PM
Yeah well, i guess you must have a repository if you want the deployment relay on checking out the source code from one. :)
Good luck mate.
February 21, 2009 at 8:49 AM
Great writeup. I've automated some of these common steps in my RoR/ec2 plugin (rubber). You should check it out if you have the chance - it has some functionality in common with Paul's setup, but more easily lets you expand your ec2 layout to accommodate multiple instances.
February 21, 2009 at 9:07 AM
Hi Matt, strange coincidence.. i was just yesterday reading about your Plugin and wanted to have another follow up on this post with it.
Keep around, i will post it soon.
February 21, 2009 at 10:00 AM
Sounds good. Let me know if you run into any problems - I use rubber for my job's production setup (20+ instances), but its been a while since I bootstrapped from bare metal, so always interested to hear if things have stopped working due to changes in external dependencies. Prefer queries on mailing list if you can.
For David: by default rubber lets you deploy from a local filesystem, so you don't need a svn repo to test it (and ec2) out.
Post a Comment