Storing Files in Ruby on Rails

Note: Mike Gunderloy has brought to my attention while writing this post, that the guys at therailsway.com already did something like that. well.. i finished it anyway, but i'm sure their post is at least half as good :).
I finished my post before reading theirs and didn't add things Koz wrote and i didn't,  so don't blame me in stealing or something. :)

Some noob at #rubyonrails (irc.freenode.net) asked a question a few days ago about some issue he had while storing uploaded files as BLOBs in his database.
I Know storing files in a database is usually a bad idea, but i still thought about the face that i was yet to see a propar guidance on how to handle files (using the correct practices of course).

Plugins
Attachement_fu - The older brother, pretty nice, better than the late file_column.

PaperClip - An "updated" attachement_fu, some people say it's better than attachement_fu because it is actually working. There is a lot more work being done on this plugin now a days, so it's better to relay on it rather than on attachment_fu.

Storing your files.

Local File System
This option can only work out for you, if you are running your app on a single server, simply set one of the plugins to store the uploaded files whereever you want, and that's it.
You can also setup a network folder if you'd like and imitate a local folder across a number of servers, but i's the same.

What you do need to be careful from, is a situation where you are saving too many files into a single folder, which will cause mainly for a slow file lookup.
Try to refine you directory structures for your upload root in order to avoid such a problem.

Amazon S3
The main reasons to use Amazon S3 are that it's amazingly scalable and incrediably cheap. Other than that it's pretty easy to manage using the plugins or some gems around there, and it's generally offloading the weight of sending files to your users an keeping your application servers busy (i wrote before on apache's x-sendfile header, that's another way).

and... still.. BLOBs
As i said before, i am not a fan of storing files in your db, seems like out of context for me, but some people like it.
If you insist (and you shouldn't) , you'll need to setup a proper migration to handle BLOBs, and the rest is up to you.

Live Free: Open Source Software indexes

1. OS Living - Great index, cut into categories.
2. Wikipedia - List of free open source software.
3. OpenSourceList - Anthoer category based list, some of the stuff here are Close Source, but still free to use.
4. 50 Apps to manage your Office with open source - This list consists over linux based programs, but i think that Ubunut is anyway the number #1 replacement for windows in offices (except mine where everyone got a mac, haha).
5. 5 Business free applications - why pink?! :)
   

Rails Facebook App Enlightments: HTTP timeouts

Since i had to handle so much crap with Facebook, Rails and the Facebooker plugin in order to get my latest application up, i decided to share a little bit with other innocent coders straying in the path of hell.

The Facebooker plugin interfaces with the Facebook API using the Net::HTTP ruby library and POSTs API requests (API call, FQL..) using the #post method (eventually, there's a whole bunch of stuff happening before that).
It sounds nice, it really does, problem is that in this specific app we have send a whole bunch of such requests to facebook, some requests return 1000-17,000+ strings back as JSON.... what makes Facebook simply.. stale.

Now getting like 3,400 Timeout exceptions raised in your production servers (adding that the CPU is on 3.2%, RAM is at 10%) is annoying. really annoying.

The "patch" we found for this issue was to ensure a specific amount of retries to be made by rescuing the Timeout Exception.

def get_my_somefing_from_fb
  retry_count = 0
  begin
    do facebook crap
  rescue TimeoutError
    if (retry_count < 5)
      retry_count+=1
      retry
    end
  end
end

Now we are watching it closely to see exactly how it helps us, if at all.

Startup Thoughts: Realty and a Dream

Everyone wants to be a big bad rock star entrepreneur, wake up to that morning when your bank account is flooded by plain cold cash, you are making millions while sitting at home and watching the download meter of your software or user count of your web site flying sky high every day.
Well, it's possible, many people have done that, and the nice thing about the web/internet world, is that there will be always, always, enough to share.
But it's a long way, and as the nature of things that take long, they tend to break our spirit, get us bored and insecure and afraid of competition.
I have gathered around a few misconceptions about startups and entrepreneurship, misconceptions that most people around this business seems to posses, but those who won't, these are the ones who break forward.

Misconception #1 - "Fast, Fast, Fast"
Well, shortly.. NO!
Nothing is more important than keeping your startup organized, Clear and planned ahead. Programming fast does not guarantee a better product, Designing, looking around a bit and gathering the baseline on which your consist your product, does guarantee a better product.
Get yourself a clear milestone plan, point out Alpha version (Standalone POC), private beta (for users you self pick) and a public beta, and stick with this plan!.


Misconception #2 - "We'll grow with it"
Scaling... people seem to forget about it, more like avoid it until the "twitter/dzone/digg affect" hit's their home page.
Web application tend to collect more users in a period of time than a downloaded software, web applications are also a single point of service (in case of SaaS), so you'd better get your system infrastructure ready and available for a user boost one day or another, viral marketing can be a dangerous thing.

Misconception #3 - "I've had enough"
i've heard people say "if this project doesn't go public in 5 months, i am out" or "i can't work with this people, i'm out". There's a word for people like that.
quitters.
and quitters don't get rich.
The people you work with may be annoying as hell, stupid as hell or useless as hell, if you truly believe in the concept of the idea, don't let anyone else's negligence to take what is yours and do everything you can (and you ALWAYS can) to get the startup up and running.

Misconception #4 - "We need more money"
Sometimes you do, sometimes you don't.
In order to get a website up and running these days, ready and eager to handle thousands of users, you only need AWS, which if you exaggerate like hell with your configuration, you'll get to 1000$ a month.
So following my previous post, you should know that one or more of your partners must be able to supply you with technical solutions (system administration, coding) so this should almost never be counted as an expense.
Most of the most successful startups were not founded, ever. keep that in mind.

How to avoid screwing up a software project

Run Programmer, Run

When developing a new application, mostly when it's being done personally and alone, can sometimes be a very complicated process. you'll need to handle a whole bunch of other stuff than just coding (marketing, server setups even investors hunt) and it can sometimes lead you to a software neglection.
Don't let it happen, run/compile your application every few days, keep it living in your head.

Sometimes you'll start work on some big shiny feature (e.g. adding a kewl Google maps integration), but stop because you hit a technical bump ("What? no maps for Israel?"), or don’t have the time to finish it ("Need to finish this Company profile by tomorrow") and The source code is left in unfinished state.
You can’t do anything with any of your code until this is fixed, and the longer you leave it, the more you’ll forget and the harder it will be to get started again.
This is called a "broken build", and is a big landmine because it impacts other peoples ability to work (And your ability to continue as well).

TIP #1: You started something? finish it before moving on to something else.
TIP #2: Stay in touch with your software.

"I Sure could use a time machine right now...."

Know these times when you wished you had a time machine? well, it can happen in the process of developing and application too.
People make mistakes, Always. When people make mistakes in the kitchen, the food comes out really bad and you call the local pizza delivery services and solve the problem. When a programmer makes a mistake or is doing a system wide change... you'll need a ready to use Plan B around.
Source Control is the software world equivalent of a time machine, you can go back to a certain version of your application and rollback any changes made to your code and application, and by that, maybe reversing a very serious threat to your code.
If you haven’t taken the plunge with revision control yet, I highly recommend looking at some of the free SVN or GIT hosting services post.

TIP #3: Save yourself, use a source control service.

"Look at this cooooooolll JQuery accordation!"

Features are fun, Javascript magic is amazing, but you'll need to focus on what's really important
Focusing on things like validation, cool eye candies or extra functions is a great way to build up a large complex code base that doesn’t do anything useful yet.
Focus on the core functionality of your software first — your main features should be complete before you start thinking about WOW stuff.
Wasting trying to think of the perfect name, designing a logo or an icon, choosing the perfect open-source license and making a website won’t get you any closer to having a working application.

TIP #4: Core functionality first, fun - later.

Throw your code away and start from scratch - The Netscape mistake

As Netscape famously discovered a few years ago, throwing away existing code to start afresh is almost never a good idea. Resist the urge and make a series of small, manageable code re-factoring instead.

TIP #5: Never sink your own boat, pickup a bucket and start pulling out water

Pull yourself together before you sit down

"mmm, Rails? or maybe Adobe Air? or maybe... maybe we'll do the cool gears thing?"
Before doing anything ask yourself a simple question - "What are you actually trying to achieve?", Spend some time with a pen and some paper coming up with a really clear vision of what you’re trying to create — e.g. screen mock-ups, basic core functionality (yes, again) and if there is a process you are trying to imitate, go through it too.

TIP #6:  If you don’t know what you’re doing from the beginning, you’ll have no chance of finishing it.

Get the right men for the job

You maybe a coding genius, a super-programmer (Mike it's you) or other superior programming entity. but unless you are a really something no one had ever seen before, you don't know anything there is to know about every aspect of developing a new software and a products.
If you can't tell the difference between red and green, get a designer to work for you. if you don't have a clue in CSS or DB infrastructure, get someone who know this art to do it.

TIP #7: Do what you know, not what you don't know.

Marketing is Important, but when it's time

Tell people about your product when you have something to show, open a product twitter account when you have something to say, don't rush yourself forward and try to stick the product or create a hype when you can't back it up by at least screenshots or an actual product on the best scenario.

TIP #8: Invite people in when you have something to offer them to eat.

The difference between a lie and a promise

Users don't mind waiting for features, they mind finding out you promised them something and didn't deliver.
Stand up to your words, and update your users about development progress and new features on the way, as long as you really intend to do it and not just saying to make them download or register.

TIP #9: Users are like your mother, lie to them, and they'll know.

Rails Conditional Eager Loading

Rails's eager loading (ScreenCast) is an important practice that saves some requests and queries when dealing with table relations in rails.
Up until now, using a condition hash to specify query was a little tricky if you use the eager loading method:
User.find(:all, :include => :jobs, :conditions => ["jobs.salary > ?",1000])

or using a hash (without eager loading)
User.find(:all, :conditions => {:name => 'eizesus'})

Using a conditions array/hash isn’t always my favorite way to write finds in Rails, but in certain cases must have when you deal with complex queries (like and "Advanced Search" feature).

In my current work,I needed to run conditions on an associated table in just such an "Advanced search" feature, so i came up with this simple and easy way to use a condition hash for eager loading methods:
User.find(:all, :include => :jobs, :conditions => {:name => 'eizesus', :jobs => {:position => 'Manager'})


or you can prepare the condition hash on the fly:
conditions = {}
conditions[:first_name] = params[:first_name] unless params[:first_name].blank?
conditions[:last_name] = params[:last_name] unless params[:last_name].blank?
conditions[:jobs] = {:position => params[:position]} unless params[:position].blank?

@users = User.find(:all, :conditions => conditions, :include => :jobs)


2 Useful CSS tips

"Don't look for water when something burns, just don't play with fire" - My Mom.

Preventing bugs before they happen, that's the key.
If you don’t prevent bug, they will start a snow ball that will force you on and on with ridiculous patch-work and what i basically call the "fire extinguishing loop", there will be a lot of them interacting with each other, making it difficult to figure out what’s what, and were it all began from.
The majority of bugs reside in JS differences between browsers and CSS and DOM interpretation, so i'll try to specify some methods and techniques we use here at my company, who make our lives a lot more easier.

CSS Resets

asterisk CSS selector:

* {margin: 0; padding: 0;}

But, it's too rough. Using this technique cause my forms to get messy and got me upset since i had to define every single padding and margin specification.
ince then, I’ve been using the popular CSS reset by Eric Meyer, which collects a set of such reset attributes, for several common HTML tags.
Of course i had to tweak it to fully match my needs, but still , it's a must-have baseline for cross browser CSS development.

Clearing floats

IE6 does not like floats too much, IE8RC1 is not a big fan either, especially when those floats are not clear.
Uncleared floats are often the cause of major bugs. however, it’s a bug that is easy to fix.
Setting an HR element with width of %1 and setting it to clear:both, will solve the problem.
<div id="container">
  <div class="floater">
    floating.
  </div>
  <div class="floater">
    floating.
  </div>
  <hr class="clear"/>
</div>
<span>I am below!!</span>

and the CSS:
#container{
}
.floater{float:left, width:60px; margin:6px; padding:6px; border:2px solid #888;}
hr.clear{clear:both;width:1%;}


ther clear:both definition orders the HR element to go down one line if the is any other element on it's left or on it's right, read more on the clear CSS attribute.

Deploying Ruby on Rails on EC2 - Deploying Oracle

Well, generally i use Oracle for my large scale applications.
Some people might say that it's not too-competible with Rails, but using the oracle_enhanced_adapter didn't pop up any special issues (it even supports Oracle's bind variables technique).
The Oracle Cloud Deployment tutorials on Oracle's website contains explanations, PDFs, tutorials and demos on how and when to deploy Oracle DB in the cloud.
Extra guides include backup and recovery practices.

Tools of the Trade: Web development tools list

Well, it's not mine, but it's great.
Dion Almaer's Web resources page includes a shiny list of web resources/tools/tips he uses when developing web applications (on PHP though, baaah.).
He indicates that some of the links are outdated, but promises to fix it. as soon as possible.

All Browsers Online Preview

Great tool, shows you how your website looks in many browsers and OS's, including screen shots.
Visit at browsershots.org

How much to charge? project estiamtion for freelancers

This neat little tool allows you to fill in a basic form with tasks and hours x rate calculation in order to know how much to charge for a project, cute, not more.

NETTUTS: JQuery for beginners

http://nettuts.com/articles/web-roundups/jquery-for-absolute-beginners-video-series/
great great tutorial, although i am (currently) a MooTools fan, i think this is a very good guide collection for JQuery.

NETTUS: Best tutorial of 2008

NETTUS just published a great tutorial roundup for 2008.
Great web development resource.

Observers, Associations and Callbacks

In my latest project, i need to notify a user via email about a certain kind of instance being created, so far normal.
This instance is created with a several of associated instances, let's say i am creating an article instance with linked categories using the virtual attribute technique (RailsCasts #16).
I created an observer for the article class to send an email with the article and categories after a successful creation of an article instance, therefore i immediately assumed that the correct callback is after_create. well, it's not.

when i used after_create, the email arrived with an empty list of categories, almost like they weren't saved, but a short trip to the console showed that the categories were created and that they are associated to the article as i wanted.

The problem resides in the order rails does this nested object creation, first the initial object is being created (the artical) and only than the association are created (after i used #build, watch the screen cast!), which causes the email to be triggered one step earlier than i wanted.

after finding this article i changed the observer's callback to after_save which apperantly is being triggers after the associations are saved as well.

i spent a lot of time on it, hope this helps.

Ajaxian: YQL - converting the web to JSON with mock SQL

Yahoo! now enables us to query a web page just like we would in sql. give it a try at YQL - converting the web to JSON with mock SQL

Periodical AJAX requests and Session Expiry

My current applicaiton requires a session expiry feature, so fat, pretty ordinary, i decided to use the limited sessions plugin, as i always do.
The problem came up when i found out, that sessions aren't being released after the desired time had passed.
What was the problem?

first we will need to understand how rails uses sessions, basically it's something like that

• session is retrieved from whereever it is. (ActiveRecordStore,CookieStore...)
• Controller#Action is being preformed
• Session is saved

now, where is the problem? when using periodical ajax calls, the session is being updated as it is after every action, so the 'updated_at' field, (required by the plugin) is updated, what causes the expiry interval (assuming it's bigger than the periodical AJAX calls's interval) to never be met. so no session expiry.

Solution

In my application.rb, i have a before_filter which updated the session with user_id in order to relate the session to the user, i migrated another field into the session table so i can keep my own track over changes, named it 'last_updated' like that

class AddCustomUpdateDateToSessions < ActiveRecord::Migration
  def self.up
    add_column :sessions, :last_updated, :datetime
  end

  def self.down
    remove_column :sessions, :last_updated
  end
end

and changed the application.rb method to update the new field, instead of the 'updated_at' field, and prevented my AJAX requests to update to the session, no need.

before_filter :update_session
  def update_session
    unless request.xhr?
      if logged_in?
        if session.model.user_id.nil?
          session.model.update_attribute(:user_id, current_user.id)
        end
        session.model.update_attribute(:last_updated, DateTime.now)
      end
    end 
  end

(yeah, i know it can be done better.) i also needed to change all the 'updated_at' references to 'last_updated' field inside the plugin's limited_sessions.rb, in order to completely take control over the session update flag. now it's working and my ajax calls do not cause the expiry interval to expire.

attachment_fu and minimagick resize bug

In one of my latest projects i was using the attachement_fu plugin to handle image uploads.
The attachement_fu plugin enables the option to select an image processor engine between ImageScience, Rmagick add MiniMagick:

1. Rmagick
   Kinda bulky, leaking but fully functional
   
2. MiniMagick
   Simple version of Rmagick, simple and small.
3. ImageSceince
   Small, nice, and quick.

i decided to use MiniMagick as my processor, i installed the gem:

 sudo gem install mini_magick

and added a specific :processor option to my Logo model.

has_attachment :content_type => :image, 
                   :storage => :file_system, 
                   :max_size => 2.megabytes,
                   :resize_to => 'x100',
                   :processor => :mini_magick

but when i was uploading an image, i was was expecting it to be resized based on the options i specified in the model. imagine how surprised i was when i found out that the image is not being resized.

checking the log found the following error:

Exception working with image: ImageMagick command (identify "/var/folders/Bv/Bvbna8-OH548ZWju8naF4U+++TI/-Tmp-/minimagick11451-0") failed: Error Given 32512

Googling it didn't help me much (except from finding other people with the same problem), so i decided to switch to ImageScience instead (RMagick is out of the question of course).

 sudo gem install image_science

this will include all most all dependencies, the freeImage dependency is also needed.
us mac/darwin users can simply use port:

  sudo port selfupdate
  sudo port install freeimage

now change the processor in the model

has_attachment :content_type => :image, 
                   :storage => :file_system, 
                   :max_size => 2.megabytes,
                   :resize_to => 'x100',
                   :processor => :image_science

and now it's working. yeah!.

Skype Weblinks - Call a skype user or send an IM message in a html page

When the user has installed Skype, one can open Skype directly from a webpage. The protocol is skype:skypename - and some other optional flags.

The example will use my skype username, but i am sure you will be able to get the hang of it.

Call a skype user
The optional flag is call - to force Skype to call somebody. It can also be left out - but if the user changed the default behavior to e.g. chat - Skype will open an IM session instead. So - go the save way and use the call - flag.

Example: call a Skype user

<a href="skype:eladmeidar?call">Call</a>

Send an Skype IM message to a Skype user

The optional flag is chat. See the example below to see the usage in a html page.
Example: send a chat message to a Skype user

<a href="skype:eladmeidar?chat">Chat</a>

Send a voicemail to a Skype user
The optional flag is voicemail - as you might have guessed.
Example: send a voicemail to Skype user

<a href="skype:eladmeidar?voicemail">Voice mail</a>

Create a Skype conference call with multiple users
This is as easy as the examples above - just more usernames before the "?" and call as flag.
Example: create a Skype conference call

<a href="skype:eladmeidar;otherfriend123?call">call 2 friends</a>

Create a conference chat with multiple users
Same as conference call - with chat as flag.
Example: open a conference chat with multiple users - in html

<a href="skype:eladmeidar;otherfriend123?chat">Chat with 2 friends </a>

i am now working on a rails plugin to implement this automated use for Rails applications

Rails 2.2: Getting Started link list

General Documentation and Guides

From railsinside.com

Rails 2.2 Release Notes - A very solid set of release notes for 2.2 with basic coverage of the new features (with short code examples and links) as well as a list of deprecated features. They were compiled by Ruby / Rails Inside's very own Mike Gunderloy!

Upgrading RubyGems to 1.3.x - Depending on your setup, Rails 2.2 may demand that you upgrade to RubyGems 1.3.x. This is not as easy as it might usually be, however. Mike Gunderloy gives some tips in case you get stuck.

Rails Security Guide - Steer clear of security issues in your Rails 2.2 applications by reading the Ruby on Rails Security Guide. Who said Rails has poor documentation? This is incredible!

Rails 2.2 Screencast - Gregg Pollack and Jason Seifer of Rails Envy put together a very solid Rails 2.2 screencast. It costs $9, but it covers a lot of ground over 44 minutes - learn about etags, connection pooling, new enumerable methods, new test helpers, and more.

Rails 2.2 - What's New - In association with EnvyCasts, Carlos Brando and Carl Youngblood present Rails 2.2 - What's New, a 118 page PDF covering all of the changes and additions to Rails 2.2. It's available in a package deal with the screencast (above) too.

InfoQ's Glance - InfoQ's Mirko Stocker takes a quick glance at some of Rails 2.2's new features.

New Features

Thread Safety - Rails 2.2 is now "thread safe." In October, Pratik Naik wrote a summary of why this is a big deal as well as some gotchas (basically, don't use class variables, use mutexes, etc.) Charles Nutter has also written What Thread-safe Rails Means which answers several pertinent questions.

Internationalization - The Rails Internationalization effort has its own homepage at http://rails-i18n.org/ which features lots of links to how-tos, tips, documentation, and demos. They also have a Google group / mailing list where you can get help, make suggestions, etc.

Basic Language Internationalization - It's a little old, but Simple Localization in Rails 2.2 gives a very quick, code-driven example of how basic internationalization works in Rails 2.2 (some of the set up is easier now, but it mostly applies).

Localization / Internationalization Demo App - Clemens Kofler has put together a demo app that shows off some of Rails 2.2's internationalization and localization features. If being knee deep in code is the best way for you to learn, jump in!

Layouts for ActionMailer - As of Rails 2.2, you can now use layouts in your ActionMailer views.

Connection Pooling - The connection pooling in Rails 2.2 allows Rails to distribute database requests across a pool of database connections. This can cause less lockups. In collaboration with a non-blocking MySQL driver, serious performance increases could result in certain situations.

Specify Join Table Conditions with Hashes - Do you need to run a find (or similar) query across a join? Now you can just specify the conditions for the joined tables in a hash, much like local tables conditions!

Limited Resource Routes - You can now limit map.resources to creating certain methods. For example, you might not want destroy or index methods - you can now specify these with :only and :except.

Memoization - Stop rolling your own memoization in Rails apps. Clemens Kofler demonstrates Rails 2.2's newly rolled-in memoization features. It's just a single method! If you have a view that calls on a calculated attribute often, this will give you some serious performance gains.

Custom Length Tokenizer for Validations - You can now specify a tokenizer of your own construction for validates_length_of validations.

Array#second through Array#tenth - If you're a bad programmer, you can now demonstrate it to the world by using the new Array#second, Array#third, Array#fourth, and so forth, methods. I've put it in my calendar to look for open source Rails apps using Array#seventh in six months time and to call them out on Rails Inside ;-)

Note: This list only takes into account some of the new features in Rails 2.2. There are a lot more! Read the release notes and the Rails 2.2 - What's New PDF to get the full picture.

Miscellaneous

restful-authentication-i18n - Want an authentication plugin for Rails 2.2 that supports internationalization? Take a look at result-authentication-i18n!

Barebones Apps - Check out Rails Inside's 7 Barebones Rails Apps to Kick Start Your Development Process.

Deploying on JBoss - You can now easily deploy a Rails app to a JBoss server. With Rails 2.2's significantly improved JRuby support, this makes rolling out Rails apps in the enterprise a breeze!

Installing Rails on Ubuntu Hardy Heron - Simon St Laurent has put together two Rails useful installation videos. One for servers, and one for the desktop.

REST for Rails 2 - Are you still in Rails 1.x land or not using REST at all? Would you like to? Geoffrey Grosenbach has put together a screencast showing you how it should be done.

A Better Rails Logo - The Rails Logo (as used at the head of this post) was created by Kevin Milden and is distrubuted under the BY-ND Creative Commons Licence. i personally like the older one.

Rails date Range

Holy Shmoly! Sometimes Rails and ruby just blows me away,

Let’s assume you are trying to find a bunch of records between a time frame.

  class Call < ActiveRecord::Base
    named_scope :by_month, lambda { |d| { :conditions  => { :date  => d.beginning_of_month..d.end_of_month } } }
  end

simply specify a range

and running (the named_scope )

  this_months_calls = Call.by_month Date.today

And Rails will return all the calls that were recorded during the current month. Love it.