Skip to content

Transactional Tests in Ruby on Rails

I’ve recently worked through a rather frustrating issue that turned out to have a simple solution.  I thought I’d share in case others out there were experiencing issues similar to mine.  Your ActiveSupport::TestCase tests (the default in Rails) are configured to run in transactions (unless they are explicitly disabled), so normally one can assume that each test runs on a clean slate of data.  It’s even supposed to work with factories like FactoryGirl (highly recommend using this gem to make your life easier while testing).  As it turns out this isn’t always true: this feature of Rails only works to a point and tends to break with even the simplest cases when running tests in a multithreaded environment.

The Problem

The issue arose when I switched my testing infrastructure over to a combination of Guard and Spork.  As a side note, so far I really like this setup as it runs the tests relevant to the change(s) made whenever your files are modified, and these tests are a bit snappier thanks to Spork.  After making this switch I noticed that my tests were ending in unexpected states (mostly leaving rows behind).  This was causing later, more complex tests to fail when they should not have been failing.

In my case I believe the issue was stemming from the multithreaded environment that is created with Spork (this is how Spork achieves its speed boost), as the tests were working without issue before the change.  It is worth mentioning that the default transactional setup of ActiveSupport::TestCase may also fail with complex tests (depends entirely on the test setup).  This can cause rows being left behind after a test just as my multithreaded issue did.  I’ve never seen this in practice myself but in researching this issue and discussing it online I encountered a few developers who have.  Just something to keep in mind.

The Fix

Thankfully the solution for this is really simple!  There is a fantastic gem out there called database_cleaner that was created to offer developers more flexibility in this situation.  database_cleaner offers a couple of different options for cleaning up changes in your database between tests.  Take a few minutes to read about them, as some are more performant than others but won’t work in all situations.  For my case I stuck with the transactions option (the most performant) but explicitly wrapped each of my tests in one using the DatabaseCleaner methods instead of relying on Rails to do it for me.  Once I installed the gem I simply had to add the necessary DatabaseCleaner calls to my test/test_helper.rb file, and my problems were solved.  My test/test_helper.rb now looks like this:

# test/test_helper.rb
ENV["RAILS_ENV"] = "test"
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'

DatabaseCleaner.strategy = :transaction

class ActiveSupport::TestCase
  # Setup all fixtures in test/fixtures/*.(yml|csv) for all tests in alphabetical order.
  #
  # Note: You'll currently still have to declare fixtures explicitly in integration tests
  # -- they do not yet inherit this setting
  fixtures :all

  # Stop ActiveRecord from wrapping tests in transactions
  self.use_transactional_fixtures = false

  setup { DatabaseCleaner.start }
  teardown { DatabaseCleaner.clean }

  # Add more helper methods to be used by all tests here...
end
 

Categories: Programming, Ruby on Rails.

doT.js Templates with Sprockets and Ruby on Rails

Browser Based Apps

With the popularity of browser based apps continuing to grow we’re seeing more and more on the web each day (here’s a list, if you’re curious).  I’ve recently jumped on board the backbone.js train and though it’s not perfect, I think this is a fantastic trend.  Browser based apps are run with JS and are completely client side (though most use some sort of backend API to fetch/save data).  As a consequence of this, your normal templating practices must change.  Your Ruby on Rails .html.erb files are no longer used (for 99% of the views), and though some setups opt to fetch raw HTML from the server you’ll mostly be fetching raw data in JSON form.  This means that you’ll need to use one of the many client side templating engines that exist.

Compiling the templates turns the HTML-based template files you write into JavaScript functions that return the HTML for the page.  These JS functions work essentially the same way as Rails save for the minor idiosyncrasies of the different engines .  The compiling can be done in the browser with JS, but you can save a lot of time by precompiling them on the server side and distributing the precompiled templates on each page load (instead of compiling them every time someone loads your app in their browser).  Sprockets (the tool that powers the Rails asset pipeline) supports precompiling of one type of JS template engine, EJS templates.

EJS templates will get the job done, but there are definitely more performant templating engines out there.  doT.js is one of those engines and it claims to be the fastest and most concise engine on the market.  I’m a big fan of doT.js myself and wanted to use it to power the JS templates in my backbone.js based browser app.  My app is powered by Ruby on Rails, and unfortunately the asset pipeline (Sprockets) doesn’t support precompiling doT.js templates.

Enter sprockets-dotjs

With sprockets-dotjs, Sprockets and the Rails asset pipeline support precompiling doT.js templates!  The templates are handled in the same way that the already supported EJS templates and handled, and do not interfere with any existing templates you may have.

Using sprockets-dotjs is simple.  Just add the gem to your Gemfile and run a bundle install.  Then in your assets/javascripts directory create template files in the format specified by the doT.js docs with the extension .jst.djs.  The gem will automatically pick up on this extension and your templates will be compiled along with the rest of your assets.

Check out the documentation on the GitHub page for more information on how to use the gem and the doT.js documentation for information on the templating engine itself.

Disclaimer

It should be noted that this is a very simple plugin with a very small set of features that was conceived and created in a very short amount of time.  Due to this fact it may not be perfect, so please keep an eye out for any issues and report them via GitHub.

 

Categories: Programming, Ruby on Rails, Web Development.

Simple Two-Legged OAuth Provider in Rails

I’ve spent the past 24 hours searching, reading and hacking my way to a very simple implementation of a Ruby on Rails based two-legged OAuth provider.  I now have a working solution and I thought I’d share seeing as I was unable to find a solid resource on this for a Rails site.

A quick disclaimer: this hasn’t been tested extensively, though I am pretty confident that this will get you off the ground for a basic set of GET/POST/PUT/DELETE requests sending along form data (Content-Type: application/x-www-form-urlencoded for PUT/POST requests).

A Bit About OAuth

Now, I’m not going to dive into the low-level details of how two-legged works, but I feel it is important to know what OAuth is trying to accomplish and what the two-legged version is doing.  OAuth is an open standard that was developed so that websites with APIs (ie Twitter) could allow users to securely grant third party websites access to their data.  OAuth accomplishes this without ever giving the user’s password to the third party.  Thus, you can authorize a specific client (TweetDeck) access to a specific user’s account information. The entire process is accomplished with a sequence of access/authorization tokens that are issued to the third party only when the end user explicitly grants access to the party.  This is what is happening when you authorize a website access to your Twitter account data.  So, traditional OAuth (aka three-legged OAuth) was created to securely grant third parties access to user data.  Two-legged OAuth, while similar to three-legged, does not control access to specific user data.  This means that you can control access on a per-client basis, but the protocol doesn’t give you user data access control.

As I said, the entire process is build upon the exchange of access keys.  Two-legged OAuth requires two keys need to be known by both the client and the server.  The first key, the public key, is transported publicly in the HTTP request and used as a client identifier.  The second key, called the secret key, is known by both the client and server but is never transported in the HTTP request.  The client uses this secret key to hash the request (which is then added to the request as a parameter) before sending it to the server. Upon receiving the request, the API looks up the client’s secret key based on the public key provided and then hashes the request with that secret key.  If the hashes match, the client is authenticated and the request is allowed.  The key here is that only the server and client know the secret key, ensuring that only the client can sign requests that the server will accept.

Note that this method only gives you a secure method to authenticate clients, the data in the request is still transported in plain text.

The Code

We’re going to use the Ruby OAuth gem to streamline this process.  The gem provides the basic request hashing functionality, so all we’ll have to do is integrate this with Rails.  To install this, simply insert one line into your Gemfile and run a bundle install:

gem 'oauth'

To get this working our Rails app needs to store API public key/secret key pairs so that we can authenticate our clients.  We’ll store the key pairs in our database, allowing a simple lookup method and the ability to store meta data about our API clients if needed.  Using this setup we could implement throttling and custom access controls, but these topics are beyond the scope of this post.  Lets first write our database migration to get the table setup; I’m going to keep this incredibly simple for the purposes of this post.

class Client < ActiveRecord::Migration
  def change
    create_table :client do |t|
      t.string :api_key
      t.string :secret

      t.timestamps
    end
  end
end

Great, run that migration to get the table into your database and we’ll move on to performing the authentication.  In order to accomplish this we’re going to add a protected method to the ApplicationController and run before processing each request using a before_filter

require 'rack'
require 'rack/request'
require 'oauth'

class ApplicationController < ActionController::Base     
  before_filter :run_oauth_check          
  # ...           

  protected           
  def run_oauth_check                 
    render :json => { :error => "Invalid request" },
        :status => 400 unless params[:oauth_consumer_key]

    client = Client.find_by_api_key params[:oauth_consumer_key]
    render :json => { :error => "Invalid credentials" },
        :status => 401 unless client != nil

    begin
      signature = OAuth::Signature.build(::Rack::Request.new(env)) do |rp|
        [nil, client.secret]
      end

      render :json => { :error => "Invalid credentials" },
          :status => 401 unless signature.verify
    rescue Auth::Signature::UnknownSignatureMethod => e
      render :json => { :error => "Unknown signature method" }, :status =>; 400
    end
  end
end

Let’s take a look at what is happening here. First, we check to make sure that oauth_consumer_key is specified in the request. This is a parameter required by the OAuth spec that contains our client’s public API key. Next, we pull the client’s data from our database so we can verify the signature with their secret key. If no client is found with the provided API key, we reject the request. Once we’ve done that, all we have to do is use the OAuth gem to build the signature. This is done by first instantiating a new Rack request object and providing it to the signature builder. Finally, we have to account for the situation where the client didn’t provide their signature method in the request with a rescue block (the OAuth spec has more information about the different request signing methods – basically they’re different hash functions).

That’s it!  Requests signed with incorrect credentials will now be rejected, and authorized requests will be allowed through.  The important takeaway from this example is the signature verification process.  I’ve included everything else as an example of its use, but so long as you can build the Rack request as we did above, you can implement this code in many different contexts.

Philip Southam has a great post on signing the requests in both Python and Ruby here. The post also contains a tutorial for this same setup in Python.

 

Categories: Programming, Ruby on Rails.

Tags: ,