I’m in the process of trying to establish some baseline performance metrics for increasing the performance of a website. I thought I’d share what I learned.
Changing the host for the test
In the application I’m testing (and in a lot of other ones that I’ve worked on) the routing is conditionally different based on the host of the request (that’s a whole ‘nother blog post for directions on how to do this). If you want to change the host for the request, its:
def setup host! 'yourproductionhost.com' end
Setting up the test environment
The guide told me that tests were run in development mode. In my experience this isn’t the case. Instead, my tests were being run in the test environment. For performance tests its even more important then normal to have a baseline of data you wish to test against. Otherwise your tests are not apples-to-apples comparisons as you improve the performance of your app.
Unfortunately, in the case of a large, complex datasets necessary to do performance testing this can be a PITA to setup. I’m using Factory Girl to make it easier.
That still leaves the problem of how to setup a test environment before performance testing. In my case, which won’t be the same as every other case, I want to load up a whole bunch of data to run a number of tests against. It take a couple of minutes to load all of this data and none of my tests change the data. Test::Unit doesn’t support setting up fixture data once per-TestCase, only once per test. But what I want is to be able to setup and teardown my data once for all the tests.
This little module did the trick. Remember to require it at the top of test/test_helper.rb and include it in the actual TestCase you want to use it in. Rspec, BTW, does this already with “setup :all,” but I’m unfortunately not using Rspec on this project.
module Test::SetupOnce module ClassMethods def setup_once; end def teardown_once; end def suite returning super do |suite| suite.send(:instance_variable_set, :@_test_klazz, self) suite.instance_eval do def run(*args) @_test_klazz.setup_once super @_test_klazz.teardown_once end end end end end def self.included(test) test.extend(ClassMethods) end end
One last note on this, if you load data in the setup_once class method, you need to turn off fixture loading from Rails or your data will get destroyed. Make sure and cleanup after yourself in teardown_once. To kill the fixtures for your TestCase, redefine the setup_fixtures and teardown_fixtures methods to be no-ops.
Pulling it all together
So here’s the test:benchmark TestCase with all the features mentioned.
require 'test_helper' require 'performance_test_help' class BenchmarkTest < ActionController::PerformanceTest include Test::SetupOnce # don't do anything with fixtures so the data sticks around def setup_fixtures; end def teardown_fixtures; end def self.setup_once # here's where you put your data to share across tests end def self.teardown_once # here's where you cleanup after yourself end def setup host! 'yourproductionhost.com' end def test_benchmark get '/' end def test_benchmark2 get '/foo' end end
One last note, the GC patch that is mentioned in the benchmarking guide gives some great info and is well worth it. I was only able to patch Ruby 1.8.6 p111, though. I would love to be able to patch the same version of ruby I’m running in production. Anyone know of other patches that reveal the same information that are portable to newer Ruby versions?