RUNNING TESTS WITH RAKE 165

rendered by the view will still need some work and a keen eye, but we know we’re done with the underlying controllers and models when the functional test passes. And what about our customer? Well, seeing us write this test first makes her think she’d like us to try using tests as a specification again in the next iteration.

That’s just one revolution through the test-driven development cycle— write an automated test before the code that makes it pass. For each new feature that the customer requests, we’d go through the cycle again. And if a bug pops up (gasp!), we’d write a test to corner it and, when the test passed, we’d know the bug was cornered for life.

Done regularly, test-driven development not only helps you incrementally create a solid suite of regression tests but it also improves the quality of your design. Two for the price of one.

12.6 Running Tests with Rake

Rake4 is a Ruby program that builds other Ruby programs. It knows how to build those programs by reading a file called Rakefile, which includes a set of tasks. Each task has a name, a list of other tasks it depends on, and a list of actions to be performed by the task.

When you run the rails script to generate a Rails project, you automatically get a Rakefile in the top-level directory of the project. And right out of the chute, the Rakefile you get with Rails includes handy tasks to automate recurring project chores. To see all the built-in tasks you can invoke and their descriptions, run the following command in the top-level directory of your Rails project.

depot> rake --tasks

Let’s look at a few of those tasks.

Make a Test Database

One of the Rake tasks we’ve already seen, clone_structure_to_test, clones the structure (but not the data) from the development database into the test database. To invoke the task, run the following command in the top-level directory of your Rails project.

depot> rake clone_structure_to_test

4http://rake.rubyforge.net

Prepared exclusively for Rida Al Barazi

Report erratum

RUNNING TESTS WITH RAKE 166

Running Tests

You can run all of your unit tests with a single command using the Rakefile that comes with a Rails project.

depot> rake test_units

Here’s sample output for running test_units on the Depot application.

depot_testing> rake test_units

(in /Users/mike/work/depot_testing)

. . .

Started

...............

Finished in 0.873974 seconds.

16 tests, 47 assertions, 0 failures, 0 errors

You can also run all of your functional tests with a single command:

depot> rake test_functional

The default task runs the test_units and test_functional tasks. So, to run all the tests, simply use

depot> rake

But sometimes you don’t want to run all of the tests together, as one test might be a bit slow. Say, for example, you want to run only the test_update( ) method of the ProductTest test case. Instead of using Rake, you can use the -n option with the ruby command directly. Here’s how to run a single test method.

depot> ruby test/unit/product_test.rb -n test_update

Alternatively, you can provide a regular expression to the -n option. For example, to run all of the ProductTest methods that contain the word validate in their name, use

depot> ruby test/unit/product_test.rb -n /validate/

But why remember which models and controllers have changed in the last few minutes to know which unit and functional tests need to be to run? The recent Rake task checks the timestamp of your model and controller files and runs their corresponding tests only if the files have changed in the last 10 minutes. If we come back from lunch and edit the cart.rb file, for example, just its tests run.

depot> edit app/models/cart.rb depot> rake recent

(in /Users/mike/work/depot_testing) /usr/lib/ruby/gems/1.8/gems/rake-0.5.3/lib/rake/rake_test_loader.rb test/unit/cart_test.rb

Started

..

Finished in 0.158324 seconds.

2 tests, 4 assertions, 0 failures, 0 errors

Prepared exclusively for Rida Al Barazi

Report erratum

RUNNING TESTS WITH RAKE 167

Schedule Continuous Builds

While you’re writing code, you’re also running tests to see if changes may have broken anything. As the number of tests grows, running them all may slow you down. So, you’ll want to just run localized tests around the code you’re working on. But your computer has idle time while you’re thinking and typing, so you might as well put it to work running tests for you.

All you need to schedule a continuous test cycle is a Unix cron script, a Windows at file, or (wait for it) a Ruby program. DamageControl5 happens to be just such a program—it’s built on Rails and it’s free. DamageControl lets you schedule continuous builds, and it will even check your version control system for changes (you are using version control, right?) so that arbitrary tasks of your Rakefile are run whenever anyone on your team checks in new code.

Although it’s a book for Java users, Pragmatic Project Automation [Cla04] is full of useful ideas for automating your builds (and beyond). All that adds up to more time and energy to develop your Rails application.

Generate Statistics

As you’re going along, writing tests, you’d like some general measurements for how well the code is covered and some other code statistics. The Rake stats task gives you a dashboard of information.

depot> rake stats

+

----------------------

+

-------

+

-------

+

---------

+

---------

+-----

+

-------

+

| Name

| Lines |

LOC |

Classes | Methods | M/C | LOC/M |

+----------------------

+-------

+-------

+---------

+---------

+-----

+-------

+

| Helpers

|

15

|

11

|

0

|

1

|

0

|

9

|

| Controllers

|

342

|

214

|

5

|

27

|

5

|

5

|

| APIs

|

0

|

0

|

0

|

0

|

0

|

0

|

| Components

|

0

|

0

|

0

|

0

|

0

|

0

|

|

Functionals

|

228

|

142

|

7

|

22

|

3

|

4

|

| Models

|

208

|

108

|

6

|

16

|

2

|

4

|

|

Units

|

193

|

128

|

6

|

20

|

3

|

4

|

+----------------------

+-------

+-------

+---------

+---------

+-----

+-------

+

| Total

|

986

|

603

|

24

|

86

|

3

|

5

|

+----------------------

+-------

+-------

+---------

+---------

+-----

+-------

+

Code LOC: 333

Test LOC: 270

Code

to Test Ratio: 1:0.8

Now, you know the joke about lies, damned lies, and statistics, so take this with a large pinch of salt. In general, we want to see (passing) tests being added as more code is written. But how do we know if those tests are good? One way to get more insight is to run a tool that identifies lines of code that don’t get executed when the tests run.

5http://damagecontrol.codehaus.org/

Prepared exclusively for Rida Al Barazi

Report erratum

PERFORMANCE TESTING	168
Ruby Coverage6 is a free coverage tool (not yet included with Ruby or Rails)
that outputs an HTML report including the percentage of coverage, with
the lines of code not covered by tests highlighted for your viewing pleasure.
To generate a report, add the -rcoverage option to the ruby command when
running tests.
depot> ruby -rcoverage test/functional/store_controller_test.rb
Generate test reports often, or, better yet, schedule fresh reports to be
generated for you and put up on your web site daily. After all, you can’t
improve that which you don’t measure.

	12.7 Performance Testing
	Speaking of the value of measuring over guessing, we might be inter-
	ested in continually checking that our Rails application meets perfor-
	mance requirements. Rails being a web-based framework, any of the var-
	ious HTTP-based web testing tools will work. But just for fun, let’s see
	what we can do with the testing skills we learned in this chapter.
	Let’s say we want to know how long it takes to load 100 Order models
	into the test database, find them all, and then process them through the
	save_order( ) action of the StoreController. After all, orders are what pay the
	bills, and we wouldn’t want a serious bottleneck in that process.
	First, we need to create 100 orders. A dynamic fixture will do the trick
	nicely.
File 114	<% for i in 1..100 %>
	order_<%= i %>:
	id: <%= i %>
	name: Fred
	email: [e-mail]
	address: 123 Rockpile Circle
	pay_type: check
	<% end %>
	Notice that we’ve put this fixture file over in the performance subdirectory of
	the fixtures directory. The name of a fixture file must match a database table
	name, and we already have a file called orders.yml in the fixtures directory
	for our model and controller tests. We wouldn’t want 100 order rows to be
	loaded for nonperformance tests, so we keep the performance fixtures in
	their own directory.
	6gem install coverage

Prepared exclusively for Rida Al Barazi

Report erratum

PERFORMANCE TESTING

169

	Then we need to write a performance test. Again, we want to keep them
	separate from the nonperformance tests, so we create a file in the directory
	test/performance that includes the following.
File 121	require File.dirname(__FILE__) + '/../test_helper'
	require 'store_controller'
	class OrderTest < Test::Unit::TestCase
	fixtures :products
	HOW_MANY = 100
	def setup
	@controller = StoreController.new
	@request = ActionController::TestRequest.new
	@response = ActionController::TestResponse.new
	get :add_to_cart, :id => @version_control_book.id
	end
	def teardown
	Order.delete_all
	end
	In this case, we use fixtures( ) to load the products fixtures, but not the orders
	fixture we just created. We don’t want the orders fixture to be loaded just
	yet because we want to time how long it takes. The setup( ) method puts
	a product in the cart so we have something to put in the orders. The
	teardown( ) method just cleans up all the orders in the test database.
	Now for the test itself.
File 121	def test_save_bulk_orders

elapsedSeconds = Benchmark::realtime do

Fixtures.create_fixtures(File.dirname(__FILE__) +

"/../fixtures/performance", "orders") assert_equal(HOW_MANY, Order.find_all.size)

1.upto(HOW_MANY) do |id| order = Order.find(id)

get :save_order, :order => order.attributes assert_redirected_to :action => 'index' assert_equal("Thank you for your order.", flash[:notice])

end end

assert elapsedSeconds < 8.0, "Actually took #{elapsedSeconds} seconds" end

The only thing we haven’t already seen is the use of the create_fixtures( ) method to load up the orders fixture. Since the fixture file is in a nonstandard directory, we need to provide the path. Calling that method loads up all 100 orders. Then we just loop through saving each order and asserting that it got saved. All this happens within a block, which is passed to the realtime( ) method of the Benchmark module included with Ruby. It brackets the order testing just like a stopwatch and returns the total time it took to save 100 orders. Finally, we assert that the total time took less than eight seconds.

Prepared exclusively for Rida Al Barazi

Report erratum

	PERFORMANCE TESTING	170
	Now, is eight seconds a reasonable number? It really depends. Keep in
	mind that the test saves all the orders twice—once when the fixture loads
	and once when the save_order( ) action is called. And remember that this
	is a test database, running on a paltry development machine with other
	processes chugging along. Ultimately the actual number itself isn’t as
	important as setting a value that works early on and then making sure
	that it continues to work as you add features over time. You’re looking for
	something bad happening to overall performance, rather than an absolute
	time per save.
	Transactional Fixtures
	As we saw in the previous example, creating fixtures has a measurable
	cost. If the fixtures are loaded with the fixtures( ) method, then all the fix-
	ture data is deleted and then inserted into the database before each test
	method. Depending on the amount of data in the fixtures, this can slow
	down the tests significantly. We wouldn’t want that to stand in the way of
	running tests often.
	Instead of having test data deleted and inserted for every test method, you
	can configure the test to load each fixture only once by setting the attribute
	self.use_transactional_fixtures to true. Database transactions are then used
	to isolate changes made by each test method to the test database. The
	following test demonstrates this behavior.
File 125	class ProductTest < Test::Unit::TestCase
	self.use_transactional_fixtures = true
	fixtures :products

def test_destroy_product assert_not_nil @version_control_book @version_control_book.destroy

end

def test_product_still_there assert_not_nil @version_control_book

end end

Note that transactional fixtures work only if your database supports transactions. If you’ve been using the create.sql file in the Depot project with MySQL, for example, then for the test above to pass you’ll need MySQL to use the InnoDB table format. To make sure that’s true, add the following line to the create.sql file after creating the products table:

alter table products TYPE=InnoDB;

If your database supports transactions, using transactional fixtures is almost always a good idea because your tests will run faster.

Prepared exclusively for Rida Al Barazi

Report erratum

PERFORMANCE TESTING 171

Profiling and Benchmarking

If you simply want to measure how a particular method (or statement) is performing, you can use the script/profiler and script/benchmarker scripts that Rails provides with each project.

Say, for example, we notice that the search( ) method of the Product model is slow. Instead of blindly trying to optimize the method, we let the profiler tell us where the code is spending its time. The following command runs the search( ) method 10 times and prints the profiling report.

depot> ruby script/profiler "Product.search('version_control')" 10

%	cumulative	self		self	total
time	seconds	seconds	calls	ms/call	ms/call	name
68.61	46.44	46.44	10	4644.00	6769.00	Product#search
8.55	52.23	5.79	100000	0.06	0.06	Fixnum#+
8.15	57.75	5.52	100000	0.06	0.06	Math.sqrt
7.42	62.77	5.02	100000	0.05	0.05	IO#gets
. . .
0.04	68.95	0.03	10	3.00	50.00	Product#find

OK, the top contributors to the search( ) method are some math and I/O we’re using to rank the results. It’s certainly not the fastest algorithm. Equally important, the profiler tells us that the database (the Product#find( ) method) isn’t a problem, so we don’t need to spend any time tuning it.

After tweaking the ranking algorithm in a top-secret new_search( ) method, we can benchmark it against the old algorithm. The following command runs each method 10 times and then reports their elapsed times.

depot> ruby script/benchmarker 10 "Product.new_search('version_control')" \ "Product.search('version_control')"

	user	system	total		real
#1	0.250000	0.000000	0.250000	(	0.301272)
#2	0.870000	0.040000	0.910000	(	1.112565)

The numbers here aren’t exact, mind you, but they provide a good sanity check that tuning actually improved performance. Now, if we want to make sure we don’t inadvertently change the algorithm and make search slow again, we’ll need to write (and continually run) an automated test.

When working on performance, absolute numbers are rarely important. What is important is profiling and measuring so you don’t have to guess.

What We Just Did

We wrote some tests for the Depot application, but we didn’t test everything. However, with what we now know, we could test everything. Indeed, Rails has excellent support to help you write good tests. Test early and often—you’ll catch bugs before they have a chance to run and hide, your designs will improve, and your Rails application will thank you for it.

Prepared exclusively for Rida Al Barazi

Report erratum