Performance Tests

(from [96497c612664976907affa899aca7165cce7ded6]) [#1 Performance Tests] only log in debug mode http://github.com/yfactorial/obj...

Test results with logging off:

Test Suite '/Users/vickeryj/Code/active_resource/build/Debug-iphonesimulator/Unit Tests.app' started at 2008-10-21 09:09:57 -0400
Test Suite 'SenTestCase' started at 2008-10-21 09:09:57 -0400
Test Suite 'SenTestCase' finished at 2008-10-21 09:09:57 -0400.
Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
Test Suite 'GTMTestCase' started at 2008-10-21 09:09:57 -0400
Test Suite 'GTMTestCase' finished at 2008-10-21 09:09:57 -0400.
Executed 0 tests, with 0 failures (0 unexpected) in 0.000 (0.000) seconds
Test Suite 'DogTest' started at 2008-10-21 09:09:57 -0400
Test Case '-[DogTest testDogCount]' passed (0.245 seconds).
Test Case '-[DogTest testDogDelete]' passed (0.549 seconds).
Test Case '-[DogTest testDogProperties]' passed (0.000 seconds).
Test Case '-[DogTest testDogSave]' passed (0.249 seconds).
Test Case '-[DogTest testDogUpdate]' passed (0.239 seconds).
Test Suite 'DogTest' finished at 2008-10-21 09:09:58 -0400.
Executed 5 tests, with 0 failures (0 unexpected) in 1.283 (1.283) seconds
Test Suite '/Users/vickeryj/Code/active_resource/build/Debug-iphonesimulator/Unit Tests.app' finished at 2008-10-21 09:09:58 -0400.
Executed 5 tests, with 0 failures (0 unexpected) in 1.285 (1.285) seconds

Test results on the iPhone with logging off:

2008-10-21 09:15:03.969 Unit Tests[497:20b] Failed to load NSMainNibFile MainWindow.
Test Suite '/var/mobile/Applications/B767C81C-089D-407F-8A54-62C4DB9303D2/Unit Tests.app' started at 2008-10-21 09:15:04 -0400
Test Suite 'GTMTestCase' started at 2008-10-21 09:15:04 -0400
Test Suite 'GTMTestCase' finished at 2008-10-21 09:15:04 -0400.
Executed 0 tests, with 0 failures (0 unexpected) in 0.005 (0.005) seconds
Test Suite 'DogTest' started at 2008-10-21 09:15:04 -0400
Test Case '-[DogTest testDogCount]' passed (4.046 seconds).
Test Case '-[DogTest testDogDelete]' passed (7.788 seconds).
Test Case '-[DogTest testDogProperties]' passed (0.000 seconds).
Test Case '-[DogTest testDogSave]' passed (3.932 seconds).
Test Case '-[DogTest testDogUpdate]' passed (4.109 seconds).
Test Suite 'DogTest' finished at 2008-10-21 09:15:23 -0400.
Executed 5 tests, with 0 failures (0 unexpected) in 19.883 (19.883) seconds
Test Suite 'SenTestCase' started at 2008-10-21 09:15:23 -0400
Test Suite 'SenTestCase' finished at 2008-10-21 09:15:23 -0400.
Executed 0 tests, with 0 failures (0 unexpected) in 0.005 (0.005) seconds
Test Suite '/var/mobile/Applications/B767C81C-089D-407F-8A54-62C4DB9303D2/Unit Tests.app' finished at 2008-10-21 09:15:23 -0400.
Executed 5 tests, with 0 failures (0 unexpected) in 19.916 (19.916) seconds

Some numbers for JSON vs XML on the device using wifi

JSON completes the test suit in around 25 seconds

XML completes the test suite in around 34 seconds.

Interestingly enough, looking at the Rails logs JSON seems to use a lot less rendering time than XML. It might be worth re-running these tests with libxml plugged into Rails:

Completed in 0.07758 (12 reqs/sec) | Rendering: 0.06712 (86%) | DB: 0.00793 (10%) | 200 OK [http://macbook/dogs.xml?(null)]

Completed in 0.04188 (23 reqs/sec) | Rendering: 0.02742 (65%) | DB: 0.01120 (26%) | 200 OK [http://macbook/dogs.json?(null)] 

Still, the time difference suggests that there is a performance improvement elsewhere. Another variable to test with would be running the server through mod_deflat or the like.

Could you use static xml/json files served via Apache for your tests to remove the Rails part from the equation?

We surely could, and probably should, but knowing that XML is slower than JSON for Rails to produce & digest is probably also valuable.