StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POiOS Tests/Specs TDD/BDD and Integration & Acceptance Testing
text
Body
copied!<p>What are the best technologies to use for behavior-driven development on the iPhone? And what are some open source example projects that demonstrate sound use of these technologies? Here are some options I've found:</p> <hr> <h2><a href="http://en.wikipedia.org/wiki/Unit_testing" rel="nofollow noreferrer">Unit Testing</a></h2> <h3><a href="http://test-unit.rubyforge.org/" rel="nofollow noreferrer">Test::Unit</a> Style</h3> <ol> <li><a href="http://www.sente.ch/software/ocunit/" rel="nofollow noreferrer">OCUnit/SenTestingKit</a> as explained in <a href="http://developer.apple.com/library/ios/#documentation/Xcode/Conceptual/iphone_development/135-Unit_Testing_Applications/unit_testing_applications.html" rel="nofollow noreferrer">iOS Development Guide: Unit Testing Applications</a> & other <a href="http://www.sente.ch/software/ocunit/#More" rel="nofollow noreferrer">OCUnit references</a>. <ul> <li>Examples: <a href="http://developer.apple.com/library/ios/#samplecode/iPhoneUnitTests/Introduction/Intro.html#//apple_ref/doc/uid/DTS40008868" rel="nofollow noreferrer">iPhoneUnitTests</a>, <a href="http://three20.info/" rel="nofollow noreferrer">Three20</a></li> </ul></li> <li><a href="https://github.com/philsquared/Catch" rel="nofollow noreferrer">CATCH</a></li> <li><a href="https://github.com/gabriel/gh-unit" rel="nofollow noreferrer">GHUnit</a></li> <li><a href="http://code.google.com/p/google-toolbox-for-mac/wiki/iPhoneUnitTesting" rel="nofollow noreferrer">Google Toolbox for Mac: iPhone Unit Testing</a></li> </ol> <h3><a href="http://relishapp.com/rspec" rel="nofollow noreferrer">RSpec</a> Style</h3> <ol> <li><a href="http://www.kiwi-lib.info/" rel="nofollow noreferrer">Kiwi</a> (which also comes with mocking & expectations)</li> <li><a href="https://github.com/pivotal/cedar" rel="nofollow noreferrer">Cedar</a></li> <li><a href="http://pivotal.github.com/jasmine/" rel="nofollow noreferrer">Jasmine</a> with <a href="http://developer.apple.com/library/ios/#documentation/DeveloperTools/Reference/UIAutomationRef/_index.html" rel="nofollow noreferrer">UI Automation</a> as shown in <a href="https://github.com/dexterous/iOS-Acceptance-Testing/tree/master/specs" rel="nofollow noreferrer">dexterous' iOS-Acceptance-Testing specs</a></li> </ol> <hr> <h2><a href="http://en.wikipedia.org/wiki/Acceptance_testing" rel="nofollow noreferrer">Acceptance Testing</a></h2> <h3><a href="http://seleniumhq.org/" rel="nofollow noreferrer">Selenium</a> Style</h3> <ol> <li><p>UI Automation (works on device)</p> <ul> <li><a href="http://developer.apple.com/library/ios/#documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/Built-InInstruments/Built-InInstruments.html#//apple_ref/doc/uid/TP40004652-CH6-SW75" rel="nofollow noreferrer">UI Automation Instruments Guide</a></li> <li><a href="http://developer.apple.com/library/ios/#documentation/DeveloperTools/Reference/UIAutomationRef/_index.html" rel="nofollow noreferrer">UI Automation reference documentation</a></li> <li><a href="https://github.com/alexvollmer/tuneup_js" rel="nofollow noreferrer" title="tuneup">Tuneup js</a> - cool library for using with UIAutomation.</li> <li><p><a href="http://developer.apple.com/library/ios/#documentation/AnalysisTools/Conceptual/WhatsNewInstruments/NewFeatures42/NewFeatures42.html#//apple_ref/doc/uid/TP40010950-CH4-SW8" rel="nofollow noreferrer">Capturing User Interface Actions into Automation Scripts</a></p> <p>It's possible to use <a href="https://github.com/cucumber/cucumber/blob/master/examples/v8/features/step_definitions/fib_steps.js" rel="nofollow noreferrer">Cucumber (written in JavaScript)</a> to drive UI Automation. This would be a great open-source project. Then, we could write <a href="http://en.wikipedia.org/wiki/Gherkin" rel="nofollow noreferrer">Gherkin</a> to run UI Automation testing. For now, I'll just write Gherkin as comments.</p></li> </ul> <p>UPDATE: <a href="http://www.zucchiniframework.org/" rel="nofollow noreferrer">Zucchini Framework</a> seems to blend Cucumber & UI Automation! :)</p> <p>Old Blog Posts:</p> <ul> <li><a href="http://alexvollmer.com/posts/2010/07/03/working-with-uiautomation/" rel="nofollow noreferrer">Alex Vollmer's UI Automation tutorial</a></li> <li><a href="http://answers.oreilly.com/topic/1646-how-to-use-uiautomation-to-create-iphone-ui-tests/" rel="nofollow noreferrer">O'Reilly Answers UI Automation tutorial</a></li> <li><a href="http://www.codeproject.com/KB/iPhone/UI_Automation_Testing.aspx" rel="nofollow noreferrer">Adi Saxena's UI Automation tutorial</a></li> </ul></li> <li><p><a href="http://code.google.com/p/uispec/" rel="nofollow noreferrer">UISpec</a> with <a href="https://github.com/twotoasters/UISpecRunner" rel="nofollow noreferrer">UISpecRunner</a></p> <ul> <li><a href="http://code.google.com/p/uispec/" rel="nofollow noreferrer">UISpec is open source on Google Code</a>.</li> <li><a href="http://code.google.com/p/uispec/wiki/Documentation" rel="nofollow noreferrer">UISpec has comprehensive documentation</a>.</li> </ul></li> <li><p><a href="http://www.gorillalogic.com/fonemonkey" rel="nofollow noreferrer">FoneMonkey</a></p></li> </ol> <h3><a href="http://cukes.info/" rel="nofollow noreferrer">Cucumber</a> Style</h3> <ol> <li><p><a href="https://github.com/moredip/Frank" rel="nofollow noreferrer">Frank</a> and <a href="https://github.com/BlueFrogGaming/icuke" rel="nofollow noreferrer">iCuke</a> (based on the <a href="http://www.slideshare.net/undees/iphone-meets-cucumber" rel="nofollow noreferrer">Cucumber meets iPhone talk</a>) </p> <ul> <li>The <a href="http://groups.google.com/group/frank-discuss" rel="nofollow noreferrer">Frank Google Group</a> has much more activity than the <a href="http://groups.google.com/group/icuke" rel="nofollow noreferrer">iCuke Google Group</a>.</li> <li>Frank runs on both device and simulator, while iCuke only runs in simulator.</li> <li>Frank seems to have a more comprehensive set of step definitions than <a href="https://github.com/BlueFrogGaming/icuke/blob/master/lib/icuke/icuke_world.rb" rel="nofollow noreferrer">iCuke's step definitions</a>. And, <a href="https://github.com/moredip/Frank/wiki/Step-Definition-compendium" rel="nofollow noreferrer">Frank also has a step definition compendium on their wiki</a>.</li> <li><a href="http://groups.google.com/group/frank-discuss/browse_thread/thread/a2c76ede9476d3db/213880d0aceb14df?lnk=gst&q=icuke#213880d0aceb14df" rel="nofollow noreferrer">I proposed that we merge iCuke & Frank</a> (similar to how Merb & Rails merged) since they have the same common goal: Cucumber for iOS.</li> </ul></li> <li><p><a href="https://github.com/square/KIF" rel="nofollow noreferrer">KIF (Keep It Functional)</a> by <a href="https://squareup.com/" rel="nofollow noreferrer">Square</a></p></li> <li><p><a href="http://www.zucchiniframework.org/" rel="nofollow noreferrer">Zucchini Framework</a> uses Cucumber syntax for writing tests and uses CoffeeScript for step definitions.</p></li> </ol> <h3>Additions</h3> <ul> <li><a href="http://www.mulle-kybernetik.com/software/OCMock/" rel="nofollow noreferrer">OCMock</a> for <a href="http://en.wikipedia.org/wiki/Mock_object" rel="nofollow noreferrer">mocking</a></li> <li><a href="http://code.google.com/p/hamcrest/wiki/TutorialObjectiveC" rel="nofollow noreferrer">OCHamcrest</a> and/or <a href="https://github.com/petejkim/expecta" rel="nofollow noreferrer">Expecta</a> for expectations</li> </ul> <h2>Conclusion</h2> <p>Well, obviously, there's no right answer to this question, but here's what I'm choosing to go with currently:</p> <p>For unit testing, I used to use <a href="http://www.sente.ch/software/ocunit/" rel="nofollow noreferrer">OCUnit/SenTestingKit</a> in XCode 4. It's simple & solid. But, I prefer the language of BDD over TDD (<a href="http://www.rubyinside.com/dhh-offended-by-rspec-debate-4610.html" rel="nofollow noreferrer">Why is RSpec better than Test::Unit</a>?) because our words create our world. So now, I use <a href="https://github.com/allending/Kiwi/pull/42#issuecomment-1768713" rel="nofollow noreferrer">Kiwi with ARC</a> & <a href="https://github.com/allending/Kiwi/issues/30#issuecomment-1550718" rel="nofollow noreferrer">Kiwi code completion/autocompletion</a>. I prefer Kiwi over Cedar because it's built on top of OCUnit and comes with RSpec-style matchers & mocks/stubs. UPDATE: I'm now looking into OCMock because, currently, <a href="https://github.com/allending/Kiwi/issues/61" rel="nofollow noreferrer">Kiwi doesn't support stubbing toll-free bridged objects</a>.</p> <p>For acceptance testing, I use UI Automation because it's awesome. It lets you record each test case, making writing tests automatic. Also, Apple develops it, and so it has a promising future. It also works on the device and from Instruments, which allows for other cool features, like showing memory leaks. Unfortunately, with UI Automation, I don't know how to run Objective-C code, but with Frank & iCuke you can. So, I'll just test the lower-level Objective-C stuff with unit tests, or create <code>UIButton</code>s only for the <a href="https://stackoverflow.com/questions/6748087/xcode-test-vs-debug-preprocessor-macros/6763597#6763597"><code>TEST</code> build configuration</a>, which when clicked, will run Objective-C code.</p> <p>Which solutions do you use?</p> <h2>Related Questions</h2> <ul> <li><a href="https://stackoverflow.com/questions/6036891/is-there-a-bdd-solution-that-presently-works-well-with-ios4-and-xcode4">Is there a BDD solution that presently works well with iOS4 and Xcode4?</a></li> <li><a href="https://stackoverflow.com/questions/5455771/sentestingkit-integrated-with-xcode-versus-ghunit-on-xcode-4-for-unit-testing">SenTestingKit (integrated with XCode) versus GHUnit on XCode 4 for Unit Testing?</a></li> <li><a href="https://stackoverflow.com/questions/4685582/testing-asynchronous-code-on-ios-with-ocunit">Testing asynchronous code on iOS with OCunit</a></li> <li><a href="https://stackoverflow.com/questions/6190486/sentestingkit-in-xcode-4-asynchronous-testing/6190553#6190553">SenTestingKit in Xcode 4: Asynchronous testing?</a></li> <li><a href="https://stackoverflow.com/questions/862244/how-does-unit-testing-on-the-iphone-work">How does unit testing on the iPhone work?</a></li> </ul>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload