If you are new to unit testing, but the source base is old, the sheer size of code to test will seem daunting. Here are some strategies for adding unit testing to old code.

• Test code before refactoring or optimizing – The purpose of refactoring and optimizing is to make the code do the exact same thing but in a different way. It is perfectly suited to being unit tested since tests you write at the beginning of the change need to pass at the end.  And, the unit tests will let you make more aggressive changes.
• Test code while learning what it does – When you're trying to learn a body of code, a unit test is a good way to check your assumptions about how the code works. This strategy also works well in training a new developer—explain the code by writing new tests for it together.
• Before fixing a bug, write a failing test – This is the perfect time to add a test since reliably reproducing the bug is a good first step. The beauty of this is that now the bug won't return.

In general, I would not make a project out of writing unit tests for old code (except during the period where you are installing the xUnit framework and getting used to it). Unit testing works best when it isn't the end, but instead a means to accomplishing something else.

It was my turn to present to the DotNetSaps. Here are the slides and code to score bowling games that I wrote during the presentation.  Get NUnit, FitNesse and look at XProgramming.com for more information on unit testing.

Note on the code: This code is not done. There are at least three bugs in it, but it passes all of the tests. This means that there are more tests to write.  Here are some that would fail:


Not to mention scoring the tenth frame correctly. Fixing those tests and finding others is left as an exercise to the reader.

Jakob Nielsen has been supporting his book, Homepage Usability: 50 Websites Deconstructed, with a few of articles on homepage design. I took the opportunity to make some changes to my own site after reading them.

The first article, Top Ten Web Design Mistakes of 2003, lists what's been annoying him recently. I did well on this one—I only needed to add some ALT text to my images (Tip #5 is Overly detailed ALT Text—oops, I actually didn't have any) and make pages stop linking to themselves (#10). The latter improvement is not directly supported in my CMS tool (CityDesk) and took a fair amount of scripting, but armed with CityDesk keywords and my keyword organizing utility, I got it done.

That article linked to The Ten Most Violated Homepage Design Guidelines. I didn't do too badly on that one either, but I decided to differently color visited links and make it more clear where you are on the site (#3). I rewrote my tag line to be more informative (#5) and made my Use a liquid layout that lets users adjust the homepage size), since that's a pet-peeve of mine.
Finally, I read Top Ten Guidelines for Homepage Usability. By this point, most of the tips were redundant. I haven't added a search yet, but I'll do that soon.

The book has over a hundred guidelines sure to improve any site.

Last week I had my first experience with a GPS based guidance system, the NeverLost from Hertz. I was struck by some of the good UI choices and want to highlight them here. For a more in-depth analysis, see this evaluation. I will limit this entry to my current blog topic of usability vs. learnability  as it applies to the disengaged user.

The designers of NeverLost certainly had to design for both usability and learnability. Few will have the time or inclination to read the manual once they get to their rental car, and many will be first time or occasional users. More importantly though, the NeverLost must be easy to use. I'm sure the designers of NeverLost understood the possibility of their device contributing to an accident, and let that be the overriding concern in all of their choices. The NeverLost is a good example of designing for disengaged users, that is, users that are doing something else while using your software. The principles of the NeverLost design have wide applicability (e.g. software for call centers or traders).

Here are some of the basic principles they have adhered to:

1. Constraining choices rather than interrupting on errors: A good choice in any application, the NeverLost takes it to an extreme. Typing is hard (you choose letters by navigating over a pick-list), so the search interface only shows letters that will actually return a result. So, for example, if you pick the letters “H-O-L-I-D” in the Yellow Pages search, only the letter “A” appears on the pick list, because the “Holiday Inn” is the only item in their list that starts with those letters. Searches never fail to return a result, and you can correct errors immediately. Of course, a keyboard or a touchscreen might have made this easier, but they would also add to the cost.
2. Speech rather than text: Essential for this kind of system since you really should not be looking at it while driving.
3. The display is easy to glance at: While driving, the display is simply the major routes with the current one highlighted and the car showing the driving direction.  It's very easy to see that you are on the right track.  All other information is via speech.
4. Anticipation rather than waiting for input:  We made two deviations from the directions. The first was missing our exit, and the NeverLost immediately calculated a new route and let us know what it was doing. The second was getting off at an exit to make a rest-stop—in that case, the NeverLost let us get back to the route ourselves without recalculating. Both choices were exactly what we wanted, so we didn't need to fiddle with the interface.
5. Mistakes are easy to correct: The NeverLost keeps a list of the destinations you have input and lets you easily go back to them. A Cancel button lets you back out of all choices. Having these features lets the NeverLost stay out of your way most of the time, because it knows that you can revert to an old state when you need to.

All of these are good ideas for any interface, and if you imagine that your user is distracted, your user interfaces will be even easier to use when they aren't. Of course, a speech interface is not right for all applications, but sounds usually are, and can help to alert users when their attention is required.
When conducting usability tests, keep in mind the environment your users are in, and try to match it. For example, don't gather users to your pristine environment. Go to them if you want to see how they really use your software and the distractions they encounter.

Usable interfaces are all about productivity for those that use your software every day. They feature a UI that anticipates you, but gets out of your way and ensures that your mistakes are recoverable.

Novices require more hand-holding. They may either be new to the application and not yet a power-user, or they may be an occasional user, or they may be watching a demo. In each of these cases, the application must be easy to learn and understand.

When you prepare your user profiles for a UI Design or a product requirements document, it may seem that these two user types are hopelessly irreconcilable, and sometimes you'll specify two completely separate UI's (e.g. wizards in addition to the “normal” UI). Sometimes this is appropriate, but on his blog, Philip Brittan discusses a product we worked on, where features supporting both types were implemented with the same UI.

I just needed to look up something in User Interface Design for Programmers by Joel Spolsky and I was reminded of a great chapter on usability testing (only in the print version). One of the insights is the difference between “usability” and “learnability”. He revisited it in another post.

For casual users, learnability and simplicity are more important than usability and power. In that sentence, by “learnability,” I mean, the ability for novices to figure out how to get tasks done rapidly. By “usability,” I mean only the ability to do tasks in a convenient and ergonomic way without making mistakes and without needing to do repetitive tasks. A data entry system that minimizes keystrokes by prefilling things and automatically jumping from field to field is more usable for experienced users, but it's harder to learn because it behaves unexpectedly to a novice.

The book is worth picking up if you are planning a formal usability test, just to see what kind of things to look for, and what to expect to accomplish. Keep in mind, that if your testers are seeing the UI for the first time, you are testing learnability, not usability.

In XP, requirements gathering is an ongoing conversation validated with acceptance testing. It makes sense, therefore, that any document that results from gathering requirements be easy to edit and give explicit instructions for validation. FitNesse takes this idea to the extreme by using a Wiki front-end to make the requirements document the living, growing entity it needs to be, and it builds in FIT acceptance testing, so that tables representing acceptance tests can be added and run directly from the document.

To use FitNesse, developers add Fixtures to their projects using the FIT framework. These Fixtures can be are added to FitNesse using path and fixture directives. Each page can embed an acceptance test using syntax like the following:

|!-CalculatorColumnFixture-!|
|button|display()|
| |0|
|1|1|
|+|1|
|2|2|
|=|3|

This example tests a calculator by pressing its buttons. CalculatorColumnFixture is a class extending ColumnFixture in FIT.  It has a member variable called button and a method called display().  For each row, button is set to the value in the left column and the return value of display() is checked against the value in the right column. In addition to this table, any text describing the functionality of the calculator can appear on the page. Advanced Fixtures are available for more complex interfaces.

The beauty of FitNesse is that it is implemented as a web-server that serves up the Wiki with the requirements documentation and acceptance tests. It adds FIT directives to the normal Wiki editing syntax and includes version tracking for each page.

Combining the aspects of collaborative document growing and automated acceptance tests, FitNesse is a great addition to the requirements process.

I use CityDesk to manage this site. One of the features is that each page can be tagged with keywords. The problem is, however, that the keywords are typed into a TextBox, not chosen from a List. I am having problems remembering all of the keywords I have decided to use, but I want to be consistent, because I plan on making pages for each keyword when I have enough blog entries (to help organize the archive page).

I wrote a small program to help me do this.  If you have the same problem, check it out.

Update: Cool. Five minutes after posting this, it's already in TK's CityDesk Help Reference—which, by the way, has some good stuff for CityDesk users.

Over at Martin Fowler's blog, I found a link to Framework for Integrated Test (FIT) from Ward Cunningham. I'm just starting to read it now, but I wanted to pass it along because it looks pretty good. Fowler writes:

But I wonder if a language designed for programming is really the right language for writing tests. The point about tests is that they operate by example. They don't try to cover how to handle any value, instead they describe specific scenarios and responses. I wonder if this implies a different kind of programming language is required. Perhaps this is the truly startling innovation in FIT.

More on this later.

If your documentation system is easy to use, add to and edit, you can grow your documentation. Esther Derby has an example. Wikis are particularly suited to this style of collaborative document growing.

Update: Martin Fowler wants us to be more precise with the word “refactoring”. Since the above article uses it in precisely the way he's talking about changing, I thought I'd add this note.

Sharing information in your organization is much easier if you use some kind of knowledge base software. For large enterprises, there are plenty of choices, most aimed at the help desk or call center, but they might be overkill and expensive if all you are trying to do is make a central repository of the bits of information in the collective mind of your company. Here are three low-cost ideas for implementing a simple knowledge repository.

• News Group Software —  There are plenty of easy to install and use newsgroup applications. I've had good experiences with Snitz, which is free and full-featured.
• • Advantages: Familiar interface, every edit is marked with author, support for alerting via e-mail is common
  • Disadvantages: Knowledge is in discussion/serial format, hard to edit old entries, linking to other entries can be hard, requires ability to install software on the server, may not support attachments
• Wiki Software — A Wiki is a website where every page is editable by the reader. The best known public example is the WikiPedia, but the concept started at the Portland Pattern Repository. It's a powerful idea, but depending on the exact software you use, it can be hard for some people to use. Here's a list of wiki implementations.
• • Advantages: Everything is editable, linking is easy, free implementations are available, pro versions track users and edits
  • Disadvantages: Can be hard to use, requires ability to install software on the server, may not support attachments
• Content Management Software — CMS tools can be as expensive as KnowledgeBase tools, but for ease of use and quality of the resulting site, they cannot be beat. I use CityDesk for this site and others (Note: as of 2007, I use RapidWeaver). It averages about $100 a user for contributors and $300 for the site designer, but for small sites, a free version is available.
• • Advantages: Complete control of resulting site, linking is easy, everything is editable, very easy to use, attachments usually supported
  • Disadvantage: Must set up templates, edits not usually logged

For some knowledge bases, a combination of these ideas can work very well—a news group for requests and a Wiki or CMS for official information, or a Wiki for internal use and a CMS for customer facing pages that need to look pretty.

I was somewhat skeptical about Visual Basic for Testers [amazon] because I thought it was going to be focused on automated GUI testing. I have no interest in reinventing WinRunner or TestComplete as a giant list of SendKeys calls. Luckily, neither did Mary Romero Sweeney. She concludes the first chapter with the advice that “Visual Basic should not be considered a substitute for the major test tools but simply a powerful adjunct to them”. Earlier in the chapter, in a section titled Automated Software Testing in the Real World (page 8), she justifies the use of VB and other general-purpose languages for testing with this:

Using Visual Basic and other programming languages on a test project are some of the real-world techniques we end up using when our high-priced automated testing tools just can't get to that important information. This is not meant as a criticism of the automated test tools on the market today [...] However, by default, these tools can't possibly keep up with every need on every test project. Most testing organizations find that at some point they need to resort to using programming-experienced personnel to write code to supplement their testing.

I have to confess that had she not been clear about this goal, I may have abandoned the book here. Confident that this book was going to offer something to add to my arsenal of testing techniques, I read on.

The next few chapters are an introduction to VB focusing on the features that a tester would be interested in (getting data from a database, automating a COM object, manipulating the registry). Experienced VB programmers will likely skip over these chapters. If you want to skim these chapters, I recommend hunting down the asides marked “Tester's Tip” and those set off with dotted lines and a bold centered title. Some of the latter of these are good software development process tips. Ms. Sweeney rightly realized that she was addressing beginning programmers and sought to instill good practice in them from the start. She offers tips on accessibility (p. 52), tab order (p. 54), setting up a directory structure (p. 62), and naming conventions (p. 77). These are all important concepts and it's never too early to learn them.

It's obvious that Ms. Sweeney actually uses VB for testing, because the examples are suited to VB's strengths, not just a hodgepodge of VB features. She spends most of her time on database, COM, registry, file I/O and other Windows API features, revisiting them in later chapters. This is the gap between unit testing (which is best written in the same language of the application) and automated GUI testing (written using off-the-shelf tools). These features are hard to test with a recorder and often best tested in the language you expect your customers to use, which in many cases is VB. If your application exposes a COM interface, for instance, it would be foolish not to use VB.

Chapter 10, Testing the Web with Visual Basic, begins with an explanation that there are tools (some free) for testing websites, but also that there is more you can do with VB. One useful example in this chapter is a testing web browser that exposes the internals of the site. I could see this being useful, for example, for verifying that specific headers are present without constantly viewing source. And since you use the IE control, you can be assured that the page will be rendered exactly as it would in IE. Taking the idea further, the browser could be a flight recorder for functional testing—logging exactly what you've done on a site, so that if you see a bug, it would be easy to reproduce.

The one critique I have of the book is that while the examples are great for learning the features of VB, they are not really testing scripts. In real testing scripts, there would not be visual confirmation—testing scripts run best without a GUI or intervention from the user, only logging information when there is something wrong. The examples are visual because of the visual nature of VB development and the fact that when learning a new language, it's easier to understand if you can see what's going on.  I would have liked to see the idea of self-verification explored more. That being said, Ms. Sweeney says in the introduction that this is not a software testing automation book or a VB manual—that it is enough of both to get started on using VB for automation, and readers are expected to be somewhat familiar with automation practice. She recommends Software Test Automation by Fewster and Graham [amazon] and Automated Software Testing: Introduction, Management, and Performance by Dustin, et al. [amazon] for learning automation practice.

Also, realizing the need to at least mention .NET, this book tacks on two chapters from a VB.NET book. They are not specifically about testing and serve to introduce a VB programmer to the many differences in between VB and VB.NET. It is somewhat of an afterthought, and might be useful to get your feet wet, but I would have liked to see some of the “Testing Tips” or other asides from the earlier chapters.

The book ends with advice directly from some professional test automators and genuinely useful appendices. Appendix D collects some interesting essays for further reading.

If you are in test automation, and running up against the limitations of the available tools, this book is great for learning how to fill that gap. Also, any tester who is interested in learning how to program will find the advice invaluable and the examples relevant to their work. The fact that Ms. Sweeney and her contributors are professional test automators imparting hard-won advice makes this book all the more useful.

Since jUnit (and the xUnit family) is just a verification mechanism, it's a natural fit for service monitoring. This article from JDJ describes using jUnit and JMX to build a service monitoring framework:

This article walks you through the process of setting up a basic service monitor and event handler for a common J2EE n-tier system. Developers of J2EE systems will be able to use JMX4ODP to create testing suites to help them develop more reliable systems. J2EE application administrators will be able to use JMX4ODP to simplify and regulate the management of deployed systems.

Why jUnit?

JUnit bills itself as a regression-testing suite for code objects, but it's not much of a leap to see it as a tool for distributed system diagnostics. JUnit runs tests by instantiating objects, invoking their methods with known inputs, and checking the output against expected returns.The article is very Java and J2EE focused, but the concepts are applicable to any service monitoring project.