Ryan Greenhall

Thoughts on Software Development

Non Descriptive Scenario Method Name

with 38 comments

This is the first entry in a series of posts that provide practical advice on how to avoid or refactor away from common functional testing smells.

Non descriptive scenario method names can easily be identified by the presence of extremely short method names that provide little or no insight into the scenario being executed. For example:

     public class SignInAcceptanceScenarios {

         @Test
         public void signIn() {
         }
     }

So, what sign in scenario is being executed here? Known user? Unknown user? The only way to find out is to read what the method is doing. Variations on this theme include partial attempts to describe the scenario. For example:

     public class SignInAcceptanceScenarios {

         @Test         
         public void signInHappyScenario() {
         }

         @Test         
         public void signInSadScenario() {
         }
     }

Using story and scenario identifiers in scenario method names should also be avoided. Who would like to hazard a guess as to what behaviour the following scenarios describe?

     public class SignInAcceptanceScenarios {

         @Test
         public void story145Scenario1() {
         }

         @Test
         public void story145Scenario2() {
         }

         @Test
         public void story345Scenario1() {
         }
     }

The use of scenario revealing method names in each of the examples would be far more helpful.

Scenario Revealing Method Names

I tend to prefer descriptive method names that describe the scenario being executed. A downside of this style is that it can lead to some very long method names. However, I would rather read a longer method name than a shorter one that has some important part of the scenario missing. Alternatively, if method names get ridiculously long then they can often be rephrased in a more concise manner. Personally I don’t worry too much about long method names. When writing scenarios I aim for an accurate and clear description of the scenario being executed.

Non descriptive scenario methods can easily be renamed, once the scenario has been understood, so that the scenario is clearly described using a scenario revealing method name. For example:

     public class SignInAcceptanceScenarios {

         @Test         
         public void shouldPresentKnownUserWithTheWelcomePage() {
         }

         @Test         
         public void shouldDeclineUnknownUserAndAskThemToTryAgain() {
         }

         @Test
         public void shouldLockAccountAfterThreeSubsequentFailedSignInAttempt() {
         }
     }

A subtle alternative to prefixing scenario method names with ’should’ is to simply provide a description of the expected behaviour. I shall leave it as an exercise for readers to decide which style they prefer. My only advice, as with most things when developing software, is to be consistent. For example:

     public class SignInAcceptanceScenarios {

         @Test
         public void knownUserIsPresentedWithTheWelcomePage() {
         }

         @Test
         public void unknownUserIsDeclinedAndAskedToTryAgain() {
         }

         @Test
         public void userAccountIsLockedAfterThreeSubsequentFailedSignInAttempts() {
         }
     }

Scenario revealing methods are a huge improvement over their less descriptive counterparts, allowing readers to quickly grasp the intended behaviour
of a given functional area. The improved SignInAcceptanceScenarios class really does serve as documentation describing the behaviour of the sign in functionality provided by the application.

Written by Ryan Greenhall

October 14th, 2008 at 7:14 pm

Posted in Uncategorized

The Rise of Modern Legacy Code

with 2 comments

Legacy code, as defined by Michael Feathers in Working Effectively with Legacy Code, is code that is without tests and difficult to change. I would like to suggest that there is a new breed of legacy code emerging with the increasing adoption of Test Driven Development and more recently Behaviour Driven Development; Modern legacy code.

These days, if you are lucky, you may encounter code bases with large suites of both unit and functional tests (perhaps I should say examples, specs or scenarios these days!). However, in some cases, although these tests serve as a reliable regression suite guarding against defects being introduced when adding new features, they may fail to clearly describe the desired behaviour of the system clearly.

In my opinion modern legacy code is code that has good functional and unit test coverage, but where the tests (especially functional) fail to clearly communicate the desired behaviour of the application increasing the cost of maintenance. This is a shame as with a little care and focus on readability it is possible to write functional tests that really do serve as documentation describing the behaviour of the application.

Should the concept of modern legacy be used as an excuse to not write functional tests? Absolutely not. Modern legacy should be a term that is used both to encourage teams to search for better ways to express their tests with the goal of improving readability and to seek out approaches that make functional tests easier to write. Modern legacy is a huge improvement over traditional legacy but there is still plenty of room for improvement in how we ensure that our applications are behaving themselves.

In a future post I plan to identify common functional test smells that I have encountered and to provide practical advice on how to spruce them up so that they describe the behaviour of the application more clearly.

In the meantime have a look at the functional tests in your application. Do they clearly describe the behaviour of your application? A good acid test is to ask a developer from another team if they can understand what your application is supposed to do by reading the functional tests for a given functional area. If you observe a puzzled stare followed by a series of WTFs then you may have some modern legacy code.

Written by Ryan Greenhall

September 30th, 2008 at 2:29 pm

Posted in development, testing

What is The Role of QA?

with one comment

Zero Defects!

What would you think of a QA who told you that they had not found a single defect in the last month?

  • Lazy?
  • Unimaginative?
  • Been on holiday?
  • An accomplished QA?

The answer to this question depends solely on how one views the role of QA in the development process. Let’s consider two hugely contrasting views of the role of QA in the context of two very different styles of development.

Team Foo

Team Foo work in a waterfall manner where each function of development: requirements, analysis, design, coding and QA are performed by different groups, in series. Communication among the different groups is achieved through documentation, with many members of the various teams having never met in person.

After a period of development, QA receive a “completed” system and have to answer the following question, often presented by management: Is the developed system fit for purpose, or in other words can we ship? This is a huge responsibility.

The sad reality for QAs working within this process is that they play no part in defect prevention to assist in ensuring the desired quality. They are only able to either detect the absence of quality through the identification of defects, or acknowledge that the system is of sufficient quality to be released. QAs therefore embark on a quest to find as many defects a possible. After all who wants to to be responsible for “signing off” a defect ridden product into production?

QAs adopt a destructive mindset and actively attempt to break the system using usage scenarios that may or may not be realistic; after all a shared understanding of the acceptance criteria for each feature has not been established. Bug reports are entered into the bug tracking system, assigned to development and the code and fix cycle begins. Bug fixes introduce new defects resulting in rising tensions and frustration between developers and QA and ultimately a late delivery.

Anyone has has worked in similar team setup will tell you that defects are common place. To make matters worse development and QA often engage in debates about whether raised bugs are in fact defects or a misinterpretation of the requirements.

Defects are expected in this environment and in some cases might be used as a measure of quality – “we have found and fixed 89 defects – we have a quality product now”. Any claim of finding zero defects would almost certainly be frowned upon and may be used as a metric to suggest poor performance.

Team Bar

Team Bar form a cross functional team comprised of business people, business analysts, developers and QAs sitting on the same floor, within earshot of one another. The team work collaboratively toward a common goal set by the business who actively work with technical people to ensure that they understand the risks and opportunities provided by the project.

QA are heavily involved in the analysis of requirements and work closely with business analysts to define the acceptance criteria that features must fulfill to be deemed complete. Prior to starting work on a story, developers engage in a conversation with QAs to ensure that the acceptance criteria is understood. During this conversation it is not uncommon for developers to say something like: “We seemed to have missed the timeout scenario. What shall we do in this case?” The QA may reply “Oh yes, I didn’t think of that, let’s retry for a maximum of three times”.

Upon completion of a feature, developers provide a demonstration to both QAs and business analysts to verify that their work fulfills the acceptance criteria defined for the feature. Any defects or missing features identified at this stage can be resolved efficiently by the original developer (or developers if pairing).

The goal for QA is to ensure that quality is built into the product from the beginning. The acceptance criteria used to judge the product’s quality is clearly communicated to the whole team to ensure that the correct product is built. QA will still independently verify that the acceptance criteria for each story has been fulfilled.

A zero defect rate would be celebrated and considered to be a result of close collaboration between QAs, business analysts and developers both prior to and throughout development of features. QAs will have worked hard to identify realistic scenarios prior to features being developed and will have clearly communicated the acceptance criteria by which features will be deemed complete.

Changing Opinions on the focus of QA

It is very interesting to note that QA communities, predating the Agile and Lean movements, acknowledge that the focus of QA has changed considerably as the discipline has matured and developed over the past fifty years.

In the paper, The Growth of Software Testing (1988) , the shifting focus of QA is categorized as follows:

Period Focus
1956 Debugging – Programs are written and then tested by the programmer until they are satisfied that any bugs have been removed.
1957 – 1978 Demonstration – Testing demonstrates that the system does what it is supposed to do.
1979 – 1982 Destruction – The focus of testing changes from proving the system does what is supposed to do, to the identification of defects.
1983 – 1987 Evaluation – Testing becomes an activity throughout the development life cycle.
1988 Prevention – The focus is on defect prevention. Tests were written to demonstrate that software satisfies its specification, to detect faults and to prevent faults.

Perhaps Test Driven Development is not so extreme after all?

Zero Defects – Really?

I have known of several projects that fit the profile of the Bar team. Did they achieve zero defects? The honest answer is no. They did, however, reduce defects to a manageable level. Unpredictable, demoralising cycles of code and fix have been long forgotten on these teams. Where a defect is found it is followed up with a root cause analysis to determine how the defect was introduced so that it can be avoided in the future.

Note that while QAs in the Bar team place heavy emphasis on defect prevention they still ensure that the behaviour required of the system is present and correct according to the specified acceptance criteria.

Summary

In both teams the goal of QA is to ensure that the system is fit for purpose and provides the features agreed with the customer. We have seen two contrasting styles of development and the different roles that QA plays in each. Team Foo focus on the identification of defects and bug reporting whereas Team Bar focus on preventing defects.

The Poppendieck’s have it right when they describe a process that regularly results in defects being found as a broken process. Defects should be a rarity rather than the norm. If identification of defects is commonplace in your organisation then you have a problem.

Software development has suffered for too long as a result of poor collaboration between QA and development. What can you do to encourage collaboration between these two groups? You could try starting with the following statement: As a company we value defect prevention over defect identification.

Written by Ryan Greenhall

September 28th, 2008 at 7:00 pm

Posted in agile lean

Behaviour Driven Development: By Example

with 2 comments

To coincide with the release of the wonderful JBehave 2, I have made available an introductory BDD article, Behaviour Driven Development: By Example, using JBehave 2 as the example BDD framework.

The goal of the article is to introduce newcomers to BDD using a worked example, documenting a BDD style development process starting with a story scenario through to working code.

Comments and suggestions for improvements are most welcome.

If you are working in a Java development team who are envious of RSpec’s Story Runner, and JRuby is a little too radical for your environment, then JBehave 2 is just what you have been waiting for.

Congratulations to the JBehave 2 team!

Written by Ryan Greenhall

September 6th, 2008 at 10:09 am

Posted in BDD