Plan

From Perl QA
Jump to: navigation, search

In the Perl and TAP world, a plan is a declaration of the number of tests you intend to perform within a testset.

Contents

Why do I need a plan?

There is a good discussion of this issue on the perl-qa list which includes comments by Michael Schwern and Ovid who are the Test::More and Test::Most maintainers respectively. Apparently this comes up every once in a while and is a bit of a contentious issue. Here are the highlights:

Reasons to not use a test plan

  1. It's annoying and takes time.
  2. It is arguably not worth the time because test scripts won't die without the test harness noticing except in some relatively rare cases.
  3. Test::More can count tests as it goes, so why should the developer?
  4. If you do use a test plan and need to skip some tests in certain situations, then you have the additional pain of needing a SKIP{} block.

Reasons to use a test plan

  1. It only takes a few seconds to do. If it takes longer, your test logic is probably too complex.
  2. If some yahoo puts an exit(0) in the code somewhere, your test will complete successfully without running the remaining test cases. An observant human may notice that the screen output doesn't look right, but in an automated test suite it could go unnoticed.
  3. A developer might accidentally write test logic so that some tests are never run.
  4. You can't really have a progress bar without knowing ahead of time how many tests will be run. This is difficult to do through introspection alone.


Alternatives

  1. Test::Simple has new functionality which allows developers to call a method called done_testing() at the end of the test script. This will eventually be moved to into the Test::More module.
  2. Test::Most also allows developers to call a method called all_done() at the end of the test script.

The alternatives fix the problem where code has an exit(0) in it. It doesn't completely fix the problem of logic which unintentionally skips tests though.

In short, its safer and more correct to use a plan, but the chances of this safe-guard actually saving the day are arguably low especially for simpler test suites. Using done_testing() is a middle ground. Its probably not a huge deal whatever your preference.

See also Eric Johnson's summary of reasons for and against test plans which is the original source of this article.

Has this feature been useful to anyone in the real world?

A few people mention that this feature has been useful to them in the real word. This includes Larry Wall. Michael Schwern says the feature originates with Larry, more than 20 years ago.

Do other languages have this feature?

None of the xUnit type testing suites has the test plan feature. I haven't come across any examples of this feature being used in any other programming language. It seems to be unique to Perl.

Why does Test::Builder/Test::More force me to declare a plan?

Why can't a program count its own tests?

See Why can't a program count its own tests.

Further Discussion

Personal tools