One of the biggest issues teams have with Specification by Example is who should
write what and when. So we need a good name that clearly says that everyone should be
involved (and that this needs to happen before the team starts programming or testing),
because we want to use acceptance tests as a target for development.
Test first
is a good
technical name for it, but business users don’t get it, and it doesn’t imply collaboration.
I propose we talk about
specifying collaboratively
instead of test first or writing accep-
tancIt sounds quite normal to put every single numerical possibility into an automated
functional test. Why wouldn’t we do it if it’s automated? But such complex tests are
unusable as a communication tool, and in Specification by Example we need to use tests
for communication. So instead of writing functional tests, let’s talk about
illustrating
using examples
and expect the output of that to be
key examples
to point out that we want
only enough to explain the context properly.
5
Key examples are raw material, but if we just talk about acceptance testing then
why not just dump complicated 50-column, 100-row tables with examples into an
acceptance test without any explanation? It’s going to be tested by a machine anyway.
With Specification by Example, the tests are for humans as well as for machines. We
need to make it clear that there’s a step after illustrating using examples, where we ex-
tract the minimal set of attributes and examples to specify a business rule, add a title and
description, and so on. I propose we call this step
refining the specification
.
6
The result of this refinement is at the same time a specification, a target for develop-
ment, an objective way to check acceptance, and a functional regression test for later.
I don’t want to call this an acceptance test because it makes it difficult to justify why
this document needs to stay in domain language, be readable, and be easily accessible.
I propose we call the result of refining a
specification with examples
, which immediately
points to the fact that it needs to be based on examples but also contain more than just
raw data. Calling this artifact a specification makes it obvious that everyone should care
about it and that it needs to be easy to understand. Apart from that, there’s a completely
different argument as to whether these checks are there to automatically accept software
or to automatically reject the code that doesn’t satisfy what we need.
7
I just don’t want to spend any more time arguing with people who’ve already paid
a license for QTP that it’s completely unusable for acceptance tests. As long as we talk
about test automation, there’s always going to be a push to use whatever horrible con-
traption testers already use for automation, because it’s logical to managers that their
teams use a single tool for test automation. Agile acceptance testing and BDD tools
don’t compete with QTP or tools like that; they address a completely different problem.
5
Thanks to David Evans who suggested this.
6
Thanks to Elisabeth Hendrickson who suggested this name.
7
http://www.developsense.com/blog/2010/08/acceptance-tests-lets-change-the-title-too
xx