Overview
Introduction

Pyccuracy is a Behavior-Driven Acceptance Testing framework (more on that in the next section).


It serves two crystal clear, yet very hard to achieve, purposes:


  1. Encouraging Test-Driven Acceptance Testing

    This means writing acceptance tests prior to the tasks that describe them. This way you take Test-Driven-Development one step further and push yourself towards the right direction when it comes to the application UI (user interface).

  2. Providing an easy and clear language to write tests

    It’s really important to write acceptance tests that prove your app works. Most of the time people don’t do it using some (or all) of the excuses below:

    1. Acceptance Tests are fragile - People say they don’t do them because they break easily.
    2. Acceptance Tests are slow - Running the acceptance tests take more time than we can afford.
    3. Acceptance Tests are hard to write - The xyz acceptance testing framework being used is too hard to write tests on.
    4. Unit Test Coverage is good enough - We have unit test coverage anyway, so why do Acceptance tests anyway?

Well, all of the above are just lame excuses on why not to write Acceptance tests.


In my point of view, if you can’t prove that something works as expected, then it doesn’t. Acceptance tests serve one purpose: Demonstrating, beyond shadow of doubt, that your application works as the customer expects.


They are real concerns, though. Let’s tackle each of them individually.


Fragility

First, the fragility issue. To determine if your tests are fragile, you first need to define a proper fragile test. In my opinion a fragile acceptance test is one that breaks even though you haven’t changed the behavior of the UI the test works on.


Fragility is often a problem with how you are writing your tests, other than the actual tests. Acceptance tests need to test behavior. It’s ok to write acceptance tests that verify that the UI is in accordance to any number of standards you define (Pyccuracy even supports that concept). Just bare in mind that these tests are not acceptance tests. They are smoke tests or whatever you want to call them, and they are going to be VERY fragile, because that’s their purpose: breaking if the UI changes. So it’s best to have as few of those as you possibly can.


Pyccuracy encourages less-fragile tests by having a concise language, yet expressful, for writing your tests.


Running Speed

Running acceptance tests is always going to be slower than unit tests, for the simple fact that they are wired tests (meaning they connect to real resources, which are always slower than mocks or stubs).


The key thing here is that slower does not need to translate into slow. Even though we rely a lot on the underlying testing framework (i.e. Selenium), we’ve optimised Pyccuracy quite a lot in order to have great performance.


We still need real project data in order to determine how slow or fast Pyccuracy really is, since our test scenarios are too simple for that purpose.


Language

With Pyccuracy, the proposition is that general-purpose programming languages might not be the best language to describe acceptance tests.


Pyccuracy provides a domain-specific language (DSL) for writing acceptance tests (you can see an example at the Sample Test section). The purpose of using a DSL is to have clear easy-to-write tests.

This enables scenarios such as a wiki that holds all your tests as well-organized pages or using the acceptance tests to enable better discussion with clients on what a story should cover.

Pyccuracy uses a Regular-Expression driven approach to the DSL, which makes it really easy to improve and refine the language.


Unit Test Coverage

Even with good unit test coverage, you still need acceptance tests.


The reason behind this is that they aim at different targets. While unit tests aim at proving that your code does what it should do, acceptance tests prove that your application does what the client expects.

Aside from that, unit tests are mocked tests, meaning that they can’t depend on any external resources, while acceptance tests are wired tests, meaning they use your application in the same way the client is going to use it.

So, saying you don’t need acceptance tests because you got good unit test coverage is nonsense.


Behavior-Driven Testing

Behavior-Driven Testing (BDT) is a concept derived from Behavior-Driven Development [1]. The information in the Behavior-Driven Development website is well worth your time.


Whereas BDD uses a parallel to Test-Driven Development [2] to specify unit tests as scenarios, BDT uses the same ubiquitous language to describe acceptance tests.


Defining the acceptance tests in a language that’s common for both developers and clients alike has many benefits:

  1. It’s very easy to define what a story really represents in terms of business value with the client.
  2. There’s no impedance mismatch between what the developers think a story is and what the client expects.
  3. The stories get auto-documented as to what their behavior is [3]. This is really important as to get new members in the team up to speed faster, as well as to increase confidence when performing refactoring to the UI.

Language Support

Pyccuracy also supports multiple languages. This has been a really important goal for Pyccuracy team since release 0.1.


If you want to write clear tests, what clearer language to use than the one you currently speak?


We understand that there are non-english speakers out there doing amazing things, and we want to empower that (the team behind Pyccuracy 0.1 is composed only of Brazilians - but we encourage anyone anywhere to join).


If your language is not covered in Pyccuracy yet, it’s really easy to create a language file. Shouldn’t take more than half an hour. Just contact us using Pyccuracy’s e-mail group [4], and we’ll point you in the right direction.


Currently Supported Languages:

  1. English
  2. PortuguÍs Brasil sem acentos (Portuguese Brazil)

Sample Test

A typical Pyccuracy test would be something like:


			As a Google User
			I want to search Google
			So that I can test Pyccuracy

			Scenario 1 - Searching for Hello World
			Given
			  I go to "http://www.google.com"
			When
			  I fill "q" textbox with "Hello World"
			  And I click "btnG" button
			Then
			  I see "Hello World - Google Search" title

			Scenario 2 - Searching for Monty Python
			Given
			  I go to "http://www.google.com"
			When
			  I fill "q" textbox with "Monty Python"
			  And I click "btnG" button
			Then
			  I see "Monty Python - Google Search" title
		

As you can see, that’s pretty clear and yet, very intentful. Without resorting to anything other than the test you could understand what’s happening.


Other than that, this test proves an aspect of google (that the specified search text should be in the title).


With a good test suite you can easily cover all business value of your application and use that as proof that your app does what it should be doing.


Conclusion

Pyccuracy should speed up the creation of acceptance tests and improve on the way your dev team communicates with the client when related to what business value and behavior the stories contain.


Footnotes

[1] Behavior-Driven Development - Know more in http://behaviour-driven.org/.
[2] Test-Driven Development - Know more in http://en.wikipedia.org/wiki/Test-driven_development.
[3] It’s all about behavior - http://behaviour-driven.org/ItsAllBehaviour
[4] Google Group for Pyccuracy - http://groups.google.com/group/pyccuracy

 
Pyccuracy is an open-source Python project with an OSI license.

The entry barrier for coding in Pyccuracy is really small, since the codebase is itself small as you can see in the stats above.

If you think you can contribute, join our dev list, check our JIRA server for the open tickets for the current release and start coding.