Jazoon 2013 – Spock: boldly go where no test has gone before

24. October, 2013

Jazoon 2013 badgeIf you look at your tests, do you see a lot of repetitive patterns? Or are you looking for a way to express your intent more easily? Then, Spock might be for you. The talk “Spock: boldly go where no test has gone before” by Andres Almiray (slides on slideshare) gave a good introduction to the framework. Here is an example:

class HelloSpock extends spock.lang.Specification {
    def "length of Spock's and his friends' names"() {
        expect:
        name.size() == length

        where:
        name     | length
        "Spock"  | 5
        "Kirk"   | 4
        "Scotty" | 6
    }
}

In a nutshell, this creates three tests, running the assertion name.size() == length for each tuple of values. Here is an example how to test a stack.

But this is just one of many ways which you can use to describe tests that Spock should execute for you. The page “WhySpock” lists 10 reasons.


Selenium vs. ZK

30. August, 2013

Testing ZK applications using Selenium can be a drag. Selenium offers a lot of tool to test traditional request-response cycle applications. But it relies heavily on stable element IDs and submitting whole forms.

ZK, on the other hand, sends just a skeleton page to the browser and from then, builds everything with JavaScript. Instead of posting a whole form, it submits individual field values (to trigger validation). And it assigns random IDs to reach element.

At first glance, the two don’t seem to be a perfect match. But there are a couple of simple tools that will make your life much more simple.

Getting Stable IDs for Tests

One solution here is to write an ID generator which always returns the same IDs for each element. But this is tedious, error-prone and sometimes impossible.

A better solution is to attach a custom attribute to some elements which doesn’t change. A beacon. If you have a central content area, then being able to find that will make your life much more simple because whatever else you might seek – it must be a child of the main content.

To do that, add a XML namespace:

    <zk xmlns:cl="client" xmlns:ca="client/attribute">

to the top of your ZUL file, you can now use a new attribute ca:data-test-id="xxx" to set a fixed attribute on an element. In Selenium code, you can now locate this element using this code:

    By.xpath( "//div[@data-test-id = 'xxx']" )

I suggest to use this sparingly in order not to bloat your DOM. But a few of them for fast moving targets will make your tests much more stable.

Debugging the DOM

Sometimes, your life would be much more simple if you could see what the DOM was when your test failed. Here is a simple trick to get a HTML fragment from the browser:

    protected JavascriptLibrary javascript = new JavascriptLibrary();

    public String dump( WebElement element ) {
        return (String) javascript.executeScript( driver,
            "return arguments[0].innerHTML", element );
    }

You can now use JTidy and JDOM to convert the fragment first into W3C XML nodes and then into JDOM elements:

    public org.w3c.dom.Document asDom( WebElement parent ) {
        String html = dump( parent );
        Tidy tidy = new Tidy();
        tidy.setShowWarnings( false );
        tidy.setErrout( new PrintWriter( new NullOutputStream() ) );
        org.w3c.dom.Document dom = tidy.parseDOM(
           new StringReader( html ), null );
        return dom;
    }

    public static org.jdom2.Document asJDOM( org.w3c.dom.Document content ) {

        // JDOM chokes on: org.jdom2.IllegalNameException: The name "html PUBLIC "-//W3C//DTD HTML 4.01//EN"" is not legal for JDOM/XML DocTypes: XML names cannot contain the character " ".
        Node docType = content.getDoctype();
        if( null != docType ) {
            content.removeChild( docType );
        }

        DOMBuilder builder = new DOMBuilder();
        org.jdom2.Document jdomDoc = builder.build( content );

        return jdomDoc;
    }

Another great use case for this is a default exception handler for tests which saves a screenshot and a copy of the DOM at the time a test fails. No more guessing why something didn’t happen.


The Difference Between Unit and Integration Tests

14. February, 2013

A unit test checks a certain feature of the software. When it fails, you usually know exactly where to look for a place to fix the problem. They are short, compact and fast.

They come at a cost, though: You have to replicate the setup code in your test.

This is an important point. To use a feature of your software, you always have some set up to do. This set up has to exist in your production code. For unit tests, you will have to copy some parts of this code into your tests because usually, the setup of the production code simply isn’t fine grained or flexible enough to be useful for tests. Think tests which check the error handling: Your production code usually can’t build mock objects that raise errors when certain methods are being invoked.

An integration test reuses the production configuration. It tests many features at once, in the same order and with the same or at least a very similar environment that your final application will use. They are high level and often much easier to set up. But that comes at a cost:

  • They are slow
  • When they fail, there will be many places which could cause the issue, so fixing them is more expensive
  • They break more often because you have more dependencies
  • Setting up a test for the “success” scenario will be simple but injecting mock objects to cause exact error states will be much harder

So be aware what kind of test you’re writing.


Jazoon 2012: Large scale testing in an Agile world

28. June, 2012

Alan Ogilvie is working at a division of IBM responsible for testing IBM’s Java SE product. Some numbers from his presentation:

  • A build for testing is about 500MB (takes 17 min to download to a test machine)
  • There are 20 different versions (AIX, Linux, Windows, z/OS * x86, power, zSeries)
  • The different teams create 80..200 builds every day
  • The tests run on heaps from 32MB to 500GB
  • They use hardware with 1 to 128+ cores
  • 4 GC policies
  • More than 1000 different combinations of command line options
  • Some tests have to be repeated a lot of time to catch “1 out of 100” failures that happen only very rarely

That amounts to millions of test cases that run every month.

1% of them fail.

To tame this beast, the team uses two approaches:

  1. Automated failure analysis that can match error messages from the test case to known bugs
  2. Not all of the tests are run every time

The first approach makes sure that most test failures can be handled automatically. If some test is there to trigger a known bug, that shouldn’t take any time from a human – unless the test suddenly succeeds.

The second approach is more interesting: They run only a small fraction of the tests every time the test suite is started. How can that possibly work?

If you run a test today and it succeeds, you will have some confidence that it still works today. You’re not 100% sure but, well, maybe 99.5%. So you might skip this test today and mark it as “light green” in the test results (as opposed to “full green” for a test that has been run this time).

What about the next day? You’re still 98% sure. And the day after that? Well, our confidence is waning fast, so we’re still pretty sure – 90%.

The same goes for tests that fail. Unless someone did something about them (and requested that this specific test is run again), you can be pretty sure that the test would fail again. So it gets light red unlike the tests that failed today.

This way, most tests only have to be run once every 4-5 days during development.

Why would they care?

For a release, all tests need to be run. That takes three weeks.

They really can’t possibly run all tests all the time.


Mocking AJAX in jQuery

31. May, 2011

When developing small web applications, it would be great if I could mock AJAX requests.

Apparently, Jonathan Sharp had the same problem and created a solution: Mock Your Ajax Requests with Mockjax for Rapid Development

Very nice. Thank you!

 

 

 


JaCoCo: Successor of EclEmma

15. March, 2011

If you’re using EclEmma to check the code coverage, you should have a look at JaCoCo. If you’re not using code coverage, yet, you really should.

JaCoCo (Java Code Coverage Library) is the successor of EclEmma where the developers applied all the lessons learned during the time with EclEmma.


Feeling like Hercules?

21. October, 2010

In the ancient mythology, Hercules slew the Hydra, a monster which could grow a new head for each one cut off. Does that remind you of your programming job? The punchline is: Hercules didn’t conquer his opponent with brute force but with brains.

So if your product sprouts two new bugs for each you fix, it’s time to take the fingers away from the keyboard and start rubbing your head.

The solution? Unit tests. Each test makes sure that a piece of code works as it should. Even if each tests looks dump and futile, in a mass, they make sure that all the code which you didn’t change still behaves as you expect.

Kudos to Thomas E. Deutsch for the idea of the Hydra.