Testing The Impossible: Inserting Into Database

5. June, 2009

Tests run slow when you need a database. An in-memory database like HSQLDB or Derby helps but at a cost: Your real database will accept some SQL which your test database won’t. So the question is: How can you write a performant test which uses the SQL of the real database?

My solution is to wrap the JDBC layer. Either use a mock JDBC interface like the one provided by mockrunner. Or write your own. With Java 5 and varargs, this is simple:

public int update (Connection conn, String sql, Object... params) throws SQLException {
    PreparedStatement stmt = null;
    try {
        stmt = conn.prepareStatement (sql);
        int i = 1;
        for (Object p: params) {
            stmt.setObject(i++, p);
        }
        return stmt.executeUpdate ();
    }
    finally {
        stmt.close ();
    }
}

Put all these methods into an object that you can pass around. In your tests, override this object with a mockup that simply collects the SQL strings and parameter arrays. You can even mix and match: By examining the SQL string, you can decide whether you want to run a query against the database or handle it internally.

This way, you can collect any newly created objects but still load some background data from the database (until you get bored and make the query methods return predefined results).

In the asserts, just collect all the results into a big String and compare them all at once.

Notes: The code above is a bit more complicated if you allow null values. In this case, you need to tell JDBC what the column type is. My solution is a NullParameter class which contains the type. If the loop encounters this class, then it calls setNull() instead of setObject().

Leave a Comment » | Software | Tagged: , , , | Permalink
Posted by digulla

Testing the Impossible: Asserting Several Values At Once

23. March, 2009

So you want to check several values at once, maybe because that helps you to locate the error more easily. Simple:

int v1 = list1.size();
int v2 = list2.size();
int v3 = list3.size();

assertEquals (
      "list1=5\n"
    + "list2=2\n"
    + "list3=0\n"
    , "list1="+v1+"\n"
    + "list2="+v2+"\n"
    + "list3="+v3+"\n"

2 Comments | Software | Tagged: , | Permalink
Posted by digulla

Verifying Results in Tests

5. March, 2009

So you finally got yourself writing a test against a database. In the test, you have to verify that a row in the table is correct, so you write:

assertEquals (
    "2008-09-16-13.50.18.000000;;;;1;2008-08-07;2008-08-07;JUNIT;2008-09-16;t0001;001;;Doe;Jane;;Street;2;Doe Jane;;;;;X;2575;John;;;;US;E;;01;;;;125;01425;0;Shop;;;DOE;JANE;JOHN;032;1;;0001010301;;;;",
    dumpRow(key));

and it sucks. Yeah, junit will notify you when something in that row is wrong and if you have a cool IDE, you can even compare the fields … but it still sucks. If one of the fields in the middle change, you have to scroll and eyeball-diff, wasting your time. The solution is pretty simple:

assertEquals (
    "2008-09-16-13.50.18.000000\n"
    + "\n"
    + "\n"
    + "\n"
    + "1\n"
    + "2008-08-07\n"
    + "2008-08-07\n"
    + "JUNIT\n"
    + "2008-09-16\n"
...
    dumpRow(key).replaceAll(";", "\n");

Instead of dumping the data in a single long string, split it into lines so you can compare fields side by side and without scrolling sideways.

Leave a Comment » | Software | Tagged: , , , | Permalink
Posted by digulla

Mutation Testing

23. February, 2009

From the website:

How do you know your test suite is “good enough”?

One of the best ways to tell is mutation testing. Mutation testing seeds artificial defects (mutations) into a program and checks whether your test suite finds them. If it does not, this means your test suite is not adequate yet.

Read more about Javalanche on the website.

Leave a Comment » | Software | Tagged: , , | Permalink
Posted by digulla

Testing the Impossible: Hardware

5. February, 2009

Victor Lin asked on StackOverflow.com:Should I write unit test for everything?

In his case, he wanted to know how to test an application which processes audio: Reads the sound from a microphone, does something with the audio stream, plays the result on the speaker. How can you possibly test a class which reads audio from a microphone? How can you test playing sound on a speaker?

As Steve Rowe pointed out: Use a loopback cable. Play a well defined sound on the speaker and check what the microphone receives.

I suggest to move this test case into a separate test suite so you can run it individually. Print a message to plug in the loopback cable before the test and wait for the user to click “OK”. This is a unit test but not an automated one. It covers the setup steps of the hardware.

The next thing you will want to do is to check the code between the mic and speaker parts. These are now no longer dependent on the hardware and therefore simple to test.

Leave a Comment » | Software | Tagged: , , | Permalink
Posted by digulla

Background Unit Testing

2. February, 2009

I just found this article: Background Unit Testing: New Evolutions in Unit Testing and IDE Integration
The idea is that your IDE should run the unit tests in the background just as it runs the compiler in the background. Compelling. There are already two implementations for Eclipse: JUnitMax from Kent Beck and Infinitest.

Leave a Comment » | Software | Tagged: , , | Permalink
Posted by digulla

Testing: Pay in Advance or Afterwards?

2. February, 2009

In a recent post, I talked about people ignoring the cost of some decision. In his blog “Joel on Software”, they talk about the same thing: How easy it is to fall into the “we must have strict rules” trap to protect ourselves against some vague  fear of failure. Only, humans are really bad at sticking to rules. Or are they? Maybe it’s just that reality doesn’t care so much about rules because things change. If you built your castle on the belief how well strong walls will protect you, the swamp around the basement is not going to care. You’re going down, chummer.

So we end up with a lot of rules which make exactly one thing simple: To assign blame. I’ve been working for a big company where we have a strict process how projects were to be set up. There were lots of documents and forms and comittees how to start a project and a lot of documents describing how to end it (put it into production, what documents to file, who to inform, you name it). It was a great process (in the sense of “big”, mind). The actual writing of the code was explained in a document which contained a single page. On that single page, they talked on how they would strive to write excellent, error free code and that they would use a proven strategy, the waterfall model.

They built a huge, shiny castle on nothing.

If you go to a bank and tell them you have lots of $$$ and you need to pay some big bill somewhere in the future, their first question will be: How you want to make that money work for you in the meantime? Just letting it rot under your desk is not very smart, right? You should invest it somewhere, so you will have $$$$$ or even $$$$$$$ when it comes to pay the bill. Which makes sense. Contrary to that, when we write software, we tend to spend our money first instead of parking it in a safe place where it can return some revenue, being ever vigilant to be able to pay as the bills show up. Which is harder than just sitting back and relying on some mythical process someone else has written on a piece of paper a long time ago.

So when you ask: “Should I write tests for all my classes? For every line of code? How should I spend my money?” Then my answer will be: I don’t know. How can I? I know nothing about your project. But I can give you some ideas how to figure it out yourself.

“Should I write tests for all my classes?” That depends on what these classes are meant for. The more low-level the code, the more tests you should have. Rule of thumb: Tests yield more result in the basement. Make sure the ground you’re building on is sound. And behaves as you expect. The upper levels are mostly built from lego bricks. They are easy to take apart and reshape. They are exchangable, so you can get away with fewer tests. But every bug in the foundation will cripple anything above it.

“For every line of code?” No. Never. 1. It’s not possible. 2. Maintaining the tests will cost more than the real code. 3. Tests are more simple than the real code but you still make a constant amount of mistakes per lines of code. So this will only drive the number of bugs through the roof. 4. Strict, fixed rules never work (note the paradox).

“How should I spend my money?” One word: Wisely. Wisely means to think about your specific problem and find the unique solution. Do you know in advance how much each piece will cost? No. So the best you can do is a staggered approach: Invest a bit of money, check how it plays out. If it works well, spend more. If it doesn’t, scratch it, learn, try something else. Which you will be able to do since you didn’t put all your money on a single horse.

So what if your three month venture into agile development didn’t really work out? All you lost is three months. Other projects are deemed a “success” after going over budget by 100%, using twice the time that was estimated (and none of them were shorter than a year). But you will still have learned something. You paid for it, that wisdom is yours.

Use it wisely.

1 Comment | Software | Tagged: , , , , | Permalink
Posted by digulla

Writing Testable Code

1. December, 2008

Just stumbled over this article: “Writing Testable Code“. Apparently, it’s a set of rules which Google uses. While I’m not a 100% fan of Google, this is something every developer should read and understand.

Leave a Comment » | Software | Tagged: | Permalink
Posted by digulla

How To Be Agile

29. November, 2008

The article “When Agile Projects Go Bad” got me thinking. I’ve talked to many people about XP and Agile Development and TDD and the usual question is: “How do we make it work?” And the next sentence is: “This won’t work with us because we can’t do this or that.”.

This is a general misconception which comes from the … uh … “great” methodologies which you were taught in school: the waterfall model, the V model, the old dinosaurs. They told you: “You must follow the rules to the letter or doom will rain on your head!” Since you could never follow all the rules, they could easily say “Told you so!” when things didn’t work out.

Agile development is quite different in this respect. First of all, it assumes that you’re an adult. That you have a brain and can actually use it. It also assumes that you want to improve your situation. It also assumes nothing else.

When a company is in trouble, it will call for help. Expensive external advisers will be called, they will think about the situation for a long time (= more money for them). After a while (when the new yacht is in the dry), they will come up with what’s wrong and how to fix it. Did you know that in most companies in trouble, the external advisers will just repeat what they heard form the people working there?

It’s not that people don’t know what’s wrong, it’s just not healthy to mention it … at least if you want to work there. So people walk around, with the anger in their hearts and the fist in the pocket and nothing will happen until someone from the outside comes in and states the obvious. Can’t happen any other way because if it could, you wouldn’t be in this situation in the first place.

Agile Development is similar. It acknowledges that you’re smart and that you know what’s wrong and that you don’t have the power to call in help. What it does is it offers you a set of tools, things that have worked for other people in the past and some of them might apply to you. Maybe all. Probably not. Most likely, you will be able to use one or two. That doesn’t sound like much but the old methodologies are pretty useless if you can’t implement 90%+. Agile is agile. It can bend and twist and fit in your routine.

So you’re thinking about doing TDD. Do you have to ask your boss? No. Do you have to get permission from anyone? No. Do you have to tell anyone? No. Can you do it any time you like, as often as you like, stop at will? Yes. If it doesn’t work for you in your situation, for the current project, then don’t use it. No harm done, nothing gained either.

But if you can use it, every little bit will help. Suddenly, you will find yourself to be able to deliver on time. Your code will work and it will be much more solid than before. You will be able to do more work in less time. People will notice. Your reputation will increase. And eventually, they will be curious: How do you do it? “TDD.” What’s that?

You win.

Be agile. Pick and choose. Pick what you think will work, try it, drop it if it doesn’t deliver. And if it works, try the next thing. Evolve. Become the better you.

Agile is not a silver bullet. It won’t miraculously solve all your issues. You still have to think and be an adult about your work. It’s meant to be that way. I don’t do every Agile practice every day. Sometimes, I don’t even TDD (and I regret every time). But I always return because life is just so much more simple.

Leave a Comment » | Software | Tagged: , | Permalink
Posted by digulla

Testing the Impossible: Rules of Thumb

19. November, 2008

When people say “we can’t test that”, they usually mean “… with a reasonable effort”. They say “we can’t test that because it’s using a database” or “we can’t test the layout of the UI” or “to test this, we need information which is buried in private fields of that class”.

And they are always wrong. You can test everything. Usually with a reasonable effort. But often, you need to take a step back and do the unusual. Some examples.

So your app is pumping lots of data into a database. You can’t test the database. You’d need to scrap it for every test run and build it from scratch which would take hours or at least ages. Okay. Don’t test the database. Test how you use it. You’re not looking for bugs in the database, you’re looking for bugs in your code. Saying “but some bugs might get away” is just a lame excuse.

Here is what you need to do: Identify independent objects (which need no other objects stored in the database). Write tests for those. Put the test data for them in an in-memory database. HSQLDB and Derby are your friends. If you must, use your production database but make the schema configurable. Scrap the tables before the test and load them from clean template tables.

So you need some really spiffy SQL extensions? Put them in an isolated place and test them without everything else against the real database. You need to test that searching a huge amount of data works? Put that data in a static test database. Switch database connections during the tests. Can’t? Make that damn connection provider configurable at runtime! Can’t? Sure you can. If everything else fails, get the source with JAD, compile that into an independent jar and force that as the first thing into the classpath when you run your tests. Use a custom classloader if you must.

While this is not perfect, it will allow you to learn how to test. How to test your work. Testing is always different just like every program is different. Allow yourself to make mistakes and to learn from them. Tackle the harder problems after the easier ones. Make the tests help you learn.

So you have this very complex user interface. Which you can’t test. Let alone starting the app takes ten minutes and the UI changes all the time and … Okay. Stop the whining. Your program is running on a computer and for same inputs, a computer should return the same outputs, right? Or did you just build a big random number generator? Something to challenge the Infinite Improbability Drive? No? Then you can test it. Follow me.

First, cut the code that does something from the code that connects said code to the UI. As a first simple step, we’ll just assume that pressing a button will actually invoke your method. If this fails for some reason, that reason can’t be very hard to find, so we can safely ignore these simple bugs for now.

After this change, you have the code that does stuff at the scruff. Now, you can write tests for it. Reduce entanglement. Keep separate issues separate. A friend of mine builds all his code around a central event service. Service providers register themselves and other parts of the code send events to do stuff. It costs a bit performance but it makes testing as easy as overwriting an existing service provider with a mock up.

Your software needs an insanely complex remote server? How about replacing this with a small proxy that always returns the same answers? Or at least fakes something that looks close enough to a real answer to make your code work (or fail when you’re testing the error handling).

And if you need data that some stubborn object won’t reveal, use the source, Luke (download the source and edit the offender to make the field public, remove “final” from all files, add a getter or make it protected and extend the class in the tests). If everything else fails, turn to java.lang.reflect.Field.setAccessible(true).

If you’re using C/C++, always invoke methods via a trampoline: Put a pointer somewhere which contains the function to call and always use that pointer instead of the real function. Use header files and macros so no human can tell the difference. In your tests, bend those pointers. The Amiga did it in 1985. #ifdef is your friend.

If you’re using some other language, put the test code in comments and have a self-written preprocessor create two versions that you can compile and run.

If all else fails, switch to Python.

Leave a Comment » | Software | Tagged: , , | Permalink
Posted by digulla

« Previous Entries
Follow

Get every new post delivered to your Inbox.

Join 102 other followers