Jazoon 2012: Why you should care about software assessment

28. June, 2012

Tudor Girba gave a presentation at the Jazoon about a topic that is very dear to him: Software assessment. To quote:

What is assessment? The process of understanding a given situation to support decision-making. During software development, engineers spend as much as 50% of the overall effort on doing precisely that: they try to understand the current status of the system to know what to do next.

In other words: Assessment is a process and a set of tools to help developers to make decisions. They typical example is a bug shows up and you need to fix it. That raises the usual questions:

  1. What happened?
  2. Why did it happen?
  3. Where did it happen?
  4. How can I fix it?

As we all know, each of these steps can be difficult. As an extreme example, someone mentioned selling software to the NSA. It crashed. The NSA calls the developer:

NSA: “There is a problem with your software.”

You: “Who am I talking with?”

NSA: “Sorry, I can’t tell you that.”

You: “Well … okay. So what problem?”

NSA: “I can’t tell you that either.”

You: “… Can you give me a stack trace?”

NSA: “I’m afraid not.”

Unlikely but we all know similar situations. Even seasoned software developers are guilty of giving completely useless failure reports: “It didn’t work.” … “What are you talking about? What’s ‘it’?”

Tudor gave some nice examples how he used simple assessment tools that allow him to query log files and sources of some application to locate bugs, locate similar bugs and help to find out why some part doesn’t behave well. Examples:

  1. An application usually returns text in the user’s language but some rare error message is always in German. Cause: When the error message was created, the code called Locale.getDefault()
  2. Several other places could be found that showed the same behavior by searching the source code for places where Locale.getDefault() was called either directly or indirectly. A test case was added to prevent this from happening again.
  3. Some cache would have a hit ratio of less than 50%. Analyzing the logs showed that two components used the same cache. When each got their own cache, the hit ratios reached sane levels.

So assessments allow you to do strategic planning by showing you all the dependencies that some part of the code has (or the whole application).

In a spike assessment, you can analyze some small part to verify that a change would or could have the desired effect (think performance).

Did you know that developers spend about 50% of the time reading code? If tools can help them understand some piece of code faster, that makes them more productive. Unfortunately, today’s tools are pretty limited when it comes to this. Eclipse can show me who calls Locale.getDefault() but it can’t show me indirect calls.

Worse: If the developer makes the wrong decision because she couldn’t see all the important facts, then these often have a huge impact.

Another important aspect is how you use metrics. Metrics are generally useful but the same is not true for every metric. Just like you wouldn’t copy unit tests from one project to the next, you need to reevaluate the metrics that you extract from each project. Some will just be a waste of time for certain projects.

My comments:

We really, really need better tooling to chop data. IDEs should allow me to run queries against my source code, collect and aggregate data and check the results in unit tests to validate design constraints.

It was also interesting to see how Tudor works. He often uses simple words which can be misleading. But when you look at the slides, then there was this graph about some data points. Most graphs show a linear Y axis with the ticks evenly spread. He uses a different approach:

Usual diagram to the left, Tudor’s version to the right

Related links:

Leave a Comment » | Conference, Software | Tagged: , , , | Permalink
Posted by digulla

Jazoon 2012: Large scale testing in an Agile world

28. June, 2012

Alan Ogilvie is working at a division of IBM responsible for testing IBM’s Java SE product. Some numbers from his presentation:

  • A build for testing is about 500MB (takes 17 min to download to a test machine)
  • There are 20 different versions (AIX, Linux, Windows, z/OS * x86, power, zSeries)
  • The different teams create 80..200 builds every day
  • The tests run on heaps from 32MB to 500GB
  • They use hardware with 1 to 128+ cores
  • 4 GC policies
  • More than 1000 different combinations of command line options
  • Some tests have to be repeated a lot of time to catch “1 out of 100” failures that happen only very rarely

That amounts to millions of test cases that run every month.

1% of them fail.

To tame this beast, the team uses two approaches:

  1. Automated failure analysis that can match error messages from the test case to known bugs
  2. Not all of the tests are run every time

The first approach makes sure that most test failures can be handled automatically. If some test is there to trigger a known bug, that shouldn’t take any time from a human – unless the test suddenly succeeds.

The second approach is more interesting: They run only a small fraction of the tests every time the test suite is started. How can that possibly work?

If you run a test today and it succeeds, you will have some confidence that it still works today. You’re not 100% sure but, well, maybe 99.5%. So you might skip this test today and mark it as “light green” in the test results (as opposed to “full green” for a test that has been run this time).

What about the next day? You’re still 98% sure. And the day after that? Well, our confidence is waning fast, so we’re still pretty sure – 90%.

The same goes for tests that fail. Unless someone did something about them (and requested that this specific test is run again), you can be pretty sure that the test would fail again. So it gets light red unlike the tests that failed today.

This way, most tests only have to be run once every 4-5 days during development.

Why would they care?

For a release, all tests need to be run. That takes three weeks.

They really can’t possibly run all tests all the time.

Leave a Comment » | Conference, Software | Tagged: , , , , | Permalink
Posted by digulla

Jazoon 2012: Development Next – And Now For Something Completely Different?

28. June, 2012

Dave Thomas gave the keynote speech (link) about how technology seems to change all around us just to show up the same, old problems over and over again.

Things that I took home:

  • In the future, queries will me more important than languages
  • Big data is big

Some comments from me:

How often were you irritated by how source code from someone else looked? I don’t mean sloppy, I mean indentation or how they place spaces and braces. In 2012, it should be possible to separate the model (source code) from the view (text editor) – why can’t my IDE simply show me the source in the way that I like and keep the source code in a nice, common format? (see Bug 45423 – [formatting] Separate presentation from formatting)

And how often have you wondered “Which parts of the code call Locale.getDefault() either directly or indirectly?”

How often did you need to make a large-scale change in the source code which could have been done with a refactoring in a few minutes – but writing the refactoring would have taken days because there simply are no tools to quickly write even simple refactorings in your IDE?

Imagine this: You can load the complete AST of your source code into a NoSQL database. And all the XML files. And the configuration files. And the UML. And everything else. And then create links between those parts. And query those links … apply a piece of JavaScript to each matching node …

Customers of my applications always want new business reports every day. It takes way too long to build these reports. It takes too much effort to change them. And it’s impossible to know whether a report is correct because no one can write test cases for them.

Leave a Comment » | Conference, Software | Tagged: , , , | Permalink
Posted by digulla

e4 is here

28. June, 2012

Unless you live under a rock or you’re not using Eclipse, you can’t possibly have missed it: Eclipse Juno (or 4.2)

If you’re unsure what happened to 3.8 or 4.0 and 4.1, here is the story in a nutshell: A small team of Eclipse developers was concerned about the state of the platform (platform is the part of Eclipse that knows what a plug-in is, how to find, install and load them and arrange them in a neat way called “perspective”). When the platform was developed 2004, it was based on VisualAge. Don’t ask. Like all “visual” tools, it had … issues. Many of them were hardcoded into the platform. For example, there are many singletons in there. Many. Testing the platform is either a nightmare or slow or both.

The e4 team decided that something needed to be done. Only, it was risky. And it would take a long time. Which meant that the PMCs probably wouldn’t approve.

Therefore, they came up with a cunning plan: Develop a new platform (e4) alongside the official Eclipse releases. e4 would get a “compatibility layer” that would allow to run the old junk (like the JDT) but it would also allow to write plug-ins in a new, clean, understandable way. For example, in 3.x, you can use this code to get the current selection:

ISelection selection = getSite().getWorkbenchWindow().getSelectionService().getSelection();
Object item = ((IStructuredSelection)selection).getFirstElement();

The same code looks a bit different in e4:

@Inject
void setSelection(@Optional @Named(IServiceConstants.ACTIVE_SELECTION) Contact contact) {
    ...
}

e4 will call this method every time the user selects an instance of Contact in the UI.

The whitepaper gives some more details. Lars Vogel wrote a pretty long Eclipse 4 RCP tutorial.

After two years, e4 was deemed stable enough to make it the default release.

Conclusion: e4 doesn’t get rid of all the problems (p2, OSGi and SWT are still there) but it’s a huge step forward.

Related links:

Leave a Comment » | Software | Tagged: , | Permalink
Posted by digulla

Java Pitfalls: Covariant Return Types

25. June, 2012

Sometimes, covariant return types (= methods overriding existing methods which return a sub-type) can bite you. Especially when you want them to have annotations.

Ralf Stuckert wrote a good blog post about the reasons: The Covariant Return Type Abyssal

Leave a Comment » | Software | Tagged: , , , , | Permalink
Posted by digulla

When Agile Fails …

18. June, 2012

When an agile approach fails, then remember rule #0: No dogmas.

Agile is all about being non-dogmatic. If a rule doesn’t work in your case, find something that works.

You need UML? Use it. UML only slows you down? Drop it. UML might have some value for you in some specific cases? Apply it in a smart way.

Leave a Comment » | Software | Tagged: | Permalink
Posted by digulla

What’s Wrong With Mass Effect 3

17. June, 2012

I’m one of those unhappy gamers (like many, many, many others) but until this video, I didn’t know why. It’s a bit hidden in the message but ultimately, it boils down to: ME 3 isn’t an RPG. It’s an ego shooter.

The main reason is that your decisions in the game don’t count. Just like Bioware ignoring the outrage of gamers, whatever you do in the game has no consequences whatsoever: After the game, everyone is dead (or will soon be dead for lack of food), the galaxy is devastated,  the citadel is gone (no place to meet anymore, so no galactic council), no travel between civilizations, the fleets who came to protect Earth are now stranded – a planet pummeled by the reapers for months. Good luck finding anything there but death.

And that’s why it’s not an RPG. In an RPG (and ME 1 and 2), Shepard was my character. I made decisions and my decisions mattered. In ME3, when the dialog options come up (if they come up at all), I often just get two ways to say the same thing. There is no choice anymore. Oh yes, I can decide to trick the Krogans or the Salarians in the early game. But in the end, all is moot.

A lot of effort went into making the fights harder. That worked; I was getting killed so often that I finally gave up on the game. But in ME2, my decisions mattered. I was Shepard. In ME3, I was just a remote controlled drone of the EA’s marketing department (“Give us all your money!”).

Leave a Comment » | Game | Tagged: , | Permalink
Posted by digulla

Re: Curing the Problem of Software Patents, by Michael Risch

16. June, 2012

Michael Risch wrote a longer piece on software patents and why we should try to make software patents work: Curing the Problem of Software Patents

Here is my answer:

I think the main problem with the patent system isn’t the ideas behind but that some people have started to abuse it. And those who didn’t are being driven to join the brawl for no other reason than protect themselves.

You quoted “Software is hardware in every sense that matters for patents.” Let’s have a deeper look at that. Software is a formalized way to write recipes (as in cooking). Software development is a translation process (as in language interpreter). We take ideas from customers and turn them into detailed instructions to be executed by a moron. A human could do it but since it would be boring, we use a computer. Note that software isn’t mathematics – it’s language (as in English). It’s very limited (because the computer is so very dumb) but we’re more authors than mathematicians.

Now, my questions are:

  1. Can you patent the work of an online interpreter?
  2. How about the words that come out of her/his mouth?
  3. What about the words that she/he hears and translates?

My answers:

#1: No. The process of translating something is a craft. Crafts as such are not patentable.

#2: Since the interpreter doesn’t add anything of value to the input (they should stay true to what is given them), the output can’t be more or less than the input. It’s the input in another form but by itself, it doesn’t make the input more or less patentable.

#3 This leads to the question: Can you patent a recipe?

And the answer here is, sadly, yes. Which leads to the question: Can you patent any recipe?

No: The recipe has to be, well, “outstanding” in some way. Unfortunately, there is no universal standard what “outstanding” means. When I teach a newbie, they will think my methods are outstanding but they are just by relative comparison of what they know and what I know.

Since “outstanding” doesn’t seem to be a good tool to make a decision, what else do we have?

Damage. One of the roots of patent law is the idea to bring more justice to the world. It was invented for the specific purpose to protect the work of “helpless” inventors so they wouldn’t be ripped off all the time.

This makes sense if you, say, develop a new medicine. As of today, you need to spend around one billion dollars for a new medicine. Without patent law, it simply wouldn’t make sense for corporations to take the involved risks, so it makes sense to apply it here.

But software patents are in a completely different league. They cost $20’000 or less to produce and they can cause hundreds of billions of damage when they are used as a weapon – which is the sole and main purpose of software patents.

Since software patents are used solely to damage society as a whole (forcing companies to invest in them, costs of defending yourself against a lawsuit no matter if it’s justified or not, costs of canceling abusive patents, costs in lost revenue when you can’t sell your product because a competitor wants more market share), they need to be outlawed.

If you fail in this task, then no new computer related products will be sold in the USA by … well … 2013 because everyone will be suing everyone and all money will flow into courts and to patent lawyers. No more software development will happen because it’s just too expensive/dangerous. Maybe someone will find a patent to “display text remotely” and take down Google or the whole Internet (at least the part that runs in the USA).

The lawsuits between Apple and Samsung should be a warning shot. These only exist to give one company a bigger market share in a saturated market. Better product doesn’t count nor how “inventive” it is. It’s just the logical, inevitable conclusion what happens when all players follow the rules that we made.

Or Oracle vs. Google. Oracle came in demanding several billion dollars and got nothing. But if they had a different judge, the outcome could have been completely different. Is that what we want?

This seems to damage companies like Swype (see “An Example” in his post). But does it really?

First of all, someone could copy their idea. But customers would only buy the copy when it was better for most of them. Do we really want to protect something that most people think worse?

Copying an idea usually leads to a new, very similar but still different idea. The “thief” adds his own wisdom to it. We call that process “learning”. Isn’t that something that we should support?

If Swype can’t patent their idea, they can still make a product and sell it. If they fail, some money will be lost. But if they succeed, no other company can sue them for billions of dollars for no other reason than to slow them down. Isn’t that better than the other way around?

We say that the Internet Year is only three months. New ideas spring up so quickly by now that the old, sluggish patent system can never keep up. Do we want to slow down progress (might be a good idea) *and* feed the trolls? How much is an idea worth protecting if it’s outdated in 9 months?

How much more money will we make/safe if software patents are outlawed and all this money goes back into R&D?

Related:

4 Comments | Software | Tagged: , , | Permalink
Posted by digulla

On The Internet, You Can Be Anything You Want

14. June, 2012

On the internet,
you can be
anything you want.

It’s strange that
so many people
choose to be stupid.

(source)

Leave a Comment » | Fun, Philosophy | Tagged: | Permalink
Posted by digulla

Better Toothbrush

13. June, 2012

Think about this: How could you improve a toothbrush?

And I’m not talking about the electronic kind; just a plan plastic toothbrush. No idea? How about adding a fountain so you don’t have to crawl into your sink!

Source: Rinsing Toothbrush WIN

Leave a Comment » | Fun | Tagged: , , | Permalink
Posted by digulla

« Previous Entries
Next Entries »