TNBT: Proactive IDEs

13. February, 2015

Imagine this situation: You’re working on some code and you get an exception when you run the unit tests. Next to the output is a link with the text: “User Joe had the same exception two months ago and fixed it with the commit b8cfda02.”

How would that work? We’re using big data for all kinds of things, tracking customer happiness, searching the Internet and discovering terrorist threats (or not).

Standard development teams have about 10 people. That means you have a super computer with 40-80 cores, 160 GB of RAM and 20 TB of disk space connected with a fast LAN in your office already. That beast is usually idling while it waits for the developers to press keys. It would be pretty simple to install a clustered log analyzer on this hardware which simply reads all the log files and reports which Maven and running JUnit test creates. It would be as simple to connect the same database to your version control. That means this system could track all the errors and exceptions that you get when you run unit tests or the whole application.

This information could then be used to detect when someone in the team gets a new exception plus the change sets which fixes them. If the system detects an exception which it has seen before, it can tell you which developer has fixed it or who is currently working on it – instead of wasting your time, you could see the code which contains the solution or ask someone who has already solved the problem.

With proper filtering, the data could be split into internal and framework code. That way, the system could report to library projects where consumers struggle most.

On the large scale of things, this system can tell you which parts of the system are most brittle.

As usual with big data, there are some downsides. The same system would tell you which developer breaks the code most often. Who writes the worst code. If your manager isn’t able to see the human value in his charges, this might not be your best bet.

Related Articles:

  • The Next Best Thing – Series in my blog where I dream about the future of software development

Jazoon 2013 – 33 things you want to do better

25. October, 2013

Jazoon 2013 badgeTom Bujok listed a lot of methods, technologies and frameworks that you should be aware about in his talk “33 things you want to do better” (slides on speakerdeck)

At the beginning he reminded us how quickly a well designed system goes bad due to hurried changes. We need to be aware of our technical debt and we need to allocate time to spend on reducing it (slides 3-12).

As an example, car batteries are easy to find. They are a replacement part, designers and engineers make it easy to find. Compare this to the configuration of your project. If you need to change it, how easy is it to find the file that needs to be changed and then the place in the file?

Another important point is skills. In most other professions, you have some mastery of a skill before you use it. You train hundreds of hours before you play your first football game. In Software, we show you a computer, we show you the programming language of the year (not necessarily this year’s). There is no time to master the tools you have to use from day one (slides 13-15).

“We are what we repeatedly do; excellence then, is not an act but a habit.” – Aristotle

Or as Wikipedia defines it:

“Habits […] are routines of behavior that are repeated regularly and tend to occur subconsciously.”

Stop wondering why you always make the same mistakes – they’re habits. Eliminate them ASAP (slide 19):

Bad Habits – Katherine Murdock “The Psychology of Habit”:

  • Recognize bad habits and eliminate them ASAP
  • The older you get the more difficult it is to remove a bad habit
  • Each repetition leaves its mark!

Turning bad habits into good ones – Dr. Michael Roussell, PhD.:

  • You can’t erase a habit, you can only overwrite one.
  • Insert the new habits into the current habit loops

Bad Habits

Configure your IDE properly and remove bad defaults. Replace “ex.printStackTrace();” with “throw new RuntimeException(ex.getMessage(), ex);” (slides 43-45).

One bad habit is empty catch blocks with “can never happen” comments. If you see one during a code review, replace it with “System.exit(-1);”. It can never happen, right? Right? (slides 46-47).

Note: I have create a “ShouldNotHappenException” for this case :-)

Another one is to make every method in a static helper class public. Maybe some of them can be package private? (slide 48)

Learn about other good habits. Read books like “Effective Java” (Joshua Bloch) and “Clean Code” (Robert C. Martin) (slide 49)

Use code reviews to notice bad habits and to spread knowledge in your team but prevent blame games. (slide 73)

Learn the keyboard shortcuts of your IDE (slide 78)

Remember (slide 79):

“Any jackass can kick down a barn, but it takes a good carpenter to build one.” – Sam Ryburn

Projects You Should Know

Project lombok and lombok-pg – In a nutshell, these hook into the Java compiler and generate additional bytecode when certain annotations are present. Bored with getters, setters, hashCode() and equals() plus a nice toString()? Use @Data (slides 21-28).

Guava (slides 29-33)) is a great library with many tools that you have been missing in Java for years. You might also want to look at commons-lang.

Want to use lambda expressions but can’t upgrade to Java 8? Then lambdaj is for you (slides 34-39).

Logging? slf4j (40-42). Especially nice when combined with the @Slf4j annotation from lombok.

Bored to write all that boiler plate code to create all those services and managers that form your app? Look at Guice or Spring.

Use Spock to make tests more compact and easier to understand. (slides 50-52)

Unitils contains all the helper functions that we always missed in JUnit and Hamcrest (53-56).

JUnitParams will help you run tests with different parameters (57-59).

Need to wait for something during a test? Awaitility will help. (60-61)

When mocking isn’t enough and you need to inject code during a test, Byteman is the tool you want to look at (62-63)

Getting bored writing boiler plate code in Java to make a compiler happy? Have a look at Groovy. (64-67)

How about adding dependencies to your scripts? Try Grape. (68-69)

Is your build a mess? Do you feel Maven is too verbose or too limiting? Gradle might be for you. (70-72)

Version control is slowing you down? Have a look at Mercurial or Git (75)

Use bash or Python to automate man-/menial work. If you’re on Windows, look at Babun or Cygwin. (76-77).


Quick: How Does Quicksort Work?

24. June, 2013

Every now and then, we need to remember how some complex algorithm works. For some, code works well. Or puzzles. Others prefer an explanation with examples. A third group likes to see what is going on.

The AlgoViz.org portal contains visualizations of all kinds of algorithms for the visually inclined.

 


Things Users Don’t Care About

8. February, 2013

Things users don’t care about” is something every software developer needs to know about.

Kudos go to Thomas E. Deutsch for finding and telling me about it.


Designing a Garbage Bin

17. December, 2012

Many of us have noticed that designing software is surprisingly hard but many don’t know why that is. The simple answer: Design is the art to balance contradicting goals.

Not convinced?

Let’s design a public garbage bin together.

What do we want?

  1. Big enough so it never spills
  2. Easy to clean
  3. Nice to look at
  4. Robust enough to withstand riots
  5. Soft enough to cushion the impact of car
  6. Long lifetime
  7. Cheap

It’s easy to see that “cheap” contradicts almost anything else. “Nice to look at” means an (expensive) artist has to build the form. Big garbage bins ain’t cheap. Easy to clean and robust mean high quality materials for hinges and locks. Easy to clean and long lifetime involve expensive surface materials and finishing.

It should be easy to lift for the cleaning crew but not for rioters. When a car hits it, the bin should give way. So these contradict each other as well.

Still not convinced? Look at my elevator example.


TNBT: Documentation Sucks

11. October, 2012

Documentation is the unloved step-mother of software development: Nobody likes it.

On the writing side, documentation is either considered a waste of time:

  • I could write code in that time
  • It won’t be valid anymore after the code changes tomorrow, anyway
  • There is no way to make sure you can trust documentation
  • Stringing sentences is hard work, especially when you want to make them easy to read, understand and interesting to follow.
  • It’s hard to connect code samples with documentation
  • If I describe too many details, readers will be bored. If I omit too many, they will be confused. There is no way to know which level of detail is good.

On the reading side, it’s a waste of time:

  • I need to solve a problem, I don’t have time to search in a huge dump of text
  • If the author doesn’t trust the documentation, how can I?
  • It will contain too many details that I already know and omit too many facts that I need to understand what is going on.

The core of the issue is that documentation and code are two different things. Documentation is, by nature, abstract. It’s at least one step removed from the solution.

Does it have to be that way?

I hope, with new technologies like Xtext or JetBrains’ Meta Programming System, we will eventually be able to turn documentation into code.

So instead of writing hundreds of lines of code to open a window, give it a size, make sure it remembers its size and position, etc., we could write:

Allow the user to edit a Customer object which has properties from foaf:Person and one or more addresses.

Users can search for Customer objects by any of the name fields.

Note that the links are part of the documentation and the code; the underlying code generator should follow them and examine the code/documentation on the other side.

Related Articles:


Software Development Costs: Bugfixing

21. July, 2012

(Second part of three; first part is here)

Software has bugsthere simply is no way to avoid them. If you can’t avoid them, maybe you can handle them efficiently? Yes, you can:

As you can see, the cost of fixing bugs rises as time passes. Why is that?

There are many reasons:

  • When you find a bug a couple of minutes after you created it, you probably still have all the information in your head that is necessary to understand and fix it.
  • If you just created the bug, no code depends on it. As soon as you start writing unit tests and more code, fixing becomes more expensive because you start to have dependencies.
  • People might have gotten used to the bug and developed workarounds. If you fix the bug, this will have an impact on them.
  • A bug found in production is likely to be reported by a customer. Customers can’t see inside of your software, so extra effort will have to be spent to determine what the actual problem is. Google for “how to report bugs
  • When a bug is discovered at the customer, it might trigger meetings and scapegoat hunting. Think of it this way: A 1 hour 8 person meeting costs about $1’000. And no bug was ever fixed in a meeting.
  • Some bugs escalate to the top-level management. Imagine for a moment what it would mean for you if their CEO called your CEO to complain about a problem you caused.
  • Bugs might break expensive things, harm or even kill people or start World War 3.

This also explains why unit testing is so much more efficient to QA testing for many kinds of bugs: It simply catches them before they spread their bad influence.

So fix your bugs early, OK?

Related articles:


Follow

Get every new post delivered to your Inbox.

Join 343 other followers