Jazoon 2012: How to keep your Architecture in good Shape?!

28. June, 2012

Ingmar Kellner presented some tips how to prevent your architecture rotting into a mess. When that happens, you will have these problems:

  • Rigidity – The system is hard to change because every change forces many other changes.
  • Fragility – Changes cause the system to break in conceptually unrelated places.
  • Immobility – It’s hard to disentangle the system into reusable components.
  • Viscosity – Doing things right is harder than doing things wrong.
  • Opacity – It is hard to read and understand. It does not express its intent well.

(Robert C. Martin)

According to Tom DeMarco, your ability to manage this depends on control. And control depends on measurements – if you can’t measure something, you can’t control it.

How rotten is your software? Look for cycle groups (some package X depends on Y depends on Z depends on A depends on X):

  • They tend to stay
  • They tend to grow
  • They are a strong smell

Ingmar showed some examples in the JDK 6 (lots of cycles) and ActiveMQ (lots of cycles in 4.x, much better in 5.0 but again growing since then).

What can you do?

Use a consistent “architecture blueprint” that makes it obvious which layer/slice can use what. In the blueprint, layers are horizontal (presentation, domain, persistence) and slices are vertical (everything related to contracts, customers, users, and finally common code).

You will need someone with the role “Architect” who “defines the architecture, thresholds for coding metrics, identifies ‘hot spots'” and developers who “implement use cases, respecting the architecture and coding metrics thresholds.” All this is verified by a CI server.

At the same time, avoid “rulitis” – the false belief that more and stricter rules makes things “better.”

Some rules you might want to use:

  • The blueprint is free of cycles
  • Package naming convention that matches the blueprint
  • Control coupling and cycles with tools
  • Use tools to control code duplication, file size, cyclomatic complexity, number of classes per package, etc.
  • Reserve 20% of your time for refactoring

Following these rules can help to reduce costs during the maintenance phase:

  • 50% less time
  • 50% of the budget
  • 85% less defects

according to a study conducted by Barry M. Horowitz for the Department of Defense.

Leave a Comment » | Conference, Software | Tagged: , , , | Permalink
Posted by digulla

Jazoon 2012: Why you should care about software assessment

28. June, 2012

Tudor Girba gave a presentation at the Jazoon about a topic that is very dear to him: Software assessment. To quote:

What is assessment? The process of understanding a given situation to support decision-making. During software development, engineers spend as much as 50% of the overall effort on doing precisely that: they try to understand the current status of the system to know what to do next.

In other words: Assessment is a process and a set of tools to help developers to make decisions. They typical example is a bug shows up and you need to fix it. That raises the usual questions:

  1. What happened?
  2. Why did it happen?
  3. Where did it happen?
  4. How can I fix it?

As we all know, each of these steps can be difficult. As an extreme example, someone mentioned selling software to the NSA. It crashed. The NSA calls the developer:

NSA: “There is a problem with your software.”

You: “Who am I talking with?”

NSA: “Sorry, I can’t tell you that.”

You: “Well … okay. So what problem?”

NSA: “I can’t tell you that either.”

You: “… Can you give me a stack trace?”

NSA: “I’m afraid not.”

Unlikely but we all know similar situations. Even seasoned software developers are guilty of giving completely useless failure reports: “It didn’t work.” … “What are you talking about? What’s ‘it’?”

Tudor gave some nice examples how he used simple assessment tools that allow him to query log files and sources of some application to locate bugs, locate similar bugs and help to find out why some part doesn’t behave well. Examples:

  1. An application usually returns text in the user’s language but some rare error message is always in German. Cause: When the error message was created, the code called Locale.getDefault()
  2. Several other places could be found that showed the same behavior by searching the source code for places where Locale.getDefault() was called either directly or indirectly. A test case was added to prevent this from happening again.
  3. Some cache would have a hit ratio of less than 50%. Analyzing the logs showed that two components used the same cache. When each got their own cache, the hit ratios reached sane levels.

So assessments allow you to do strategic planning by showing you all the dependencies that some part of the code has (or the whole application).

In a spike assessment, you can analyze some small part to verify that a change would or could have the desired effect (think performance).

Did you know that developers spend about 50% of the time reading code? If tools can help them understand some piece of code faster, that makes them more productive. Unfortunately, today’s tools are pretty limited when it comes to this. Eclipse can show me who calls Locale.getDefault() but it can’t show me indirect calls.

Worse: If the developer makes the wrong decision because she couldn’t see all the important facts, then these often have a huge impact.

Another important aspect is how you use metrics. Metrics are generally useful but the same is not true for every metric. Just like you wouldn’t copy unit tests from one project to the next, you need to reevaluate the metrics that you extract from each project. Some will just be a waste of time for certain projects.

My comments:

We really, really need better tooling to chop data. IDEs should allow me to run queries against my source code, collect and aggregate data and check the results in unit tests to validate design constraints.

It was also interesting to see how Tudor works. He often uses simple words which can be misleading. But when you look at the slides, then there was this graph about some data points. Most graphs show a linear Y axis with the ticks evenly spread. He uses a different approach:

Usual diagram to the left, Tudor’s version to the right

Related links:

Leave a Comment » | Conference, Software | Tagged: , , , | Permalink
Posted by digulla

Jazoon 2012: Large scale testing in an Agile world

28. June, 2012

Alan Ogilvie is working at a division of IBM responsible for testing IBM’s Java SE product. Some numbers from his presentation:

  • A build for testing is about 500MB (takes 17 min to download to a test machine)
  • There are 20 different versions (AIX, Linux, Windows, z/OS * x86, power, zSeries)
  • The different teams create 80..200 builds every day
  • The tests run on heaps from 32MB to 500GB
  • They use hardware with 1 to 128+ cores
  • 4 GC policies
  • More than 1000 different combinations of command line options
  • Some tests have to be repeated a lot of time to catch “1 out of 100” failures that happen only very rarely

That amounts to millions of test cases that run every month.

1% of them fail.

To tame this beast, the team uses two approaches:

  1. Automated failure analysis that can match error messages from the test case to known bugs
  2. Not all of the tests are run every time

The first approach makes sure that most test failures can be handled automatically. If some test is there to trigger a known bug, that shouldn’t take any time from a human – unless the test suddenly succeeds.

The second approach is more interesting: They run only a small fraction of the tests every time the test suite is started. How can that possibly work?

If you run a test today and it succeeds, you will have some confidence that it still works today. You’re not 100% sure but, well, maybe 99.5%. So you might skip this test today and mark it as “light green” in the test results (as opposed to “full green” for a test that has been run this time).

What about the next day? You’re still 98% sure. And the day after that? Well, our confidence is waning fast, so we’re still pretty sure – 90%.

The same goes for tests that fail. Unless someone did something about them (and requested that this specific test is run again), you can be pretty sure that the test would fail again. So it gets light red unlike the tests that failed today.

This way, most tests only have to be run once every 4-5 days during development.

Why would they care?

For a release, all tests need to be run. That takes three weeks.

They really can’t possibly run all tests all the time.

Leave a Comment » | Conference, Software | Tagged: , , , , | Permalink
Posted by digulla

Jazoon 2012: Development Next – And Now For Something Completely Different?

28. June, 2012

Dave Thomas gave the keynote speech (link) about how technology seems to change all around us just to show up the same, old problems over and over again.

Things that I took home:

  • In the future, queries will me more important than languages
  • Big data is big

Some comments from me:

How often were you irritated by how source code from someone else looked? I don’t mean sloppy, I mean indentation or how they place spaces and braces. In 2012, it should be possible to separate the model (source code) from the view (text editor) – why can’t my IDE simply show me the source in the way that I like and keep the source code in a nice, common format? (see Bug 45423 – [formatting] Separate presentation from formatting)

And how often have you wondered “Which parts of the code call Locale.getDefault() either directly or indirectly?”

How often did you need to make a large-scale change in the source code which could have been done with a refactoring in a few minutes – but writing the refactoring would have taken days because there simply are no tools to quickly write even simple refactorings in your IDE?

Imagine this: You can load the complete AST of your source code into a NoSQL database. And all the XML files. And the configuration files. And the UML. And everything else. And then create links between those parts. And query those links … apply a piece of JavaScript to each matching node …

Customers of my applications always want new business reports every day. It takes way too long to build these reports. It takes too much effort to change them. And it’s impossible to know whether a report is correct because no one can write test cases for them.

Leave a Comment » | Conference, Software | Tagged: , , , | Permalink
Posted by digulla

Zurück vom Perry Rhodan WeltCon 2011

3. October, 2011

(Due to the topic, this post is in German)

Der Con ist vorbei, Zeit für einen Rückblick: Eine Menge nette Leute getroffen, die gar nicht so verrückt sind, wie man meinen möchte. Unter Computer-Freaks zumindest findet man häufiger merkwürdigeres.

Freitag fing der Con relativ spät an und gleich mit einem Problem: Der Ticket-Verkäufer hat nicht alle Adressdaten an die Veranstalter weitergeleitet, daher habe ich nur so einen schäbigen Ausweis ohne gedruckten Namen *schnüff*

Dann habe ich meine Zeit so ca. 50-50 zwischen Vorträgen und Quatschen aufgeteilt.

Ich habe den Vortrag wider das Expo gesehen (wo man schön die Spannung zwischen Autor und Expokrat mitbekommen hat).

Leider ist der Rückblick auf Gucky ausgefallen 😦

Den ESA-Vortrag habe ich mir gespart; war vielleicht ein Fehler. Andere Besucher fanden ihn gut. Ich habe mir in der Zeit einen Illustrator für meinen Roman “Haul” (link oben) gesucht, was gegessen (lecker … ist anders) und lange mit zwei anderen Fans geratscht.

Zurück zum Musensaal, wo ich durch glückliche Überziehung noch die letzten beiden Fan-Filme mitbekam. Beeindruckend. Leider habe ich die “Kneipenszene” verpasst, aber ich hab Michael Thurner scharf gemacht. Vielleicht gibts die Filme bald auf Youtube. Würde sich auf jeden Fall lohnen.

Der PR-Film war eine Enttäuschung. Der Projektleiter wollte sich in keiner Richtung festlegen, aber auch den Eindruck vermeiden, sich nicht festlegen zu wollen. Wird wohl noch ein paar Jahre dauern. Ein Fan meinte hinterher, er würde einen Animationsfilm bevorzugen, weil man da alles machen könnte. Ausser glaubhafter Mimik. Aber die Zeit arbeitet für uns. Nach den Fanfilmen zumindest zu Urteilen wird es in 5-10 Jahren einen Film geben, halt vielleicht nicht von den Leuten, von denen wir es erwarten 🙂

Das weitere Programm hat mich nicht so interessiert, also wieder unter Leute mischen.

Samstag

Zu lange geschlafen. Bin gerade zum Ende des Quiz “Wer wird Galaxionär?” angekommen. Die letzten Fragen waren echt fies: “Wer hat das Thanatos Programm ausgelöst?” (Delorian) und dann “… und zwar wann? 4., 5. oder 5. September 1469 NGZ?” *lol* Haben die Terraner aber richtig beantwortet (es war natürlich der 5. September 😉 ) und damit trotz enormen Rückstand doch noch gewonnen.

Kurz was gegessen (draussen).

Einen Teil von “Negasphäre” gesehen. Wieder ein sehr beeindruckendes Werk von Raimund Peter wie auch seine Raumschiffmodelle im Vorraum (CREST II und die Festung der Inquisition, Bilder unten).

 

 

 

Lange mit Risszeichnern über Copyright und Release-Early-Release-Often diskutiert.

Frank Borsch getroffen, der sich bei Fans Feedback zu NEO geholt hat. Mein Fazit: Die meisten Fans (und ich) sind zufrieden mit NEO. Dazu gibts aber noch einen Extra-Post.

“Kurs 3000” war nicht so vage, wie befürchtet. Perry geht konsequent seinen Weg und damit dem hohen Mächten aus den Keks. Mal sehen, wie die damit umgehen.

“Unterwegs in fremden Universen” war dagegen kein Brüller (auch wenn die anwesenden Autoren sich Mühe gegeben haben).

Dann um 19:00 zum Kaffeeklatsch gegangen. Hatte eigentlich Leo Lukas erwartet (wollte ihm Rückmeldung zum Thema LEUCHTKRAFT geben), aber wegen einer Terminverwechslung Kai Meier und Marie Sann angetroffen. War dann aber doch lustig und informativ. Frau Sann auf Turbo Defiant Kimecan hingewiesen – vielleicht gibt es ihre Comics in Zukunft auch als eBook.

PR Doku teilweise gesehen (hab die DVD gekauft). Lustig: K. N. Frick meint “Ich hab keine Ahnung was Perry für Musik mag. Oder seine Lieblingsspeise.” Eigentlich sollte es da doch ein Datenblatt geben? F. Borsch meinte “Perry ist keine Figur sondern ein Symbol.” Was wohl erklärt, warum er im Roman so blass wirkt. Schade eigentlich.

Spät nach Hause gewankt; die Nacht auf Lepso habe ich mir gespart.

Sonntag

Gerade rechtzeitig zum Vortrag über Kosmophysik (und vor allem Biologie) und Raumfahrt der ESA eingetroffen. Interessante Informationen über DNA-Bausteine und Bruchstücke in Meteoriten, Bakterien auf Landefüssen von Raumsonden und in der Marsoberfläche. Bei einer nicht-repräsentativen Umfrage unter Wissenschaftlern an einer Konferenz (Handzeichen) glauben ca. 70%, dass es heute Leben auf dem Mars gibt, aber unser Wissen über den Planeten ist sehr, sehr rudimentär. Wir können nicht mal seine Oberfläche so genau überwachen, um sicher sagen zu können ob es da jemals regnet. Dunkle Streifen tauchen ab und zu auf, aber keiner kann genau sagen, was das ist (dunklerer Staub oder feuchter Sand).

Dann einige Informationen zu Leben im Weltall und auf anderen Planeten, z.B. über Spektralanlyse. Ozon etwa kann nur entstehen, wenn es Leben gibt, weil anorganische Chemie den Sauerstoff irgendwann komplett verbraucht. Oder zu den Kosten: Wenn eine bemannte Mission 100 Millionen Dollar kostet, dann wird dieses Geld immer noch auf der Erde ausgegeben – Raketen werden (noch) nicht von Ausserirdischen auf dem Mond gebaut. Das Geld ist also nicht “verloren” – aber möglicherweise bald wir, wenn wir keinen Weg finden unseren Lebensraum zu erweitern. Die Utopie die Menschheit zum Achtsamen Umgang mit Ressourcen zu erziehen ist noch unwahrscheinlicher als eine Landung der Arkoniden.

Auf der Auktion konnte ich “Der Goldene Mann” ersteigern – mit diesem Roman begann meine grosse Liebe zur SF. Das zweite “Item” – eine Szene aus einem PR-Roman – war mir zu teuer. Aber ich gebe nicht auf 🙂 Irgendwann steht auf einem PR-Roman mein Name 😉

Ende

Schön wars. Bei einigen Punkten hat man deutlich gemerkt, dass die Organisatoren noch Lehrgeld bezahlen, aber ich finde so etwas macht die Cons … menschlicher als die Perfektion, die man vielleicht in Amerika kennt. Einige mir wichtige Punkte sind ausgefallen, aber ich hatte den Eindruck, dass sich jeder viel Mühe gegeben hat und daher hatte ich kein Problem damit. Das kaum jemand pünktlich fertig wurde war eher ein Zeichen von Begeisterung als schlechter Planung. Zumindest die allgegenwärtigen Schlümpfe vom WeltConTeam haben versucht die Leute auf der Bühne Zeichen zu geben, dass sie zum Ende kommen müssen, aber die waren so in der Materie versunken … 😉

Schade, dass es nun viele Jahre bis zum nächsten Con dauert.

 

Leave a Comment » | Conference | Tagged: , | Permalink
Posted by digulla

Perry Rhodan

29. September, 2011

For the next four days, I’ll be in Mannheim, Germany at the PERRY RHODAN-WeltCon 2011 (their caps).

If you didn’t know, Perry Rhodan is the “most successful science fiction book series ever written” (Wikipedia) with over one billion copies sold in Germany alone.

Make you wonder why it isn’t as famous as StarTrek or Star Wars or Babylon 5. Well, there was never a TV series.

1 Comment | Conference | Tagged: , | Permalink
Posted by digulla

denkfest.ch

9. September, 2011

Yesterday, I attended denkfest.ch, “Four days of science, critical thinking & intelligent entertainment.”

The first event was a discussion about “skeptic blogging” where seven bloggers talked about what they do, where they get their ideas from and how they handle feedback.

For example, if you blog about a controversial topic like religion, you will get some heated feedback. The stance of the bloggers was that you won’t sway the die-hard believers. But a lot more people are just curious or want a second opinion. For those, “independent” blogs are a great help. I quote that because no one of us is really independent and that is something we all need to keep in mind.

There is a part of your brain which filters anything coming in through the senses long before the facts reach the consciousness. It’s easy to forget this since it’s so handy normally. Without it, it would be impossible to concentrate on something. While reading this text, you would have to listen to what the people next to you are talking about, how the seat below you feels like, or the clothes on your skin, what the air smells like and not to forget all those words on this page that you don’t want to read all at the same time – your eyes see all of them all the time, but the filtering makes sure that you can understand them one at a time.

In the Q&A session, someone said that the Internet became one of the most important tools in the Arabic world today. As many of you probably know, there were huge advances in science in Arabia while the Christians suppressed any free thought (6th to 15th century). Without those people,  astronomy, mathematics and chemistry wouldn’t be what they are today. We’d use Roman numerals instead of Arabic ones. Astrology would try to avoid launching probes to the moon lest we found “something”.

But religious fundamentalism led to a ban on thinking in the Arabic world. There was a comment that religion led to a lot of wars. That isn’t true. Greed and power lead to war, religion is just a handy tool in this context to rally emotions. Saying “we kill them because they are assholes/heretics/infidels” always gets better reception than saying “we kill them because they are more wealthy.” The latter is like saying “We’re too dumb to make a living, so let’s make someone else miserable, too.”

As Vince Ebert puts it, “if the Islamists were true to their belief, they’d attacked the Twin Towers not in air planes but with a battering ram.”

There was also some talk about what is religion and what is science.

I think religion is what you know for sure while science is what you know not. I tried to make this clear in a comment I gave in the Q&A session. When there is a light switch on the wall, the religious person tends to believe that it works while the scientist tries to come up with an experiment that proves that it doesn’t. While a religious person might flip the switch to “prove” that it works, a scientist will install sensors near the lamp and the switch to see if the time between flipping the switch and the reaction is always the same – someone else could operate the real switch. Or they’ll follow the wiring. They might influence the voltage on the wires to see if it’s the same on both ends to make sure this wire actually goes to the light bulb.

This might sound ridiculous but the problem with experiments is that they are only “true” as long as someone can’t prove them wrong. Einstein isn’t right, it’s just that so far no one could falsify his theory and since a lot of smart people tried, it’s probably a good theory in the sense that it’s resilient. Astrology wasn’t ever tried. No astrologer took a million people, followed them for years, noting down interesting events in their lives to see if there is a correlation to celestial “bodies” like the “houses” into which we are born. They always take each person individually, see “oh, here I was right, so my theory works” and they’re happy. If their theory fails, they don’t ask how they could conduct an experiment to verify anything, they just think “oh, well, outlier.”

Another good point by Lars Fischer: Most scientific publications are hard to understand. So if you don’t understand something, it’s probably scientific because the frauds try hard to make their lies easy to grasp.

Florian Freistetter has another summary of the event in his blog.

1 Comment | Conference | Tagged: , , | Permalink
Posted by digulla

Jazzon 2011, Day 3

26. June, 2011

The day started with a keynote from a M$ guy:

Leave a Comment » | Conference | Tagged: | Permalink
Posted by digulla

Jazzon 2011, Day 3 – How to become a famous author and publish a book: Using Freemium Content with a Profit – Pouline Middleton

26. June, 2011

How to become a famous author and publish a book: Using Freemium Content with a Profit – Pouline Middleton

Pouline showed us the obstacles in which you run when you try to publish your own book. It confirmed my own conclusions: Unless your name is Stephen King or J. K. Rowling, publishers aren’t really for you. They most often want to own your work but selling it is more of a second thought.

Instead of using the common channels to sell her book, she chose a freemium model. You can read the book as blog posts, by email or buy it from herself (she eventually founded her own publishing company Fiction Works.

The fun (or sad) part is that she made much more money this way than she could have hoped for if she had used the traditional channels.

Here is an example. Say you have 1’000 die hard fans (not so hard to come by when the Internet has almost 1 billion users). Each of them buys from you for $100. Again not so much. That gives you $100’000.

1 Comment | Conference | Tagged: , , , | Permalink
Posted by digulla

Jazzon 2011, Day 3 – Web Security: Develop. Penetrate. Smile. – Matt Raible

26. June, 2011

Web Security: Develop. Penetrate. Smile. – Matt Raible

Matt demonstrated how to “implement authentication in your Java web applications using Spring Security, Apache Shiro and good ol’ Java EE Container Manager Authentication. You’ll also learn how to secure your REST API with OAuth and do it all securely with SSL.”

Nothing spectacular but the usual mix of nice code and how to avoid the most common pitfalls.

Some things to remember: Firewalls don’t work, not even if they’re stateful and inspect the HTTP stream.

If you’re interested in web app security, you should have a look at OWASP. Right now, there are a lot of non-developers there. What we all desperately need is web frameworks which make it more simple to configure a secure web app correctly than configuring a normal web app.

Links:

1 Comment | Conference | Tagged: , , , , | Permalink
Posted by digulla

« Previous Entries
Next Entries »