21. June, 2015
I’m back from SoCraTes Day Switzerland where I help a Code&Hack session called “Testing the Impossible”.
The session is based on this Mercurial repository de.pdark.testing.
- Space Shuttle – They Write the Right Stuff
- Why wasn’t the bug caught by QA?
- Why did the bug escape dev?
- Personal Bug Diary
- Team Culture: Strengths va. Blame Game
- Miscommunication with Customers
- Anyone can press the “Red Button“
- Make failures cheap. Fail fast.
I’m wrong. I have learned something.
- Bug-.hunting takes time and mindshare.
- If you found a bug, write a test for it
How do I test randomness?
- Test small pieces
- Fix things that varies (IDs, timestamps, etc.)
How do I test a DB?
- Layer between DB and App code which allows to switch between real DB and mock
- Verify generated SQL instead of executing it
- Allows to test thousands of combinations easily
- Useful for code that builds search queries
- Run SQL against real DB during the night (CI server)
- H2 embedded DB can emulate Oracle, MySQL, PostgreSQL, …
- Test at various depths (speed vs. accuracy)
- Testing Walrus dives deep!
23. December, 2014
Eine kleine Hilfe für alle Leser der c’t, welche einfach(er) einige Artikel archivieren wollen: Das verlinkte Script unten sortiert das Inhaltsverzeichnis im Archiv nach Seitenzahl wenn man auf “aktuell” klickt.
3. November, 2014
Thomas Schuetz from Protos Software showed some surprising similarities between embedded and finance systems: Both need to run a long period of time without human interaction, they shouldn’t show “odd” behavior, and you can’t simply shut them down to look for a bug.
Granted, lives are much less at risk when a financial software crashes (as opposed to, say, a pacemaker). So at first glance, the strict safety rules which apply to embedded systems seem too strict for financial software. But safety is built on top of reliability. And we very much want reliability in any system we build.
An important tool here is tracing. Tracing is the pedantic brother of logging. The goal is to collect enough data to simulate the state of the system at any point in time.
In a demonstration, he showed a demo for project eTrice. In a mix of a textual and UI editors, he created a simple application with two objects that could send data back and forth. Since everything is based on EMF, changes on one side are immediately reflected on the other. As a free bonus, you get a sequence diagram of the whole process by clicking a button after the application has finished.
12. November, 2013
The article “WORKING IN A WANNABE-AGILE TEAM” points out a common problem in agile: It really exposes you and most people simply are prude.
Unlike many people want to make you believe, they are aware of their own flaws and how much a certain process humiliates them – it’s a skill everyone adopts at an early age, and therefore almost completely subconscious.
Since there is no way to be agile without looking at the team’s issues, the solution is to offer them something else instead.
In my experience, the easiest foot to get into the door is testing. When customers ask for features, ask “How would you know that it’s working correctly? Can you give me an example?” Yay, acceptance tests for free.
People struggling with a piece of code that fails all the time in interesting ways? “How do you know it’s wrong? What would be right? Maybe we could write a piece of code that makes sure it stays right from now on?” Yay, unit testing.
“Can you write a test?” “You can’t test that!” “…. You wrote software that can’t be tested? Seriously?” “… No, of course you could test it but …”
The best part: It focuses on solutions. When suggesting tests, no one can get into the blame game. Everyone can get involved. Customers, managers, developers, everyone understands tests. And they offer the most value for the least investment.
When people have started testing, they become interested in other things as well: Agile planning. Listening to the customer. That’s when you can start to change the culture – you now have some trust that you can spend.
12. April, 2013
A short summary of the presentation “UI Best Practices & Trends” held by Roland Studer of We Are Cube (slides).
- UIs are often bloated, confusing and frustrate users.
- A major factor is that features are often requested by people who won’t have to use the software later i.e. by (self) “important” managers. A variation of “Nothing is impossible if you don’t have to do it yourself.”
- “Don’t make me think” by Steve Krug (Amazon)
- “The interface is your product” (“Getting Real – The Book“). Customers don’t care about the code or the effort which went into the product, they only see what you show them on the screen.
- “Show me where I am”
– breadcrumbs, highlight tabs, use tree navigation
- “Use the F-Pattern”
– eyes travel left to right, top to bottom in Western cultures
- “Emphasize in Tables”
– gray out less important values, make important ones bold, group with color, use gauges instead of numbers
- “Affordance (sic): Show what’s interactive”
– Make it obvious for users to see where they can click
- “Make frequent uses accessible”
– Show only the most important actions as buttons (max. 5).
– Put the other, less signification actions in a popup menu.
- “Caution with icons”
– “A picture says more than 1000 words but which words exactly?”
– Icons can be confusing, hard to remember and they depend on culture/training.
– Offer at least user preferences to replace them with text or show text in addition to the icon or use tooltips.
- “Emphasize primary actions”
– Make the button stand out which the user most likely wants to click next or reduce visibility of less likely options (like “[Submit] [Cancel]”).
– Be careful that the visual cue doesn’t conflict with “this button is disabled”
- “Avoid OK / Cancel”
– Instead of asking “Is it OK to delete?”, ask “This object is still be used. [Delete Anyway] [Keep]”
- “Use undo if you can”
– Avoid confirmation dialogs. No one ever reads them.
– The user should never be able to do anything that isn’t reversible in some way.
– Allows users who got confused with an icon (see #6 above) to unto destructive action on touch screens.
- “Avoid left aligned form labels”
– The eye is only able to align elements when they are close enough.
– Right align labels, rely on browser search for locating elements if you have too many of them (which you shouldn’t)
– Or put the label on top of the fields (but this works better for forms where you have to fill in every single field)
- “Be nice to user input”
– Support “slang” like “next Tuesday” or “now” in date fields
– Keep illegal values so the user can edit them
- “Prevent errors”
– Calendar popups
– Offer a list of valid inputs in a drop-down list or use radio buttons
- “Instant Feedback”
– Show what has been entered correctly as well as what’s wrong.
– Display validation errors immediately (when the focus is lost) instead of waiting for the user to click on a “Submit” button
– Use tooltips on disabled buttons to explain how to enable them
- “Help users recover from errors”
– Avoid “There have been errors” dialogs; they aren’t useful in any way.
– Don’t tell what’s wrong, tell how to fix the error. If you don’t know how to fix it, will the user?
- “Responsive Web Design” – adjust the layout to available screen space. “Mobile first” helps to concentrate on the core features of your application. “Reduce to the max”, avoid distractions, show only the absolute minimum of data. Anything on the page begs to be noticed, so even if the user doesn’t read it, it will still take a tiny bit of concentration to ignore!
- There is a trend towards flat and minimal design. Example: http://basecamp.com/ where a lot of things on the page are interactive (i.e. responds to clicks) but almost everything looks like plain text.
- “Actions only on hover” – Show available actions only when the user hovers over some part of the page. Drawback: Doesn’t work on touchscreens since the user can’t “hover”
- “Implicit feedback” – Show a “saved successfully” message. Give hints in text fields.
Always keep in mind:
Developer != User
Product Owner != User
UI Guru != User
Do usability testing. If you can’t, try installing the product on a laptop and ask someone in the corridor to use it for five minutes. A little is better than nothing. You’re blind to your own prejudices.
26. February, 2013
David Blake posted a current overview of Man in the Middle type attacks: 15 Surprising Ways You Could Fall Victim to a Man in the Middle Attack
- Key-loggers (hard- and software)
- Browser plugins
- Cameras (a.k.a Shoulder Surfing)
- Wireless attacks
14. February, 2013
A unit test checks a certain feature of the software. When it fails, you usually know exactly where to look for a place to fix the problem. They are short, compact and fast.
They come at a cost, though: You have to replicate the setup code in your test.
This is an important point. To use a feature of your software, you always have some set up to do. This set up has to exist in your production code. For unit tests, you will have to copy some parts of this code into your tests because usually, the setup of the production code simply isn’t fine grained or flexible enough to be useful for tests. Think tests which check the error handling: Your production code usually can’t build mock objects that raise errors when certain methods are being invoked.
An integration test reuses the production configuration. It tests many features at once, in the same order and with the same or at least a very similar environment that your final application will use. They are high level and often much easier to set up. But that comes at a cost:
- They are slow
- When they fail, there will be many places which could cause the issue, so fixing them is more expensive
- They break more often because you have more dependencies
- Setting up a test for the “success” scenario will be simple but injecting mock objects to cause exact error states will be much harder
So be aware what kind of test you’re writing.