The Coverity Scan 2011 Open Source Integrity Report (registration necessary) says: “Open source quality is on par with proprietary code quality, particularly in cases where codebases are of similar size.”
Which isn’t that surprising considering that it’s the same people who write both.
But there are a couple of hard number in the report which are interesting:
Linux 2.6 has about 0.62 defects per 1000 lines of code (KLOC) which Coverity says “is roughly identical to that of its proprietary codebase counterparts.” They can’t tell names but I guess the counterparts are Windows and Mac OS X. They have 0.64 defects per KLOC.
The industry average is 1.0 defects per KLOC which matches well with my (more anecdotal) knowledge that the best software developers make about 3-4 mistakes per KLOC of which 75% are found during development.