Oxymoron: “Bug-Free Software”
- The $500 million dollar European satellite carrying Ariane-5 rocket blew up 37 seconds into its first launch – because of a one line software bug.
- The $1.4 billion US Air Force B-2 bomber wouldn’t fly on its maiden flight — thanks to a software bug.
- Later another B-2 bomber crashed and burned due to another software bug.
“If debugging is taking bugs out of software – then programming must mean putting bugs in . . .” – Doug Goodall, Assembly Language Poet, 1989 (1)
This article is to let us ponder the complexity and potential flaws of computer programs used in astrophysics and cosmology.
Rule #1. There is no such thing as Bug-Free Software
According to Carnegie Mellon University’s CyLab Sustainable Computing Consortium — “Commercial software typically has 20 to 30 bugs for every 1,000 lines of code.” (“KLoC” stands for “Thousand Lines of Code”) Another researcher broadens that estimate to “about 15 – 50 errors per 1000 lines of delivered code.” (Code Complete, Steve McDonnell, 2004) (2)
Please do not take this auxiliary “rule of thumb” on “lines of code” too literally. Lines of Source code can vary by a magnitude (10 times) or more depending on which language you use and how you count lines (e.g. assembly code or a high level language). And of course inexpert programmers can push the number of errors way up. What it does tell you is the relative magnitude of the program which indirectly indicates its complexity and likelihood of serious problems.
Some people claim they have software with zero bugs. That may be possible when you are dealing with only a few lines of software code, however it is my experienced opinion (shared by a lot of world class programmers) whenever your code is larger than a hundred lines or so – it will have bugs.(3)
Rule #2. It is Impossible to Know How Many UnDetected Bugs are in a computer program.
Please keep in mind the difference between “Reported Bugs” and unknown or “Undetected Bugs.” While you must add the two together to get the total bugs; by definition we simply cannot know how many or how serious “Undetected Bugs” exist. We can only make
wild . . . ahem . . . educated estimates.
What does that mean for you and me? Well, consider the most popular operating system in use as of Sept 2012 – Windows XP with hundreds of millions of users. According to one source XP contains some 45 million lines of source code. Using the guidelines above and pretending Windows has only one-tenth of the minimum bugs of other commercial programs – at best that still calculates to considerably more than 5,000 bugs in Windows XP. (Microsoft itself reported Vista had some 5,000 known bugs just prior to release.)
Ok, lets relate computer bugs back to Cosmology.
Now as a cosmology aficionado you know that cosmic microwave “background” is one of the four main pillars presumably holding up the Big Bang conjecture. If you perused this earlier article “Cosmic Microwave Radiation Surprise” you would know that it is impossible to get a map of the controversial cosmic microwave “background” claim until you erase / subtract all the foreground microwave sources – primarily Milky Way stars and other galaxies.
Here’s a leading cosmologist’s breathtakingly opaque definition of Foreground microwaves (stated in the negative no less) “A Foreground is an effect whose dependence on cosmological parameters we cannot compute accurately from first principles at this time.” from “Overview of foregrounds and their impact,” Tegmark et all, 1999
And you would know that while we can make Microwave Foreground maps directly from the “image” data with no calculations or processing — to get any “background” mapping we need serious computer processing to subtract the foreground.(4)
Lets see a show of hands for those who would like to guess — “How many lines of source code are in the computer program used to establish the sky-covering microwave radiation as ‘background?'”
What do you estimate ? Ten thousand lines of code . . . twenty thousand ?
If so – that would be a minimum of 30 to as many as 1,000 predicted bugs.
Answer: But that’s not even close. How about two-thirds of a million lines of code?
Next Question: How long does that complex computer program have to run before we can see the results? 10 minutes?, an hour . . . 3 hours?
Answer: Five Months.
Each run of the program on each year of COBE data takes about five months. (5)
“How long does it take the software to process the COBE data? Because of the complex I/O required to input data, create numerous intermediate scratch files, and delete files to save space which was at an absoulte (sic) premium, it took about 20 hour of CPU time to process a single weeks data. A total of 5 months would be required to process a single year of data from all three instruments!”- Dr. Sten Odenwald
Question 3: How do you verify a computer program that takes five months to run — using more than 10,000 subroutines . . .
Answer: No one has enough time or funding to verify a computer program that takes five months to run. You test all the subroutines as well as you can and hope the whole system works without any misleading mistakes, absurd assumptions, egregious errors or fatal flaws.
Notably, commercial software has the benefit of having hundreds to millions of people testing the software millions of times and reporting bugs. COBE software, at best, had a handful of people running it; and they only ran the complete program a few times.
Question 4: Did the COBE software have any reported bugs?
Answer: At one time the Bug List had more than 11,700 known problems.(5) It is unclear if all those bugs were fixed before the deadline for software delivery.
Question 5: Does this make you wonder if the COBE software had any serious bugs ?
Answer: But COBE was a space program with world class programmers . . .
And so was Ariane.
Notes and References:
1. Doug Goodall once wrote a Personal Computer communications program in 64 bytes of assembly code. You read that right, that is not a mis-print. I did not mean 64 kilobytes. Sixty-four TOTAL bytes of assembly code. If poetry is distilled wisdom – that’s programming poetry.
3. How to be a zero-bug programmer, “Programmers Forum,” stackexchange.com
4. “Computing Challenges of the Cosmic Microwave Background,” Crittenden, Jaffe, Knox (1999)
Regarding the B2 Bomber programming error:
“moisture in three port transducer units ‘distorted data introduced by a B-2 Spirit’s air data system’ which led to flawed information entering the bomber’s flight control computers. The aircraft was reacting to inaccurate airspeed and a ‘perceived’ negative angle of attack. This resulted in an ‘uncommanded 30 degree nose-high pitch-up on takeoff,‘ according to the Air Force.”
7. Nobel Laureate Prof Richard Feynman’s Observations on the Challenger Space Shuttle Failure “Appendix F.” (Excellent analysis of risk predictions in the context of reality.)