Rocket Explosion From Software Bug (Ariane 5)
Oxymoron: “Bug-Free Software”
- The $1.4 billion US Air Force B-2 bomber wouldn’t fly on its maiden flight — thanks to a software bug.
- Later another B-2 bomber crashed and burned due to another software bug.
“If debugging is taking bugs out of software – then programming must mean putting bugs in . . .” – Doug Goodall, Assembly Language Poet, 1989 (1)
This article is to let us ponder the complexity and potential flaws of computer programs used in astrophysics and cosmology.
Rule #1. There is no such thing as Bug-Free Software
B-2 Bomber Crash due to a Software Bug
According to Carnegie Mellon University’s CyLab Sustainable Computing Consortium — “Commercial software typically has 20 to 30 bugs for every 1,000 lines of code.” (“KLoC” stands for “Thousand Lines of Code”) Another researcher broadens that estimate to “about 15 – 50 errors per 1000 lines of delivered code.” (Code Complete, Steve McDonnell, 2004) (2)
Please do not take this auxiliary “rule of thumb” on “lines of code” too literally. Lines of Source code can vary by a magnitude (10 times) or more depending on which language you use and how you count lines (e.g. assembly code or a high level language). And of course inexpert programmers can push the number of errors way up. What it does tell you is the relative magnitude of the program which indirectly indicates its complexity and likelihood of serious problems.
Some people claim they have software with zero bugs. That may be possible when you are dealing with only a few lines of software code, however it is my experienced opinion (shared by a lot of world class programmers) whenever your code is larger than a hundred lines or so – it will have bugs.(3)
Rule #2. It is Impossible to Know How Many UnDetected Bugs are in a computer program.
Please keep in mind the difference between “Reported Bugs” and unknown or “Undetected Bugs.” While you must add the two together to get the total bugs; by definition we simply cannot know how many or how serious “Undetected Bugs” exist. We can only make wild . . . ahem . . . educated estimates.
What does that mean for you and me? Well, consider the most popular operating system in use as of Sept 2012 – Windows XP with hundreds of millions of users. According to one source XP contains some 45 million lines of source code. Using the guidelines above and pretending Windows has only one-tenth of the minimum bugs of other commercial programs – at best that still calculates to considerably more than 5,000 bugs in Windows XP. (Microsoft itself reported Vista had some 5,000 known bugs just prior to release.)
Ok, lets relate computer bugs back to Cosmology.
Now as a cosmology aficionado you know that cosmic microwave “background” is one of the four main pillars presumably holding up the Big Bang conjecture. If you perused this earlier article “Cosmic Microwave Radiation Surprise” you would know that it is impossible to get a map of the controversial cosmic microwave “background” claim until you erase / subtract all the foreground microwave sources – primarily Milky Way stars and other galaxies.
Here’s a leading cosmologist’s breathtakingly opaque definition of Foreground microwaves (stated in the negative no less) “A Foreground is an effect whose dependence on cosmological parameters we cannot compute accurately from first principles at this time.” from “Overview of foregrounds and their impact,” Tegmark et all, 1999
And you would know that while we can make Microwave Foreground maps directly from the “image” data with no calculations or processing — to get any “background” mapping we need serious computer processing to subtract the foreground.(4)
Serious Software Bug
Lets see a show of hands for those who would like to guess — “How many lines of source code are in the computer program used to establish the sky-covering microwave radiation as ‘background?'”
What do you estimate ? Ten thousand lines of code . . . twenty thousand ?
If so – that would be a minimum of 30 to as many as 1,000 predicted bugs.
Answer: But that’s not even close. How about two-thirds of a million lines of code?
More precisely 654,000 lines of computer code and 11,156 subroutines (and at one time some 11,700 known software bugs).(5)
Next Question: How long does that complex computer program have to run before we can see the results? 10 minutes?, an hour . . . 3 hours?
Answer: Five Months.
Each run of the program on each year of COBE data takes about five months. (5)
Question 3: How do you verify a computer program that takes five months to run — using more than 10,000 subroutines . . .
Answer: No one has enough time or funding to verify a computer program that takes five months to run. You test all the subroutines as well as you can and hope the whole system works without any misleading mistakes, absurd assumptions, egregious errors or fatal flaws.
Notably, commercial software has the benefit of having hundreds to millions of people testing the software millions of times and reporting bugs. COBE software, at best, had a handful of people running it; and they only ran the complete program a few times.
Question 4: Did the COBE software have any reported bugs?
Answer: At one time the Bug List had more than 11,700 known problems.(5) It is unclear if all those bugs were fixed before the deadline for software delivery.
Question 5: Does this make you wonder if the COBE software had any serious bugs ?
Answer: But COBE was a space program with world class programmers . . .
Rocket Explosion From Software Bug (Ariane 5)
And so was Ariane.
________________________
Notes and References:
1. Doug Goodall once wrote a Personal Computer communications program in 64 bytes of assembly code. You read that right, that is not a mis-print. I did not mean 64 kilobytes. Sixty-four TOTAL bytes of assembly code. If poetry is distilled wisdom – that’s programming poetry.
Redmond, Wa.: Microsoft Press, 2004. 960 pages. ISBN: 07356196703. How to be a zero-bug programmer, “Programmers Forum,” stackexchange.com
4. “Computing Challenges of the Cosmic Microwave Background,” Crittenden, Jaffe, Knox (1999)
5. The COBE Program, The Astronomy Cafe, Dr. Sten Odenwald
Regarding the B2 Bomber programming error:
“moisture in three port transducer units ‘distorted data introduced by a B-2 Spirit’s air data system’ which led to flawed information entering the bomber’s flight control computers. The aircraft was reacting to inaccurate airspeed and a ‘perceived’ negative angle of attack. This resulted in an ‘uncommanded 30 degree nose-high pitch-up on takeoff,‘ according to the Air Force.”
7. Nobel Laureate Prof Richard Feynman’s Observations on the Challenger Space Shuttle Failure “Appendix F.” (Excellent analysis of risk predictions in the context of reality.)
English
Pingback: Cosmic Microwave Radiation Surprise | Cosmology Science © 2011-2013 David Dilworth
This is staggering. It upsets me. If what you say is correct microwave background is probably imaginary.
Thank you for your note.
Your conclusion may be a bit strong for me. Instead let me suggest this leaves the COBE claims unproven – not disproven.
What Microwave background researchers must do to clear this up is clearly and unambiguously define “background” and then explain how they detect it, or how it behaves, separately from the known, and easy to observe, foreground.
With what I understand so far regarding the microwave radiation from millions of galaxies in every square degree of sky, and the very poor angular resolution of all microwave “cameras” including COBE, WMAP and Planck — it appears logically impossible.
However, I’m willing to review any evidence and explanation that helps.
“Complex Computer Programs Weaken Cosmic Microwave “Background” Claim”.
No. This isn’t an idea that arise “above the minimum threshold for a scientific claim”. You can’t find any science references to software errors being a problem of principle, and indeed your only reference supports that seeing how it discuss challenges of processing large data sets. A challenge since met, by the way – WMAP has just made its 9 year data release, the final one of many successful such.
No specific experiment can “weaken” an observation or its theory, only test it. And because such observations are independently tested in independent experiments, there is no principle problem with leaking valves, software errors or noise sources. Science 101.
It would be nice to know what “claim” is alluded to, seeing how most observations and theories based on the many details in the CMB are firmly tested and not untested claims.
“Arianne” – Ariane, french for the daughter of Minos of Crete.
Thank you for the historical reference to Ariane and raising a good point related to WMAP’s new data release. However, that’s a minor distraction, since this article is really about calculations on COBE’s data to better explain why “Microwave Background Must be Calculated, Does not Clearly Exist” in the article “Cosmic Microwave Radiation Surprise.”
In response, to make it clearer I’ve improved the article title so it is clearly specific to COBE and duplicated a vital reference on the COBE software bugs to the bottom of the page.
1. With all due respect, the assertion that “No specific experiment can ‘weaken’ an observation or its theory” is a sophisticated example of how an argument improperly reverses the burden of proof and responsibility.
The burden of responsibility and proof is on the proponents of microwave “background” to define it adequately. The burden is not, as your note tries to invert, on skeptics with reasonable concerns.
Only after “background” is unambiguously defined, does the burden switch to skeptics.
In spite of that responsibility reversal (even though the CMBR folks have not yet unambiguously defined “background”), this article provides more than sufficient warrant for the position that COBE software is unlikely to be error free because microwave “background” mapping calculations are gigantic (654,000 lines long), complex (11,00 subroutines), and difficult to impossible to verify (5 months to process one year of data).
“The cooled remnant of the hot big bang” is not an adequate definition. If one wants to claim there is a microwave “background” it must be described by how it is physically detected differently from foreground microwave radiation in some way: presumably either by location or character. (You can read my understanding of how it is defined here.)
2. Unfortunately, contrary to your opinion the facts and ideas in this article can and do weaken the COBE calculation of “background” radiation, because the “background” claim depends on a computer software project that at one time had 11,744 individual bugs reported (SPRS).
If I had found any evidence-based claim that the 11,744 known COBE software bugs were all resolved by their deadline, the article might be unnecessary – even though (as the article explains) it is inherently impossible to know if all (other / unreported) software bugs are solved.
It is my experienced opinion that there were/are bugs in the COBE software programs. The big question is — Are any of them as serious as those that caused the Air Force B-2 Bomber crash or the Ariane rocket failure ?
By far, more importantly how did the COBE software subtract the microwave radiation from tens of thousands of foreground galaxies in each and every pixel of COBE data ?
3. The implication that somehow this article is an experiment is invalid. This article merely illustrates a reasonable concern that should have been raised years ago but, as far as I can find, no one ever did. I hope you are not asserting that we cannot raise any concerns with COBE’s gigantic, complex and difficult to verify software program.
If you want a hypothesis for this article it would be “the COBE software that led to the claim that there is a microwave background is unlikely to be error free.” That’s a testable claim that can potentially be falsified with evidence and knowledge by the COBE team.
4. “Background Radiation” is not an Observation
The assertion “No specific experiment can ‘weaken’ an observation or its theory” is misleading at best.
COBE’s “background radiation” claim was not an “observation” in any reasonable sense – it is a calculation. While Foreground microwaves can be easily observed, imaged and mapped, COBE’s “background” required at least five months of intensive calculations for each year’s worth of data to produce their maps.
Pingback: Java’s Security Problems are Designed-in. Problems are Not Solvable. I Do NOT Use ROES. | Dilworth Information Systems