Dave's Archives

Entries tagged with “science”

Hopes for 2012

Here's a bit of everything for the new year — some hopes for what we could and should be doing as a nation, in no particular order.

We must address the asylum seeker debate with decency, maturity and humility. We should accept many more refugees, and at the same time encourage other countries to do so too. There's really very little downside to this, save for the political ramifications of xenophobia. The world collectively might not be interested in finding a safe home for all its refugees, and so if we let refugees come to us, they will certainly continue to do so. We might prefer that their lives were not further jeopardised by the journey, but, having arrived, it's an utter perversion of human decency for us to turn them away, no matter how much we'd like to discourage further risky voyages. We must not create disincentives that rely on penalising innocent people; we have no right to play chess with human beings.

We must get some perspective on the economy; it is not a blanket reason for putting aside all other problems. Yes, it's important. No, we are not teetering on the edge of starvation. Panic is precisely the thing that causes economic problems in the first place. Basically, let the disinterested economic experts make rational, progressive decisions based on careful, objective modelling, and ensure there is a safety net for the poor. Everyone else, suck it up.

We must continue to insist that our politicians get off their conservative arses and legalise same-sex marriage. This is truly a no-brainer. The arguments against it are utterly, unequivocally spurious, and will dissipate like so much hot air once the requisite legislation is passed. Nobody opposes same-sex marrige for any substantive reason, but basically just "because". Once legalised, the whole "debate" will be relegated to the inane murmurings of ineffectual dinosaurs. (Do politicians fear a backlash from voters angry that their marriages are suddenly devoid of meaning following the gender requirements being dropped?)

For the love of humanity can we please redouble efforts to improve the health and living standards of those living in remote Aboriginal communities? Of course it won't be done in a year. It's not just about grand rhetorical gestures — though these have their place — and it's certainly not about sending in the army. We have a lot of smart, dedicated people who have been on the case for some time, and surely by now we've learnt a thing or two about what can usefully be done, given sufficient government funding.

The climate change debate is not over, and won't be for decades. We must not lose sight of the fact that the goal, in the end, is zero (or even negative) carbon emissions. The purpose of a carbon price is not simply to reduce emissions, but ultimately to price them out of existence. To make this work, alternatives must exist. Australia should, by all rights, be a world-leader in solar energy. We could be a world-leader in all kinds of renewable energy. Surely there is much more scope for public and private funding of renewable energy research. We might only contribute 1.5% of the world's greenhouse gas emissions, but renewable energy research could help reduce everyone's emissions, not just our own.

Recycled drinking water — get used to the idea, people. Water efficiency is vastly more important than your squeamishness; there's really no rational objection. Even now, the water you drink has already passed through the digestive tracts of a trillion different organisms, without any technological assistance. Water recycling is the lowest-hanging fruit for securing our water supplies (especially in places like Perth that are drying out). Why would we ignore it in favour of energy-intensive desalination  or enormous engineering works to transport water from thousands of kilometres away? Yes, we can build wind farms, tidal generators, etc. to power desalination plants, but we could be using that power to replace coal, not just to replace water.

The location of the Square Kilometre Array (SKA) — the world's largest telescope, and one of the world's largest scientific projects — will be decided in 2012: either Australia or South Africa. Let's step back from the parochial contest. Australia might not get it, but would this be such a terrible outcome, all things considered? Maybe Africa would benefit more from this project than Australia. Besides the raw economics, the presence of such visible, cutting edge science must have some inspirational effect. Scientists can travel, but for young Africans trying to discern their opportunities in life, a local SKA would surely leave an impression.1

Finally, in an Olympic year, let's not lose sight of our non-sporting heroes. A nation defined by sport is a nation not defined by its doctors, lawyers, scientists, engineers and other professionals. Sport is exciting, and important in its own way, but not really on the same scale as curing illness, defending human rights, exploring the universe and creating things that have never existed before.

Now, you lot, get started on that while I take a holiday.

  1. The same would be true of young Australians, but we are relatively spoiled for choice. []

→ 1 Comment Tags: , , , , , , , ,

The Galileo gambit movement

I've had another sudden fit of pseudo-artistic buffoonery.

I stumbled across the Galileo Movement largely by way of Wendy Carlisle's Background Briefing report:

In February this year a new group emerged: the Galileo movement. Its scientific advisers are the who's who of the international climate sceptics movement. Its patron is the powerful Sydney radio personality Alan Jones. The Galileo movement is aiming to kill the carbon tax, and it's aiming to do this through attacking the science of climate change.

This is a fabulously un-self-aware group of climate change denialists who liken their cause to that of Galileo, and who purport to offer the Real Truth of the Earth's climate. They are the living epitome of the Galileo gambit, which itself is much older and is described by RationalWiki as follows:

They made fun of Galileo, and he was right.
They make fun of me, therefore I am right.

I feel the following diagram adequately summarises the situation:

This is all very Australia-centric, of course, and wholly political, though they claim otherwise. The stated purpose of the Galileo Movement is merely to stop Australia's carbon tax - an entirely political goal - not to actually redress the horrific corruption of science they claim to be occurring. The "corruption of science" seems more like an excuse for their own political predicament than an actual problem that must be solved. If such systemic corruption of the scientific process was real, after all, it would be far worse a problem than any mere tax1. However, the movement's scientific literacy is clearly razor-thin, with adorable statements like this:

We care about freedom, security, the environment, humanity and our future.

The Galileo Movement's co-founders are retirees Case Smit and John Smeed. Their business backgrounds are in science and engineering - science's real-world application. Their experience is in environmental protection and ensuring air quality.

At first they simply accepted politicians' claims of global warming blamed on human production of carbon dioxide (CO2). When things didn't add up, they each separately investigated. Stunned, they discovered what many people are now discovering: climate claims by some scientists and politicians contradict observed facts.

Here's another theory: Case Smit and John Smeed have never been involved in actual climate-related research at all, but through some intrepid Googling discovered that People On The Internet were having arguments. A stunning revelation indeed. Not having any particular notion of what real science is actually supposed to look like, they simply believed those people who appeared to be more outraged. Just as in Galileo's time.

  1. Denialists rarely realise the scale of the allegations they so casually make; if they did, they would have to confront their implausibility. []

→ 1 Comment Tags: , , , ,

Poor persecuted Monckton

His Great and Wondrous Beneficence the Lord Christopher Monckton did, after all, give a lecture at Notre Dame University. Attempts (initiated by Natalie Latter) to dissuade Notre Dame from lending Monckton its credibility did not come to fruition, though drawing attention to his Lordship's rank lunacy is always a small victory in itself.

As the letter puts it:

We all support academic freedom and the freedom to express our ideas and beliefs. However, Notre Dame University has a responsibility to avoid promoting discredited views on an issue of public risk. Notre Dame's invitation to Lord Monckton makes a mockery of academic standards and the pursuit of evidence-based knowledge.

This has been laughably characterised as an attempt to "gag" Monckton (who has a minor obsession with characterising people as fascists and war criminals, suggesting for instance that climate scientists ought to stand trial for genocide). Does anyone honestly think that Monckton actually could have been gagged?

This call to preserve academic standards morphed (perhaps predictably) into a spurious fight for free speech. Tell me, dear reader: when was the last time you exercised your apparently fundamental democratic right to give a public lecture at a university?1 Do you believe that you have that right; that a university has a duty to invite you to give a lecture if you see fit to give one? Why should Monckton be afforded this privilege, when clearly "ordinary" members of the public are not?

Some, such as Professor Chris Doepel at Notre Dame, argue that all points of view must be heard. This is the refrain we hear from creationists asking that "Intelligent Design" be taught in schools. It's a convenient rhetorical tool for engineering doubt. The consensus of virtually all the relevant experts, arrived at by considering the entire gamut of objective data collection and analysis conducted over decades, is made to look like only one set of opinions, rivaled by another set of opinions formed simply by making things up. Doepel makes the following self-refuting remark:

The university does not take a view one way or the other on the positions advocated by Christopher Monckton.

But that is a position on Monckton. An individual person might legitimately claim not to know enough to form an opinion2, but it beggars belief that a university - a place wherein truth is uncovered and disseminated - would have formed no position on one of the most outspoken and controversial figures of our time. A refusal to condemn Monckton's views, for an institution that cannot possibly claim ignorance of what he stands for, is effectively an endorsement of those views. We certainly know where Notre Dame stands on legitimate climate research and climate action, then.

Others (such as the Fremantle Mayor Brad Pettitt) believe we should just let Monckton speak, and take the time to refute his claims. But this is to accept the false dichotomy that either he be allowed to speak wherever he likes, at any institution, or we tie him up in the basement. Monckton was clearly never in any danger of actually being silenced, not even if Notre Dame had heeded the call to preserve its academic integrity. Universities have credibility in the first place precisely because they discriminate between views supported by evidence and views not so supported (the same as scientific journals, and the scientific process in general). One can delude oneself into thinking that this is somehow undemocratic, but then reality isn't democratic. At some point, for the sake of advancing the human cause, we must stand up and pass judgement; not on each other, but on our ideas. Science, technology, economics, etc. are not served simply by sitting and listening politely and "fairly" to endless regurgitations of refuted arguments. We have the media and the Internet for that; universities should know better.

Some believe we should just ignore Monckton. However, the man is steering the public debate in ways that are fundamentally detrimental to the prospects for sensible policy making. We cannot just ignore him. Academic institutional credibility aside, he already has all the media coverage anyone could dream of. This isn't the result of some PR folly by his critics, but rather his oratory skills and the cozy hardline ideological relationship he has with some very loud and obnoxious media personalities.

It is incumbent upon academics to preserve the integrity of their institutions, and to confront misinformation that threatens to derail rational decision making. Free speech is a right, no doubt, but credibility must be earned.

  1. It is entirely possible, I suppose, that you have indeed given a public lecture at a university, but I think you'll agree that it's not exactly a right. []
  2. I sometimes admire those willing to admit ignorance rather than pick whichever view "feels" better. []

→ 5 Comments Tags: , , , ,

Who is Dennis Ambler?

Continuing on (a bit) from my last post, I'm going to examine another of Dennis Ambler's articles for the Science and Public Policy Institute (SPPI). This one is mostly a long rambling swipe at lots of different and very accomplished individuals, and not (as in the other case) an outright attempt to reinvent the laws of mathematics and statistics.

Here, Ambler focuses on a report written for the American National Academies, called Advancing the Science of Climate Change. He is apparently responding to an article in the Washington Post by Sherwood Boehlert, who quotes a key line from the report's summary:

A strong, credible body of scientific evidence shows that climate change is occurring, is caused largely by human activities, and poses significant risks for a broad range of human and natural systems.

This is the official, considered position of the American National Academies, and closely resembles a Q&A document released by the Australian Academy of Science:

The Earth’s climate has changed. The global average surface temperature has increased over the last century and many other associated changes have been observed. The available evidence implies that greenhouse gas emissions from human activities are the main cause. It is expected that, if greenhouse gas emissions continue at business-as-usual rates, global temperatures will further increase significantly over the coming century and beyond.

Ambler doesn't (here, at least) address the Australian statement, or any of the other statements issued by other national science academies and other scientific organisations around the world. We'll set that aside for now.

His point appears to be that the Panel on Advancing the Science of Climate Change (presumably responsible for the first quote) is stacked with advocates* of anthropogenic global warming (AGW). He goes into great detail about the nefarious activities of various board members, such as being IPCC authors, members of NGOs, members of advisory boards, having links to the United Nations and being either corporate- or government-funded. Scandalous, I know.

However, Ambler's entire argument rests on the notion that the aforementioned organisations have already somehow been discredited. That may be true in his head, because the IPCC says all the things he doesn't like to hear. However, there is no objective reason to think that the IPCC is not essentially fulfilling its intended function: the careful, objective assessment of the science behind climate change and its implications. His argument also relies on the assumption from the outset that the AGW consensus does not exist. If consensus is real, then it's hard to imagine what could be wrong with having such a panel "stacked" with those who've poured their energies into addressing the problem.

Ambler's other complaint is one I've heard repeated for other advocates of climate action - that they're not climate scientists:

I doubt a dissenting voice on “the science” is ever heard in their deliberations. As can be seen, climate scientists are very much in the minority. It seems that a mix of economists, social scientists, engineers, NGO’s and corporations in receipt of government funding, form the main strength of these particular committees.

I doubt a dissenting voice on "the science" would ever be heard no matter how many climate scientists you added (Ambler has tried to argue against the existence of the consensus, but not too successfully.) Nevertheless, at first glance, it might seem common sense that climate change panels ought to be populated entirely by climate scientists, until you realise that climate science only identifies the existence of the problem. Climate science says nothing about the humanitarian, economic, or even environmental effects of climate change, and it certainly does not say what we might do, as a society, to prevent or mitigate them. This is not a problem with climate science - it simply reflects the cross-disciplinary nature of climate change.

Climate change denialism does not appear to recognise the distinction between all these facets. It does not accept, for instance, that someone can be qualified to talk about reducing CO2 emissions unless they can also convincingly explain the data used to establish the effects of CO2 in the first place. To an armchair "sceptic", these two discussions belong in the same discipline. Except they don't. There is a very good reason why the Stern and Garnaut Reviews, for instance, were written by economists rather than climate scientists. Modeling economic costs just isn't part of climate science; the issue transcends disciplines.

In setting out to undermine all work on climate change**, climate denialists become Jacks of all trades and masters of none. They perceive the entire concept of climate change as being the domain of a single discipline (climate science), because they don't realise the depth of analysis that underlies each part of that picture. Analysing the effects of greenhouse gases on temperature could be a life's work, for instance. Analysing the effects of temperature increases on agricultural practices is another life's work, as is modelling the economic costs and benefits of reducing CO2 emissions, and so on. The true experts each spend their time on a relatively small part of the problem, but paying enormous attention to detail. They rely on other experts to fill in the gaps where needed, because no single person can be an expert in everything. Meanwhile, denialists skip lazily across the entire scope of the problem and engage only in shallow commentary and nitpicking. It would be difficult to comprehend just how many different disciplines are crossed when you naïvely believe that you (and/or those you follow) possess the entire range of necessary expertise.

Thus, the many genuine scientific, humanitarian, political and economic debates regarding climate change are, in denialist circles, mashed crudely into just one big issue, adjudicated solely by climate scientists (except for all the ones who write those terrible papers about hockey stick graphs; they don't count).

Just for fun, let's examine the scientific credentials of some SPPI contributors, starting with Dennis Ambler himself. Ambler puts his name to 14 of the last 100 articles (at the time of writing), and thus appears to be the most prolific recent contributor to the SPPI collection. However, try as I might, I cannot find any biographical information on the man at all. Even SPPI's own Personnel page neglects to mention him. There are no details of his history, qualifications, accomplishments, collaborations, involvements with other organisations, or even interests. "Dennis Ambler" might as well be a pseudonym for all I can tell.***

Next in line is Christopher Monckton, named as the author of 10 of the last 100 articles. Fortunately he is mentioned in the Personnel page, as being an expert on virtually everything, despite not possessing any qualifications at all on anything remotely resembling science or economics. It's worth a read.

Then there's Ross McKitrick, with 4 out of 100 articles. He's an economist, which I hope doesn't put him offside with Ambler.

The remaining articles were contributed by a slew of authors with (I presume) only a tangential relationship to SPPI itself. I won't discuss them, except to note that the current President of the Czech Republic Václav Klaus is among them. Doubtless he too specialises in climate science. It's all about expertise, you understand.


* It's hard to maintain the correct wording here, since it can be a little unwieldy. Nobody advocates climate change - that's precisely what we don't advocate.

** Climate denialism, in aggregate form, opposes every facet of the science on climate change - virtually every finding of every paper - which would be quite a remarkable occurrence if we held it to be intellectually honest. Individuals may accept certain parts of the science to varying extents, and often claim that "nobody" seriously disputes those parts (e.g. that the Earth is currently warming), but in reality every single detail is disputed in some corner or other. (The only exceptions to this are the occasional papers written by denialist champions like Steve McIntyre.)

*** To be fair, it would be hypocritical of me to suggest that all this information is a pre-requisite for making public comments. However, I'm just a pseudo-anonymous blogger, while Dennis Ambler is backed by an "Institute", conferring a facade of expert credibility.

→ 4 Comments Tags: , , , , ,

Consensus Bashing

The Science and Public Policy Institute certainly does provide a lot of hilariously twisted commentary on climate change.

Two years ago (January 2009), Doran and Zimmerman (D&Z) published a paper based on Zimmerman's masters thesis. Unsurprisingly, they found that the vast majority (97%) of climate scientists think climate change is real and human-induced. This kind of thing really, really irritates climate change denialists, and so we have Dennis Ambler from SPPI launching into a blisteringly woeful attack on the survey.

He sets the tone with this:

[The survey] was roundly de-bunked at the time by several commentators and it would have been forgotten and consigned to its proper place in the dustbin, if it hadn't been continually
quoted by activists as fact.

If you're going to claim that a paper has been "roundly debunked", a little elaboration would not go astray. Some of us might just be curious about just what arguments were put forth, and you're not giving us much to go on. Also, if we're going to lend it such credence, I would expect some sort of expertise to be involved in this debunking, not just a vague reference to unspecified "commentators".

He then bemoans the surveying of experts as a means of assessing scientific opinion:

This is not arcane knowledge for the select priesthood, this is science and we can read scientific papers and apply quality judgements to them, whether we be specialists or not.

No, Ambler, you really can't. I know this for two reasons:

  1. Those who've genuinely tried to read and understand technical papers in a field they don't work in will know just how much of an uphill battle it can be. There's unfamiliar jargon, horrendous equations, often enormous amounts of assumed background knowledge, and frequently little attention paid to overall readability. These papers are written for a very narrow audience, and you can't just plant the flag of egalitarianism and ignore all the hard work that goes into building the necessary expertise.
  2. Even if you were equipped to read and understand technical papers from any discipline, the sheer quantity of them would make the task logistically impossible. They don't just dribble out one or two at a time every news cycle. There's countless thousands (possibly millions) of them, and nobody (scientists included) can ever hope to read them all. That's why we have surveys. Even researchers themselves rely on survey papers, for instance, to make sense of their own fields.

The remainder of Ambler's article demonstrates his unique inability to "read scientific papers and apply quality judgments". I say "unique" because D&Z's paper is actually quite short and accessible. Given a modicum of education and common sense, there really isn't much of an excuse for not understanding it.

We are also told that only 5% of the original sample responses were climate scientists, so if we pragmatically apply those proportions we end up with just 141 from the US, 9 from Canada and just 6 from 21 countries around the world, hardly a global consensus.

Is there some significance to the 5% figure? The survey was a broad look at the opinions of Earth scientists. Climate scientists just form an important subset of that population, and it's hardly the fault of the authors or anyone else if the proportion happens to be 5%.

Moreover, Ambler knows he can estimate the number of respondents from each country, but he seems not to understand that the very same mathematical device is the reason you don't need to ask everyone in the world. So long as you have a representative sample (and consulting a database of Earth scientists, as D&Z did, would seem to be perfectly acceptable), you can generalise your findings. If 97% of your sample believes X, and your sample is representative of a given group (e.g. climate scientists), then you infer that about  97% of the overall group believes X as well. This is the entire basis of surveys. If this statistical logic did not hold, surveys would not exist.

Why so few non-American climate scientists? That's just a result of the database used by D&Z, coming as it did from the American Geological Institute. There's no reason to think that American and non-American climate scientists are likely to have any specific, major points of professional disagreement, so this shouldn't be a problem.

We find that they originally contacted 10,257 scientists, of whom 3,146 responded, less than a 31% response rate. “Impending Planetary Doom” was obviously not uppermost in the minds of over two thirds of their target population.

31% is a very good response rate, in my experience. I would not have raised an eyebrow if it was only 10% (except that the authors would then have been less well-equipped to draw conclusions). It's silly to start attributing reasons for non-response, because by definition you don't have the data. It's certainly very silly to suggest that 7111 scientists don't care about the issue merely because they failed to fill out a questionnaire on it. Perhaps they were too busy actually working on the problem!

Ambler does us a service by linking to D&Z's summary paper, but he's a bit of a cheapskate:

The paper is behind a pay wall but there is a comprehensive summary here.

It's obvious from reading Ambler's article that his own investigative skills cannot penetrate this "pay wall". Despite describing the summary as "comprehensive", he repeatedly complains about missing details. For example:

There is little detail of how many peer reviewed papers are needed to qualify as a specialist, it could by their definition be just two papers, one of which needs to be on climate change. What a poor example of scientific enquiry this survey really is.

The one-and-a-bit-page summary paper does not, of course, include all the information from the 141-page thesis. The price for an electronic copy of Zimmerman's thesis is only $US 2, hardly a prohibitive sum. I bought a copy myself just so that I could write this post. Zimmerman provides an extensive explanation of the process of verifying whether survey respondents are, in fact, active publishers on climate science (page 16). However, this kind of nit-picking was never going to undo the rather stark results.

There were supposed to have been nine questions asked, but we are only given sight of two of them.

Again, this is what you get if you only read a summary. For those interested, the full set of nine questions consisted of four opinion-related questions and five demographic questions:
  1. When compared with pre-1800's levels, do you think that mean global temperatures have generally risen, fallen, or remained relatively constant?
  2. Do you think human activity is a significant contributing factor in changing mean global temperatures?
  3. What do you consider to be the most compelling argument that supports your previous answer?
  4. Please estimate the percentage of your fellow geoscientists who think human activity is a contributing factor to global climate change.
  5. Which percentage of your papers published in peer-reviewed journals in the last 5 years have been on the subject of climate change?
  6. Age
  7. Gender
  8. What is the highest level of education you have attained?
  9. Which category best describes your area of expertise?

Ambler of course takes issue with the first two questions. For question 1:

Has it got warmer since pre-1800 levels? This really depends on the time period referred to. Do they mean the Little Ice Age, when disastrously cold temperatures caused massive loss of life and untold hardship? Of course temperatures are now warmer than that desperate period in climate history. Is that what they would wish to regard as normal?

Climate denialists often have a mild obsession with two proposed fluctuations in global temperature over the last few hundred years. They explain away the current warming trend by saying that we're merely coming out of a cold period (the Little Ice Age), and that temperatures have been warmer in the past (the Medieval Warm Period), but the evidence for either of these is rather limited. This is related to another pet denialist obsession: the "hockey stick" graph, which shows that the current warm temperatures are unprecedented over at least the last millennium. It is essential denialist lore that the hockey stick has been discredited. In reality, it has numerous independent replications.

For question 2, on whether human activity is a factor:

This is the classic closed question, in that it implies mean global temperatures are being changed and someone must be responsible.

First, respondents are not asked this question if they previously said that temperatures remained relatively constant; so no, the question does not assume temperatures are being changed. It certainly does not assume that "someone must be responsible" - I have no idea how Ambler could have read that into it.

About half-way through his article, Ambler makes his biggest departure from reality, and one that cannot be excused by lack of information. This divergence begins as follows:

Of [the 3146 respondents], only 5% described themselves as climate scientists, numbering 157. The authors reduce that by half by only counting those who they classed as “specialists”.

The authors do no such thing. They categorise their 3146 respondents by field (climatology, geology, etc.) and whether more than 50% of their recent published papers were related to climate science. In the media, the most widely-reported statistics are, appropriately enough, for actively-publishing climatologists. However, this categorisation does not omit anyone, but merely provides more detailed information.

It is disingenuous to now use the “climate scientists” as a new population sample size. The response figure of 3,146 is the figure against which the 75 out of 77 should be compared and in this case we get not 97% but just 2.38%.

Ambler appears not to notice that there are statistics on the complete set of 3146 respondents, not just the 77 who happen to be actively-publishing climate scientists. Ambler's 2.38% is the proportion of respondents who agree that humanity has an influence on climate and who also happen to be climate specialists. If you think that climate change is real but you're not a specialist, Ambler is counting you in the total population but not in the "yes" pile (and so by implication in the "no" pile).

That's dishonesty if ever I've seen it. D&Z expressly state in their summary paper that 90% of respondents overall agreed that temperatures have risen, and 82% agreed that humanity was a factor. Ambler expressly ignores these statistics and then tries to reverse engineer them using profoundly broken mathematics.

The original number contacted was 10,157 [sic] and of those, 69% decided they didn’t want any part of it, but they were the original target population. When the figure of 75 believers is set against that number, we get a mere 0.73% of the scientists they contacted who agreed with their loaded questions.

Ambler now wants to count non-respondents in the total as well, making the unsupportable implication that they would have said "no". This is utter nonsense and is a complete corruption of general survey methodology. You use the data you have - that's how science works - not by making assumptions about the data you don't have.

Ambler ends by ridiculing media reports that quite fairly echo D&Z's findings. Apparently they're not privy to his powers of deduction, and neither am I.

→ 1 Comment Tags: , , , ,

Open source science

Slashdot notes an article from the Guardian: "If you're going to do good science, release the computer code too". The author is, Darrel Ince, is a Professor of Computing at The Open University. You might recognise something of the mayhem that is the climate change debate in the title.

Both the public release of scientific software and the defect content thereof are worthwhile topics for discussion. Unfortunately, Ince seems to go for over-the-top rhetoric without having a great deal of evidence to support his position.

For instance, Ince cites an article by Professor Les Hatton (who I also cite on account of his recent study on software inspection checklists). Hatton's article here was on defects in scientific software. The unwary reader might get the impression that Hatton was specifically targetting recent climate modelling software, since that's the theme of Ince's article. However, Hatton discusses studies conducted from 1990-1994, in different scientific disciplines. The results might still be applicable, but it's odd that the Ince would choose to cite such an old article as his only source. There are much newer and more relevant papers; for instance:

S. M. Easterbrook and T. C. Johns (2009), Engineering the Software for Understanding Climate Change, Computing in Science and Engineering.

I stumbled across this article within ten minutes of searching. While Hatton takes a broad sample of software from across disciplines, Easterbrook and Johns  delve into the processes employed specifically in the development of climate modelling software. Hatton reports defect densities of around 8 or 12 per KLOC (thousand lines of code), while Easterbrook and Johns suggest 0.03 defects per KLOC for the current version of the climate modelling software under analysis. Quite a difference - two orders of magnitude, for those counting.

Based on Hatton's findings of the defectiveness of scientific software, Ince says:

This is hugely worrying when you realise that just one error — just one — will usually invalidate a computer program.

This is a profoundly strange thing for a Professor of Computing to say. It's certainly true that one single error can invalidate a computer program, but whether it usually does this is not so obvious. There is no theory to support this proclamation, nor any empirical study (at least, none cited). Non-scientific programs are littered with bugs, and yet they are not useless. Easterbrook and Johns report that many defects, before being fixed, had been "treated as acceptable model imperfections in previous releases", clearly not the sort of defects that would invalidate the model. After all, models never correspond perfectly to empirical observations anyway, especially in such complex systems as climate.

Ince claims, as a running theme, that:

Many climate scientists have refused to publish their computer programs.

His only example of this is Mann, who by Ince's own admission did eventually release his code. The climate modelling software examined by Easterbrook and Johns is available under licence to other researchers, and RealClimate lists several more publicly-available climate modelling programs. I am left wondering what Ince is actually complaining about.

Finally, Ince seems to have a rather brutal view of what constitutes acceptable scientific behaviour:

So, if you are publishing research articles that use computer programs, if you want to claim that you are engaging in science, the programs are in your possession and you will not release them then I would not regard you as a scientist; I would also regard any papers based on the software as null and void.

This is quite a militant position, and does not sound like a scientist speaking. If Ince himself is to be believed (in that published climate research is often based on un-released code), then the reviewers of those papers who recommended publication clearly didn't think as Ince does - that the code must be released.

Ince may be convinced that scientific software must be publicly-auditable. However, scientific validity ultimately derives from methodological rigour and the reproducibility of results, not from the availability of source code. The latter may be a good idea, but it is not necessary in order to ensure confidence in the science. Other independent researchers should be able to confirm or contradict your results without requiring your source code, because you should have explained all the important details in published papers. (In the event that your results are not reproducible due to a software defect, releasing the source code may help to pinpoint the problem, but that's after the problem has been noticed.)

There was a time before computing power was widely available, when model calculations were evaluated manually. How on Earth did science cope back then, when there was no software to release?

→ 2 Comments Tags: , , ,

Peer review

I've stumbled across yet another "ClimateGate" article (by way of James Delingpole), this one going right for the jugular of science: peer review. The author is journalist Patrick Courrielche, who I hadn't come across until now.

Courrielche argues that peer review is kaput and is being replaced by what he calls "peer-to-peer review", an idea that brings to mind community efforts like Wikipedia. This has apparently been catalysed by "ClimateGate", an event portrayed by the denialist community as something akin to the Coming of the Messiah.

Courrielche asserts that peer review is a old system of control imposed by the "gatekeepers" of the "establishment", while peer-to-peer review is a new system gifted to us by the "undermedia". Courrielche has very little time for nuance in the construction of this moralistic dichotomy, and clearly very little idea why peer review exists in the first place.

It should be noted from the start (and many an academic will agree) that peer review is a flawed system. It's well known that worthwhile papers are rejected from reputable journals from time to time, while the less reputable journals have the opposite problem. Nevertheless, there is a widely-recognised need for at least some form of review system to find any weaknesses in papers before publication. It seems obvious that the people best placed to review any given piece of work are those working in the same field. Peer review acts both as a filter and a means of providing feedback (a sort of last-minute collaborative effort). The reviewers are not some sort of closed secret society bent on stamping their authority on science, as Courrielche seems to imply. Anyone working in the field can be invited by one relevant journal or another to review a paper, and it's in a journal's best interests to select the best qualified reviewers.

Courrielche sticks the word "review" on the end of "peer-to-peer" so that it can appear to fulfill this function. The premise seems to be that hordes of laypeople are just as good, if not better, at reviewing a given work than those who work in the relevant field. This is really just thinly-veiled anti-intellectualism. How can a layperson possibly know whether the author of a technical paper has used the appropriate statistical or methodological techniques, or considered previous empirical/theoretical results, or made appropriate conclusions?

That's why papers are peer-reviewed. Reputable journals get their reputation from the high quality (i.e. usefulness and scientific rigour) of the work presented therein, as determined by experts in the field. Barring the very occasional lapse of judgment, the flat earth society, the intelligent design movement, the climate change denialists, and any number of other weird and wonderful parties are prevented from publishing their dogma in Science, Nature and other leading journals. There's no rule forbidding such publication; that's just what happens when you apply consistent standards in the persuit of knowledge. Ideologues are frequently given an easy ride in politics, and it clearly offends them that science is not so forgiving.

However, Courrielche appears to be more interested in describing how the "undermedia" is up against some sort of vast government-sponsored conspiracy to hide the truth. His tone is one of rebellion, of exposing the information to the media, and doing battle with dark forces trying to prevent its disclosure. Even if such a paranoid fantasy were true, it has nothing to do with peer review. Peer review is not a means of quarantining information from the public, but simply a way of deciding the credibility of that information. In reality, the information is already out there, and in fact it's always been out there (just not necessarily in the mass media). The problem is not the lack of information, but the prevalence of disinformation. We are all free to ignore the information vetted by the peer review system, but we don't because it's intrinsically more trustworthy than anything else we have.

Courrielche makes mention of the "connectedness" of the climate scientists, as if mere scientific collaboration is to be regarded with deep suspicion. Would he prefer that scientists work in isolation, without communicating? This is quite blatantly hypocritical, because his peer-to-peer review system is based on connectedness.

Well, sort of. I also suspect that most of the many and varied denialist memes floating around have not resulted from some sort of collective intelligence of the masses, but from a few undeserving individuals exalted as high priests by certain ideologically-driven journalists. There is nothing "peer-to-peer" about that at all.

From my point of view, what Courrielche describes as the "fierce scrutiny of the peer-to-peer network" is more like ignorant nitpicking and groupthink. There are no standards for rigour or even plausibility in the many of the discussions that occur in the comments sections of blog sites. Free speech is often held sacrosanct, but free speech is not science.

The denialists are up against much more than a government conspiracy. They're up against reality itself.

Comments Off Tags: , ,

The colloquium

An "official communication" from early June demanded that all Engineering and Computing postgraduate students take part in the Curtin Engineering & Computing Research Colloquium. Those who didn't might be placed on "conditional status", the message warned.

A slightly rebellious instinct led me to think of ways to obey the letter but not the spirit of this new requirement. Particularly, the fact that previous colloquiums have been published online introduced some interesting possibilities:

  • a randomly-generated talk;
  • a discussion of some inventively embarrassing new kind of pseudo-science/quackery; or
  • the recitation of a poem.

In the end I yielded, and on the day (August 25) I gave a reasonably serious and possibly even somewhat comprehensible talk on a controlled experiment I'd conducted on defect detection in software inspections.

A while afterwards, I received in the mail a certificate of participation, certifying that I had indeed given the talk I had given. It felt a little awkward. Giving a 15 minute talk isn't something I'd have thought deserving of a certificate. It might be useful for proving that I've done it, since it now appears to be a course requirement, but a simple note would have sufficed.

Interestingly, I later received another certificate, identical except that my thesis title had been substituted for the actual title of my talk. In essence, I now have a piece of paper, signed personally by the Dean of Engineering, certifying that I've given a talk that never happened.

Comments Off Tags: , ,