Tim Oren's Due Diligence

Head of Global Warming Research Unit Steps Down

Dr. Phil Jones, head of the East Anglia Climate Research Unit, and star of the leaked e-mail and document set from that institution, has stepped down pending an independent review of the unit and its research. So ClimateGate (hate that name) claims its first head on the science side. (Apparently the head of the opposition in Australia fell this week, largely due to a cap-and-trade scheme perhaps exacerbated by the ongoing scandal.)

I have been following the developments, reading relevant blogs as well as skimming a chunk of the e-mails and related text documents. I've also read most of the famous 'HARRY READ ME' file, containing the tribulations of a programmer trying over a period of three years to repeat the derivation of the CRU's temperature database from a poorly documented and sometimes missing set of programs and data. I've made a few comments elsewhere, but not posted here, as those with a lot more familiarity with the climatology data and personalities have entered the conversation, and others a lot more current in the software engineering community, e.g., Eric Raymond, have started tearing apart the released code.

I have drawn a few conclusions, though:

• No one, including those from CRU, has suggested that these files are a fake or have been 'salted', so I can let the latter worry go. Given (for instance) the dearth of 'chatter' or housekeeping type e-mails, it's still a safe bet that they are a careful selection from the available material. Leading to:


• This was an inside job, a leak. No random hacker grabbed these files. It's a dossier assembled over time by someone who knows just where the skeletons have been buried. I suspect that CRU is now 90+% positive who did the deed, but is going to go real easy under the fear (or knowledge) that enough has been held back to make any prosecution into an even worse debacle.


• The main story of the e-mails is the deliberate corruption of the scientific process. From blackballing opponents' articles, to getting non-cooperative editors fired, to specifically declaring their readiness to alter the peer review process to keep skeptics' views out of the IPCC discussions. This isn't rebutting criticism, it's attempting to rule it out a priori. The dishonesty seen here just reeks, and casts a deserved pall over everything published by the CRU and other implicated scientists.


• The public (or media) have an incorrect impression of 'peer review'. I've served on a number of conference program committees, albeit in different domains, and done reviews for journals as well, so I feel qualified to say: 'Peer review' is an initial screening of submissions to see if they are fit to enter the conversation. You're looking for things such as citation of the relevant literature and knowledge of prior art, significance of the result, (non)repetition of previous work, a minimal level of readability, and making sure that the submission is not completely in left field (e.g., in computer science claiming to have solved a known NP-complete problem, without an exhaustive demonstration). No one expects a reviewer to attempt to duplicate the described results. Peer review does not validate the submission. Attempting to confirm, rebut, or refine a paper is another project, worthy of another publication - that's how the conversation proceeds. Attempting to prop up the validity of any model by calling it 'peer reviewed' is dishonest, particularly when the peer review process is corrupted into a stone wall against the unorthodox.


• The main story of HARRY_READ_ME.txt and the related program files and comments is of an incompetent data and code management process. 'Harry' (or perhaps his coder) has my entire sympathy. Back in my grad student days I also implemented models and converted datasets, in FORTRAN even. Projects that aim to get a one-off result determined and published don't tend to be encumbered by documentation and source code maintenance, let's say, particularly when untaken by amateur coders. But somewhere along the line at CRU, what might have started as a grad student's climatology study passed the point of collapse for a project run out of the hip pocket. CRU's unwillingness to grapple with this before their results became Big Science speak very poorly to their competence, or honesty, depending on your views. Either way, not a good testimony for something on which to base global and national policy.

I came to this with skeptical views about catastrophic human-caused warming (CAGW in the jargon). There have been gross perturbations of climate in the past, and yet, here we are. Obviously there are homeostatic processes at work in climate that we understand incompletely at best. Arguments about irretrievable 'tipping points' and runaways leave me cold, and somewhat suspicious of those making them.

On the other hand, I have found the notion of a human impact on warming to some extent to be quite plausible. CO2 has known greenhouse properties, and there's no getting around that we've pumped out a lot of it. That plausibility is now shaken. I've seen a chunk of the research process, and it's a dog's breakfast. The credibility of those pointing out that we haven't accounted for things like solar radiation fluctuations or the effects of urbanization on climate records increases accordingly. I'm reset back to "not proven", which is certainly not going to make me support gross governmental interference in personal liberties and the global economy in the name of global warming.

What I've seen so far of the 'modeling' process is also dismaying. Much of what I've encountered is modeling only in the very grossest sense, consisting of things like cross-correlations and other purely statistical procedures. Such things are notoriously vulnerable to problems like over-fitting, or divergence when dealing with conditions that are uncommon, and therefore under-sampled. (Like, for instance, CO2 driven warming.) It's dismaying to see so little embedded theory and learning process in such a critical domain. I plan on pursuing this somewhat further, in hopes that some of the vaunted redundant sources to CRU do rest on some attempt at theory and a real process of corroboration or refutation.

A Scandal In East Anglia

(With apologies to A. Conan Doyle.)

Breaking, but not in the MSM just yet: A hacker (or leaker) has copied over 60Mbytes (compressed) of e-mail correspondence, documents, data files and programs from the University of East Anglia Climate Research Unit (CRU), the laboratory of Dr. Phil Jones, and one of the primary sources for data and analysis supporting the global warming hypothesis. The zip file has now been removed from the Russian server where it was first posted, but has predictably appeared at numerous other places on the net. Jones has now confirmed that the leaked data appears to be genuine.

E-mails posted in whole or part at WUWT and other sites appear to describe data manipulations that may or may not be misleading. They are, however, quite explicit is discussing how to manipulate the scientific review and publication processes to prevent dissenting views from appearing. With the release out for only a few hours, there's not been enough time for a full forensic or scientific review of the included documents, so we'll have to stayed tuned and cautious. Even if the bulk of the release is genuine, there's always the chance that it has been 'salted'. With literally trillions of dollars at stake with prospective cap-and-tax and other AGW 'mitigations', there is plenty of incentive here for skullduggery.

My only observation so far is that the e-mail release is obviously cherry-picked, given the number of messages compared to the time period involved. Without seeing all, or a representative block of the total messages, we don't know how or why they were selected. The original Russian posting described this as a 'random selection', implying there's more data out there. The implied threat of further release now hangs over the CRU; someone out there may be learning from Breitbart's tactics of dribbling out the damning data.

Update: On Drudge, Fox, Guardian and BBC now. You can amuse/inform yourself by searching or reading the leaked e-mails as long as it lasts. 'FOI' is reputed to be a good place to start.

(More recent posts here and here.)

High Beta Personalities?

An interesting article in The Atlantic describes studies of a so-called 'orchid' gene, believed to indirectly code for a tendency to ADHD as well as anti-social behavior, violence and 'externalizing' (what we'd have called 'acting up' back in the day). The relevant evolutionary question is: If humans as both as groups and individuals generally enhance their survival chances by social cohesion, why haven't these negative alleles been selected out? Noting in passing that studies in which phenotypes are evaluated by necessarily subjective questionnaires make me twitchy, the studies cited in the article put forward a hypothesis with some experimental support: The same gene that creates the downside behavior is highly responsive, in a positive sense, to more supportive environments during childhood. It can apparently lead to either a problem child, or a super-achiever, depending.

Roughly 20% of the population have the 'orchid' allele in question. Leading me to wonder: What's the proportion of carriers in Silicon Valley in general, or in the entrepreneurial or venture capital contingents in particular? What kind of selection might be going on right under our noses?

Against Buzzword Heat-Death

As I've observed before, tech buzzwords that escape into general usage tend to lose their semantics at the hands of wannabes and the media. Commendably, an article at Ars Technica not only notes the phenomenon as it applies to 'cloud', but does something about it by recounting the history and actual meaning behind the term. (And yes, this is another example of the new media kicking the MSM's behind.)

And I should also give further credit-where-due, in noting that Ars has become one of my favorite feeds, due to its willingness to sponsor in-depth pieces, rather than the regurgitated press releases or shallow reviews of the product du jour that prevail at other outlets. Further case in point, this ongoing series on genome sequencing. Well done!

"Smart Money" Indeed

This is both a gross privacy violation and a fascinating result. Seems some Finnish researchers managed to get their hands on that country's records of IQ tests of its soldiers. They then joined that data with several public databases to correlate the intelligence tests with not only income, but stock market participation and portfolio structure. Multiple regression and factors analysis ensued. The punchline:

...the consumption of smarter individuals, rather than the wealthy, may be what drives the pricing kernel. This latter conclusion is not currently featured in formal neoclassical models of asset pricing. However, it is consistent with neoclassical folk wisdom—that economic behavior, divorced from utility optimization, is irrelevant for market efficiency because savvy investors determine asset prices.

How politically incorrect. How fascinating. Read the whole thing. (Hattip: Tyler Cowen.)

Synthetic Biology Primer

If you're trying to keep up with the rapid progress in synthetic biology or genomics generally, here's a nice get-you-up-to-speed primer (PDF) from Scott Mohr. Dating from mid-2007, it only contains one of three sections advertised in the front matter, that on 'Molecular Biology for Novices', but that's 73 pages of reasonably dense material right there.

I grew up on "Blue Book Biology" a 'few years' ago, and amazingly it's still in print, in its ninth edition. That plus the intervening years of perusing the odd Scientific American (more recently, American Scientist) article left me able to follow things like the Church and Venter talk, but lacking confidence that I had the whole story. Mohr's piece is a good level set for those with similar aging or informal backgrounds. What's interesting is that the understanding of what is going on in the cell hasn't changed that much from my high school BSCS days, but the understanding of how it happens has grown immensely more sophisticated and - increasingly - modifiable.

Great Synthetic Biology Briefing

If you've any interest in geonomics, synthetic biology, or engineered evolution, you should check out the nearly six hours of lectures sponsored by Edge.org, presented by Drs. George Church and Craig Venter (mostly the former). Unless you've been following these areas month-by-month, you're out of date. Progress in gene sequencing has accelerated to a 10x improvement per year. Venter and others have figured out how to borrow naturally occurring mechanisms to aid the assembly of much larger artificial genomes. Results are being obtained in days that would formerly take months, or were in the realm of science fiction. If you haven't time for the whole thing, try segment four for the densest presentation of new developments, though you'll need some prior background to keep up. This one of the best up-to-speed briefs you're ever going to find for free. Hats off to the Edge guys.

The Roving Eye: Correct Diagnosis, Cussing Is Good For You, Artifacts Of The Week

DRM moribund. Nearly five years ago, I wrote about digital rights management as a (negative) business diagnostic. It always creates user disvalue, so your only analytical task is to figure out whose business model is in the process of failing. Now it seems that the RIAA finally gets it, too. Now perhaps they can spend some time figuring out how they are going to build something that will actually add value. Not that I was feeling bad about it to begin with, but I regard this as final vindication for all of the so-clever DRM business plans that I've consigned to the circular file or bit bucket over the years.

A wart on Jupiter. How cool is this? An amateur Aussie astronomer noticed a new black spot on Jupiter over the weekend, which may have been caused by a meteor or comet impact (like Shoemaker-Levy in 1994) that escaped other's notice. More discussion here. Observations and analysis still coming in. Update: Confirmed.

Construction workers and mechanics have always known this. But now some scientists think they've proved it: Expletives reduce pain. The English researcher says: ""I would advise people, if they hurt themselves, to swear." Having pulled a back muscle during a volunteer trail build project this weekend, I should be cussing the proverbial blue streak, but unfortunately the positive effects seem to be transient.

Not Photoshop. From the Pink Tentacle blog, Japanese rice paddy art. Apparently by planting rice with variously colored foliage. Some of these should show up on Google Earth. Coords, anyone? And in case you need some recon capability to check it out, here's the Blackbird flying again, sort of. Awesome job!

Ammo Shortages And The Beer Game

If you have anything at all to do with shooting sports, even having a family member or friend who's a devotée, you've heard about the ammo shortages. Pretty much all the popular and some not-so-popular rifle and pistol calibers have become scarce and pricey. Causes ascribed to the ammo drought range from dark government conspiracy to war stockpiling to metals shortages. The truth is likely more prosaic.

This post at the 'Books, Bikes and Boomsticks' blog lays out some of the basics of ammo production and the current supply and demand situation. To simplify (read the post for details), there's a fairly fixed supply capacity, some portion of which can be converted amongst calibers. There's been a spike in retail demand since last fall, probably caused by a combination of fears of civil disorder in the wake of economic collapse and/or moves against ammo supplies by the leftist Obama administration.

This all brought back memories of a little simulation that I've encountered in the past, the Beer Game, which was originated at the MIT Sloan School back in the '60s. Details of the game are given here, but basically it simulates the production, distribution and sale of beer, from manufacturer through distributor, wholesaler, and finally retailer. Players take on one of these roles, and the only communication allowed amongst them is placing their next week's order for fulfillment from up the chain. There's a scoring function that penalizes for holding inventory, but even more for not having stock on hand to fill an order.

After setting up the players and roles and initial inventories, the moderator (or computer, these days) does exactly one more thing: Presents a sharp spike in demand to the player representing the retailer.

Chaos ensues. Every time. It's not giving anything away to say that the result is always gross oscillation in both inventory levels and ordering, which gets worse and worse as demand is aggregated up the supply chain. That dynamic instability is built into the system structure, and all it takes is a sharp transient in the input to kick it off.

I first played this as a graduate student in systems science in the late 70s, and the experience stuck with me. I encountered it again in the late 90s as a participant in an executive workshop, and even though I knew what was about to happen, there was nothing that could be done about it without breaking the information exchange rules of the game. It did make it a lot funnier, however.

The point, of course, is that rule about limited information exchange. After you've been through a session of the Beer Game, you're not going to argue about the value of visibility of inventories and pull-through rates throughout supply and distribution chains. (There's also a meta-message: The best people, with the best practices and data, will still end up SOL if they're working in a system that's structured contrary to their goals.)

Now go back and read the blog post about ammo supply, if you didn't already. Once you get outside of Wal-Mart, and maybe Cabela's, it's a fragmented sector of small retail gun shops, with little to no IT infrastructure and working through at least one level of distribution, along with a smattering of specialist online vendors. Which all got hit with a demand spike back last fall. In the case of the ammo shortages, it's neither incompetence nor conspiracy, it's the nature of the beast. (And maybe some bright entrepreneur can figure out how to make a buck by fixing it.)

<PSA> Boom first, beer later. </PSA>

Note From A Friend in Mexico City

An e-mail from an acquaintance south of the border includes the following:

"Information is confusing and not helping at all. If we extrapolate the (real) cases we know of and the information we are getting it does not make sense. We should have more cases registered."

That fits with what I've been guessing: The only way to reconcile the apparent lethality of the swine flu outbreak in Mexico with the lack of same outside the borders is if there are a lot more unreported minor cases in Mexico. The official Mexican statistics suggest a 1/10 fatality rate, but they seem to be based on patients who present themselves at hospitals or other medical facilities, as opposed to those who spend a miserable few days stuck at home. Otherwise, you've got to believe the virus somehow magically becomes less virulent when it crosses the border.

On the good side, that suggests an actual mortality rate much less than 1/10. On the bad side, it also suggests there's a much larger disease pool in Mexico than reported, with correspondingly lower changes of controlling its spread - if those chances ever existed.

(A couple of good blogs for following the flu outbreak are Effect Measure and Aetiology.)