Tim Oren's Due Diligence

Akbar and Jeff's Climate Modeling Hut

Know how to run a spreadsheet? Want to learn how Mann, the CRU, et. al. did it?

Iowahawk doffs his jester's cap, picks up his keyboard and displays his mad day-job skilz, to show us how to make a hockey stick chart on our own. And for the advanced students, some 'exercises for the reader' to show how shaky that conclusion may be.

If you care at all about the AGW debate, this is the post to read this weekend. Even if you just read it as a gloss on the jargon, it's well worth the 15 minutes to do so.

Or All The Seas Without Oysters

One of the fun things about about chasing AGW links, as a balance to all the heated rhetoric and wretched code, is wandering into corners of science I would never otherwise encounter. Today's find is a study by Woods Hole Oceanographic Institute investigating the effects of increased CO2 levels on shell-building sea creatures. (Some fraction of CO2 dissolves into water, creating the relatively weak carbonic acid, which nonetheless has the potential to screw up deposition of shells built of calcium, which is normally dissolved by acids.)

The study used gaseous CO2 concentrations well beyond those currently observed to see what might happen if we keep dumping it into the atmosphere. It seems that some of the critters did in fact have trouble growing their shells under extreme conditions, but others liked it just fine and even grew heavier shells. Us crab and lobster munchers are in good shape, you oyster and uni fans, maybe not so much.

In a way, this isn't very surprising. It's well known that most organisms have collections of genes that are normally repressed and inactive, but might become active depending on conditions in the environment or internal to the individual. The critters we eat for seafood dinner have been around in more-or-less similar forms for much longer than Homo saps. Their evolutionary history could very easily include atmospheric and climatic conditions well beyond our current experience, and they may have genes evolved during those periods sitting around ready to become active at need. Might be some interesting research possibilities there when sequencing gets even cheaper.

(Found via Open Markets.)

A Good Layman's Analysis of the ClimateGate Scandal

I was in the process of writing an analysis of the disclosures from a general systems point of view, but someone beat me to it, which is just great. The second half of the post discusses why the disclosed data issues are so damaging to the modeling enterprise, the first half puts the scientific, economic and political issues into context. Very nicely done, and credible from my POV, though colonists may have to stumble over some UK-isms. (What, for instance, is "a hiding to nothing"?) (*) It's posted by the pseudonymous "Pedant General", who means he/she can pull rank on me.

(Thanks to SDA.)

Update: Here's another good piece at the WSJ from an MIT meteorology professor. Generally skeptical, but focuses on both what is generally accepted by both sides, and on the issues with modeling rather than just the data quality problems. A sample:

The main statement publicized after the last IPCC Scientific Assessment two years ago was that it was likely that most of the warming since 1957 (a point of anomalous cold) was due to man. This claim was based on the weak argument that the current models used by the IPCC couldn't reproduce the warming from about 1978 to 1998 without some forcing, and that the only forcing that they could think of was man. Even this argument assumes that these models adequately deal with natural internal variability—that is, such naturally occurring cycles as El Nino, the Pacific Decadal Oscillation, the Atlantic Multidecadal Oscillation, etc.

(Via Maggie's Farm.)

Update 2: And another good list of modeling issues from Coyote. Meanwhile, these guys have the code from a prominent climate model and are starting to examine it and decide on a platform to bring it up. The Army of Davids is beginning to assemble; we're going to end up with transparency one way or another.

*(Belated update: Apparently a "hiding to nothing" is a colloquial way of saying something along the lines of "you're doomed". Derivation unknown, but possibly from "getting their hides thrashed until there's nothing left" or "hiding in a nonexistent spot, and being, therefore, failed fugitives". Thanks to 'Nortius Maximus' for the "English" lesson.)

Head of Global Warming Research Unit Steps Down

Dr. Phil Jones, head of the East Anglia Climate Research Unit, and star of the leaked e-mail and document set from that institution, has stepped down pending an independent review of the unit and its research. So ClimateGate (hate that name) claims its first head on the science side. (Apparently the head of the opposition in Australia fell this week, largely due to a cap-and-trade scheme perhaps exacerbated by the ongoing scandal.)

I have been following the developments, reading relevant blogs as well as skimming a chunk of the e-mails and related text documents. I've also read most of the famous 'HARRY READ ME' file, containing the tribulations of a programmer trying over a period of three years to repeat the derivation of the CRU's temperature database from a poorly documented and sometimes missing set of programs and data. I've made a few comments elsewhere, but not posted here, as those with a lot more familiarity with the climatology data and personalities have entered the conversation, and others a lot more current in the software engineering community, e.g., Eric Raymond, have started tearing apart the released code.

I have drawn a few conclusions, though:

• No one, including those from CRU, has suggested that these files are a fake or have been 'salted', so I can let the latter worry go. Given (for instance) the dearth of 'chatter' or housekeeping type e-mails, it's still a safe bet that they are a careful selection from the available material. Leading to:


• This was an inside job, a leak. No random hacker grabbed these files. It's a dossier assembled over time by someone who knows just where the skeletons have been buried. I suspect that CRU is now 90+% positive who did the deed, but is going to go real easy under the fear (or knowledge) that enough has been held back to make any prosecution into an even worse debacle.


• The main story of the e-mails is the deliberate corruption of the scientific process. From blackballing opponents' articles, to getting non-cooperative editors fired, to specifically declaring their readiness to alter the peer review process to keep skeptics' views out of the IPCC discussions. This isn't rebutting criticism, it's attempting to rule it out a priori. The dishonesty seen here just reeks, and casts a deserved pall over everything published by the CRU and other implicated scientists.


• The public (or media) have an incorrect impression of 'peer review'. I've served on a number of conference program committees, albeit in different domains, and done reviews for journals as well, so I feel qualified to say: 'Peer review' is an initial screening of submissions to see if they are fit to enter the conversation. You're looking for things such as citation of the relevant literature and knowledge of prior art, significance of the result, (non)repetition of previous work, a minimal level of readability, and making sure that the submission is not completely in left field (e.g., in computer science claiming to have solved a known NP-complete problem, without an exhaustive demonstration). No one expects a reviewer to attempt to duplicate the described results. Peer review does not validate the submission. Attempting to confirm, rebut, or refine a paper is another project, worthy of another publication - that's how the conversation proceeds. Attempting to prop up the validity of any model by calling it 'peer reviewed' is dishonest, particularly when the peer review process is corrupted into a stone wall against the unorthodox.


• The main story of HARRY_READ_ME.txt and the related program files and comments is of an incompetent data and code management process. 'Harry' (or perhaps his coder) has my entire sympathy. Back in my grad student days I also implemented models and converted datasets, in FORTRAN even. Projects that aim to get a one-off result determined and published don't tend to be encumbered by documentation and source code maintenance, let's say, particularly when untaken by amateur coders. But somewhere along the line at CRU, what might have started as a grad student's climatology study passed the point of collapse for a project run out of the hip pocket. CRU's unwillingness to grapple with this before their results became Big Science speak very poorly to their competence, or honesty, depending on your views. Either way, not a good testimony for something on which to base global and national policy.

I came to this with skeptical views about catastrophic human-caused warming (CAGW in the jargon). There have been gross perturbations of climate in the past, and yet, here we are. Obviously there are homeostatic processes at work in climate that we understand incompletely at best. Arguments about irretrievable 'tipping points' and runaways leave me cold, and somewhat suspicious of those making them.

On the other hand, I have found the notion of a human impact on warming to some extent to be quite plausible. CO2 has known greenhouse properties, and there's no getting around that we've pumped out a lot of it. That plausibility is now shaken. I've seen a chunk of the research process, and it's a dog's breakfast. The credibility of those pointing out that we haven't accounted for things like solar radiation fluctuations or the effects of urbanization on climate records increases accordingly. I'm reset back to "not proven", which is certainly not going to make me support gross governmental interference in personal liberties and the global economy in the name of global warming.

What I've seen so far of the 'modeling' process is also dismaying. Much of what I've encountered is modeling only in the very grossest sense, consisting of things like cross-correlations and other purely statistical procedures. Such things are notoriously vulnerable to problems like over-fitting, or divergence when dealing with conditions that are uncommon, and therefore under-sampled. (Like, for instance, CO2 driven warming.) It's dismaying to see so little embedded theory and learning process in such a critical domain. I plan on pursuing this somewhat further, in hopes that some of the vaunted redundant sources to CRU do rest on some attempt at theory and a real process of corroboration or refutation.

A Scandal In East Anglia

(With apologies to A. Conan Doyle.)

Breaking, but not in the MSM just yet: A hacker (or leaker) has copied over 60Mbytes (compressed) of e-mail correspondence, documents, data files and programs from the University of East Anglia Climate Research Unit (CRU), the laboratory of Dr. Phil Jones, and one of the primary sources for data and analysis supporting the global warming hypothesis. The zip file has now been removed from the Russian server where it was first posted, but has predictably appeared at numerous other places on the net. Jones has now confirmed that the leaked data appears to be genuine.

E-mails posted in whole or part at WUWT and other sites appear to describe data manipulations that may or may not be misleading. They are, however, quite explicit is discussing how to manipulate the scientific review and publication processes to prevent dissenting views from appearing. With the release out for only a few hours, there's not been enough time for a full forensic or scientific review of the included documents, so we'll have to stayed tuned and cautious. Even if the bulk of the release is genuine, there's always the chance that it has been 'salted'. With literally trillions of dollars at stake with prospective cap-and-tax and other AGW 'mitigations', there is plenty of incentive here for skullduggery.

My only observation so far is that the e-mail release is obviously cherry-picked, given the number of messages compared to the time period involved. Without seeing all, or a representative block of the total messages, we don't know how or why they were selected. The original Russian posting described this as a 'random selection', implying there's more data out there. The implied threat of further release now hangs over the CRU; someone out there may be learning from Breitbart's tactics of dribbling out the damning data.

Update: On Drudge, Fox, Guardian and BBC now. You can amuse/inform yourself by searching or reading the leaked e-mails as long as it lasts. 'FOI' is reputed to be a good place to start.

(More recent posts here and here.)

High Beta Personalities?

An interesting article in The Atlantic describes studies of a so-called 'orchid' gene, believed to indirectly code for a tendency to ADHD as well as anti-social behavior, violence and 'externalizing' (what we'd have called 'acting up' back in the day). The relevant evolutionary question is: If humans as both as groups and individuals generally enhance their survival chances by social cohesion, why haven't these negative alleles been selected out? Noting in passing that studies in which phenotypes are evaluated by necessarily subjective questionnaires make me twitchy, the studies cited in the article put forward a hypothesis with some experimental support: The same gene that creates the downside behavior is highly responsive, in a positive sense, to more supportive environments during childhood. It can apparently lead to either a problem child, or a super-achiever, depending.

Roughly 20% of the population have the 'orchid' allele in question. Leading me to wonder: What's the proportion of carriers in Silicon Valley in general, or in the entrepreneurial or venture capital contingents in particular? What kind of selection might be going on right under our noses?

Against Buzzword Heat-Death

As I've observed before, tech buzzwords that escape into general usage tend to lose their semantics at the hands of wannabes and the media. Commendably, an article at Ars Technica not only notes the phenomenon as it applies to 'cloud', but does something about it by recounting the history and actual meaning behind the term. (And yes, this is another example of the new media kicking the MSM's behind.)

And I should also give further credit-where-due, in noting that Ars has become one of my favorite feeds, due to its willingness to sponsor in-depth pieces, rather than the regurgitated press releases or shallow reviews of the product du jour that prevail at other outlets. Further case in point, this ongoing series on genome sequencing. Well done!

"Smart Money" Indeed

This is both a gross privacy violation and a fascinating result. Seems some Finnish researchers managed to get their hands on that country's records of IQ tests of its soldiers. They then joined that data with several public databases to correlate the intelligence tests with not only income, but stock market participation and portfolio structure. Multiple regression and factors analysis ensued. The punchline:

...the consumption of smarter individuals, rather than the wealthy, may be what drives the pricing kernel. This latter conclusion is not currently featured in formal neoclassical models of asset pricing. However, it is consistent with neoclassical folk wisdom—that economic behavior, divorced from utility optimization, is irrelevant for market efficiency because savvy investors determine asset prices.

How politically incorrect. How fascinating. Read the whole thing. (Hattip: Tyler Cowen.)

Synthetic Biology Primer

If you're trying to keep up with the rapid progress in synthetic biology or genomics generally, here's a nice get-you-up-to-speed primer (PDF) from Scott Mohr. Dating from mid-2007, it only contains one of three sections advertised in the front matter, that on 'Molecular Biology for Novices', but that's 73 pages of reasonably dense material right there.

I grew up on "Blue Book Biology" a 'few years' ago, and amazingly it's still in print, in its ninth edition. That plus the intervening years of perusing the odd Scientific American (more recently, American Scientist) article left me able to follow things like the Church and Venter talk, but lacking confidence that I had the whole story. Mohr's piece is a good level set for those with similar aging or informal backgrounds. What's interesting is that the understanding of what is going on in the cell hasn't changed that much from my high school BSCS days, but the understanding of how it happens has grown immensely more sophisticated and - increasingly - modifiable.

Great Synthetic Biology Briefing

If you've any interest in geonomics, synthetic biology, or engineered evolution, you should check out the nearly six hours of lectures sponsored by Edge.org, presented by Drs. George Church and Craig Venter (mostly the former). Unless you've been following these areas month-by-month, you're out of date. Progress in gene sequencing has accelerated to a 10x improvement per year. Venter and others have figured out how to borrow naturally occurring mechanisms to aid the assembly of much larger artificial genomes. Results are being obtained in days that would formerly take months, or were in the realm of science fiction. If you haven't time for the whole thing, try segment four for the densest presentation of new developments, though you'll need some prior background to keep up. This one of the best up-to-speed briefs you're ever going to find for free. Hats off to the Edge guys.