Daubert and the Admissibility of Climate Models as Evidence in a Court of Law

February 29th, 2012 by Roy W. Spencer, Ph. D.

I get involved in so many things, sometimes I forget to let people know when something significant happens.

A few months ago, a paper I co-authored with attorney Brooks Harlow appeared in Energy Law Journal entitled: An Inconvenient Burden of Proof? CO2 Nuisance Plaintiffs Will Face Challenges in Meeting the Daubert Standard

I had always wondered how climate models, which are the ultimate source of “evidence” of anthropogenic global warming, could meet the rather stringent guidelines that judges use when deciding whether expert testimony should be allowed in court. Then, one day I received a phone call from an attorney (Brooks Harlow) who had been wondering the same thing, and we agreed to collaborate on a paper for submission to a law journal.

According to Wikipedia, “The Daubert standard is a rule of evidence regarding the admissibility of expert witnesses’ testimony during United States federal legal proceedings.

While the Supreme Court’s decision in Connecticut v. AEP effectively shut the federal court doors to federal common law nuisance claims, about half of the states rely on Daubert, and CO2 litigation can be expected in the coming years in various states.

There are 5 relevant standards in the Daubert test of admissibility, but which the judge does not necessarily have to follow rigorously. Again from Wikipedia:

  • 1. Empirical testing: the theory or technique must be falsifiable, refutable, and testable.
  • 2. Subjected to peer review and publication.
  • 3. Known or potential error rate.
  • 4. The existence and maintenance of standards and controls concerning its operation.
  • 5. Degree to which the theory and technique is generally accepted by a relevant scientific community.
  • In my opinion, two of the standards above (#1 and #3) are not met by climate models. In standard #1, climate models are not testable in any rigorous sense in what they are being used for: prediction of future climate change. We will not know whether they are right or wrong until the future happens.

    And even then, we can’t be sure that warming which ends up being observed in nature occurred for the same reason the models predicted it. Just because they create average weather that is somewhat realistic does not mean they can tell us how average weather might change in the future.

    And, contrary to what you might have heard, there is no obvious “fingerprint” of human warming. Natural warming from a slight change in cloud cover would look the same.

    Also, the oft-mentioned missing “hot spot” of warming in the upper troposphere would be the theoretically expected result of ANY warming; its presence (or absence) does not demonstrate causation as to the source of the warming.

    After all, the twenty-something climate models are all over the map in their predictions, and if one or two happen to be correct, what about all the other climate models which are also based upon “physical principles”, but which were wrong in their predictions? At some point, we have to admit that given enough different model predictions, one or two models are bound to be close to correct just by chance.

    Standard #3, which I also believe is not met, is related to standard #1 just discussed: the methodology should have a known error rate. In other words, how likely is a climate model to be correct based upon its success in previous predictions?

    “Predictions” of the past (e.g. what happened in the 20th Century) don’t really count because those are more exercises in curve fitting than prediction. What if the modelers did not know what the 20th Century temperature variations looked like, and were asked to use models to “hindcast” what happened. Would they be able to? Not likely.

    And since future warming (if it occurs) is a one-of-a-kind event, we won’t know for many years whether any of the climate models are correct even once, let alone for a statistical sample of independent predictions.

    If the climate system is always changing anyway, as some of us believe, you have a 50% chance of being right in a prediction of a warming trend by just flipping a coin.

    While some might also argue whether one or more of the other Daubert standards are not met, I thought I would address the two most obvious ones.

    I yearn for the day when the “scientific consensus” defenders are cross-examined in court. No longer will they be allowed to get away with demanding they be believed just because they are experts. That kind of attitude does not get you very far in court.


    32 Responses to “Daubert and the Admissibility of Climate Models as Evidence in a Court of Law”

    Toggle Trackbacks

    1. Dan says:

      For this very reason climate models will not be set before a court of law.

      Instead policy is placed into rules which are not subject to direct congressional oversight.

      An example is the EPA expanding the definition of a hazardous substance to include green house gases.

    2. You are most correct here. In addition to not being able meet anything close to the standards for evidence in a law court the models and the modeling process do not meet the standards of science as understood from philosophers such as Popper.

    3. Olavi says:

      Can we sue Mann, Jones, Ternberth, Rahmstorff & co after their all prediktions have failed, because that’s going to happen.

      I predict at least 0.5c cooling in next decade.

    4. John says:

      If climate models/predictions become admissible in court and are considered equivalent with facts, and observations the country and it’s legal system will have departed from rational analysis in favor of delusional speculation. Psychics will argue that their predictions should be considered evidence in court as well, with the same rational. Hopefully our court system doesn’t devolve into insanity.

    5. Pasgetty says:

      I am really glad you took time to do this Roy.

    6. Svend Ferdinandsen says:

      Dear Roy.
      The climate science does not themself believe in the models for prediction of temperature or even climate.
      First of all, they use an average over all the Earth, but even then the models deviate too much, so they use ensembles of various kinds.
      By doing that they could just use simple forcings. It’s hiding the simple basis for the prediction, to use very complex models in that way.

      In december i found this question to Gavin Schmidt and his answer:

      If a single simulation is not a good predictor of reality how can the average of many simulations, each of which is a poor predictor of reality, be a better predictor, or indeed claim to have any residual of reality?” Gavin Schmidt replied:

      “Any single realisation can be thought of as being made up of two components – a forced signal and a random realisation of the internal variability (‘noise’). By definition the random component will uncorrelated across different realisations and when you average together many examples you get the forced component (i.e. the ensemble mean).”

      The quote also appears on the NCAR GIS Climate Change Scenarios FAQ webpage:

      “Climate models are an imperfect representation of the earth’s climate system and climate modelers employ a technique called ensembling to capture the range of possible climate states. A climate model run ensemble consists of two or more climate model runs made with the exact same climate model, using the exact same boundary forcings, where the only difference between the runs is the initial conditions. An individual simulation within a climate model run ensemble is referred to as an ensemble member. The different initial conditions result in different simulations for each of the ensemble members due to the nonlinearity of the climate model system. Essentially, the earth’s climate can be considered to be a special ensemble that consists of only one member. Averaging over a multi-member ensemble of model climate runs gives a measure of the average model response to the forcings imposed on the model. Unless you are interested in a particular ensemble member where the initial conditions make a difference in your work, averaging of several ensemble members will give you best representation of a scenario.”

      I would like if you could take this inconsistency up in a blog.

      • Edmh says:

        Assessing the Actual Temperature effect of Man-made CO2 or can Gavin Schmidt do sums?

        In spite of the IPCC assertions that essentially all the warming since 1850 is wholly due to Man-made CO2 emissions, there is a wider range of published and peer-reviewed opinion that differs on the actual level of the impact of Man-made additions of CO2 to the atmosphere.

        One well-accepted view is provided by CDIAC, the Carbon Dioxide Information Analysis Centre of the U.S. Department of Energy (DOE).

        The CDIAC figures can be transposed into parts of the 33°C total Greenhouse Effect are as:

        Water Vapour ~95% of effect ~31.35°C

        Greenhouse Gases ~5% ~1.65°C

        Carbon Dioxide at 390 ppmv ~75% non H2O GHGs ~1.24°C

        Natural CO2 ~86% (50% CO2 >1850 Man-made) ~1.07°C

        Man-made CO2
        Current 2010 Worldwide ~14% of 390 ppmv ~0.17°C

        Other Greenhouse gases ~25% non H2O GHGs ~0.41°C

        The CDIAC figure of ~0.17°C for Man-made influence since 1850 is roughly a quarter of the measured temperature rise of ~0.7°C since then: it accords well with the notion that the temperature increase can be allocated roughly 50%/50% solar influence and 50%/50% natural / Man-made CO2 emissions.

        A value in the region of 0.17°C for the effect of worldwide emissions to date has been accepted as reasonable in correspondence with Professor David MacKay, the chief scientific advisor of the UK Department of Energy and Climate Change, DECC.

        On the other hand, Gavin Schmidt of NASA GISS , one of the topmost scientists involved in the Catastrophic Anthropogenic Global Warming cause, in a recent paper

        http://pubs.giss.nasa.gov/docs/2010/2010_Schmidt_etal_1.pdf

        clearly asserts that:
        • only 75% of the Greenhouse effect is attributable to water vapour and clouds
        • 100% of the increase in CO2 emissions since 1850 (110 ppmv) is Man-made
        • Following these numbers through and accounting for the effect of other Greenhouse gases, results in a Man-made temperature rise between 1850 and 2010 of ~2.21 °C.

        As the reported temperature increase since 1850 is only ~0.7°C in total, his result is in error.

        So this premier scientist supporting the concept of alarming greenhouse warming markedly exaggerates the Man-made effect by more than threefold than the actual rise and is 15 fold the CDIAC Man-made temperature estimate.

        He obviously did not carry out the trivial sums that would have shown that his figures grossly exaggerate Man-made influence on temperature.

    7. Greg says:

      Vancouver Canada and California have already passed carbon tax laws that will result in higher prices for just about everything for their citizens. The California law (AB-32) was challenged by another ruling (Prop.23) that tried to prevent its passage until unemployment levels dropped from 12.3% to 5.5% in a vain attempt to prevent further damaging the economy. Prop. 23 went down in defeat, so Cap & Trade is now legal in California.

      I just read that Governor Brown is expected to dip into the Cap & Trade pot of gold to the tune of $1B in an effort to reduce California’s massive debt. This is expected to be challenged in court. Your paper “An Inconvenient Burden of Proof? CO2 Nuisance Plaintiffs Will Face Challenges in Meeting the Daubert Standard” may well be used to stop this seemingly illegal tax increase.

    8. John says:

      Roy,

      According to your post California is subject to the Frye standard. The attachment to your post states:

      For example, the California Supreme Court held that “the Frye test does not demand ‘absolute unanimity of views in the scientific community. . . . Rather, the test is met if use of the technique is supported by a clear majority of the members of that community.’”

      It seems California and a great deal of the country is devolving into complete speculative irrationality and a witch doctor’s view of reality. Apparently, if you can find a majority of scientists and or witch doctor’s with degrees to support an opinion, the opinion becomes admissible as fact. It’s not that the standard should be that an “absolute unanimity of views in the scientific community” become the basis of evidence. Rather, empirical data/observation (i.e. facts) should be required before even thinking about assigning responsibility in a case. CO2 nuisance cases apparently require none of that. While it is a fact that we all emit CO2, CO2 is considered a “green house gas” and satellites purport to indicate some warming since 1979 their is no empirical evidence that any one person’s (company’s or nation’s) CO2 emission contributed for or against any observed weather/climate event. All conclusions to the contrary are the rational equivalent of the witch doctor spinning the proverbial finger bone of blame.

      The CO2 finger bone of blame however is really under the control of the witch doctor’s/puppet masters in the AGW community (and sympathizers in the government) who as a recent Guardian article regarding Peter Gleick suggested are sometimes willing to fabricate falsehoods, and apparently others to seize the unearned property of others to obtain their goals. Their is no scientific, constitutionally legal or ethical basis for any of this activity. The nations legal foundation is under suicide watch. We will see where the finger bone of blame spins next.

      • John says:

        This is just a note regarding my previous post. Please note that I am not personally “denying” the green house effect in this post or even satellite evidence that recent warming since 1998 is consistent with the green house effect. Nor am I denying that human beings are responsible for the emission of green house gasses and/or that CO2 and such gasses may have played a role in recent warming. However, even admitting all of the above does not prove causality in regards to recent warming and/or any observed climate or weather event. As Roy pointed out, there are alternative explanations for observable climate/weather events. Therefore, any assignment of legal culpability upon an individual, company or nation would by nature involve a great deal of speculation and lack an empirical basis.

    9. G. E. Pease says:

      Would the following rationalizations stand up in a California or Canadian court of law?

      “The data doesn’t matter. We’re not basing our recommendations on the data. We’re basing them on the climate models.” -Prof. Chris Folland, Hadley Centre for Climate Prediction and Research
      “The models are convenient fictions that provide something very useful.” -Dr David Frame, Climate modeler, Oxford University
      “It doesn’t matter what is true, it only matters what people believe is true.” -Paul Watson, Co-founder of Greenpeace
      “Unless we announce disasters no one will listen.” -Sir John Houghton, First chairman of the IPCC
      “No matter if the science of global warming is all phony … climate change provides the greatest opportunity to bring about justice and equality in the world.” -Christine Stewart, former Canadian Minister of the Environment
      Read more in Singer’s piece: http://www.americanthinker.com/2012/02/climate_deniers_are_giving_us_skeptics_a_bad_name.html#ixzz1nnMsrA1y

    10. Christopher Game says:

      Dr Spencer, I see you have cited the Wikipedia. I plead guilty to making some edits to the Wikipedia. I am of the opinion that the Wikipedia is sometimes right and sometimes wrong. There is no way of knowing which except by your own checks of the statements from other sources.

    11. L Nettles says:

      The Leona Marshall Libby model seems to be holding up well.

    12. Christopher Game says:

      In the present blog thread it has not been explicitly mentioned, that I have noticed, that Drs Spencer and Braswell have evidence from satellite data that the observations of radiative energy balance of the earth are distinctly at variance from the estimates made of that balance by the AOGCM climate models. My limited understanding is that this is because the climate models underestimate the emission of terrestrial thermal radiation and/or underestimate the reflection of sunlight by clouds or other factors in response to land-sea surface temperature change. I do not know whether this finding by Drs Spencer and Braswell is consistent with the earthshine observations, but I think it probably is. This seems to me to be of cardinal importance and should be borne in mind.

      Reading the list of five items that Dr Spencer cites from the Wikipedia, I do not see mention of the question of whether the models are actually in fact in agreement with reliable and decisive empirical observations. I see that they must be testable, but I do not see that they must actually pass the tests. Perhaps there is some further legal input needed here?

      I do not agree with what I read Dr Spencer as saying, that it is logically necessary that the models must be tested against the presently unrealized future; for present purposes that is logically impossible.

      It is however logically necessary that their predictions should have been tested against a logically sufficient or adequate range of outcomes which were future when the predictions were made; this is a fundamentally necessary and primarily important logical requirement for any decisive empirical test of a physical theory.

      It seems to me unlikely or perhaps impossible that the models can pass the latter test because we do not have enough knowledge of the facts and laws of nature to enable us to reliably determine a logically sufficient or adequate range of outcomes. For example, we do not know well enough why there is a correlation between the activity of the sun and the changes of the climate; for an adequate test of the models, the mechanisms of that correlation would need to be reliably known, in order to determine a logically sufficient or adequate range of outcomes. The same applies to the fact that we do not adequately understand the generation and perishing of clouds.

    13. KuhnKat says:

      Dr. Spencer,

      you say

      “Also, the oft-mentioned missing “hot spot” of warming in the upper troposphere would be the theoretically expected result of ANY warming; its presence (or absence) does not demonstrate causation as to the source of the warming. ”

      I believe the issue is worse than that. You claim, and so do many warmers, that ANY source of warming will cause a Hot Spot. I understand the logic and how it is based in the THEORY. Can you show us the “Hot Spot” in the data for about 1975 throughabout 2000??

      If this theory is based in sold physics, AND if there was actual warming, should’t it be possible to see this “Hot Spot” in the data?? if not we are left with a few possibilities. The earth didn’t have a GAT warming, the theory is falsified, or both.

    14. Bil Danielson says:

      The Mises organization has an interesting analysis – linked here

    15. Dr. Strangelove says:

      Dr. Spencer,

      I do not believe AGW on the basis of climate models. I think they are unreliable in predicting the past and future. But do you think atmospheric CO2 and sea level data in past 20,000 years are unreliable? What’s the margin of error?

    16. Doug Cotton says:

      I would say there is a good chance of a group action in Australia by companies affected financially by the carbon tax which will apply from July.

      Any takers are welcome to contact myself (in Sydney) as I may get involved as a witness. [email protected]

    17. cementafriend says:

      In Australia a number of prominent (so-called) scientists made predictions in 2006 & 2007 about climate in parts of Australia particularly about temperature increase and drought. These predictions were a basis for politicians to waste huge sums of money building desalination plants (now not completed or shut down and rusting away) to augment water supply to cities. The predictions based on climate models have been proved wrong by floods around the country. Naturally, occurring El Nino cycles (which have not been built into models) are being blamed for the failure of the predictions.
      In contrast to what Gavin Schmidt (who has little understanding of the engineering subject -heat&mass transfer) says, a pile of rubbish is still a pile of rubbish no matter how many items in it maybe useful to someone for some purpose. “Garbage in, Garbage out” is the common term.
      Beside complexities such as cloud formation and the effect of clouds on heat transfer (including evaporation & condensation) the climate scientists ignore the vast amount of research perform by the likes of Prof Hoyt Hottel who certainly understood more about radiative heat transfer than any physicist, engineer or scientist during his lifetime and more than all the so-called present climate scientists (including Gavin Schmidt & the rest of the team) put together.

    18. Lewis Guignard says:

      Gentlemen,

      Here you talk around the court system and propose that ‘expert’ testimony be of some ‘testable’ type.

      I contend, and it is easily shown, that the court system is not in the least concerned with truth or fairness, but in politics. Certainly tort lawyers are, in general, only concerned with how much money might be made from some particular suit, and truth is something which only gets in their way – it is not what they are seeking.

      So the question is how will large companies (where the money is) – power companies, – then the perhaps the industrials – be attacked in order to take the money they make – through years of hard work and effort – to be given to the plaintiffs lawyers? That is the only question.

      What Dr. Spencer alludes to – truth and accuracy – is a side show to be worked around.

    19. Christopher Game says:

      Response to the post of Doug Cotton of March 2, 2012 at 7:45 PM.

      Since he may be a witness, I trust to help against the AGW gang, may I point Doug Cotton to my post at http://www.drroyspencer.com/2012/02/more-musings-from-the-greenhouse/#comment-35696 in response to his request for a pointer to empirical information about the absorptivity of the land-sea body for thermal radiation from the atmosphere.

    20. Jim says:

      As Jerry Pournelle has noted several times on his blog (www.jerrypournelle.com) there are two possible tests that can be used to falsify climate models.

      a) Using best-estimated data from 100 Years Before Present and forecasting to today; or,

      b) Using current conditions and running the models with inputs and forcings reversed and seeing if they can reproduce past conditions. (Which should work without bias if all thermodynamic analyses are cyclic.)

      While Jerry doesn’t cite his sources, he says that the climate models in use consistently fail both tests.

    21. Dr. Strangelove says:

      AGW will not pass the Daubert test because the evidence presented to support it is results of climate models. IMO this evidence is the weakest.

      The stronger evidence is sea level data. Instead of Daubert test, I prefer Bayesian inference which is used in science, engineering, medicine and law. Bayesian inference can show that AGW is a more compelling hypothesis than natural variability of climate.

    22. Concept excellent, I support.

    23. I really like your composing fashion, do keep on creating!

      I will be back again! Commented by – Jamila Gaestel

    24. Norm Kalmanovitch says:

      Climate models operate based on flux and not actual energy so they are incapable of producing output in the form of temperature. To do this a climate sensitivity factor is introduced to convert watts/m^2 into degrees C of warming.
      To date there has been no verification of this climate sensitivity factor to a scientific standard and since all forcing from climate models is positive for increasing CO2 and the climate sensitivity factor is also positive in the form of so many degrees C for each watt/m^2 of forcing; the increase in CO2 of 2ppmv/year for the past decade should have produced an increase in global temperature over the past decade.
      The average of UAH MSU and RSS MSU posted on the friends of science website (www.friendsofscience.org) shows a cooling trend of 0.05°C/decade since 2002 proving conclusively that the climate models are false because either the forcing from the models or the climate sensitivity factor should be of the opposite sign.
      At the other end of the climate models there is no scientific justification for the CO2 forcing parameter and without this essentially fabricated parameter there is no other input for the effect from CO2 into the climate models.
      A third point is that the climate models predict a decrease in OLR from the 57.1% increase in CO2 emissions over the past three decades but the satellite measurements of OLR do not even show a detectable decrease let alone the 0.782W/m^2 decrease in OLR predicted for the models according to the CO2 forcing parameter.
      Since the models are completely wrong on all counts and neither the CO2 forcing parameter nor the Climate Sensitivity factor have any scientific justification; climate models do not provide any valid evidence for AGW but only provide evidence for scientific fraud which is what is driving the climate change issue

    25. D Cotton says:

      Within about 24 hours there will be a new paper Radiated Energy and the Second Law of Thermodynamics at http://principia-scientific.org/ explaining why the models are wrong.

      In over 6,000 words it covers a wide range of reasons why carbon dioxide can have no warming effect and only a slight cooling effect.

      This is only the sixth paper to be accepted by this organisation which is dedicated to the truth in science.

    26. economy says:

      I was 12 years old when, i started going to lions.My mom would drop me off 10.30am, and pick up at 10.30pm.Those times, i wish icould have back.The best ,was C.J Hart,doing his sked on his scooter.It will allways be my memories of Lions ,that will stay with me forever. Thanks to all of you,for making it so great.

    27. Doug Cotton says:

      The paper I referred to above is now published on about six sites.

      Some may wish to read the paper and also my response to Jeff Condon’s weak attempt to rebut the paper. You’ll find a screen capture of my response in a link near the top of my Home page at http://climate-change-theory.com

      It seems he found it so embarrassing that all he could do was delete it from his site and ban me from commenting in answer to his garbage.