Jump to content
Sign in to follow this  
jethro

New Research

Recommended Posts

That there were serious flaws not picked up at review, that Mann refused to allow release of the computer code and mathematical techniques actually used on the proxy data, and the whole McIntyre saga of uncovering the truth, is there in the public domain for all to see.

That Mann is singled out is very unfortunate. Scientific computing as a body of work is neglegently poor. A piece by Professor Ince, here, provides a good overview.

And if you can't get your head around that, take a quick look, here, where some scientific software was found to have a precision of one significant digit, where the scientists concerned believed there was six.

Mann is a climatologist. That he might have been embarrassed about his source code is actually to his credit, not detriment; since he is aware of it's inherent shortcomings and his (very) limited ability to author good source code. I think it's probable that M&M use something like MatLab which is a commercial off-the-shelf package of which source-code is not available. Are you clamouring for commercial companies to publish their source-code if it used in the preparation of a scientific paper? DO you think Microsoft will release the source-code to their cashcow, Excel?

The only way out of this is for journals to insist that source-code is submitted along with the raw data, and the paper, and the source code is reviewed by a competent computer scientist and rejected if faulty.

And as for commercial software - it should NOT be used, since the goal of commercial software is to make money. A bunch of universities should get together and fund an open library designed by scientists, and written by skilled programmers that they all share.

Edited by VillagePlank

Share this post


Link to post
Share on other sites

The paper is certainly not a vindication of Wegman's plagiarised political rubbish, nor does it appear to be a successful vindication of McKitrcik or MacIntyre's so far failed attempts to discredit one of the many 'hockey sticks' in existence. The NAS (infinitely superior to the Wegman garbage) critcised Mann's statistics but vindicated his results. Others subsequently reproduced his results several times over with different methodologies, and Mann addressed the NAS criticisms in the 2008 paper. Those who think that McShane and Wyner have demonstrated anything more than the fact that statisticians really ought to consult with climate scientists before attempting to publish on climate science should read carefully the criticisms outlined at Deep Climate:

http://deepclimate.o...and-wyner-2010/

"So there you have it. McShane and Wyner’s background exposition of the scientific history of the “hockey stick†relies excessively on “grey†literature and is replete with errors, some of which appear to be have been introduced through a misreading of secondary sources, without direct consultation of the cited sources. And the authors’ claims concerning the performance of “null†proxies are clearly contradicted by findings in two key studies cited at length, Mann et al 2008 and Ammann and Wahl 2007.These contradictions are not even mentioned, let alone explained, by the authors." [Deep Climate]

I'm not qualified to comment on all the details of the statistical reconstructions, but I can comment that the choice of favoured review literature was awful to the point of being deliberately awful. It would not pass muster for an undergraduate essay.

Lots of serious statistical issues raised by Deep Climate - anyone defending the paper in detail might have their work cut out it would appear.

RealClimate also has a passing nod at it, with hints that there may be more to come:

http://www.realclima...-it-yourselves/

I like Deltoid's take on it, here:

http://scienceblogs....mcshane_and.php

The most interesting part there is Martin Vermeer's early comment - one of the many errors in the paper is a failure to adjust for high latitude skewing of results - hence the slanting shaft of their hockey stick.

But seeing as they purport to discredit palaeoclimate temperature reconstruction altogether, much more serious are the statistical failings pointed out at Deep Climate and RealClimate - it looks very much like this paper will be added to the list of those that fails to dislodge even one of the many hockey sticks on the rack from it's place.

EDIT: Interesting second paper there VP - though it's 13 years old. Any sign of those problems being remedied? But the main issues with software such as those you pointed out are resolved by replication of results by different methodoloiges. In the case of 'Hockey sticks', tree rings, boreholes, lake sediments, glaciers, ice cores, corals and stalagmites all provide independent evidence, as do the reconstructions using different combinations of these sources and different methodologies. Computing issues then don't come into it IMHO.

Edited by sunny starry skies

Share this post


Link to post
Share on other sites

The paper is certainly not a vindication of Wegman's plagiarised political rubbish, nor does it appear to be a successful vindication of McKitrcik or MacIntyre's so far failed attempts to discredit one of the many 'hockey sticks' in existence. The NAS (infinitely superior to the Wegman garbage) critcised Mann's statistics but vindicated his results. Others subsequently reproduced his results several times over with different methodologies, and Mann addressed the NAS criticisms in the 2008 paper. Those who think that McShane and Wyner have demonstrated anything more than the fact that statisticians really ought to consult with climate scientists before attempting to publish on climate science should read carefully the criticisms outlined at Deep Climate:

http://deepclimate.o...and-wyner-2010/

"So there you have it. McShane and Wyner’s background exposition of the scientific history of the “hockey stick†relies excessively on “grey†literature and is replete with errors, some of which appear to be have been introduced through a misreading of secondary sources, without direct consultation of the cited sources. And the authors’ claims concerning the performance of “null†proxies are clearly contradicted by findings in two key studies cited at length, Mann et al 2008 and Ammann and Wahl 2007.These contradictions are not even mentioned, let alone explained, by the authors." [Deep Climate]

I'm not qualified to comment on all the details of the statistical reconstructions, but I can comment that the choice of favoured review literature was awful to the point of being deliberately awful. It would not pass muster for an undergraduate essay.

Lots of serious statistical issues raised by Deep Climate - anyone defending the paper in detail might have their work cut out it would appear.

RealClimate also has a passing nod at it, with hints that there may be more to come:

http://www.realclima...-it-yourselves/

I like Deltoid's take on it, here:

http://scienceblogs....mcshane_and.php

The most interesting part there is Martin Vermeer's early comment - one of the many errors in the paper is a failure to adjust for high latitude skewing of results - hence the slanting shaft of their hockey stick.

But seeing as they purport to discredit palaeoclimate temperature reconstruction altogether, much more serious are the statistical failings pointed out at Deep Climate and RealClimate - it looks very much like this paper will be added to the list of those that fails to dislodge even one of the many hockey sticks on the rack from it's place.

SSS

Oh so now real climate is a site to visit when climate audit is not ....... .

Complete claptrap ... but I expected no less.

You proclaim garbage for one report against another ......yet they both undermine the Mann Papers principal workings .... don't you get it !!!

That the NAS critisised the statistical measures but then came away with that the results were 'probably correct' was based on the available other study data at the time which used the same flawed sets of proxy data that Mann had used. Have a look at the correspondance that was undertaken by McIntyre at the time (its available at climate audit). All reports have heavily critisised the techniques used ...... its taken nearly 10 years for the whole thing to be uncovered and undone, when this should have been obvious much sooner.

The whole point was that the techniques were flawed and actually mined the data for 'hockey stick' shapes weighting those few proxy series that showed what they were looking for above all others using the discredited tree-ring proxies. That was the point then, and that is the point now. This recent paper just shows once again what a complete load of tosh it was.

Here is a reminder of what exactly was going on with the data:

http://www.uoguelph....APEC-hockey.pdf

Y.S

Edited by Yorkshiresnows

Share this post


Link to post
Share on other sites

EDIT: Interesting second paper there VP - though it's 13 years old. Any sign of those problems being remedied? But the main issues with software such as those you pointed out are resolved by replication of results by different methodoloiges. In the case of 'Hockey sticks', tree rings, boreholes, lake sediments, glaciers, ice cores, corals and stalagmites all provide independent evidence, as do the reconstructions using different combinations of these sources and different methodologies. Computing issues then don't come into it IMHO.

Yup - quite agree: it's my opinion.

Having looked at thousands upon thousands of lines of poor and faulty code, written by people who get paid to write it, I can only assume that as in other industry areas, scientific computing has exactly the same problem. An obvious example, of a technique which seems ultimately neglected is Interval Arithmetic. It's understandable why it hasn't be used (computers once weren't that fast!) but the excuse is wearing thin, now.

WRT to the M&W paper, it seems to me that the blogosphere is attributing conclusions that the paper doesn't make. As I said before, my take on it, is that paleoclimate reconstructions is fraught with danger, even when completed by professional statisticians. The paper even concludes that their fit is not good, too (it doesn't follow the steep up curve of the recent instrumental record) Perhaps it's a little bit of confirmation bias on my part ...

I can nevertheless agree with the sentiment, though, even if the method they used to get to it is flawed to some degree or another since, in my view, selection of the proxies, and the test period, seem arbitrary. Of course, I am happy to be shown that it is driven by the domain ...

(EDIT - WRT to interval arithmetic - I am the only person I know who has ever used it in production systems ... take from that what you want :girl_devil:)

Edited by VillagePlank

Share this post


Link to post
Share on other sites

That Mann is singled out is very unfortunate. Scientific computing as a body of work is neglegently poor. A piece by Professor Ince, here, provides a good overview.

Mann is a climatologist. That he might have been embarrassed about his source code is actually to his credit, not detriment; since he is aware of it's inherent shortcomings and his (very) limited ability to author good source code.

The only way out of this is for journals to insist that source-code is submitted along with the raw data, and the paper, and the source code is reviewed by a competent computer scientist and rejected if faulty.

Hi V.P,

The point in all this was that Mann refused to show how he worked the data. Information was asked for repeatedly but he would not release it in regards to the 98/99 papers. You can see some of what went on here:

http://www.uoguelph....APEC-hockey.pdf

Or check out the climate audit site which has a ton of information.

There was no disclosure in either paper to the verification (correlation) statistics that couteracted his choice (his team had used a single measure that showed a correlation where other more obvious measures had not). No discussion of this, or to what this could mean to the outcome was discussed.

If any paper is demonstrating something as important and controversial as what the Mann papers were (basically re-writing climatic history), you'd think that the Journal would endeavour to see how the data was used to generate the results.

Yes, agree with your last point, for papers that are heavily reliant on statistical manipulations of the data, stataticians and computer programmers should be included on the review pannel ....... which is where the NAS, Wegman reports as well as Mcintyre's work all comes in.

Y.S

Edited by Yorkshiresnows

Share this post


Link to post
Share on other sites

There's only one way to sort this out, I reckon, and that's for me to give it a go, myself ...

Share this post


Link to post
Share on other sites

Yes please VP.

Just started - have to rewrite all of it since commercial copyright concerns rear their head ....

public struct Interval{    private double _a;    private double _b;    public double a     {         get { return _a;}        set {_a = value;}    }    public double b    {        get { return _b; }        set { _b = value; }    }    public Interval(double a, double     {        _a = a;        _b = b;    }    public static Interval operator +(Interval a, Interval     // [a,b] + [c,d] = [a + c, b + d]    {        Interval tmp = new Interval(0, 0);        tmp.a = a.a + b.a;        tmp.b = a.b + b.b;        return tmp;    }    public static Interval operator -(Interval a, Interval [img]http://nwstatic.co.uk/forum/public/style_emoticons/<#EMO_DIR#>/cool.gif[/img]    // [a,b] − [c,d] = [a − d, b − c]    {        Interval tmp = new Interval(0, 0);        tmp.a = a.a - b.b;        tmp.b = a.b - b.a;        return tmp;    }    public static Interval operator *(Interval a, Interval [img]http://nwstatic.co.uk/forum/public/style_emoticons/<#EMO_DIR#>/cool.gif[/img]    // [a,b] × [c,d] = [min (ac, ad, bc, bd), max (ac, ad, bc, bd)]    {        Interval tmp = new Interval(0, 0);        tmp.a = Minimum(a.a * b.a, a.a * b.b, a.b * b.a, a.b * b.[img]http://nwstatic.co.uk/forum/public/style_emoticons/<#EMO_DIR#>/cool.gif[/img];        tmp.b = Maximum(a.a * b.a, a.a * b.b, a.b * b.a, a.b * b.[img]http://nwstatic.co.uk/forum/public/style_emoticons/<#EMO_DIR#>/cool.gif[/img];        return tmp;    }    public static Interval operator /(Interval a, Interval [img]http://nwstatic.co.uk/forum/public/style_emoticons/<#EMO_DIR#>/cool.gif[/img]    //[a,b] / [c,d] = [min (a/c, a/d, b/c, b/d), max (a/c, a/d, b/c, b/d)]    {        Interval tmp = new Interval(0, 0);        tmp.a = Minimum(a.a / b.a, a.a / b.b, a.b / b.a, a.b / b.[img]http://nwstatic.co.uk/forum/public/style_emoticons/<#EMO_DIR#>/cool.gif[/img];        tmp.b = Maximum(a.a / b.a, a.a / b.b, a.b / b.a, a.b / b.[img]http://nwstatic.co.uk/forum/public/style_emoticons/<#EMO_DIR#>/cool.gif[/img];        return tmp;    } /// snip}

Of course, this is all cursory :) (no checking for div by zero, for instance, and no contract terms - give me a chance! as well as the assumption that b>a etc etc)

Edited by VillagePlank

Share this post


Link to post
Share on other sites

Just started - have to rewrite all of it since commercial copyright concerns rear their head ....

What is 'it' again? :whistling:

Share this post


Link to post
Share on other sites

What is 'it' again? :whistling:

Reconstruction based on intervals, not native types ( and learning C#, too )

Edited by VillagePlank

Share this post


Link to post
Share on other sites

No offence intended VP, but there's a very good reason why I'm publishing/gardening career based - what you've posted above makes absolutely no sense to me but I'm happy to take your word on whatever it shows.

Share this post


Link to post
Share on other sites

Contains error(s):

Although climate models contain parameters that may be tuned, climate models are not really fit to observations. If that were the case, the models would all reproduce perfectly the observed global trend. We all know this is not the case, and that the spread is quite large.

GCMs are calibrated against the instrumental record- they are 'fit'. Otherwise how would you test their skill before making a certainty about prediction? This doesn't imply that it would perfectly reproduce the instrumental record primarily because there are, possibly, an infinite amount of 'things' available from which to deduce temperature. Since we can't process an infinite amount of parameters, it will *never* fit exactly.

When I say infinite, I, rather colloquially, consider the processing of every electron coming into contact with the Earth's climate more or less infinite. Of course, it isn't, but for all intents and purposes, it might as well be ...

Quick question: should I therefore, and necessarily, discard all of the other comment in that blog because of that glaring error? That's a trick question, in case you haven't guessed :whistling:

No offence intended VP, but there's a very good reason why I'm publishing/gardening career based - what you've posted above makes absolutely no sense to me but I'm happy to take your word on whatever it shows.

I rather suspect that you, rather than I, get the benefit of better company down the local hostelry.

Edited by VillagePlank

Share this post


Link to post
Share on other sites

Contains error(s):

GCMs are calibrated against the instrumental record- they are 'fit'. Otherwise how would you test their skill before making a certainty about prediction? This doesn't imply that it would perfectly reproduce the instrumental record primarily because there are, possibly, an infinite amount of 'things' available from which to deduce temperature. Since we can't process an infinite amount of parameters, it will *never* fit exactly.

When I say infinite, I, rather colloquially, consider the processing of every electron coming into contact with the Earth's climate more or less infinite. Of course, it isn't, but for all intents and purposes, it might as well be ...

Quick question: should I therefore, and necessarily, discard all of the other comment in that blog because of that glaring error? That's a trick question, in case you haven't guessed :whistling:

I rather suspect that you, rather than I, get the benefit of better company down the local hostelry.

I think he is saying something close to what you are: 'are not really'...

No offence intended VP, but there's a very good reason why I'm publishing/gardening career based - what you've posted above makes absolutely no sense to me but I'm happy to take your word on whatever it shows.

I've been much criticised here for taking the word of climate scientists...

Share this post


Link to post
Share on other sites

I rather suspect that you, rather than I, get the benefit of better company down the local hostelry.

You want to bet? There's nothing quicker to make eyes glaze over than talk of aphids....

I've been much criticised here for taking the word of climate scientists...

I'm happy to accept any criticisms Dev, I know my limits and am happy to admit that what VP is talking about, is utter gibberish to me.

Share this post


Link to post
Share on other sites

You want to bet? There's nothing quicker to make eyes glaze over than talk of aphids....

Well, exponentiation by squaring (a way of raising a number to a power - which I am 'doing' now for intervals) seems to help people I know with sleeping disorders.

Edited by VillagePlank

Share this post


Link to post
Share on other sites

Well, exponentiation by squaring (a way of raising a number to a power - which I am 'doing' now for intervals) seems to help people I know with sleeping disorders.

I promise I'll at least try to learn and understand that lot.

Share this post


Link to post
Share on other sites

I promise I'll at least try to learn and understand that lot.

Not really relevant to climate apart from the implementation of the underlying arithmetic, really, but if you do give it a go PM me if you get stuck.

Share this post


Link to post
Share on other sites

Thank you.

Think I still suffer from block out and panic syndrome, brought about by my old Maths Masters fondness for creeping up behind you and whacking your desk with a meter rule whenever he caught you daydreaming; my mind instinctively goes blank when faced with stuff like that.

Share this post


Link to post
Share on other sites

Think I still suffer from block out and panic syndrome, brought about by my old Maths Masters fondness for creeping up behind you and whacking your desk with a meter rule whenever he caught you daydreaming; my mind instinctively goes blank when faced with stuff like that.

(Poor) mathematics teachers have to shoulder one hell of a lot of blame for a lot of peoples fear of mathematics. I see it in my wife, I see it in my kids (although I'm working very hard on that one) I see it in my colleagues, and I see it on here as well as elsewhere.

Do you know what I see? I see passive acceptance that people just 'can't do it' Yet they can, and they don't know that they can. Consider adding three hours to 11pm. What's the time? Clearly 2am. That's modulo arithmetic If you can add the three hours you can do it. You don't need to know about integer rings and such like; it is enough to be able to perform the maths. And everyone I know can do it.

Can you cook a cake? Even if you do it badly, and realise you do it badly (like me - thank goodness for wives) you still know the basics of algorithm analysis since to cook a cake you must consider not only the steps required to do it but the timings and efficiences involved.

Most people know a lot more maths than their maths teacher ever let them realise.

Which makes me upset. Really.

EDIT: The first step, I think, is to realise that it works conceptually, and not visually. I mean when you plant a lawn (and you do a time series analysis against minimum temperatures of greater than 5C - did you realise that's time series analysis over seasonal patterns?) do you know the applied biology and applied chemistry that puts the strength in those first shoots of grass? Perhaps. But it's not required - it is enough to know the correct tools for the job, to complete the job: you don't need to know about autoregession and stable variance in order to figure out the best time to lay the lawn: you're head does it for you. You don't need to know about adenosine monophosphate (AMP) in human muscle activity to realise that you are going to be knackered the day you lay the lawn. But you 'know' it nevertheless.

Edited by VillagePlank

Share this post


Link to post
Share on other sites

That's the first maths lesson that has made sense for a very long time. Thank you.

And I make a mean cake.....http://nwstatic.co.uk/forum/public/style_emoticons/<#EMO_DIR#>/biggrin.gif

Share this post


Link to post
Share on other sites

As folk have pointed out I'm no more than a commentator/observer but I am acutely aware that ,for some, it takes a while for pennies to drop (and many varied approaches) but once the penny has dropped then your own enthusiasm will drive you to understand your topic (and all that involves.......even the maths and science some folk got burned by in their youth) and so I continue to 're-phrase' the science (as I come across it) that interests me in the hope it can spark others into interest.

I used to do this by my music in the hope that I'd touch someone with more nous than me who would then go on to effect change.......still waiting......

Whilst we should aspire to being a fluent in our calculus as V.P. we should also not beat ourselves up (nor allow other to do so) if we haven't found the right way in for us yet. We must hope for patience in our teachers!

Share this post


Link to post
Share on other sites

I agree completely with that last post VP! The hard part is that it's very difficult to convince people that they are able to do something when they have convinced themselves they can't...

It's a pity that the M&W paper fails so badly. It would have seemed to me an opportunity for statisticians to collaborate with relevant climate scientists and do the best possible job. Unless of course McShane and Wyner didn't want to come to the same conclusions as the climatologists... I find it amusing, in a way, that they can do such a poor job of reviewing the literature or understanding the methodology applied by Mann, Wahl and others, yet claim to overthrow a raft of independently verified climate reconstructions. And from what I can see, and from what others better qualified than me can see, they do an awful job of the actual statistics in the process. The only place they then get it published is of course a journal that clearly has no idea about why the paper is bad - which raises questions about how that particular journal is screening papers!

And the only people who of course see it as flawless statistics perfectly applied to the topic are... Watts, MacIntyre and their acolytes, who will be horrendously critical of every piece of climate science , except something that happens to say what they believe, however bad a piece of science it is. :nea: Confirmation bias? Of course. Denial? Written all over their faces. When will they learn that to be truly sceptical you have to be sceptical of every piece of science, including the ones you initially think ought to be right?

Share this post


Link to post
Share on other sites

I agree completely with that last post VP! The hard part is that it's very difficult to convince people that they are able to do something when they have convinced themselves they can't...

It's a pity that the M&W paper fails so badly. It would have seemed to me an opportunity for statisticians to collaborate with relevant climate scientists and do the best possible job. Unless of course McShane and Wyner didn't want to come to the same conclusions as the climatologists... I find it amusing, in a way, that they can do such a poor job of reviewing the literature or understanding the methodology applied by Mann, Wahl and others, yet claim to overthrow a raft of independently verified climate reconstructions. And from what I can see, and from what others better qualified than me can see, they do an awful job of the actual statistics in the process. The only place they then get it published is of course a journal that clearly has no idea about why the paper is bad - which raises questions about how that particular journal is screening papers!

And the only people who of course see it as flawless statistics perfectly applied to the topic are... Watts, MacIntyre and their acolytes, who will be horrendously critical of every piece of climate science , except something that happens to say what they believe, however bad a piece of science it is. :) Confirmation bias? Of course. Denial? Written all over their faces. When will they learn that to be truly sceptical you have to be sceptical of every piece of science, including the ones you initially think ought to be right?

What a ridiculous post. I simply cannot believe anybody could so miss the key points of this so superbly well as you do. Stunning

So every statatician who has looked at and then critisised the Mann papers .... are competely wrong ........ ?!!!!

Absolutely priceless, please please enlighten me. What exactly is incorrect in all the independant summations. What is incorrect in the Wegman and NAS reports, the studies carried out by Mkintyre and now Blakeley and McShane that has got it so completely wrong. They all come up with broadly the same key points ...... must be a 'conspiracy' !!

Love your quote in bold ....... couldn't apply to you could it.

You continuously rubbish one set of work over another ...... because it serves your view point. Sorry this does not wash with me, or should it with anybody else.

If you cannot see that this recent paper finds serious flaws and backs up what others have already indicated in regards to the use of temperature proxies ... even when using the same proxy series as used in Mann 2008 and for which it is known there are the usual suspects in there (Bristlecone pine series and the Tilanjer Lake sediments) and still shows that you could just as easily have come up with an opposite answer ..... then well...... no need to go any further.

Why don't you read the paper. I have and I am no statatician, but you can clearly see the general thrust of what they attempted and concluded. Have a look at section 3.3 titled 'Validation Against Pseudo-proxies and then section 3.5 Variable selection: True proxies versus Pseudo-Proxies page 15 on up to the graph on page 30. This graph shows three back-casts of the data. One showing and replicating what Mann reported, the other two, more or less the opposite .... Only the one using regression of a single principal component provides the 'hockey stick' ...... when they used regression on ten proxy prinicpal componets and a separate two-stage modal ....... no hockey-stick.

Read the conclusions on page 41.

Here's a nice comical look at it all: http://wmbriggs.com/blog/?p=2773

Y.S

Edited by Yorkshiresnows

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Sign in to follow this  

×
×
  • Create New...