Model Output Discussion; into 2015

[blizzard81]

ECM op is an absolute nightmare! The azores high is much higher than the 00z and I challenge anyone to show a more hideous chart for coldies than the 192hr ECM which has just come out. It has put me off my Guinness!!

Edited January 1, 2015 by blizzard81

[SMU]

Thanks Steve - superb post.  You have called a spade a spade.

 

Do you think the Strat warming may have an impact going forward?

 

Of Course if we have a Strong warming yes- it will help reverse the westerlies, however just gentle warming will probably not be the trigger we are looking for.

 

ECM 216 still showing the jet digging.

http://www.meteociel.fr/modeles/ecmwf/runs/2015010112/ECH1-216.GIF?01-0

 

S

Edited January 1, 2015 by Polar Maritime
To remove deleted part of quoted post.

[blizzard81]

Of Course if we have a Strong warming yes- it will help reverse the westerlies, however just gentle warming will probably not be the trigger we are looking for.

ECM 216 still showing the jet digging.

http://www.meteociel.fr/modeles/ecmwf/runs/2015010112/ECH1-216.GIF?01-0

S

cmon ecm 240, give us some hope lol!

Edit - Nope! And if anyone says look at that north east canadian high at day 10, I swear I will spit!!

Edited January 1, 2015 by blizzard81

[nick sussex]

It looks like the ECM has once again been barking up the wrong tree and has proved this winter that it's not up to much past T144hrs.

 

Solid agreement on the PV turning into Terminator before weakening slightly towards day ten. A grim picture for coldies with output that reminds us of last winters horror show. Regardless of whether the background signals are better this winter its whats delivered that is the true test.

 

And its been an abject failure on that front, one  snowfall that didn't effect most people and a few decent frosts does not a winter make.

 

We just have to hope that something happens soon to remove the PV from its current location.

[Pennine Ten Foot Drifts]

Well, if we're reduced to desperate straw cluthcing, here's a (small) straw from the GFS 12z:

 

 

Problem is it's at T372, so it's so far into FI it might as well be next winter !!!

[blizzard81]

It looks like the ECM has once again been barking up the wrong tree and has proved this winter that it's not up to much past T144hrs.

 

Solid agreement on the PV turning into Terminator before weakening slightly towards day ten. A grim picture for coldies with output that reminds us of last winters horror show. Regardless of whether the background signals are better this winter its whats delivered that is the true test.

 

And its been an abject failure on that front, one  snowfall that didn't effect most people and a few decent frosts does not a winter make.

 

We just have to hope that something happens soon to remove the PV from its current location.

I totally agree Nick, the ecm ops this season have been frequently woeful past 144hrs. Over amplification, under estimation of the azores high, over estimation of the azores high, take your pick.

[nick sussex]

I totally agree Nick, the ecm ops this season have been frequently woeful past 144hrs. Over amplification, under estimation of the azores high, over estimation of the azores high, take your pick.

I've been following the ECM for many years and can say that its performance over the last two winters has been shocking at day 7 onwards.

 

Of course its under a microscope during the winter but when push comes to shove it pulls out too many colder solutions that are neither supported by the other outputs and never verify. I would rather it just didn't show them to begin with.

 

This used to be the reserve of the GFS which with its four outputs would often go off into fantasy land but really the ECM now wears that crown!

[bluearmy]

I'm a bit confused by the dissing of the ECM op post day 7. It's an op. Why place any store by it post day 7? Do you do that with the gfs or gem? This is why we have ensembles.

If it's in good agreement with the ensembles/spreads at day 7 then by all means use it as a useful extrapolative tool but to hang on it's every run post day 7?

[blizzard81]

I've been following the ECM for many years and can say that its performance over the last two winters has been shocking at day 7 onwards.

 

Of course its under a microscope during the winter but when push comes to shove it pulls out too many colder solutions that are neither supported by the other outputs and never verify. I would rather it just didn't show them to begin with.

 

This used to be the reserve of the GFS which with its four outputs would often go off into fantasy land but really the ECM now wears that crown!

Absolutely, the GFS was the joker in the pack. If it wasn't the op showing the outlandish options, it was a selection of the ensembles. That mantle is now in the hands of the ecm. Even when the op is going mild outlier, the control goes frigid. Lost count how many times the blue control run has gone frigid this season and guess what? None have verified.

[johnholmes]

It looks like the ECM has once again been barking up the wrong tree and has proved this winter that it's not up to much past T144hrs.

 

Solid agreement on the PV turning into Terminator before weakening slightly towards day ten. A grim picture for coldies with output that reminds us of last winters horror show. Regardless of whether the background signals are better this winter its whats delivered that is the true test.

 

And its been an abject failure on that front, one  snowfall that didn't effect most people and a few decent frosts does not a winter make.

 

We just have to hope that something happens soon to remove the PV from its current location.

 

Come on a bit of objectivity, beyond 144h when it way outscores any other model day in day out. Are you saying that GFS and other post 144h are superior? Odd if you are as the GFS update was done with a request for ECMWF help as 'their' data suggests ECMWF (that is NOAA) that ECMWF is better in its accuracy.

 

And as ba suggests using an op at that time distance is bound to lead to errors and much turmoil on here. Why not use ensembles, that is what the professional turn to to support or otherwise what the ops suggests. It is why I use the anomaly charts, yes they are means and it takes a while to get reasonably accurate at intrepreting what they are suggesting but better than the regular ups and downs the ops give folk on here. Much less fun I suppose bu then so many on here get annoyed when they do not show what is wanted.

Still it takes all sorts so best I shut up and let us all follow our preferences I suppose.

I just wish we had reliable data such as the link below to show scienitific evidence rather than conjecture?

http://www.emc.ncep.noaa.gov/gmb/STATS/html/new_acz6.html

I am NOT suggesting ECMWF is right more times than it is wrong at 240h or somewhere between 144h and 240h just curious to know how much less accurate than GFS, GEM etcit is?

Edited January 1, 2015 by johnholmes

[bluearmy]

And the ecm control for de bilt often goes frigid under stagnant surface conditions over there. It's not necessarily an indicator of a cold run for us.

[fergieweather]

It looks like the ECM has once again been barking up the wrong tree and has proved this winter that it's not up to much past T144hrs.

???? But NO topline clusters out to T+360 since at least Xmas have been anything reliably categorised (some transient mid-run ridging aside) other than +ve NAO, mobile westerly regime (with frequent signal for pronounced cyclonicity). EC Monthly has been equally steadfast all winter and signposting accurately thus-far. So I wholly disagree. We've yet to see all today's 12z EPS, obviously. But thus far they've been very consistent an an indicator of broad steer or assessing likely M-R reliability of each DET run.

[nick sussex]

I'm a bit confused by the dissing of the ECM op post day 7. It's an op. Why place any store by it post day 7? Do you do that with the gfs or gem? This is why we have ensembles.

If it's in good agreement with the ensembles/spreads at day 7 then by all means use it as a useful extrapolative tool but to hang on it's every run post day 7?

 

I admire your defence of the ECM but even you would accept that somethings not quite right with it over recent winters. And I don't hang on any operational run post day 7 onwards but if you compare its performance operational wise its been poor over recent winters.

 

The reason I don't slate the GEM is because its a poor model to begin with and its upgrade has done zip to improve its performance. The GFS perhaps benefits from a flat bias because its less likely to show cold scenarios, the fact is the ECM likes to be seen as the best model and overall it gets number one upto day 6.

 

But in terms of its performance over recent winters as recently as the drama pre Christmas it has been lacking. No amount of spin from the ECM public relations unit will alter my view! I've seen every ECM output during the winter months for over 12 years and am just calling it as I see it.

[blizzard81]

And the ecm control for de bilt often goes frigid under stagnant surface conditions over there. It's not necessarily an indicator of a cold run for us.

Has it been frigid (in the Netherlands sense) at all this season in Debilt? Also, all the scenarios of trough diving into Europe, whether it is shown by op, control or ensemble means. These have not verified at all.

[bluearmy]

Perhaps if you have less high expectations of the ecm op fi, you will be less disappointed?

Because of its higher res its prone to making a wrong call early on into a hugely bad call later on.

The same will become apparent on the gfsp (it already is tbh)

Re the ecm fi - there are too many with selective memories aswell. I recall the ECM calling this flatter period quite well and asking nick where the fi amplification was?

I'm struggling to work out what it's done so terribly wrong now anyway? If you are upset this evening then you had very unrealistic expectations to begin with.

Edited January 1, 2015 by bluearmy

[johnholmes]

I have asked this before but how about someone does actual checks on ECMWF outputs on the Op past 144h against GFS and see how it shows. Maybe those that knock ECMWF will be proved correct but until a genuine check is done then we have no proof unless somehas access to this type of check?

[nick sussex]

???? But NO topline clusters out to T+360 since at least Xmas have been anything reliably categorised (some transient mid-run ridging aside) other than +ve NAO, mobile westerly regime (with frequent signal for pronounced cyclonicity). EC Monthly has been equally steadfast all winter and signposting accurately thus-far. So I wholly disagree. We've yet to see all today's 12z EPS, obviously. But thus far they've been very consistent an an indicator of broad steer or assessing likely M-R reliability of each DET run.

I'm not disagreeing with your post, the issue isn't the ensembles generally although they have also been a bit operational led at times. The issue is one of a clear operational bias which is blatantly obvious to most people in here , we're not imagining this!

 

Regardless of what day it occurs at in the modelling its an issue, I don't think we should give the ECM a  get out clause like give the operational a free pass to go off into the sunset past 168hrs because we have the ensembles.

[ANYWEATHER]

The general weather pattern at 500 mbs for the days ahead remains the same from the gfs and ecm ; An over riding low to the North of the Uk with higher pressure squashed to the south of the uk , but even the azores high finds opportunity to find windows of fine weather to effect the Uk from time to time. That's up to ten days time, detail will change and there is always a "slight" area of possibility even for the south to see some wintry precip, but the North of the Uk will see the best conditions and great news again for the Scottish ski industry.

[johnholmes]

Nick this is a serious question, see my above post asking for someone to help show if this 'bias' is correct.

I simply have not the time with all my other commitments but would you take it up please?

[bluearmy]

And what is it does have bias in fi? It's called fi for a reason. The problem is with us as much as it with the model.

Can we move on now? Or are we now going to biff the navgem?

[IDO]

Come on a bit of objectivity, beyond 144h when it way outscores any other model day in day out. Are you saying that GFS and other post 144h are superior? Odd if you are as the GFS update was done with a request for ECMWF help as 'their' data suggests ECMWF (that is NOAA) that ECMWF is better in its accuracy.

 

And as ba suggests using an op at that time distance is bound to lead to errors and much turmoil on here. Why not use ensembles, that is what the professional turn to to support or otherwise what the ops suggests. It is why I use the anomaly charts, yes they are means and it takes a while to get reasonably accurate at intrepreting what they are suggesting but better than the regular ups and downs the ops give folk on here. Much less fun I suppose bu then so many on here get annoyed when they do not show what is wanted.

Still it takes all sorts so best I shut up and let us all follow our preferences I suppose.

I just wish we had reliable data such as the link below to show scienitific evidence rather than conjecture?

http://www.emc.ncep.noaa.gov/gmb/STATS/html/new_acz6.html

I am NOT suggesting ECMWF is right more times than it is wrong at 240h or somewhere between 144h and 240h just curious to know how much less accurate than GFS, GEM etcit is?

 

Hi John.

The ECM op past D7 is very poor when there is shannon entropy. This is more noteworthy because of it's usual clarity and consistency. It shows statistical bias when there is uncertainty and goes amplification crazy.

 

The most recent D10 verifications highlight this:  

 

Looking at the last 4 D10 verifications it was beaten by the GFS each time and on 2 occasions it was whacked. Once by 0.32. These are not just one offs, it is a regular feast of dire verifications when there are changes. It only beats the GFS when the pattern is more rigid because of GFS known bias (statistical) with over doing the Atlantic. Of course the hardest things for models to pick up are changes in the pattern and GFS beats ECM regular.

 

I am not surprised that the GFS P has also been awful during this uncertain period, bearing in mind the guidance from ECM. The GFS P threw up Scandi, Greenland and cold UK highs. The GFS P, as recently as the 12z yesterday was miles from its mean, and we should see both models perform badly in high Shannon entropy if they kill the GFS op.

 

The latest height bias for D7 ECM verification (4 runs) shows how it has performed so badly with respect to over amplification of the pattern:

 

 

I have not used D7+ ECM op charts, when there is uncertainty, for over 18 months, because they are so bad. The ensembles are better but sometimes they will blindly support the op, so caution at times there.

 

Whats that about Lies, damn lies and statistics; the verification stats is a case in point.

[ANYWEATHER]

Calm down folks ,its New Years Day, there is plenty of threads to point to youre opinion 

[johnholmes]

Hi John.

The ECM op past D7 is very poor when there is shannon entropy. This is more noteworthy because of it's usual clarity and consistency. It shows statistical bias when there is uncertainty and goes amplification crazy.

 

The most recent D10 verifications highlight this:  cor_day10_HGT_P500_G2NHX.png

 

Looking at the last 4 D10 verifications it was beaten by the GFS each time and on 2 occasions it was whacked. Once by 0.32. These are not just one offs, it is a regular feast of dire verifications when there are changes. It only beats the GFS when the pattern is more rigid because of GFS known bias (statistical) with over doing the Atlantic. Of course the hardest things for models to pick up are changes in the pattern and GFS beats ECM regular.

 

I am not surprised that the GFS P has also been awful during this uncertain period, bearing in mind the guidance from ECM. The GFS P threw up Scandi, Greenland and cold UK highs. The GFS P, as recently as the 12z yesterday was miles from its mean, and we should see both models perform badly in high Shannon entropy if they kill the GFS op.

 

The latest height bias for D7 ECM verification (4 runs) shows how it has performed so badly with respect to over amplification of the pattern:

 

NOAA_NWS_NCEP_WPC_MODEL_DIAGNOSTIC_-_INTERACTIVE_MODEL_BIAS_PAGE 9.png

 

I have not used D7+ ECM op charts, when there is uncertainty, for over 18 months, because they are so bad. The ensembles are better but sometimes they will blindly support the op, so caution at times there.

 

Whats that about Lies, damn lies and statistics; the verification stats is a case in point.

 

thank you very much for that IDO that is exactly the sort of scientific check I was looking for, will note that address-thanks again.

sod it, the link is not a link to the actual site, can you drop it in here for me please although I will go and see if I can find it-thanks?

Edited January 1, 2015 by johnholmes

[fergieweather]

I'm not disagreeing with your post, the issue isn't the ensembles generally although they have also been a bit operational led at times. The issue is one of a clear operational bias which is blatantly obvious to most people in here , we're not imagining this!

 

Regardless of what day it occurs at in the modelling its an issue, I don't think we should give the ECM a  get out clause like give the operational a free pass to go off into the sunset past 168hrs because we have the ensembles.

But why would we be using a bespoke medium range tool out at medium range, by solely employing deterministic run? It's always used with close scrutiny of all 51 stamps in HRES to T+168 and the cluster scenarios beyond. I'm sorry, but email any of the key national forecast centres from USA to Australia and I assure you they'll all feedback that ECMWF suites are their principal and most valued tool for the task. Identifying and dismissing likely flights of fancy in all models into 6-15d period is core work each day at medium range desks all across the globe. Their staff don't go into breakdowns over this... it's inevitable that we go through ebb-flow cycles of reliability at that range and that's just accepted. It's precisely why products like DECIDER exist!

[nick sussex]

Nick this is a serious question, see my above post asking for someone to help show if this 'bias' is correct.

I simply have not the time with all my other commitments but would you take it up please?

IDO I think has already shown the bias in some charts he posted, the problem John is if you take the stats as a whole over the year you might get similar figures for both the GFS and ECM.

 

The issue is more one of " fit for purpose" let me explain. Generally we view the outputs through a microscopic lens in here particularly in winter. I make no secret of my liking for extremes and I think many in here are the same, the time when we most want reliability is in the winter.

 

I want the ECM to be as reliable as possible even if that's dishing out mild runs, the issue for the ECM is an overamplification problem past day 6.

 

We don't need a statistical review because many of the people in here have noted the same and to be honest I don't think I'd be best to do this anyway. My past experience educationally and career wise encompasses the more " qualitative" side of things, you know the " how did that make you feel" side of things.

 

I'd be happy though to conduct a study into the emotional impact of mild weather on snow lovers! :smiliz19: