Model Output Discussion 1st January 2014-06z onwards.

[fergieweather]

Do people actually properly read Ians forecasts? the dates he mentioned are the 20th January onwards we're at the 6th January, if I'm reading this right the UKMO don't discount a drier spell before then and and possibilities of it turning colder but then suspect the jet will fire up and flatten any high pressure. Perhaps I've got the wrong end of the stick here but that's my take on things!

They lean towards broadly unsettled into 10-15d period but with settled phases especially to S/SE; increased incidence frost and fog (latter exacerbated due to saturated catchments). The only clear trend, they say, is for no sign of mildness.

[draztik]

They lean towards broadly unsettled into 10-15d period but with settled phases especially to S/SE; increased incidence frost and fog (latter exacerbated due to saturated catchments). The only clear trend, they say, is for no sign of mildness.

 

Which is broadly inline with the current GEFS mean.

[SnowBallz]

Another excellent post Draztik and I must admit to sharing your confusion over the constant critisism of GFS. Of course it get things wrong at times and no doubt it is often more progressive and extreme re zonal pattern in particular. However as far as I'm concerned (not really interested in the so called verification stats) it has completely out shone ECM since early Autumn and it did so frequently across Summer too, especially post T+144hrs.  

 

I guess the only plausible explanation for the amount of abuse it receives is the fact it often shows what most don't want to see in Winter, but if it were a living thing I'm sure it would have successfully sued for deformation of character long ago....and won handsomely

 

Hi Shedhead,

 

What I would say, as the strongest possible response to your post, is that the NOAA themselves are not happy with the performance of neither the GFS or its ensemble compatriot, GEFS.  To that end, it is undergoing a major overhaul in April, in order to correct a number of errors which have been identified in the physical model.  In addition, it is also worth stating that all mathematical models generate bias, and the GFS is no different in that regard; it will - for example - display a predisposition to explosive cyclogenesis, and this is mainly due to an issue around resolution depth, coupled with a data issue regarding SST.  If you read the narrative that Nick often references from NOAA discussion, you will - quite often - see commentary to this effect.

 

With respect to the verification statistics, I would wholly disagree with your approach.  The statistical basis for verification is an accepted industry standard, and it is through that which respective model performance is objectively assessed.  There is no agenda at play; the statistical results are derived from the output of the models, so - if they don't verify - then it is the model that is at fault, not the statistics.  I think an important point to make here is that - in a scientific environment, there is no room or for that matter value, in assessing using subjective interpretation; there needs to be benchmarks and criteria, and that is what a standard reanalysis model delivers.

 

The ECMWF - in tandem with UKMO - are world-leading pioneers in meteorology, and this isn't lost on NOAA or other worldwide counterparts.  There is actually a fairly broad consensus right across the board, and I think you'd be surprised that there really isn't this competitive nature - it's a lot more scientific and progressive than that.  For example, I know that there are (at least) two ECM colleagues over in Reading who are currently advising NOAA on how to best transition from the current iteration of GFS over to the new one in April.  That's so that the NOAA have a seamless implementation of the operational model which, in time, the ECM can also make use of and naturally compare itself against.

 

My personal - and I stress personal - view of the GFS, is that it contains too many consistent and permanent flaws, in order for me to attribute it too much credit.  That is not to say that I completely discount it - that would be foolish - but it's about weighing it up, relative to its peers; there will be times when the GFS performs better, and there will be other times when it performs quite obviously poorer.  But that's mathematical modelling.  There is one model, for instance, that is nigh-on useless through the first third of winter, and that is because its physics engine relies so heavily on stratospheric data.  Thereafter, its performance appears to be quite exceptional.  It is not within my gift to say which one, but I'm sure Ian knows which one I'm referring to.  Equally so, it is evident that some models perform better through different seasons, and this can be understood through how every model will have different elements of teleconnective reasoning.

 

So, it isn't really the case that there is one model that is in any way to be considered a panacea - that really isn't the approach that is used by the Met Office (or the NOAA, for that matter)  The approach is to objectively assess all outputs, and to then apply probabilistic reasoning against it, in order to draw a forecast.  That approach abrogates against the sort of bias which is often found to be at the source of most known human errors.

 

I hope that helps, or at least serves to perhaps dispel a myth or two?

 

SB

[Biggin]

...

Edited January 6, 2014 by Osbourne One-Nil
then report those posts.

[nick sussex]

Just to add my two pence worth to the GFS v ECM debate, the proof really is in how much they're used by forecasters.

 

I have followed every single model discussion from NCEP for the last few years and the ECM and especially its ensemble mean is by far the most used by USA forecasters, and you have to bear in mind that its not great for them to have to rely on the Euro model because they are essentially saying they don't think much of their own model.

 

It would be a bit like the UKMO using the GFS above their own and the ECM model!

 

In terms of recent performance I think you have to separate out the operationals upto T144hrs and then after that, the ECM's main problem in recent winters has been over amplifying the upstream pattern but at T168hrs onwards this is a completely different scenario from the GFS which has many times been totally wrong within T144hrs.

 

Often I'll read in the discussions that the ECM operational is used to day 5/6 then they switch to the ensemble mean, even the makers of the GFS accept it has a progressive bias within the supposed normally reliable timeframe.

 

In the USA they often talk of impact events and these are where the models really earn their corn, because this type of situation impacts business etc but more importantly peoples lives.

 

This is where you really want your model to be good and on the money, in recent episodes the ECM has wiped the floor with the GFS, Im' not saying the GFS isn't a good model, its  better than many global models and deserves its place in the big 3.

 

However the current PV dropping into the eastern USA was not modelled by the GFS initially because it didn't have the trigger for that, the inland runner low which helped to pull that brutal cold south and so for American forecasters it failed on this impact event.

 

Regardless of what we think of the ECM over Europe all I can say in terms of upstream its far more respected than the GFS, apparently the GFS is undergoing some upgrade, so perhaps it will pick up its performance over the USA.

[shedhead]

Hi Shedhead,

 

What I would say, as the strongest possible response to your post, is that the NOAA themselves are not happy with the performance of neither the GFS or its ensemble compatriot, GEFS.  To that end, it is undergoing a major overhaul in April, in order to correct a number of errors which have been identified in the physical model.  In addition, it is also worth stating that all mathematical models generate bias, and the GFS is no different in that regard; it will - for example - display a predisposition to explosive cyclogenesis, and this is mainly due to an issue around resolution depth, coupled with a data issue regarding SST.  If you read the narrative that Nick often references from NOAA discussion, you will - quite often - see commentary to this effect.

 

With respect to the verification statistics, I would wholly disagree with your approach.  The statistical basis for verification is an accepted industry standard, and it is through that which respective model performance is objectively assessed.  There is no agenda at play; the statistical results are derived from the output of the models, so - if they don't verify - then it is the model that is at fault, not the statistics.  I think an important point to make here is that - in a scientific environment, there is no room or for that matter value, in assessing using subjective interpretation; there needs to be benchmarks and criteria, and that is what a standard reanalysis model delivers.

 

The ECMWF - in tandem with UKMO - are world-leading pioneers in meteorology, and this isn't lost on NOAA or other worldwide counterparts.  There is actually a fairly broad consensus right across the board, and I think you'd be surprised that there really isn't this competitive nature - it's a lot more scientific and progressive than that.  For example, I know that there are (at least) two ECM colleagues over in Reading who are currently advising NOAA on how to best transition from the current iteration of GFS over to the new one in April.  That's so that the NOAA have a seamless implementation of the operational model which, in time, the ECM can also make use of and naturally compare itself against.

 

My personal - and I stress personal - view of the GFS, is that it contains too many consistent and permanent flaws, in order for me to attribute it too much credit.  That is not to say that I completely discount it - that would be foolish - but it's about weighing it up, relative to its peers; there will be times when the GFS performs better, and there will be other times when it performs quite obviously poorer.  But that's mathematical modelling.  There is one model, for instance, that is nigh-on useless through the first third of winter, and that is because its physics engine relies so heavily on stratospheric data.  Thereafter, its performance appears to be quite exceptional.  It is not within my gift to say which one, but I'm sure Ian knows which one I'm referring to.  Equally so, it is evident that some models perform better through different seasons, and this can be understood through how every model will have different elements of teleconnective reasoning.

 

So, it isn't really the case that there is one model that is in any way to be considered a panacea - that really isn't the approach that is used by the Met Office (or the NOAA, for that matter)  The approach is to objectively assess all outputs, and to then apply probabilistic reasoning against it, in order to draw a forecast.  That approach abrogates against the sort of bias which is often found to be at the source of most known human errors.

 

I hope that helps, or at least serves to perhaps dispel a myth or two?

 

SB

Hi Snowballz - thanks for your clear and detailed reply. I fully appreciate what you have said and I can't/wouldn't argue with the points you made, but as I made clear from the off I was talking from a personal perspective.

[Biggin]

Two excellent posts re model performance and quality from Nick n SnowBallz Thankyou.

[fergieweather]

Snowballz post very concisely outlines the considered professional views on the inter-model 'debate': a great post and the sort of value that this forum benefits from.

[bluearmy]

the ecm public data used to only go to T168. if it still did , i doubt we would be having this debate. i think the problem lies less with the model and more with those who use its output too 'literally'. that comment is also relevant to gfs. if you appreciate the bias of the different models then there is much that you can garner from their runs and that includes gfs. if asked, i bet most on here would say that ukmo is the most reliable model. i think that proves my point.back to the future - i take draztik's viewpoint on board (though i would again point out that on this occasion we have all the models bar gfs amplifying the pattern markedly.)probably, ecm will again prove to have been over amplified on the ops (the ens mean, once it had picked up that there would be some amplification to our east, likely to again provide the best guidance). gfs will prove to have been over progressive and flat. however, because gfs didnt show a screaming beasterly (neither has ecm btw), many will think that ecm has climbed down when it will more likely be gfs that has had to change its lw pattern the most. and there again, it will be us who are gulity rather than the modelling.

[Sunny Kent]

As a lurker here for several years and only occasionally posting - can I say a 'big thanks' to posters Ian, Snowballz, Gibby and Nick for their informative and welcome input into this forum. It has certainly become a really authoritive voice. Long may it continue!

[swilliam]

As a lurker here for several years and only occasionally posting - can I say a 'big thanks' to posters Ian, Snowballz, Gibby and Nick for their informative and welcome input into this forum. It has certainly become a really authoritive voice. Long may it continue!

Don't forget Steve Murr

[The Eagle]

My 2 cents. The ECMWF is Europe's primary model funded and sustained by met offices around Europe. It's our model paid for with our money and is the best in the world. For me it's the first model I always go to and the last I always check. The GFS is only over regarded because so many of the parameters are free to view. Which is fine. Be rather boring if most of it wasn't available. The others are just unreliable - GEM/JMA/NAVGEM and the UKMO as well all be it a little further out - beyond 72 hrs usually it's cuckoo land. The ECM has always been superior to all of them.

[weather eater]

Just to add my two pence worth it’s pretty obvious that the ECM outscores the GFS, the stats don’t lie, however, one of the problems with that truth is that people seem to then go on and draw the conclusion that.

1. The ECM trumps the GFS every run, when it does not and.

2. If the ECM says cold synoptics are coming our way, then it’s almost a dead cert, also not true.

[warrenb]

I always check the ECM followed by the METO and then the JMA (which I have found to be very good as well).

[festivalking]

Shall we get back to discussing the charts then......

Edited January 6, 2014 by Osbourne One-Nil
That's fine, but not what you're doing there either!

[nick sussex]

Overall today we're still in limbo as to what might happen going forward. Essentially here what happens on a farm in Kansas will either nudge the outputs towards the colder drier option or the more average set up, either way it at least looks like the jet will slow and bring a bit more drier weather.

 

In terms of the synoptics near the UK theres not much to choose between them at T96hrs but then its how the models deal with the pattern over the central USA that will cause that ripple effect in western Europe.

 

I always advise newcomers to the site to view the outputs from the NH set up and not just Europe because then this really shows the fluidity in terms of how patterns evolve and the effects that changes over that farm in Kansas can have for us in Europe. So to hightlight this I've done a few paint jobs and compared the NH set up between the ECM and GFS 00hrs runs:

 

Firstly the ECM at T96hrs:

 

 

Then the T120hrs chart:

 

 

Compare these with the GFS for the same times:

 

 

T120:

 

 

You can see the difference upstream near that Kansas Farm!

 

You can see the effect on both the PV and also the dig of the jet to the west of the UK.

 

In effect here the model that has the correct pattern over the USA will be the one that verifies for western Europe, because of this type of set up a westwards correction and also more dig south of the jet to the west of the UK will lead to the colder outcome, versus the flatter solution.

 

In terms of cold watch heres the latest ECM ensembles for De Bilt the last three shown so you can see the trend: So starting with the 00hrs yesterday and going back to last nights 12hrs and then the 00hrs from today:

 

 

 

 

There is a small drop off in terms of colder solutions for De Bilt but still enough to keep the coldies amongst us interested. The wind ensembles show the split going forward, both the operational and control run develop a shortwave which runs through se'wards and causes that swiftish change in wind direction:

 

 

In terms of tonights outputs and given the timeframes I think whatever trend we see tonight will be the decisive one in terms of whether its  colder  with a chance of some trough disruption further west and even a little snow, or whether its a more average/cooler drier set up.

 

 

 

 

 

 

 

[Purga]

Difference this time though is ECM is not screaming easterly just showing potential of Scandi high, long way to go yet before anyone can say for sure what way this is going to go.

All I can say is there is more potential for cold weather lovers than there has been all Winter and nothing more than that.

If I lived off the charts showing 'potential for cold weather' I'd be a millonaire!

[weather eater]

Thank you for that bit of info, I asked a similar question a few weeks back as to whether the models were organic and "learnt" as they went along and gained new information. I would have thought by now with they available computing power that a model would have been able to do this by now. Is this what GloSea5 does?  

Work called so it’s taken a while to get back to you, in regards GloSea5, IF will have to answer that because I don’t know the answer.

What I would say is this, while it may appear to be only common sense for the models to take past patterns into account, in reality there are good reason why they don’t. Among them would be that the climate is interconnected, so even today, if we had a synoptic pattern across the Atlantic, UK and Europe that looked exactly the same as let’s say Jan the 6^th 1947, that pattern today would not go on and develop in exactly the same way as it did in 1947. For it to have even a remote chance of doing so, the synoptic pattern and all the other factors that drive climate would have to be an exact match worldwide, to those that existed on that day in 1947, even then it would still be highly unlikely that the pattern would evolve in exactly the same way past a couple of days if that, given that there are so many factors that are influencing the climate all of the time, some of which we simple don’t have the ability to measure. My own opinion is that while a model that referenced past patterns might be useful for short range and local forecasting, long range I feel it would make matters worse, by potentially making the wrong kind of corrections to projected forecasts on a frequent basis, not only that but the computing power would have to be far greater than it already is.

Edited January 6, 2014 by weather eater

[Weather Boy]

Snowballz post very concisely outlines the considered professional views on the inter-model 'debate': a great post and the sort of value that this forum benefits from.

 

Hopefully this isn't totally off topic, so can I ask this?

 

To what extent, if any, do the models share the data on which they are based?  I assume that each model hasn't got its own private collection of weather balloons, for example, so I assume that they must share to some extent.

 

...And presumably those that share more closely are more likely to chime with the same output?  Who shares with whom?

[weather eater]

Hopefully this isn't totally off topic, so can I ask this?

 

To what extent, if any, do the models share the data on which they are based?  I assume that each model hasn't got its own private collection of weather balloons, for example, so I assume that they must share to some extent.

 

...And presumably those that share more closely are more likely to chime with the same output?  Who shares with whom?

They must use the same sources but it’s the software that does the calculations that counts, the ECM model simply does it better, more often.

[MR EXTREMES]

I noticed earlier on the bbc news website john Hammond talking about the vortex bringing record low temps into the states.

is it possible that this vortex could relocate into the northern alantic into Greenland again or does this not work this way.

 

it seems to be doing a great job of throwing the models into uncertain territory.

[Man With Beard]

Looking at fax charts, I see 510dam halfway across the Atlantic, probably the remnants of the record cold in the USA. Probably one of those rare occasions where the UK could get snow off a very strong westerly. Sods law then that this is the first Atlantic storm in 3 weeks that is programmed to stall out west, on account of a Scandi High not strong enough to bring in the cold from the east! Only in the UK eh!

[Downpour]

Shedhead is 100% wrong about GFS in summer. It constantly tried to bring in the Atlantic, and he backed it, only for the high to win out, leaving us with a great summer. Shed was led astray during summer by the GFS, and constantly forecast breakdowns that never came. Short memories!

[Snowman.]

Back to the models..

 

6z12z

 

I'll let you decide

[ARNOTT ad]

And on top of that Paul Hudson at lunchtime said that there isn't even the slightest glimpse of any cold or snowy weather in the near future and beyond