Playing with Data

Personal Views Expressed in Data

Forecast Soundings: A Look to the Future

Those who are regular readers of this blog know by now that my research interests are centered on operational meteorology, data visualization, and data mining. For the 2011 Hazardous Weather Testbest (HWT) Experimental Forecast Program (EFP), I was able to combine my interests by creating a means of viewing model forecast soundings from various convection-allowing models, for the same location, simultaneously. Now, I didn’t create the viewing tool, but created a way to properly format millions of lines of data to utilize the BUFKIT sounding program. An example of one of these ensemble soundings is shown below.

As you can see, the ensemble sounding illustrates several different forecast scenarios that are not apparent when looking at a single sounding. In severe weather forecasting, these sometimes subtle differences in an atmospheric profile can lead to vastly different results. By looking at an ensemble of solutions, forecasters can begin to gauge the variability of the numerical guidance, as well as the predictability. Researchers also benefit. In the HWT this year, we learned about how different model physics impact the atmospheric profile, which, in turn, might introduce systematic biases into the forecast. Knowing this allows researchers to develop better ensembles and better models.

Creating this data set was not an easy process. I had to read in text-files containing the sounding information (example located at the end of the post) for each forecast sounding points — 1146 per model — in each of the 18 numerical models. At that point I needed to be able to compute various thermodynamic parameters from the soundings, so I rewrote most of the thermodynamic and kinematic routines in the Storm Prediction Center’s (SPC) sounding viewer, NSHARP (National Skew-T and Hodograph Analysis and Research Program), in Python. This allowed me to compute a thermodynamic and kinematic parameters in a wide variety of ways for the HWT-EFP, all consistent with what Storm Prediction Center forecasters are used to using in operations.

After computing the necessary thermodynamic fields, I then created sounding files in BUFKIT format for each of the 1146 sounding sites for each of the 18 models. Lastly, I combined the 18 model forecasts for each of the 1146 sites into a single file and output 1146 more text files containing the ensemble sounding files which could then be read by E-BUFKIT. A sample image is above. All-in-all this took about 1 hour of computation time on 18 separate computers. That’s a lot of data to crunch through!

After doing this for the HWT-EFP, I thought to myself, “Why stop there?”. Since the end of the EFP, and with the help of John Hart of the Storm Prediction Center, I have begun work on creating an ensemble sounding viewer — written entirely in Python — based on the SPC’s sounding program (NSHARP). Why Python? Because Python allows the program to be cross-platform, meaning anyone who installs Python on their computer can use the program.

The ultimate goal is to create this program and then release it to the atmospheric community as an open source project. This would allow researchers, forecasters, and hobbyists to be able to use the program to view model soundings in both a deterministic and ensemble framework. Additionally, by having a community supported sounding tool, it is my hope that researchers in the climate community will also use this program for their research. If the meteorology community adopts a single sounding tool, scientists, forecasters, and hobbyists will be able to quickly compare parameters from climate models, weather models, and observational data, knowing that the values were computed in the same manner. This would be a huge boost to comparing historical, current, and future events. Not to mention allow for consistency between datasets.

I’m still in the early stages of development. My hope is to be able to present an alpha version of this program at the American Meteorological Society’s Annual Meeting in New Orleans next January. I’ll have my work cut-out for me…

In the mean time, a screen shot from this new sounding tool is shown below. This prototype supports active read-out, with on-the-fly interpolation as one moves the mouse over the sounding. The read-out is listed at the bottom of the sounding.

I’m interested in knowing what people think, so feel free to leave me feedback!

Sample Sounding Text File

Visual Comparison: 3-4 April 1974 and 27-28 April 2011

By the afternoon of 28 April 2011 it was fully apparent that the unthinkable had happened. In an era of unprecedented communication abilities, a single tornado outbreak took the lives of more people than all the tornadoes over the past several years combined – in broad daylight no less. In the days the followed, many tried to place this event into historical context. Nearly every one defaulted to the 3-4 April 1974 “Super Outbreak”.

The Super Outbreak was nothing short of impressive from a meteorological point of view. 148 tornadoes, 319 fatalities, over 13-states, in 24-hours. Never before, and not until this April, had anything even close to the scale of this tornado outbreak had ever been recorded. By comparison, the tornado outbreak of 27-28 April 2011 has an unofficial count (undertaken by several of us at the Storm Prediction Center) of over 174 tornadoes (done via Public Information Statements) and 259 fatalities attributed to these tornadoes. (Unfortunately, the death toll is considerably higher, I simply have been unable to place all the fatalities to the corresponding tornado at this time.)

From the standpoint of the number of tornadoes recorded and the number of fatalities, these two tornado outbreaks are in a class by themselves (in the “modern” tornado database starting in 1950). In the days that followed, I created a set of figures for internal NWS/SPC/NSSL use to compare the two tornado outbreaks. The images show all reported tornado tracks, color coded based on intensity and the counties are color-filled based on the number of fatalities that occurred within that county’s boundaries. A simple, quick look through the two events shows that the 3-4 April 1974 event covered a much larger area than the 27-28 April 2011 event, although there is considerable overlap between the two events. Several counties experienced fatalities in both events; in fact, Marion County, Alabama was unfortunate to have had a F/EF-5 tornado, and large loss of life, in both of outbreaks (1974: Guin, AL; 2011: Hackleburg, AL). Lastly, each figure has a table of the number of tornadoes and corresponding fatalities, broken down by EF-Scale (the 2011 event is still “preliminary” and subject to change). (Note, higher resolution images, for “zooming” are available by request.)

Meteorologists (and others) can, and will, debate for years as to which event was “more impressive”. I know what my thoughts are, but I’ll spare you those. However, please feel free to leave your thoughts in the comments.

The two images above are on the same background. This means if you download both of them and flip back and forth between the two, the only things that should change are the county colors and tornado tracks. Below is a zoomed in version of the 27-28 April 2011 event, complete with NWS County Warning Areas and County Names denoted.

Long Hiatus Ends

A lot has taken place the last few months and this has prevented me from being able to blog. Since my last post the United States has experienced a devastating tornado outbreak (27 April 2011 in the southeast), the deadliest tornado since 1947 (22 May 2011 in Joplin, MO), and a violent tornado outbreak in the more tradition area of Oklahoma (24 May 2011). What makes this year remarkable is the number of tornadoes that have hit heavily populated areas, which has contributed to the number of direct tornado fatalities being well over 500. It’s certainly been an emotional year for meteorologists. Also during my blogging hiatus, the National Severe Storms Laboratory and the Storm Prediction Center held another successful Experimental Forecast Program. The datasets generated will provide researchers ample opportunities for discovery.

This post is short, but serves to end my blogging drought. In the coming days, weeks, and months, I hope to share what’s been keeping me busy. Here’s to getting back into the habit of putting my thoughts in words.

Historical Arkansas Tornadoes


UPDATED: 11:30 PM CDT on 25 April 2011
The statistics have been updated to better account for tornadoes that crossed state boundaries. This added 32 tornadoes and 9 fatalities to the Arkansas counts. None of these additional tornadoes were rated violent tornadoes (F/EF-4 or greater). The Arkansas Tornado CSV file has been updated as well.

Several large tornadoes moved through Arkansas this evening, with the worst damage as of this writing coming from the town of Vilonia, AR. Several television stations from Little Rock are now reporting that parts of Highway 64 “are missing”. Anytime reports of pavement being scoured, the talk inevitably turns to discussion about tornado ratings, and comparisons to other pavement scouring tornadoes. Personally, I believe any talk of ratings tonight is premature, however, I went ahead and pulled the Arkansas historical record to help put this event in perspective.

Officially, there has never been a F/EF-5 tornado in Arkansas. Let me repeat that. There has never been an F/EF-5 tornado in Arkansas. In fact, there have only been 26 F/EF-4 tornadoes. If this tornado is rated a violent tornado (EF-4 or EF-5), it will rank among the strongest 2% of all tornadoes to strike Arkansas. If we limit to only tornadoes between 1980 and 2010, only 1.3% of al tornadoes were “violent” tornadoes. Even though most tornadoes are not violent tornadoes, almost half of all tornado fatalities in Arkansas (53) resulted from these 13 EF-4 tornadoes. The fact that there have been so few reported fatalities as of this writing (only 1), is a testament to the National Weather Service in Little Rock and their excellent warnings.

Arkansas Tornadoes and Fatalities (1981 – 2010)

  • F/EF0 : 363 (36.37%)……..0 fatalities (0.00%)
  • F/EF1 : 376 (37.68%)……..1 fatalities (0.76%)
  • F/EF2 : 172 (17.23%)…….24 fatalities (18.18%)
  • F/EF3 : 74 (7.41%)………54 fatalities (40.91%)
  • F/EF4 : 13 (1.30%)………53 fatalities (40.15%)
  • F/EF5 : 0 (0.00%)………..0 fatalities (0.0%)

Total : 998 tornadoes and 132 fatalities

In terms of all Arkansas tornadoes:

All Arkansas Tornadoes (1950 – 2010)

  • F/EF Unknown : 16 (1.01%)
  • F/EF0 : 436 (27.47%)
  • F/EF1 : 568 (35.79%)
  • F/EF2 : 387 (24.39%)
  • F/EF3 : 154 (9.70%)
  • F/EF4 : 26 (1.64%)
  • F/EF5 : 0 (0.00%)

Total: 1587 tornadoes

Here are all Arkansas F/EF-4 tornadoes:

All Arkansas F/EF-4 Tornadoes (1950 – 2010)

  • 21 March 1952 @ 14:40 (2:40 PM)
  • 21 March 1952 @ 16:50 (4:50 PM)
  • 21 March 1952 @ 17:00 (5:00 PM)
  • 21 March 1952 @ 17:30 (5:30 PM)
  • 21 March 1952 @ 18:00 (6:00 PM)
  • 19 December 1957 @ 12:44 (12:44 PM)
  • 10 April 1965 @ 18:26 (6:26 PM)
  • 03 April 1968 @ 18:30 (6:20 PM)
  • 19 April 1968 @ 15:12 (3:12 PM)
  • 15 May 1968 @ 20:36 (8:36 PM)
  • 15 May 1968 @ 20:45 (8:45 PM)
  • 26 May 1973 @ 22:30 (10:30 PM)
  • 28 March 1975 @ 19:52 (7:52 PM)
  • 02 April 1982 @ 17:20 (5:20 PM)
  • 24 December 1982 @ 17:00 (5:00 PM)
  • 15 March 1984 @ 17:40 (5:40 PM)
  • 15 March 1984 @ 19:17 (7:17 PM)
  • 14 April 1996 @ 18:10 (6:10 PM)
  • 01 March 1997 @ 14:20 (2:20 PM)
  • 01 March 1997 @ 14:47 (2:47 PM)
  • 01 March 1997 @ 15:25 (3:25 PM)
  • 01 March 1997 @ 15:41 (3:41 PM)
  • 01 March 1997 @ 18:20 (6:20 PM)
  • 16 April 1998 @ 1:50 (1:50 AM)
  • 21 January 1999 @ 17:50 (5:50 PM)
  • 05 February 2008 @ 16:49 (4:49 PM)

Keeping Perspective – Preliminary vs. Final Tornado Reports

Over the last few days, the southern United States has endured a significant severe weather event that took the lives of a still increasing number of people. While many communities are still trying to sift through the wreckage, meteorologists, “meteorologists”, chasers, and other weather enthusiasts have taken to Facebook and Twitter to discuss what has happened. Unfortunately, a lot of misinformation is floating around. Here’s my quick attempt to clarify some of this information.

As of this writing, 248 preliminary tornadoes have been reported via the Storm Prediction Center’s Preliminary Storm Report webpage over the three days 14-16 April 2011. Much has been made about this number. Unfortunately this number contains many duplicate tornadoes, and potentially even some tornadoes that never were. Hence the label “Preliminary”. Over the next few weeks, National Weather Service Offices throughout the south will be conducting damage surveys to determine the number of actual tornadoes to the best of their ability. There is no doubt that this has been a significant three-day tornado outbreak. However, until the official numbers are released via the National Weather Service’s Storm Data publication, people should exercise extreme caution in trying to quantify where this week’s severe weather outbreak ranks in history.

It used to be the case that the preliminary number of tornadoes underestimated the number of actual tornadoes. However, near March of 2006 the pattern reversed itself with the number of preliminary tornadoes typically overestimating the number of actual tornadoes. For more information regarding preliminary vs. final tornado reports, please read this blog post on the topic by Harold Brooks, which can be found on the United States Severe Weather Blog.

In 2008, the final tornado count was roughly 75-80% of the preliminary count. Based on the number of spotters now reporting tornadoes, it’s not out of the question to assume this ratio is now closer to 70%, if not lower. Based on this an estimated guess to the number of final tornadoes might be closer to 198 (80%), 186 (75%), or even 173 (70%). Since I know people want to know below are the top three-day tornado counts on record using the final tornado numbers. The date given is the last day of the three days used in the count.

Three Day** Final Tornado Counts (1950 – 2010)

  • 20040530: 184
  • 20030506: 183
  • 20040531: 172
  • 19740403: 166
  • 19920617: 152
  • 20030508: 148
  • 19740404: 147
  • 20070506: 136
  • 20070505: 135
  • 20030510: 135

** The day listed is the end of the three days used in the counting.

Update 1PM CDT 18 April 2011:
It should be noted that the 75% ratio mentioned above was computed prior to the SPC removing a simple time and space filter. As such, the current ratio will most likely be even lower.