Hindcasts
Author: Jason Fleming (GitHub ID: jasonfleming)
Retrospective analyses (i.e., "hindcasts") of historical tropical cyclone events are an important part of model development as well as risk assessment. The ASGS is an ideal system for producing hindcasts because it already contains features for working with tropical cyclone data including BEST track files as well as the Forecast/Advisories that are issued by the National Hurricane Center.
Creating hindcasts of historical hurricanes with the parametric Generalized Asymmetric Holland Model (GAHM) is a three step process:
- Select a storm to hindcast and create the associated input data
- Configure the ASGS properly to run the hindcast
- Execute the ASGS and possibly manipulate the input data feed to make the hindcast progress
These steps are detailed in the sections below using Hurricane Ike of 2008 as a case study.
September 13, 2008 - Hurricane Ike makes landfall at Galveston as a large Category 2 hurricane. Its large size contributes to a storm surge that is as high as 20ft (6.1m), which inundates many of the barrier islands off the Texas coast. Many structures on the Bolivar Peninsula are destroyed. Most notably, in Gilchrist all but one house is destroyed by the storm surge. The storm also causes heavy rains where it makes landfall, peaking at 18.9in (480mm). At one point, the storm knocks out power to as many as an estimated 2.6 million people.[ Ike is one of the most destructive hurricanes ever to hit Texas and one of the deadliest, causing $19.3 billion in damages and killing 84 people.
Ike advisory 01 was issued almost two weeks before landfall when the storm system was still in the central Atlantic (see below). Since this time period will not be relevant to the target area (the Texas Gulf coast), the Operator will need to make a decision on when the retrospective period should start. This decision will also need to be informed by initialization concerns, including the development of a fully developed wave field, as well as any potential coastal water level set up.
I normally set the start date by examining the graphics issued with each forecast/advisory, and take the associated physics into account, along with looking at the location and forward speed of the storm. For Ike, I selected the point of emergence of the storm off the coast of Cuba into the Gulf of Mexico in advisory 36 as a reasonable starting point, as shown below.
Choosing the end time of the retrospective period should take several factors into account, including the "drain down" time period after the storm has passed, as well as the total length of the retrospective. Some storm events don't have much drain down time in the BEST track (i.e., the storm dissipates shortly after landfall) and some have so much that it would make the hindcast too long to be practical. Looking at Ike in particular, the final advisory was issued at 4amCDT on Sunday 14 September 2008, which gives us about one day of drain down time and a total hindcast length of 5 days (see below).
Qualitatively, our hindcast plan will be to
-
run a tides-only initialization to the point that we want to start including meteorological forcing from Ike
-
set up a BEST track file and forecast/advisory file that represents the time that we want our hindcast to end
Conceptually, the timeline looks like the following for our Ike hindcast:
BEST track BEST track
START END
|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Advisory 36 |
|**********************| |
START END | Advisory 53
| | |********************|
| | START END
| |
|---------------------v|^=======================v|
COLDSTARTDATE HINDCAST HINDCAST
START END
Where the v indicates that a hotstart file was written, and the ^ indicating that a hotstart file was read.
In order to accomplish this, we will configure ASGS to start on the COLDSTARTDATE; it will perform a tides-only
run for a number of days equal to HINDCASTLENGTH and then start looking for a forecast/advisory. We
will feed it Ike advisory 53 and the full BEST track file. It will see that it needs to perform a catch-up
nowcast from the end of the tidal initialization until the start time for advisory 53. We will also set SCENARIOPACKAGESIZE=0 so that it does not actually try to run a forecast.
In order to start the retrospective modelling at a time corresponding to advisory 36, we need to take a look at the BEST track file for the storm, and the data that corresponds to the start date we've selected. Forecast/advisory 36 was issued at 11pmEDT on Tuesday 9 September 2008, which corresponds to 03Z on Wednesday 10 September 2008. Therefore, the last available BEST track data at the time that this advisory was issued was at 00Z on 10 September 2008. We now need to find the full BEST track file for this storm and look to this time as our starting point.
The official archive of BEST track files contains links to BEST track files by year. Ike occurred in 2008, in the Atlantic basin (prefix al) and was the 9th tropical cyclone of the season, so the file to look for is bal092008.dat.gz.
Inside the file, we find this time on line 100 (i.e., the first line below that starts with AL, 09, 2008091000,...):
AL, 09, 2008090918, , BEST, 0, 227N, 833W, 65, 966, HU, 34, NEQ, 155, 150, 105, 170, 1008, 275, 15, 80, 0, L, 0, , 0, 0, IKE, D, 12, NEQ, 180, 180, 90, 180
AL, 09, 2008090918, , BEST, 0, 227N, 833W, 65, 966, HU, 50, NEQ, 100, 75, 30, 90, 1008, 275, 15, 80, 0, L, 0, , 0, 0, IKE, D, 12, NEQ, 180, 180, 90, 180
AL, 09, 2008090918, , BEST, 0, 227N, 833W, 65, 966, HU, 64, NEQ, 20, 0, 0, 30, 1008, 275, 15, 80, 0, L, 0, , 0, 0, IKE, D, 12, NEQ, 180, 180, 90, 180
AL, 09, 2008091000, , BEST, 0, 231N, 840W, 65, 968, HU, 34, NEQ, 160, 150, 110, 180, 1006, 200, 15, 80, 0, L, 0, , 0, 0, IKE, D, 12, NEQ, 180, 210, 150, 240
AL, 09, 2008091000, , BEST, 0, 231N, 840W, 65, 968, HU, 50, NEQ, 100, 75, 30, 90, 1006, 200, 15, 80, 0, L, 0, , 0, 0, IKE, D, 12, NEQ, 180, 210, 150, 240
AL, 09, 2008091000, , BEST, 0, 231N, 840W, 65, 968, HU, 64, NEQ, 30, 0, 0, 20, 1006, 200, 15, 80, 0, L, 0, , 0, 0, IKE, D, 12, NEQ, 180, 210, 150, 240
AL, 09, 2008091006, , BEST, 0, 234N, 846W, 70, 964, HU, 34, NEQ, 170, 150, 115, 180, 1006, 200, 15, 85, 0, L, 0, , 0, 0, IKE, D, 12, NEQ, 180, 210, 150, 240
Based on the above, and considering that we want our initialization period, a.k.a. "tidal spinup", to last 30 days (HINDCASTLENGTH=30.0) and to end at 00Z on 10 September 2008, we see that we should set COLDSTARTDATE=2008081100 (00Z on 11 August 2008). When in doubt doing date math, you can always used the little dateCalculator.pl command line utility:
asgs (default)> dateCalculator.pl --input-date 20080910000000 --ddays -30.0
The resulting date is 2008 08 11 00 00 00.
The first line in the BEST track file, indicating the time the system spent as an "INVEST" is as follows:
AL, 09, 2008090106, , BEST, 0, 172N, 370W, 30, 1006, TD, 0, , 0, 0, 0, 0, 1011, 250, 90, 0, 0, L, 0, , 0, 0, INVEST, D,
The last line in the Ike BEST track file is as follows:
AL, 09, 2008091512, , BEST, 0, 472N, 711W, 35, 986, EX, 34, NEQ, 0, 180, 150, 0,
So the data we can use with the GAHM model ends at 12Z September 15 2008.
Now that we have the BEST track file, we will also need file to represent Ike advisory 53. This file must exist and be valid to prevent parse errors from the ASGS, but the actual contents will not be used, since it represents a forecast and we are not planning to run a forecast.
We can now update our conceptual timeline for the retrospective model runs with actual dates and times.
BEST track BEST track
Start End
2008090112 2008091512
|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
|********************|
Advisory 53 Advisory 53
Start End
HINDCAST HINDCAST
COLDSTARTDATE START END
2008081100 2008091000 2008091512
|--------------------->+========================>|
The text advisories from the NHC archive website can be used for forecasts. These files can be found at the following location:
http://www.nhc.noaa.gov/archive/
Just click on the year to see all the storms for that year.
Then click on the name of the storm that the ASGS should run. The list of all the forecast/advisories that were issued for that storm will be shown.
Then click on the first forecast/advisory that should run, and "Save Page As..." to save the html (just the html) to a file. This file could be fed to the ASGS, but some additional editing should be performed, as described below. Repeat this step for all the advisories in the hindcast.
Once all the html files for the forecast/advisories in the hindcast have been
obtained, a mock RSS feed should be created. Take the text of each
forecast/advisory from the html files insert it into an index-at.xml file. An
example index-at.xml file from advisory 07 of TS Beryl 2012 can be found in the
doc subdirectory of the ASGS installation directory.
mimic the real time Operational environment for the ASGS, by providing the data that would normally be available in real time from the National Hurricane Center.
For example, to run Ike advisories 42, 44, and 46, the procedure would be as follows:
-
use a text editor to pluck out the actual text of the forecast/advisory from each html file;
-
use a text editor to embed that text into a template index-at.xml file, being sure to also update the advisory number;
-
end up with three files named something like ike.42.index-at.xml, ike.44.index-at.xml, ike.46.index-at.xml.
Now, a hindcast file will also be needed to go along with each forecast file. In a real storm, the hindcast data end before or just after the start of the forecast when each advisory is issued.
So, in the case of Ike, a set of bal092008.dat files is needed that each end at the appropriate times for the forecasts. To support ASGS test efforts in prior seasons, the author has created a bunch of files like this for various storms here:
https://github.com/StormSurgeLive/storm-archive
Hopefully this archive will be of use to the ASGS Operator community.
So at this point, an inventory of files called (for example)
ike.XX.index-at.xml and ike_advisory_XX.btk (one set for each advisory,
where XX is the advisory number) has been developed.
In normal operation, the ASGS contacts remote web and ftp sites to collect the real time data that it requires. During a test, however, the Operator will be supplying these files. If the ASGS Operator has administrative access to a web server and ftp server, these can be set up to mimic the web and ftp sites of the NHC, and the ASGS can be configured to go to the mock sites instead of the real ones.
However, it is simpler to reconfigure the ASGS to look to its local filesystem for the mock files, thus eliminating the requirement for having web and ftp servers handy. A sample set of test configuration parameters for external data sources is as follows:
STORM=
YEAR=
TRIGGER=rssembedded
RSSSITE=filesystem
FTPSITE=filesystem
FDIR=/path/to/index-at.xml
HDIR=/path/to/bal'STORM'YEAR'.dat
Using Ike as an example, 'STORM' would be set to 09 and 'YEAR' would be set to 2008. Please see the section on External Data Sources above for more detail on these parameters.
Once the mock advisories have been developed and the ASGS has been configured, it is up to the Operator to play the role of the NHC and "issue" new hindcast/nowcast data and new forecast advisories. Step-by-step instructions for doing this are as follows:
-
Go to the directory where the mock RSS feed for the forecast/advisories are stored.
-
Make a symbolic link to the first advisory in the test (because the ASGS is hardwired to look for the index-at.xml file), e.g.:
ln -s ike.42.index-at.xml index-at.xml
- Make a symbolic link to the hindcast data that correspond to the first advisory in the test (because the ASGS is hardwired to look for a file that is formatted as bal 'STORM' 'YEAR' .dat), e.g.:
ln -s ike_advisory_42.btk bal092008.dat
-
Start up the ASGS normally. It will grab the NHC data from the local filesystem (whichever advisory the symbolic links point to).
-
To issue a new advisory, manually update the symbolic link for the hindcast to the next hindcast file, then manually update the symbolic link to the next forecast file. For example:
rm bal092008.dat ; ln -s ike_advisory_44.btk bal092008.dat
rm index-at.xml ; ln -s ike.44.index-at.xml index-at.xml
Always update the symbolic link to the hindcast first, because the ASGS will be looking for a change in the forecast data to tell it when a new advisory has been issued. If the forecast link is updated first, the ASGS will detect it so quickly that it will start its new cycle before the hindcast link can be updated. This will just cause the ASGS to use the old hindcast/nowcast data.
A shell script could be written to update these links periodically to issue the mock advisories; the process of issuing advisories doesn't have to be manual.