QUALITY MONITORING
THE PRODUCT VALIDATION PROGRAMME
Introduction

This cluster was introduced from the beginning of the project to structuralize the product validation activities of several counties.

For all the products generated by the project, the product validation group is responsible:

- to monitor the progress in product quality as further development evaluating statistical scores and case study analysis on the base of comparison between satellite products and ground data;

- to provide validation service to end-users publishing on the H-saf web-page the statistical scores evaluated and the case studies analysed;

- to provide online quality control to end-users: quality is provided inside the product, with the resolution of IFOV, to be directly used and also included in automatic routines;

- to monitor operational features of the products as actual arrival, timeliness, intelligibility, etc..;

- to provide ground data service inside the project for algorithm calibration and validation activities;

- to investigate the H-saf product impact in end-user applications as Civil Protection activities for emergency management, precipitation event alerts, street monitoring, water balance evaluation, etc.

The partecipants to Group of Product Validation Programme

The cluster is coordinated by the italian Civil Protection Department (DPC). The DPC is an expert user of near-real-time observations commonly used in the hydrological field, closely linked to national and local meteorological services. The DPC is actually involved as main user of national and international spatial projects.

The Product Validation Programme is composed of experts from the National Meteorological and Hydrological Institutes of Austria, Belgium, Bulgaria, Finland, France, Germany, Hungary, Italy, Poland, Romania, Slovakia, and Turkey. Hydrologists, meteorologists, and precipitation, snow and soil moisture ground data experts, coming from these countries are involved in the product validation activities. ECMWF takes also part of the Product Validation Programme.

Structure of product validation programme

Fig. 1

The fig.1 shows the work breackdown structure (WBS) of Product Validation Programme. The different WPs included in Product Validation Programme are coordinated by the person responsible of the product validation programme (fig.2) through continuous contacts via e-mail, call phone and at least one technical meetings per year to exchange experiences, problem solutions and evaluation of possible improvement of the validation methodologies.

The results of the Product Validation Programme are published in this web page developed by DPC and it is continuously updated with the last validation results and studies coming from the validation cluster activities.

It has been decided recently to introduce Working Groups to solve specific items of validation procedure. The coordinator and the participants of the working group are members of the Validation Programme or experts of the institutes involved in the validation activities. The results of the WG are discussed inside the Validation Programme Group before introducing them in the common validation methodology.

A short descriptions of the activities, here limited to main headings, it is reported in the Project Plan.

Validation Programme Group

Fig. 2

Fig. 3

Fig. 4

BELGIUM
Name of the institute: Royal Meteorological Institute (IRM)
Link of the institute: www.meteo.be
Contact point for the PP validation activity: Emmanuel Roulin, emmanuel.roulin@oma.be - Angelo Rinollo, angelo.rinollo@oma.be
Institute PP validation activity description
Common validation activity

The continuous statistics, multi-category contingency tables, and the probability distributions are prepared on a monthly basis for the different precipitation products. From the beginning of the development phase of the project, it has been chosen at RMI to focus on a validation area centered over location of the weather radar at Wideumont, whatever the product so that the scores could be compared image by image. For the microwave products either from conical or cross-track scanners, this requires a procedure to select the files.

From a local point of view, rain rates products based on microwave sensors onboard of Low Earth Orbit satellites are characterized by a varying coverage and projection. To make the statistics comparables from one file to the other, a validation domain has been defined which is a square of 230 km ´ 230 km centered on the Wideumont radar location and only the products covering entirely this common area have been considered. To be more precise, for every product file, a sub-set of lines and columns including the common square has been extracted. Then, the radar data have been up-scaled to the projection of the sub-set of pixels and compared with the product estimates.

Left: Gaussian filter; right: sketch of the up-scaling procedure. The circle corresponds to the range of the weather radar. The square in the middle is a common area such that it is entirely included in the selected PR-OBS-2 files. The grey rectangle, the tilted dark grey rectangle and the black ellipse are explained in the text.

The up-scaling of the radar data is performed taking the footprints of the microwave sensors into account. In the above figure, the Gaussian filter corresponding to the first scan position of the AMSU-B antenna (PR-OBS-2) is represented on the left. The filtering procedure is organized as follows (see the figure, on the right). First, a part of the radar image is selected (grey). Then, the radar data (0.6 km resolution are re-sampled onto a tilted grid (2 km resolution) where the Gaussian filter is ³ 1% of maximum (dark grey). Tilting depends on the scan position and on the satellite overpass mode. Finally the Gaussian filter is applied. The black ellipse corresponds to half power. Additional information about the up-scaling equations and about the tilting of the PR-OBS-2 pixels can be found in Van de Vyver and Roulin (2008).

For PR-OBS-3 and PR-OBS-5, a sub-set of lines and columns has been also extracted which comprises the common validation area. The up-scaling has been simply performed by averaging the radar values included in each pixel in the SEVIRI projection.

For the validation of PR-OBS-5 using raingauge data, the ground data have been interpolated as explained in the Section on ground data and the comparison has been performed between the product estimate and the nearest interpolated grid point over a domain corresponding to the Walloon region in Belgium. Finally, the scores of the continuous statistics, the contingency tables and the probability distribution functions have been prepared on a monthly basis according to the rules common to all the teams involved in the precipitation products validation.

Scale Recursive Estimation

As specific development, we have investigated an application of scale recursive estimation (SRE) to assimilate rainfall rates during a storm estimated from the data of two remote sensing devices. These are ground based weather radar and space-born microwave cross-track scanner (PR-OBS-2). Our approach operates directly on the data and does not require a pre-specified multi-scale model structure. We introduce a simple and computational efficient procedure to model the variability of the rain rate process in scales. The measurement noise of the radar is estimated by comparing a large number of datasets with rain gauge data. The noise in the microwave measurements is roughly estimated by using up-scaled radar data as reference. Special emphasis is placed on the specification of the multi-scale structure of precipitation under sparse or noisy data. The new methodology is compared with the latest SRE method for data fusion of multi-sensor precipitation estimates. Applications to the Belgian region show the relevance of the new methodology (Van de Vyver and Roulin, 2009).

Case studies

The visual inspection of hydrographs (graphs of changes river discharge over time) is a convenient way to select interesting case studies. Usually, all precipitation products are inspected during the 1-2 days corresponding to the selected storm. It happens that some case studies are also selected on the base of the joined time-series of scores and rainfall amounts. For the microwave products (PR-OBS-1 and PR-OBS-2), this is possible because we select images that cover a common validation area and scores can be calculated for each selected scene. For instance, the case study of 14 May 2009 for PR-OBS-1 was selected because the rain covered a large area and also the Equitable Threat Score was significantly high. This allows exploring the potential value of the products.

For each case study, the synoptic situation is analyzed and described by a weather forecaster, information like the cloud classification and lightning maps are collected, and the maps of the products are compared with the corresponding maps of the up-scaled weather radar data.

Ground data used with instrument description and map

The validation results for Belgium were obtained by comparison of the rain rates products with weather radar data and of the cumulated precipitation products with either cumulated weather radar data or raingauge data. The following table summarizes the ground data used as well as the domain over which the validation extends. The last row has been included but refers to results to be presented in the report on hydrological validation.

Product   Ground data Validation domain
PR-OBS-1 MW Conical Wideumont Radar >= 230 km x 230 km
PR-OBS-2 MW Cross-Track Wideumont Radar >= 230 km x 230 km
PR-OBS-3 IR+MW Rapid Update Wideumont Radar >= 230 km x 230 km
PR-OBS-5 Cumulated 24h Cum. Wideumont Radar >= 230 km x 230 km
PR-OBS-5 Cumulated 24h SETHY Raingauges Walloon Region
PR-OBS-5 Cumulated 24h RMI Daily Raingauges Test Catchments
Weather Radar

Belgium is well covered with three radars. New radar is currently under construction in the coastal region. These are Doppler, C-band, single polarization radars with beam width of 1° and a radial resolution of 250 m. Data are available at resolution of 0.6, 0.66 and 1 km horizontal resolution for the Wideumont, Zaventem and Avesnois radars respectively. Currently, only the Wideumont radar has been used.

Meteorological radar in Belgium

Raingauge

Several raingauge networks are managed in Belgium. RMI has a dense network of daily raingauges and an increasing network of automatic weather stations equipped with tipping bucket gauges. Other networks are operated by the Regional Authorities in charge of rivers. For the validation of the PR-OBS-5, we have used hourly data from the SETHY (now Direction de la Gestion Hydrologique Intégrée, Ministry of the Walloon Region) raingauges which are quality controlled daily at RMI. The daily data are gathered and checked with 1.5 to 2 month delay. These later data are mainly used in the hydrological validation programme.

[left] RMI raingauges: daily (+) and AWS (*) --- [right] SETHY raingauges (*)

For the validation of the PR-OBS-5 cumulated rainfall product, a validation with raingauge data has been performed, in parallel to the radar validation. The reference data used are hourly rain gauge records from the SETHY (Walloon Region) network. The network includes 89 automatic non-heated stations and 3 heated stations (in coincidence with non-heated ones). Only the non-heated stations have been considered, for the sake of uniformity. The data have been interpolated in onto a 5 km ´ 5 km grid, following the Barnes method. The sensitivity parameter in the Barnes procedure has been set to 108, considering the fact that the mean distance between every station and its closest neighbor is roughly 104 m. The interpolation procedure is iterative. If the mean squared difference between the source field and the interpolated field falls below 0.01 mm h-1, or if the improvement is below 1% between two steps, the procedure is stopped, otherwise it goes on for a maximum 20 iterations. The result is a series of files with interpolated data, one per hour.

The quality of the interpolated data has been checked for several months in the following way: the interpolation is calculated taking into account all the stations except one, and the value corresponding to the missing station is estimated. The procedure is repeated for all the stations. A set of 89 reconstructed values is obtained, and compared with the measured data. The verification refers to the period from August to November 2008. The interpolation is first assessed in its capacity to reconstruct the rain / no rain field. Taking 0.01 mm h-1 as a threshold, the probability of correct rain (POD) is 0.79, the false reconstruction (FAR) is 0.07 and the equitable threat score (ETS) is 0.71. Then, statistical scores are calculated on a monthly basis. The bias ranges from 0.06 to 0.14 mm h-1, the root mean square from 0.37 to 1.00 mm h-1 and the mean relative error from 0.09 to 0.19 for mean observed values of 0.50 to 1.00 mm h-1.

As preliminary test for the hydrological validation of PR-OBS-5, the data of the daily raingauge stations have been interpolated using the Thiessen polygons method and spatially averaged over the two test catchments. The values obtained have been compared with the corresponding cumulated values from satellite.

Miscellaneous information

For the analysis of test cases, additional information has been used like the cloud types identified using the SAF-NWC tools, the expertise of weather forecasters to select and analyze the synoptic conditions, the SAFIR maps of lightning impacts.

Ground data quality
Weather Radar

The data of the weather radar are controlled in three steps. First, a long-term verification is performed as the mean ratio between 1-month radar and gauge accumulation for all gauge stations at less than 120 km from the radar. The second method consists in fitting a second order polynomial to the mean 24 h (8 to 8 h LT) radar / gauge ratio in dB and the range; only the stations within 120 km and where both radar and gauge values exceed 1 mm are taken into account. The third method is the same as the second but is performed on-line using the 90 telemetric stations of the SETHY (Ministry of the Walloon Region). Corrected 24 h images are then calculated. New methods for the merging of radar and raingauge data have been recently evaluated (Goudenhoofdt and Delobbe, 2009).

Raingauge

The hourly data from the SETHY are quality controlled daily at RMI. The daily precipitation data from the climatological network are controlled and made available with 1.5 to 2 months delay.

References

Goudenhoofdt, E. and Delobbe, L.: Evaluation of radar-gauge merging methods for quantitative precipitation estimates, Hydrol. Earth Syst. Sci., 13, 195-203, 2009.
Van de Vyver, H., and E. Roulin, 2008. Belgian contribution to the validation of the precipitation products: methodology developed and preliminary results. Proceedings of the 2008 EUMETSAT Meteorological Satellite Conference, Darmstadt, Germany, 8 - 12 September 2008, 8 pp.
Van de Vyver, H., and E. Roulin, 2009. Scale recursive estimation for merging precipitation data from radar and microwave cross-track scanners. J. Geophys. Res., 114, D08104, doi: 10.1029/2008JD010709.

BULGARIA
Name of the institute: National Institute of Meteorology and Hydrology NIMH - BAS
Link of the institute: www.meteo.bg - hydro.bg
Contact point for the PP validation activity: Gergana Kozinarova, g.kozinarova@gmail.com, gergana.kozinarova@meteo.bg.com
Institute PP validation activity description
Introduction

The National Institute of Meteorology and Hydrology (NIMH) at the Bulgarian Academy of Sciences is the official hydrometeorological service in Bulgaria. Its primary mission is to provide meteorological and hydrological information and products to different organizations and users in Bulgaria. Its duties comprise both operational and applied research activities. Hydrological and meteorological observations, data acquisition and telecommunication, monitoring the air, surface and ground water, meteorological and hydrological forecasts, maintenance of data base, scientific researches, numerical and statistical modelling are the general duties of NIMH.

The products PR-OBS-5 will be validated by comparison with rain rates estimated from rain gauge data.

Study area river Varbitsa watershed up to the hydrometric station Djebel

The Varbitsa River is situated in the South Bulgaria and is part of the Arda's catchment - its right tributary. Its springs are situated just below the peak Martazyan in the South-East Rodopi Mountain. The length is 98 km, the catchment area is around 1200 sq. km. and flows in the Studen Kladenets Dam. The tortuosity coefficient is around 2.40 and the average river bed slope is 11 ‰. The highest point of the catchment is around 1390 m and the lowest is around 200 m. the average height of the catchment is around 550 m a.s.l.(fig.1)

Fig. 1. Study area and location of the meteorological stations

The catchment is influenced by Mediterranean type of climate with comparatively mild, wet winters and hot, dry summers. In the upper part the catchment is covered by forest lands and in the lower part the river bed becomes wide and covered by sands. There are two periods of peak discharges - in February-March and October-November. The peaks in February-March are usually formed by combination of rainfalls and snowmelting, which sometimes could lead to floods around the valley especially in the lower part of the river. The catchment is monitored by 2 stations for discharge measurements on the main river, 5 rain/snow stations, 2 climatic stations, and 3 automatic stations. List of the stations is given below in table 1.

Automatic Station with raingauge type :
1. Tipping bucket with heating (measures the precipitation with increments of 0.1 mm) - all but Rozhen and Hvoyna - quality index of the measurements (between 1 and 10) - 7-8.

Table 1. Meteorological stations used in the modeling and verification

Station with raingauge type :
1. Tipping bucket with heating (measures the precipitation with increments of 0.1 mm) - all but Rozhen and Hvoyna - quality index of the measurements (between 1 and 10) - 7-8.

Study area river Chepelarska watershed down to the hydrometric station Bachkovo

Tested area is situated in the western Rhodopi Mountain. The study area is a part of Chepelarska river and the watershed is 824.9 km2. Chepelarska reka, also called Chaya takes its source from Snezhanka Peak (around 1,700 m above sea level) in the Rhodopes. It is one of the three largest rivers in the Rhodopes. The river's length is about 87 km. The Chepelarska river runs through the town of Chepelare, Bachkovo, as well as the second largest city of Plovdiv province Asenovgrad before flowing into the Maritsa (fig.1). The average altitude of the watershed is 1241m. Bachkovo village is 30.6 km from the mouth of the river. The largest part of the area is occupied by forest habitats consisting mainly of coniferous and mixed forests.

Fig. 1. Study area and location of the meteorological stations

Precipitations are varying over the basin from 600 to 1200 mm per year according to the altitude and with strong Nord-South gradient. Potential evapo-transpiration is below 700 mm per year, decreasing to 500 mm with altitude. Therefore the available runoff water is above 50 up to 100 mm per year.

The main discharge [m3/s] statistical characteristics for the period 1961 - 1998 for the river Chepelarska at Bachkovo are given in table 1.

Table 1. Statistical characteristics

Monthly distribution of the river flow for the same period in % is given in table 2.

Table 2.

Table 3. Meteorological stations used in the modeling and verification

Station with raingauge type :
1. Tipping bucket with heating (measures the precipitation with increments of 0.1 mm) - all but Rozhen and Hvoyna - quality index of the measurements (between 1 and 10) - 7-8.
2. Weighing type measurement with heating rim (measures the precipitation with increments of 0.1 mm) - Rozhen and Hvoyna stations - quality index of the measurements (between 1 and 10) - 8-9.

Study area river Iskar watershed up to the hydrometric station Novi Iskar

The study area is part of the Iskar river basin up to the hydrometric station Novi Iskar. The Iskar is the largest Bulgarian tributary of the Danube, with a total length of 368 km and a catchment area of 8 684 km². Its headwaters lie up in the passes of the Rila Mountain. The mainstream flows through the outskirts of Sofia, and through the Balkan Mountains. The study area is a part of Iskar river watershed and is 3558,462 km2. The main tributaries are Lesnovska river, Vladaiska river, Bankenska river and Blato river (Fig. 1).

Fig. 1. Study area and location of the meteorological stations

Two big towns are on the this part of the river - Sofia and Novi Iskar. The adequate and on time flood forecasting information is very important in flood prevention of these towns. Floods were a major problem for Bulgaria in 2005. In study area the biggest floods occurred in June and August 2005.

Artificial Neural Networks (ANN) are applied for forecasting of streamflow at a given hydrologic station (18700 r. Iskar, Novi Iskar) based on observations at the hydrometric station and rainfall values at rain gauge stations located in the study catchment. Artificial Neural Networks treat the hydrological system as a black box and try to find a relationship between historical inputs (rainfall, temperature, etc.) and outputs (runoff). Recently Artificial Neural Networks are widely used as a potentially useful way for modeling nonlinear system. The prediction of variables as runoff, river levels etc. are major problem in hydrology.

Flow in rivers is extremely complex process that is influenced by many factors such as watershed topology, vegetation cover, soil types, river bad characteristics, groundwater aquifers, precipitation distribution, snowmelt, rural and urban activities.

The study area is located in West Bulgaria at foot of Stara Planina Mountain. The climate is moderate continental.

Annual accumulated precipitation is approximately 600 mm. The highest rainfalls occur in months of May and June and are about 85 mm. The lowest rainfalls occur in February and March and are about 30 mm. Winter accumulated precipitation is approximately 110 mm. Average snow pillow is about 10 cm. and days with snow are about 50 - 80 days per year. Accumulated precipitation during spring season is about 160 mm, during summer is about 180mm and during autumn about 140 mm.

Mean annual temperature is +80C - +90C. Mean temperature during winter is about - 20C, during spring is about +90C, during summer is about +200C and during autumn is about +100C.

The wet period is during spring and beginning of the summer. The dry period is during summer and autumn. Winter specific discharge is 4 l/sec/km2, spring specific discharge is 7,5 l/sec/km2, summer specific discharge is l/sec/km2, and autumn specific discharge is 0,5 l/sec/km2.

Daily mean discharge data measured at the hydrological station, daily accumulated precipitation data and daily average temperature measured at the meteorological stations are used for ANN modeling. The length of records is 15 years covered period 1991-2006 (Table 1).

Table 1. Meteorological stations used in the modeling and verification

The measurements are provided using precipitation gauges type Wild. The gauges are placed at a fixed height - 1 m above a ground surface and measure liquid (rain) and solid precipitation and have no wind shields.The site of official precipitation gauges are chosen to minimize the wind's disturbing influences on the precipitation measurements. The PGs measure daily precipitation total observed one per day at 7 o'clock in the morning. In case of solid precipitation the quantity is measured by melting.

The average density of the network is about 20 - 50 km².

GERMANY
Name of the institute: Federal Institute of Hydrology (BfG)
Link of the institute: www.bafg.de
Contact point for the PP validation activity: Thomas Maurer: thomas.maurer@bafg.de, Peer Helmke: helmke@bafg.de
Institute PP validation activity description

The products PR-OBS-1, PR-OBS-2, PR-OBS-3, PR-OBS-5 have been validated by comparison with rain rates estimated from rain gauge data and ground based radar calibrated with rain gauge measurements.

Products from regular processing chain and from reprocessing have been evaluated by means of continuous statistics and selected case studies. Case studies were selected for periods of outstanding performance to demonstrate maximum possible impact at that stage of development.

Ground data used with instrument description and map

Rain gauge

A network of about 1400 rain gauge stations was used to validate H-SAF precipitation product. Synop, ttrr and climate stations were jointly evaluated to set up a data set of high spatial density.

Radar Data

Radar Data

As an alternative source of validation data RADOLAN RW (Routine procedure for an online calibration of radar precipitation data by means of automatic surface precipitation stations "ombrometers") was used in the German validation activities.

RADOLAN is a quantitative radar composite product provided in near-real time (via ftp) by DWD to BfG. Radar data are calibrated with hourly precipitation data from automatic surface precipitation stations. For a description of the radar network see http://www.dwd.de/RADOLAN. The process chain from the five-minute-interval radar signals to the final hourly precipitation product is presented in the following figure. RADOLAN data of hourly precipitation (sampling period hh:51 min to (hh+1):50 min) have a precision of 0.1 mm/h and cover the whole territory of Germany with a spatial resolution of 1 km.

RADOLAN

Ground data quality

Both types of ground data, rain gauge and radar, are checked for plausibility and data consistency.

HUNGARY
Name of the institute: Országos Meteorològiai Szolgálat-Távérzékelési Osztály- (HMS)
Link of the institute: www.met.hu
Contact point for the PP validation activity: Eszter Lábò: labo.e@met.hu
Institute PP validation activity description
Common validation tasks within the HSAF Precipitation Products Calibration/Validation Group

The common task of the Precipitation Validation Group is the statistical validation. The up-scaling method has a key role in this process (see paragraph "Time and space alignment"). We can only compare the radar, satellite, rain gauge, and lightning if we have previously determined the coherent values from different sources. The data is collected into one file containing the following information to calculate statistic values for each pass of the DSMP and NOAA satellite over Hungary for H01 and H02; for 15 minutes samples for H03, and for accumulated total precipitation information (over 3, 6, 12, 24 hours) for H05:

- radar-derived precipitation intensity
- the satellite product values derived by H-SAF.

Statistical continuous scores (SME, SAME, CORR, STD) are calculated between the satellite-based precipitation and radar data for each month since December 2007.

We also generate multi-categorical tables according to the classes determined by common agreement by the validation group and the hydrologists. The rain rates are divided into 11 classes. Statistical rain/no-rain scores (ACC, POD, FAR, HSS, CSI, TSS) are calculated between the satellite-based precipitation and radar data for each month since December 2007.

Besides, distribution functions are created for each month. The number of cases fallen into each category is presented for both satellite and radar data. We have to be aware that it only gives good information on the rain rate distribution according to the satellite and the radar separately, and the results cannot be compared within the categories themselves.

Time and space alignment

Time and space alignment of the data from different sources, represented on different scales - pixel size of radar and satellite data is not the same. Especially in the case of microwave measurements, we have large footprints that need to be treated carefully. The time alignment can be solved by simply matching the closest data available in time to the H-SAF products (this causes maximum 5-minutes divergences). There are advanced methods to solve the up-scaling of ground and/or the down-scaling of satellite precipitation data. The OMSZ-Hungarian Meteorological Service uses the techniques discussed and proposed by the members of the validation group: the radar data for comparison with microwave measurements (H01 and H02) is up-scaled by use of Gaussian weighting functions (Eszter Lábò, Judit Kerény: Precipitation products: Calibration and Validation case studies in Hungary, FIRST H-SAF WORKSHOP, poster presentation). For H03 and H05, the radar pixels on the satellite pixel are averaged (4 to 6 radar pixels on one MSG pixel).

Independent validation tasks of the OMSZ

As the visual comparison between radar and satellite data is an inevitable tool for the understanding of the validation results of H-SAF products, because it:

- provides an overall picture of the H-SAF products (resolution, territories seen, general values),
- highlights similarities and differences in the structures and in the values,
- helps to monitor the meteorological situation as well.

Therefore, we constantly prepare case studies for Hungary in different meteorological situations.

- The first task in the preparation of case studies is the selection of the periods with significant rain amounts. This is done by the plotting of the radar and satellite rainfall estimate sums on a daily basis of each month.
- The second task to the preparation of case studies is the collection of all relevant data to the period in question. This means the collect of radar, lightning data, and of MSG images (10.8µm or Airmass RGB) and SAFNWC products (Cloud Type, Precipitable Clouds) which correspond to the time period investigated.

To visualize the different products and to make subjective comparison we use the Hungarian Advanced Weather Workstation (HAWK) developed by OMSZ-Hungarian Meteorological Service which can interpret all kind of meteorological products together.

Ground data used with instrument description and map

In Hungary, about 90 automatic stations work, where 10-min precipitation is measured by tipping bucket rain gauges. This data is used to correct the accumulated precipitation radar data

The automatic rain gauge network in Hungary

The main data used for validation in Hungary is the data of meteorological radars. There are three C-band dual polarized Doppler weather radars operated routinely by the OMSZ-Hungarian Meteorological Service.

The location and coverage of the three meteorological Doppler radars in Hungary

Year of installation Location Radar type Parameters measured
1999 Budapest Dual-polarimetric Doppler radar Z, ZDR
2003 Napkor Dual-polarimetric Doppler radar Z,ZDR,KDP,ΦDP
2004 Poganyvar Dual-polarimetric Doppler radar Z,ZDR,KDP,ΦDP
Ground data products used

Instantaneous precipitation data

Precipitation intensity is derived from radar reflectivity with the help of an empirical formula, the Marshall-Palmer equation. From the three radar images a composite image over the territory of Hungary is derived every 15 minutes applying the maximum method in order to make adjustments in overlapping regions.

Accumulated precipitation data

For the 3h and 6h accumulated products, we use a special method as well: we interpolate the 15-minutes measurements for 1-minute grid by the help of displacement vectors also measured by the radar, and then sum up the images which we got after the interpolation. It is more precise especially when we have storm cells on the radar picture, because a storm cell moves a lot during 15 minutes and thus we do not get continuous precipitation fields when we sum up only with 15.minutes periods. This provides satisfying results. However, there is still a need for rain-gauge adjustment because there are obviously places (behind mountains) that the radar does not see.

Correction by rain gauges

The non-corrected precipitation field can be corrected by rain gauge measurements. As recent researches have shown it is only possible to produce adequate precipitation fields by the correction of raw radar data at time scales of the order of a few hours or more, thus we do not make corrections to 15 minutes radar data. In our institute, we only use a correction for the total precipitation over a 12 hour period.

The radars are corrected with rain gauge data every 12 hours and 24hours. The correction method using rain gauge data for 12 hour total precipitation consists of two kinds of corrections: the spatial correction which becomes dominant in the case of precipitation extended over a large area, whereas the other factor, the distance correction factor prevails in the case of sparse precipitation. These two factors are weighted according to the actual situation. The weighting factor depends on the actual effective local station density, and also on the variance of the differences of the bias between radar and rain gauge measurements. On the whole, we can say that our correction method is efficient within a radius of 100 km from the radar. In this region, it gives a final underestimation of about 10%, while at bigger distances; the underestimation of precipitation fields slightly increases. Besides, we also produce 12 hour total composite images: first the three radar data are corrected separately, and then the composite is made from them. The compositing technique consists of weighting the intensity of each radar at a given point according to the distance of this point from the radars.

Ground data quality

The quality of the radar measurements is influenced by mainly two factors:

- the accuracy of the Marshall-Palmer equation in deriving rain rates;
- the beam blockage which causes the lack of precipitation in areas behind mountains.

The Hungarian radar data bears the consequences of both problems. The Marshall-Palmer calculations can result in a factor of multiplying or dividing by 2; whereas the beam blockage can result in serious underestimation of precipitation amounts (e.g. behind the Börzsöny mountains at the north of Budapest).

Besides, the Hungarian radar data is filtered from WLAN signal, which is also a source of false signals in the radar.

A filter to disregard signals below 7dBz is also applied because in general, these data is not coming from real rain drops, but false targets.

ITALY
Name of the institute: Università di Ferrara (UniFe)
Link of the institute: www.unife.it/dipartimento/fisica
Contact point for the PP validation activity: Federico Porcù: porcu@fe.infn.it - Marco Petracca: marco.petracca84@libero.it
Institute PP validation activity description

The calibration and validation activity at the Department of Physics of the University of Ferrara is carried on by using as ground reference the raingauge Italian network, made available by the Italian DPC. The dataset, described below, is pre-processed in order to get an equispaced using the Barnes approach (Barnes, 1964) as revised by Koch et al (1983).

The Barnes objective map analysis scheme is a computationally simple, Gaussian weighted-averaging technique which assigns a weight to a datum solely as a known function of distance between datum and grid point. The weight wm in assigned according to the distance rm between the datum f(xm,ym) and the (i,j)-th grid point as wm = exp (-rm2/k), where k is a parameter that determines the shape of the filter response function. Since the Barnes scheme is Gaussian in nature, then an "influence radius" R may be thought as that radius where wm = e-1.

The sensitivity parameter (k) in the Barnes procedure has been set to 108, which is 104 squared, considering the fact that the mean distance between every station and the closest one is roughly 104 m. The interpolation procedure is iterative. If the mean squared difference between the source field and the interpolated field falls below 0.01 mm2, or if the improvement (decrease of the mean squared difference between source field and interpolated field) is below 1% between two steps, the procedure is stopped, otherwise it goes on and is stopped after 20 steps.

The result is a series of files with interpolated data, one per hour. The output of the interpolation is a 5x5 km equispaced grid on a 288 lines by 240 column matrix. A total of about 11,000 grid-points over land have been made available for comparison with all the H-SAF products.

The statistical parameters requested for validation are computed by matching satellite estimates with corresponding interpolated raingauges grid, using different approaches for the different H-SAF precipitation products.

The H-SAF precipitation products are regularly downloaded from the CNMCA ftp-site and locally stored in BUFR format. The data pre-processing includes the de-BUFR of the data and the selection of the part of satellite overpass over the area covered by the raingauges grid.

References

Barnes, S.L.,1964: A technique for maximizing details in numerical weather map analysis. J. Appl. Meteorol., 3, 396-409.
KOCH, S. E., M. DESJARDINS, P. J. KOCIN, 1983: An interactive Barnes objective map analysis scheme for use with satellite and conventional data. - J. Climate Appl. Meteor. 22, 1487-1503.

Ground data used with instrument description and map

The dataset has changed during the project, in terms of number of stations, sampling time and data quality/availability. Since January 2009, the dataset has stabilized a little more, and its basic features are summarized in the table below.

Number of stations 1500 to 1800
Type of instrument Tipping bucket raingauge
Sensitivity of the instrument 0.1 or 0.2 mm/h
Average distance between nearest stations 9.5 km
Data failure ~ 10%
Sampling time 60 minutes
Italian raingauge network main characteristics

The network covers now the whole Italian territory, although with non homogeneous density, including the major Islands, as it is shown in figure below, where green points indicate the position of the raingauges.

Figure 1. Distribution of the raingauges of the Italian raingauge network

The data are archived and made available on a DPC ftp server on monthly basis, after a variable number of weeks from the month end.

The product from the raingauge network is the ran amount cumulated over 60 minutes.

Ground data quality

The data are quality checked by the regional administrations who take care of the network in the Italian Regions, before to send the data to the DPC. No further processing is made.

POLAND
Name of the institute: Institute of Meteorology and Water Management (IMWM)
Link of the institute:
Contact point for the PP validation activity: Bozena Lapeta: bozena.lapeta@imgw.pl
SLOVAKIA
Name of the institute: Slovak Hydrometeorological Institute (SHMÚ)
Link of the institute:
Contact point for the PP validation activity: Ján Kaňák: jan.kanak@shmu.sk
Institute PP validation activity description

Ground data used in Slovak Hydrometeorological Institute for validation of H-SAF precipitation products

Data Sources raingauges radars
instrument characteristics telemetry and mechanic 2 radars: beam width 1 degree, radial resolution 1000 m, C-band doppler radars
time resolution Depends on type; 1 min, 10 min, 1hour, 6, 12, 24 hours precipitation intensity every 15 min till Aug 2009, 10 min after Aug 2009 cumulative precipitation 1, 3, 6, 24 hours
spatial distribution whole national territory, see attached map for distribution of gauges whole national territory, see attached map of radar network coverage; limitation in some regions due to terrain obstacles
number of stations total 684 gauges,98 operational, 586 climatological, 37 hydrological 2 radar sites
data quality check in automatic analysis unreasonable data are removed from processing visual control on regular basis

Map of raingauge stations used for calibration/validation of HSAF products in Slovak Hydrometeorological institute.
- operational stations (98), blue - climatological stations (586), red - hydrological stations (37).

Map of Slovak meteorological radar network: Shadowed big circles show maximum radar range (240km for Malý Javorník and 200km for Kojšovská hoľa radar site). Dark smaller circles show precipitation effective radar range (100km)

Real coverage of Slovak radar network for precipitation measurement estimated by digital terrain model.

TURKEY
Name of the institute: Istanbul Technical University, Meteorology Department (ITU), Middle East Technical University (METU)
Link of the institute:
Contact point for the PP validation activity: Ahmet öztopal: oztopal@itu.edu.tr Zekai Şenz: zsen@itu.edu.tr
Institute PP validation activity description
Introduction

Due to the time and space structure of precipitation and to the sampling characteristics of both the precipitation products and observations used for validation, care has to be taken to bring data into comparable and acceptable range. At a given place, precipitation occurs intermittently and at highly fluctuating ratesVarious maps, time series analysis, statistical and probabilistic methodologies are employed in the validation procedure classically, but some additional new aspects such as the spatial coverage verification model of point semivariogram (PSV) approach (Şen and Habib, 1998) are proposedfor usagein this work.

On the other hand, calibration includes philosophical aspects concerning the use of tools for validation (raingauges, radar, numerical models …) and their relative merits; techniques to bring observations comparable including upscaling, downscaling, etc and structuring of the results of the validation activity.

Ground Observation Sites

193 Automated Weather Observation Station (AWOS) located in the western part of Turkey are used for the validation of the precipitation products. The location of the AWOS sites is shown in Figure 1.

Figure 1. Spatial location of the 193 Awos sites used for the cal/val activities

Various numbers of meteorological and agricultural parameters are observed in each Awos site depending on the site type. Temporal resolution of each observation is either 1 minute or 10 minutes. Among those, precipitation parameter is observed in every Awos site with 1 minute the temporal resolution.

Ground Data Quality

High quality of the ground data is critical for performing the validation of the precipitation products. The validation results or statistics can provide meaningful feedbacks for the product developers and additionally the products can be used reliably only if there is a confidence present about the ground data at a certain level. For this reason, some predefined quality assurance (QA) tests are considered for the precipitation data in order to define the confidence level. First of all, a flagging procedure is defined as described in Table 1.

QA Flag Value QA Status Brief Description
0 Good Datum has passed all QA Test
1 Suspect There is concern about accuracy of datum
2 Failure Datum is unstable

Table 1. QA flags descriptions (modified from Shafer et al. (1999))

The predefined QA test for the precipitation data are as follows.

Range Test

Range Test is used to see if any individual precipitation observation falls within the climatological lower and upper limits. The test procedures applied for the test are as follows.

IF LimLower = < Obserj,t <= LimUpper THEN Obserj,t flag is "Good"

IF Obseri > LimUpper OR Obserj,t < LimLower THEN Obserj,t flag is 'Failure'

LimLower and LimUpper thresholds are separately determined for each station on a monthly basis. At any specific site, all the observed monthly data is considered for determination of the upper and lower limits. By applying this test, each observation is flagged either by "Good" or "Failure" label depending on the comparison tests mentioned above.

Step Test

Range test is used to see if increment/decrement between sequential observations in time domain is in acceptable range or not. The test procedure applied for this test is,

IF |Obserj,t-Obserj,t-1| < Stepj THEN Obseri,t flag is "Good"

IF |Obserj,t-Obserj,t-1| > Stepj THEN Obseri,t flag is "Suspect"

Stepj threshold is determined again for each site on a monthly basis. For each site, the dataset containing the absolute difference of the sequential observations is determined by considering the observations for the matching month. The 99.9 % cumulative histogram value of the dataset is set as the Stepj threshold for the related site and the month.

Persistence Test

Persistence test is used to determine if any group of observations are due to instrument failures. The test procedure applied is defined as,

IF T < Δ THEN Flag for all Obser in T : "Good"

IF T > Δ THEN Flag for all Obser in T : "Suspect"

where T is the total number of the sequentially repeating observations forward in time and Δ is the possible maximum number of sequentially repeating observations. As in the other two tests, Δ threshold is determined for each site on a monthly basis. For any site, the data belonging the same month is taken into account to determine the repeating number of the sequential observations. Then, 99.9 % cumulative histogram value of the repeating number dataset is assigned as the Δ amount for the corresponding site and the month. Since there is a high possibility of having no-precipitation data (zero), the sequential zero observations are excluded in this test during the determination of the Δ threshold amount and application of the test.

QA Test procedure

By applying the control procedures of the QA test mentioned above, each individual precipitation observation receives three flags, each referring to the corresponding test. For the corresponding observation if all the test flag is not "Good" then the observation is excluded not to participate in the validation process.

Validation Methodology

Each precipitation product within the H-SAF project represents a foot print geometry. Among these, H01 and H02 products represent an elliptical geometry while H03 and H05 represent a rectangular geometry. On the other hand, the ground observation (rain-gauge) network consists of point observations. The main problem in the precipitation product cal/val activities occurs in the dimension disagreement between the product space (area) and the ground observation space (point). To be able to compare both cases, either area to point (product to site) or point to area (site to product) procedure has to be defined. The area to point comparison seems easier. The basic assumption in such approach is that the product value is homogenous within the product footprint. An illustration is shown in Figure 2 where satellite foot print (FOV) centers of the H02 product, an elliptical footprint for the corresponding center (area within the yellow dots) and Awos ground observation sites are presented. The comparison statistic can be performed just by taking the sites in the footprint area into consideration. This approach is reasonable on the average but less useful in representing the spatial variability of the precipitation. The comparison is not possible in case of not having any site within the footprint area.

Figure 2. H02 product footprint centers with a sample footprint area as well as the Awos ground observation sites.

Alternatively, the point to area approach is more appealing for the realistic comparison of the precipitation product and the ground observation. This approach is simply based on the determination of the true precipitation field underneath the product footprint area. To do so, the footprint area is meshed and precipitation amounts are estimated at each grid point by using the precipitation observations at the neighboring Awos sites as shown in Figure 3. A 3x3 km grid spacing is considered for the products with elliptical geometry while 2x2 km spacing is considered for the products with rectangular geometry. At each grid point, the precipitation amount is estimated by,

(1)

where Zm is the estimated value and W(ri,m) is the spatially varying weighting function between the i-th site and the grid point m

Figure 3. Meshed structure of the sample H02 product footprint.

Determination of the W(ri,m) weighting function in Equation 1 is crucial. In open literature, various approaches are proposed for determining this function. For instance, Thiebaux and Pedder (1987) suggested weightings in general as,

(2)

where R is the radius of influence, r is the distance from center to the point and a is a power parameter that reflects the curvature of the weighting function. Another form of geometrical weighting function was proposed by Barnes (1964) as,

(3)

Unfortunately none of these functions are observation dependent but suggested on the basis of the logical and geometrical conceptualizations only. They are based only on the configuration, i.e. geometry of the measurement stations and do not take into consideration the natural variability of the meteorological phenomenon concerned. In addition, the weighting functions are always the same from site to site and time to time. However, in reality, it is expected that the weights should reflect to a certain extent the regional and temporal dependence behavior of the phenomenon concerned.

For the validation activities, the point cumulative semi-variogram technique proposed by Şen and Habib (1998) is used to determine the spatially varying weighting functions. In this approach, the weightings not only vary from site to site, but also from time to time since the observed data is used. In this way, the spatial and temporal variability of the parameter is introduced more realistically to the validation activity.

References

Barnes, S.L. 1964: A technique for maximizing details in numerical weather map analysis, J. App. Meteor., 3, pp.396-409.

Thiebaux, H.J. and Pedder, M.A. 1987: Spatial objective analysis, Academic Press, 299 pp.

Sen Z and Habib, Z. Z. 1998: Point cumulative semivariogram of areal precipitation in mountainous regions, J. Hydrol., 205, 81-91.

Shafer, M.A., C.A. Fiebrich, D.S. Arndt, S.E. Fredrickson, and T.W. Hughes, 1999: Quality assurance procedures in the Oklahoma mesonet. J. Atmos. Oceanic Technol., 17, 474-494.

CONTACTS
Main Contact Point Silvia Puca: silvia.puca@protezionecivile.it
Web page contact point Emanuela Campione: emanuela.campione@protezionecivile.it
Corrado De Rosa: corrado.derosa@protezionecivile.it
Product Validation and Value Assestment Silvia Puca: silvia.puca@protezionecivile.it
Emanuela Campione: emanuela.campione@protezionecivile.it
Product Monitoring and NRT feedback Veronica Casartelli: veronica.casartelli@protezionecivile.it
Emanuela Campione: emanuela.campione@protezionecivile.it
Provisioning of Ground Data Gianfranco Vulpiani: gianfranco.vulpiani@protezionecivile.it
Antonio Gioia: antonio.gioia@protezionecivile.it
Corrado De Rosa: corrado.derosa@protezionecivile.it
STATISTICAL ERROR DESCRIPTION
CONTINUOUS STATISTICS
Continuous verification scores for coast area

We indicate with:

  • sat1 sat2… satk… the precipitation values estimated by satellite products belonging class z in coast area
  • true1, true2,… truek,… the precipitation values observed by radar/rain gauges inside class z in coast area
  • N the number of satellite /radar/rain gauge derived precipitation cases belonging to precipitation class z in coast area