Being certain of uncertainty: Getting the most from weather and climate forecast models.

Take home message

• The Bureau of Meteorology’s Agriculture Program has been engaging with the grains industry and responding to feedback from growers and advisers since 2017. Research and face-to-face engagement has shown that there is a need for insights into computer models, both short-term weather models, and long-term climate models
• With funding from Agricultural Innovation Australia, through investment from the Grains Research Development Corporation (GRDC) and other rural Research and Development Corporations (RDCs) via the Agri-Climate Outlooks (ACO) project, a dedicated service has been established which publishes region and commodity-specific forecasts on Youtube to assist with understanding of long-term outlooks beyond 7-days. . The Agriculture Program delivering the Agri-climate outlooks project aims to positively impact on-farm business management and grains-specific decisions such as autumn sowing.
• The insights captured by the Agriculture Program and continued engagement with industry have shaped the decision support service to date. The case studies presented outline the value of  drawing on the expertise of the Bureau, which in turn helps to ground-truth the long-term forecast for improved accuracy.
• As growers and advisors plan for the upcoming autumn season, the Agriculture Program will continue to provide relevant insights that bridge the gap between short-term forecasts and long-term forecasts. We invite growers and advisors to keep up to date with the analysis of the forecasts, by subscribing to the grains climate video briefings (Bureau of Meteorology Agriculture YouTube playlist) and by getting in touch with the team via email at agriculture@bom.gov.au, or in-person at field days and seminars.

Background to computer models

Computer models are the main tool that meteorologists use to forecast weather and climate. They divide the atmosphere into grid boxes, both vertically and horizontally, in order to simulate the atmosphere and project forward in time. Computer models can provide forecasts within the next hours to days (short-term models) to weeks and months out with probabilistic scenarios (long-term models).

Today, there are many different computer models, each requiring immense supercomputing power. Some of the large meteorological agencies across the world produce their own models. Table 1  shows some of the major short-term and long-term computer models currently in operation, including the ACCESS Australian model. ACCESS stands for Australian Community Climate and Earth-System Simulator, and it is based on the UK Met Office’s Unified Model.

Table 1. Comparison of different international computer models.

Results

The Agriculture Program has recently increased its service to the agriculture sector with investment in the Agri-Climate Outlooks (ACO) project from the agriculture industry and other RDCs, including GRDC. Research conducted by the Agriculture Program has found that there is a need for insights into computer models, both short-term weather models, and long-term climate models. These insights can provide additional perspectives when making risk management and high-value decisions.

The meteorologists in the Agriculture Program, through understanding the physics of the atmosphere and the complexity of computer models, aim to provide insights that connect the physical interactions of the atmosphere to modelled scenarios. In doing this, they  provide targeted insights to inform high-value decisions.

This is demonstrated in a specific NSW case study from November 2023 (see below). This case study demonstrates the limitations of using one source without meteorological analysis or understanding of the biases within the computer modelling systems for weather and climate risk assessment.

Which computer model performs best, and why do they tell different stories?

There is no one computer model that can be definitively described as being 'the best'. Each model has its own limitations and biases which in turn affect its accuracy. This is true for both short-term and long-term models. In a perfect world, a computer model would simulate the atmosphere and ocean perfectly, with observations feeding in from every point on earth. This would require computing power and resources well beyond our current capabilities. Instead, computer models use approximations, while other factors come into play. These factors can include:

• Grid box size and model resolution, as shown in Table 1
• Geographic region, including topography and representation of the land
• Initial conditions used
• Timeframe for forecast
• Equations used to represent atmospheric processes
• Computing power and availability of resources

Based on these factors, models can produce different outcomes for the same weather or climate events. Different models and resources will be covered in more detail during the presentation.

Case study: Storm outbreak across eastern Australia – November 2023

The Agriculture Program attended the Cropping Solutions Seminar in Narrabri in July 2023. Through conversations with growers and advisers at the seminar and on the ground, staff learned that Yr.no is a popular online resource for grain growers in New South Wales as it provides an 'exact' expected rainfall amount up to 10 days ahead. However, there is not a widespread understanding of how these forecasts are produced, nor of the computer models behind the forecast, which can limit their effectiveness in decision making.
An example of this came from canola growers near Narrabri who relied on this online source of information for a rainfall forecast more than 7 days in advance. These growers made decisions to sow 1,000 ha of canola based on this one rainfall forecast. As the day of the forecast rainfall approached, the rainfall expectation fell dramatically (from over 60 mm to just 2 mm) and this resulted in significant losses due to non-viable crops.

In contrast, the Bureau's 7-day forecast uses a blend of computer models that have been analysed by meteorologists, and so the rainfall forecast shows a probable range rather than one single rainfall amount. Beyond 7-day forecasts, the Bureau's ACCESS-S climate model displays long-term forecasts that show rainfall totals for a given level of chance, and various other displays for example, the chance of exceeding a certain rainfall amount over 3 days. Long-term forecasts are probability-based and are designed to be used as one of several planning tools within risk management and decision-making. The greatest benefits of using Bureau long-term forecasts will accrue from use over several seasons or years. The  meteorologists in the Agriculture Program use a range of long-term forecasts that include other international models as part of the analysis to provide a forecast beyond 7 days.

To demonstrate the limitations of using one source without meteorological analysis or understanding of the biases within the computer modelling systems for weather and climate risk assessment, a case study was developed that investigated the effectiveness of forecasts from Yr.no, in November 2023. This was to demonstrate how one model forecast can change significantly over time, and therefore not provide the best decision outcome if used in isolation without long term analysis and verification to confirm the model simulation.

In late November 2023, several surface troughs combined with tropical moisture to produce rain and storms over eastern Australia, peaking on 24–25 November. An analysis of forecasts displayed on Yr.no captured how the forecasts changed at Dubbo and Tamworth, and was then compared to observed rainfall amounts.

Forecast on Yr.no

Figure 1 shows how the forecast for 25 November evolved at Dubbo and Tamworth between 20 November (5 days before the storm event) to the day prior.

At Dubbo (top line) the forecast indicated more than 20 mm until 24 November, at which point the forecast fell to 10 mm. Similarly in Tamworth (bottom line), the forecast rain amount dropped in the lead up to 25 November.

The reason why the Yr.no forecast jumped around is because thunderstorms are localised events and are spatially much smaller than the grid resolution of many models. Computer models only give the forecast for a point location based on how the model has resolved precipitation at that grid point. However, actual observed rain may form 5 or 50 km away. A computer model should be viewed as simulating the environment.

Figure 1. Evolution of the forecast at two different locations on Yr.no (taken from yr.no between 20 and 24 November 2023)

Bureau of Meteorology forecast

The grains climate video update (November Grains Climate Outlook), released on 8 November 2023, discussed the increased risk of rain and storms across eastern Australia more than 2 weeks in advance, during the second half of the month, due to surface troughs and moisture from the north. This included annotations within the video which helped to explain the outlook and provide context. Specifically, the video stated that "the inland trough over inland parts of Queensland and NSW… [would] lead to an increase in shower and thunderstorm activity. Figure 2 shows a screenshot lifted from this video.

This video update went further and compared both the Australian ACCESS-S climate model, as well as the international range of long-term climate models. By doing this, the Agriculture Program provided a scientifically rigorous method of analysis, in addition to a meteorologist's understanding of the interactions of weather systems, such as airmass evolution and surface low pressure troughs.

Figure 2. AgDS video briefing screenshot from 8 November 2023 (Bureau of Meteorology Agriculture YouTube).

Rain observations

Rain observations from 25 November 2023 were as follows:

• Dubbo 0.2 mm
• Tamworth 31 mm

Thunderstorms developed during the day and swept over parts of the Murray-Darling Basin. The Bureau issued Severe Thunderstorm Warnings for broad areas of NSW. Figure 3 below demonstrates how some locations received high totals, for example Wellington, NSW (38mm) while others nearby received very little, for example Dubbo (0.2mm), which is 46 kilometres from Wellington.

Figure 3. 24-hour rainfall totals for 25 November 2023 (observations taken from www.bom.gov.au).

Findings and conclusion

There are many computer models and resources available to growers and advisers today. Some of these can be popular as they provide the information that growers seek. Over-reliance on resources that provide a quantitative numerical forecast outcome based on one model only, without analysis or verification, can increase the error in decision assessment or risk analysis. This is demonstrated in the above example of canola sowing in northern NSW, and the case study of the outcome from the popular website, yr.no.

To gain a fuller perspective on weather and climate for risk management decisions, it is important to consider additional analysis, rather than to rely on one resource. The Bureau uses a blend of computer models to provide a rigorous analysis, and weather-dependant organisations are encouraged to use a range of sources as well. In particular, the Bureau's Agriculture Program staff can provide insights that connect the physical interactions of the atmosphere to modelled scenarios, with the objective of providing the best information to support weather dependant agriculture businesses to understand weather and climate-related risk scenarios. The role of meteorologists is to understand the complex science of weather and climate and provide additional information to support on-farm high-value decisions.

The Agriculture Program will continue to engage with the agriculture industry in 2024 to provide direct, relevant weather and climate analysis and insights. Staff will support research to understand model bias, verification, and model physics in international climate models where appropriate. Growers and advisors are invited to keep up to date with the analysis and the long-term forecast, by subscribing to the Grains Climate Outlook video briefings, and by contacting the team via email at agriculture@bom.gov.au, or in-person at field days and seminars.

References

Bureau of Meteorology Agriculture YouTube playlist.

November Grains Climate Outlook - NSW & Qld - YouTube

Yr.no. Linked accessed 20-25 November 2023.

Acknowledgements

The research undertaken as part of this project is made possible by the significant contributions of growers through both trial cooperation and the support of the GRDC, the author would like to thank them for their continued support.

The Bureau of Meteorology, in partnership with Agricultural Innovation Australia (AIA) are delivering the 4-year ACO project, which involves collaboration across 10 rural Research and Development Corporations. This includes the Grains Research Development Corporation (GRDC). This project will help to deliver improved seasonal climate and water outlooks for use by the agricultural sector, and create value by helping producers make better, and more informed decisions.

Contact details

Jonathan How
Bureau of Meteorology
700 Collins St Docklands 3008 Vic
Ph: 03 9669 4000
Jonathan.how@bom.gov.au

Date published
February 2024