Dear Colleagues,

I would like to bring to your attention the UQ workshop that will be held this September in Italy. Below is the workshop description copied from the workshop website available at https://frontuq18.org/.

The second  edition of the outreach workshop “Frontiers of Uncertainty Quantification” (FrontUQ) organized by the GAMM Activity Group on Uncertainty Quantification will be held on 5-7 September 2018 in Pavia, Italy, and will be focused on Uncertainty Quantification in Subsurface Environments (see this website for first edition of the workshop, which was held in Munich, Germany, September 6-8, 2017).

Subsurface environments host natural resources which are critical to the needs of our society and to its development. The diversity of problems encountered in subsurface modeling naturally calls for multi-disciplinary research efforts, with contributions from a wide range of fields including, e.g., mathematics, hydrology, geology, physics and biogeochemistry. Complex mathematical models and numerical techniques are then often required to tackle the simulation of coupled processes which are ubiquitous in relevant applications. In this context, our knowledge of the structure and properties of subsurface porous media is of critical importance to parameterize these models, but yet is incomplete, the first and simple reason being that the subsurface itself is not easily accessible to our direct observation. Estimation of input parameters is then plagued by uncertainty and so are consequently target output variables. Designing effective numerical methods dealing with uncertainties becomes crucial in every phase of the workflow, from the solution of inverse problems for the identification of the flow and transport properties to the forward propagation of the uncertainties, the sensitivity analysis and the design of management policies.  This workshop gathers researchers with different backgrounds active in the area, presenting both advances in dedicated uncertainty quantification techniques and real-world test cases.

Ming

Grand Challenges:

Substantial progress has been made in the last several decades for quantification and communication of hydrologic uncertainty (uncertainty quantification here is in a broad sense, including parameter estimation, sensitivity analysis, uncertainty propagation, and experimental data and data-worth analysis for uncertainty reduction). For example, due to development of public-domain software (e.g., PEST, UCODE, and DREAM), uncertainty quantification using regression and Bayesian methods has become a common practice not only in academic but also in consulting industry and govern agencies. However, the hydrologic uncertainty community are still facing several grand challenges that have not been fully resolved, and new challenges emerge due to changing hydrology and environmental conditions. Below are three grand challenges that may be addressed in the coming decade:

(1)               Software development: Software for supporting decision-making and for communicating uncertainty with decision-makers and stakeholders is still needed. Given that there have been a number of software in the public domain, it appears to be necessary to launch an effort for community software development such as building libraries of uncertainty quantification, visualization, and communication. An effort is also needed to closely collaborate with software developers of physical models, so that uncertainty quantification can be built as a module of the modeling software for efficient and effective operations.

(2)               Information and knowledge extraction from data: While new technologies of data across multiple scales collection are always needed, it is of tantamount importance to develop methodologies that can extract information and knowledge from data. This includes identification of new and overlooked data needs (e.g., water management data such as water use), revisit of existing data (e.g., those collected by NASA or NOAA but have not been analyzed), and development of machine learning and deep learning methods suitable to hydrologic research. Machine learning is a hot topic in many research fields, and its value to hydrologic uncertainty quantification (especially on reducing model structure uncertainty) has not been intensively explored. A particular need

(3)               Computationally efficient algorithms: Uncertainty quantification nowadays mainly relies on Monte Carlo approaches, which is computationally expensive particularly for new models that are more complex than models several decades ago. Computationally efficient algorithms (e.g., parallel computing and surrogate modeling) will enable us to conduct more comprehensive and accurate uncertainty quantification. The effort of algorithm development requires close collaboration with scientists in other disciplinaries such as applied mathematics, statistics, and computational science.         

We feel that the research field of hydrologic uncertainty is in its transition stage in two sense. First, substantial progress has been made in the past but we need to finish the last mile. For example, we have developed many methods for uncertainty quantification, but need to work on efficient and effective communication of uncertainty to decision-makers and stakeholders. In addition, we are facing new challenges of developing more advanced methodologies to make a full use of existing data and emerging computational hardware and algorithms.

Grey's view on grand challenge.

As for a ‘grand challenge’, I see the biggest open challenge as about how to build models that learn. That is, how do we leverage the power of machine learning and modern inference techniques for learning multi-scale physical and emergent principles in watersheds and complex systems? How do we construct models that allow for direct learning form data, but also allow us to prescribe what we do know about the biogeophysics of ecohydrological systems?

Grey's Writing on Game Changers for Hydrologic Uncertainty Analysis. This is a great start. We may build a list, and then select for the top three or top ten.

First successful automatic calibration of a hydrology model:
Duan, Q., S. Sorooshian, and V. Gupta (1992), Effective and efficient global optimization for conceptual rainfall-runoff models, Water Resour. Res., 28(4), 1015–1031,

First use of real machine learning (ANNs) for hydrological prediction:
Hsu, K., H. V. Gupta, and S. Sorooshian (1995), Artificial Neural Network Modeling of the Rainfall-Runoff Process, Water Resour. Res., 31(10), 2517–2530, doi:10.1029/95WR01955.

First computer-based land surface model:
Charney, J.G., Halem, M., and Jastrow, R. (1969) Use of incomplete historical data to infer the present  state of the atmosphere. Journal of Atmospheric Science, 26, 1160–1163.
Manabe, S., 1969. Climate and the ocean circulation. 1. The atmospheric circulation and the hydrology of the Earth’s surface. Mon. Weather Rev. 97(11), 739–774].

First global 1-km LSM or hydromet simulation:
Kumar, S. V., C. D. Peters-Lidard, Y. Tian, P. R. Houser, J. Geiger, S. Olden, L. Lighty, J. L. Eastman, B. Doty, P. Dirmeyer, J. Adams, K. Mitchell, E. F. Wood and J. Sheffield, 2006. Land Information System - An Interoperable Framework for High Resolution Land Surface Modeling. Environmental Modelling & Software, Vol. 21, 1402-1415.

Who had the first hydro DA paper? Jackson (1981) and Bernard (1981) apparently had the first direct insertion papers, but Milly had the first KF application
Jackson, T.J. et al. (1981) Soil moisture updating and microwave remote sensing for hydrological simulation.  Hydrological Sciences B., 26(3), 305–319.
Bernard, R., Vauclin, M., and Vidal-Madjar, D. (1981) Possible use of active microwave remote sensing data for prediction of regional evaporation by numerical simulation of soil water movement in the unsaturated zone. Water Resources Research, 17(6), 1603–1610.

Milly, P.C.D. (1986) Integrated remote sensing modelling of soil moisture: sampling frequency, response time, and accuracy of estimates. Integrated Design of Hydrological Networks – Proceedings of the Budapest Symposium, 158, 201–211.

The call for physically-based models to be used in application
Milly, P. C., Betancourt, J., Falkenmark, M., Hirsch, R. M., Kundzewicz, Z. W., Lettenmaier, D. P., & Stouffer, R. J. (2008). Stationarity is dead: Whither water management?. Science, 319(5863), 573-574.

Our community’s start to uncertainty quantification
Beven, K., & Binley, A. (1992). The future of distributed models: model calibration and uncertainty prediction. Hydrological processes, 6(3), 279-298.

Information theory for hypothesis testing
Gong, W., Gupta, H. V., Yang, D., Sricharan, K., & Hero, A. O. (2013). Estimating epistemic and aleatory uncertainties during hydrologic modeling: An information theoretic approach. Water resources research, 49(4), 2253-2273.

First multiparameterization model
Clark, M. P., Slater, A. G., Rupp, D. E., Woods, R. A., Vrugt, J. A., Gupta, H. V., ... & Hay, L. E. (2008). Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models. Water Resources Research, 44(12).

I attended a tele-conference today (3/12/2018) organized by Jeffrey McDonnell, the President of the AGU Hydrology Section, for the Section’s Technical Committee (TC) chairs. There are a number of items that I would like to share with you and, at the meantime, to ask for your inputs. 

AGU is planning for a number of activities for this year’s AGU Centennial celebration. One of them is to identify the breakthroughs that have been made in the last contrary. WRR (and actually all AGU journals) will have a special issues for hydrologic “game changers”, e.g., paradigm-shift concepts, innovative sensor techniques, and computational algorithms and/or software. The identification of breakthrough is more like a review of academic history of hydrology. How were the breakthroughs initiated? How did they get there? Which paper or papers were they originally published? What have we learned during the breakthrough-making process? For our TC, it would be interesting that we identify the game changers for hydrologic uncertainty analysis. Please come up with one or two breakthroughs, and justify why you think that they are truly breakthroughs.

Another activity is to identify grand challenges for AGU communities. This links to the effort of Unresolved Problem in Hydrology (UPH) initiated recently by the International Association of Hydrological Science, and more information of UPH can be found at https://iahs.info/IAHS-UPH.do. It would be interesting to identify the uncertainty-related grand challenges and to also offer some solution from your own perspective. The list of grand challenges may be useful for organizing a Chapman Conference in next couple of years focusing on UQ. Again, please come up with one or two grand challenges and offer your insights for addressing the challenges.

We always focus on a narrow range of problems related to our research, and these AGU requests help us think something BIG. I personally view it as a great opportunity to reexamine our own research and the research of the UQ community, so that the TC can offer thoughtful and insightful guidelines to the UQ community.

Three last but not least notes:

(1)   The AGU nomination deadline is 3/15/2018. Please nominate our colleagues for the hydrology section awards.

(2)   AGU has started accepting session proposals, and the deadline is 4/18/2018.

(3)   The hydrology section is exploring the idea of TC-led sessions, i.e., a session proposed by each TC for promoting the TC theme research. Should you have ideas for TC-led sessions, please let me know.

Please feel free to comment on this post, and add your inputs to the identification of game changes and grand challenges.

Which session to submit to? Hydrologic Uncertainty at AGU Fall Meeting 2017

Uncertainty is a multi-faceted topic. To help in choosing a session to submit to at the AGU Fall Meeting 2017, we've put together a shortlist of sessions related to characterizing uncertainty, living with uncertainty, and reducing uncertainty.

Comments and questions about specific sessions are welcome, including any we may have missed.

The early abstract submission deadline is 26th July 2017.

Characterizing uncertainty

Uncertainty analysis (UA) and Sensitivity Analysis (SA) methods

• H042: Diagnostics, Sensitivity, and Uncertainty Analysis of Earth and Environmental Models
• H128. Understanding the Interface between Models and Data 

Parameter estimation and data assimilation

• H039. Data integration, inverse methods, and data valuation across a range ofscales in hydrogeophysics
• H066. Hydrologic Data Assimilation
• S010. Frontiers of uncertainty quantification in geoscientific inversion
• NG001: Advances in Data Assimilation, Predictability and Uncertainty Quantification

Ensemble methods

• H008: Advances in Hydrologic Prediction to Support Water and Energy Applications
• H146. Weather/Climate Ensembles and Statistical Downscaling for Hydrologic prediction systems: Methods, Process Understanding and Applications 

Stochastic modelling

• H116. Stochastic Modeling of the Hydrosphere and Biosphere
• NG011. Stochastic Modelling in Atmosphere, Ocean, and Climate Dynamics

Living with uncertainty

Risk management and Robust Decision Making

• H110: Science to Action: What is the Role of Climate Science and the Climate Scientist in Robust Decisions?
• H135. Water and Society: Implications of Climate and Hydrologic Forecasting in Risk Mitigation and Management
• PA029: Science to Action: Resilient Decision Making in the Midst of Uncertainty
• H139. Water and Society: Water Resources Management and Policy in a Changing World

Uncertainty in Decision Support Systems

• H133. Water and Society: Advances in Decision Support for Climate Adaptation in Environmental Systems
• H099. Quantifying uncertainty in assessment of freshwater, ecosystem and agricultural sustainability under climate change and urbanization (eLightning)

Reducing uncertainty

Data-based, machine learning modelling approaches

• H022: Applications of machine learning in hydrology
• H031. Computational Intelligence and Machine Learning Methods in Water and Environmental Management 
• H082. Machine Learning Applications in Earth Science and Remote Sensing 
• H089. Multiscale, multifidelety, and hybrid machine-learning methods for flow and transport in hydrologic systems
• H094. Prediction of Hydrologic Behavior from Sparse Information Using Statistical and Machine Learning Techniques 
• A028. Combining Physical Simulation and Machine Learning across Geophysical Sciences 

Advancing process-based modelling

• H067. Hydrologic Dynamics, Complexity and Predictability: Physical and Analytical Approaches for Improving System Understanding and Prediction
• H073. Advances in experimental techniques, validation of modelling tools and uncertainty in predictions from pore to field scale
• H088. Multi-hypothesis modeling of ecological and hydrological systems
• H106. Role of Process Representations on Prediction of Hydrologic States and Fluxes

Uncertainty about Uncertainty

I like to point to Keith Beven's (1987) conference paper, titled ‘Towards a new paradigm in hydrology’, as a place for new hydrologists to start to develop an understanding of uncertainty in the hydrological sciences[1]. In that paper Keith argued that “little to no success” had been made against the fundamental problem of developing theories about how small-scale complexities lead to large-scale behavior in hydrological systems[2].

The paper discussed what hydrologists might do about this situation, and in the last two paragraphs Keith made two predictions:

·      First, that hydrology of the future will require a theoretical framework that is inherently stochastic to deal with the “value of imperfect observations and qualitative knowledge in reducing predictive uncertainty.” 

·      Second, that hydrologists would not actually develop this type of theoretical framework, but instead would capitalize on the (then) emerging power of desktop computing to approximate uncertainty; leading to results that “may not be pretty,” but which are “realistic in reflecting both our understanding and lack of knowledge of hydrological systems.”

Of course, both of his predictions were essentially correct. In 2007 Jeff McDonnell wrote “to make continued progress in watershed hydrology … we need to … explore the set of organizing principles that might underlie heterogeneity and complexity” (McDonnell et al., 2007). Jeff went on to describe several possibilities for ‘moving beyond’ heterogeneity and process complexity, but the point is nevertheless clear: the problem remains unsolved. Similarly, Keith wrote last year that “our perceptual model of uncertainty is now much more sophisticated … but this has not resulted in analogous progress in uncertainty quantification” (Beven, 2016).

The situation in hydrology right now is that we understand that macro-scale behaviors of watersheds are governed by small-scale heterogeneities, but we don’t have a theory about how this works, and we don’t have any fundamental theory that allows us to (reliably) quantify predictive uncertainties related to these processes.

Instead, what we have are ad hoc strategies for obtaining numbers that seem like they might be related with uncertainty. One example of this is the recent proliferation of multi-parameterization modeling systems that allow the user to choose between a variety of options for different flux parameterizations. An example is the Structure for Unifying Multiple Modeling Alternatives; Clark et al. (2015) wrote that “[SUMMA] provides capabilities to evaluate different representations of spatial heterogeneity and different flux parameterizations, and therefore tackle the fundamental modeling challenge of simulating the fluxes of water and energy over a hierarchy of spatial scales.” It’s unclear why predicting with a variety of different parameterizations of scale-dependent processes allows us to ‘tackle’ scale-related challenges: we still lack a fundamental theory of hydrologic scaling, and making predictions with several different parameterizations is not in any way reflective of the actual nature of our lack of knowledge about the principles and behavior of hydrologic systems.

Other methods that we often use for uncertainty quantification suffer from similar problems. Bayesian methods allow us to do precisely one thing: inter-compare or average several different competing models. Bayesian methods, and indeed any methods for model inter-comparison or model averaging, are fundamentally incapable of helping us to understand the difference between our family of models (i.e., those models that are assigned finite probability by the prior) and the real system. Gelman & Shalizi (2013) give a philosophical treatment of this problem that is worth reading.

It has been suggested that we might use empirical methods to develop probability distributions over different components of predictive imprecision (e.g., Montanari and Koutsoyiannis, 2012), but this type of approach assumes stationarity not only in those aspects of the hydrological system that are captured by the model, but also stationarity in the relationship between model error and those parts of the hydrological system that are not captured by the model.

The point is that uncertainty is tautologically inestimable, and so it is really no surprise that Keith’s second prediction came true – there was never any real possibility to develop a rigorous theoretical basis for understanding uncertainty, scale-dependent or otherwise. More than that, the methods that we have come up with to approximate uncertainty don’t actually do that at all – at least not in any way that is fundamentally or theoretically reliable. I propose the following challenge: provide a theorem that proves a bounded, asymptotic, or even consistent relationship between any quantitative estimator and real-world uncertainty under evaluable assumptions. I offer the standard wager for scientific controversy[3]: a bottle of Yamazaki 12 year, or comparable.

Until we have such a theorem, I propose that it is not useful to talk about uncertainty quantification – approximate or otherwise, – because none of our estimators are related to real uncertainty in any systematic way.

Instead, I predict a new paradigm change in hydrology. I suspect that within the next 30 years, the conversation in hydrology will be about information rather than uncertainty. The reason for this is that while it is impossible to estimate uncertainty (even approximately), it is possible to obtain at least bounded estimates of information measures (Nearing and Gupta, 2017). The main project of science seems to be about comparing the information contained in observation data with the information provided by a hypothesis-driven model. Similarly, the problem of scaling under heterogeneity seems to be fundamentally about cross-scale information transfers, rather than about uncertainty. Given recent work in basic physics (e.g., Cao et al., 2017), I suspect that it will not take three more decades for us to discover real scaling theories under this type of perspective.

Of course, we will always want to know the reliability of our model predictions, and for this reason the concept of uncertainty will never go away completely. I propose, however, that the tractable and meaningful challenge is to understand the actual predictive precision implied by our hydrological theory and hypotheses. At present, we do not do this. Current practice is to build models that are over-precise, and then append ‘uncertainty’ distributions to their predictions. This method of dealing with a lack of complete knowledge stems fundamentally from our Newtonian heritage. Essentially all process-based hydrological models are expressed as PDEs, which must admit solution in order to make a prediction.

There are two problems with building dynamical models as PDEs. First, such models make ontological predictions (predictions about what will happen), whereas what we actually want are epistemological predictions (predictions about what we can know about what will happen). The uncertainty probabilities that we append to our models are the latter, and they are what we actually need for both hypothesis testing and decision support. But these probabilities are not the product of actually solving our model equations. Even if our model is a stochastic PDE, the random walk component is simply an ad hoc appendage on the drift function. Sampling model inputs or different model structures does not actually tell us anything about our lack of knowledge associated with any of those model structures. Models built as PDEs simply do not solve for anything that represents what we can know from our physical theory and hypotheses.

The second problem is that a PDE only provides a prediction if it can be solved. This requires that we prescribe values (or distributions) over all parameters contained in our hypothetical parameterizations, some of which are impossible to measure and otherwise difficult to estimate. It would be exciting to have a method for constructing models that allows us to assign values to only those parameters that we feel we actually have some information about.

But there is, in principle (although I have no example of such), a way to build this type of model. Instead of expressing conservation principles using differential equations, we could express them as symmetry constraints on probability distributions. To do this, we might specify a Bayesian network such that each node is a random variable representing a particular scale-dependent quantity at a particular time and location within the modeled system; conservation laws could then be used to effectively rule out large portions of the joint space of values over these random variables. By imposing conservation laws and other physical principles as constraints on joint probability distributions, our models would fundamentally solve for what we can know about the future or unobserved behavior of a dynamical system conditional on whatever information (theories, hypotheses, data) are used to build the model. In principle, anything that we do know, or wish to hypothesize, could be imposed as a constraint on the joint distribution over a family of random variables representing different aspects of system behavior. 

Although such a strategy would not allow us to measure epistemic uncertainty (uncertainty is always and still inestimable), at least it would allow us to know what information we actually have about the behavior of hydrologic systems. This would be a very different way of approaching model building than appending uncertainty distributions to PDE solutions, and would allow us to actually quantify the information content of our scientific hypotheses and models.

So perhaps my predictions about paradigm change will not come to pass. I am, after all, essentially arguing against two of the most fundamental concepts in our science: that we should not use Bayesian methods to evaluate models, and that we should not use differential equations to build models. I do suspect that I am right about both of these things, in the sense that our science (indeed, any science of complex systems) would accelerate by abandoning these ideas in favor of information-centric philosophies and methods, but perhaps it will take longer than 30 years to demonstrate that such a substantial change is necessary. 

------

Beven, K. 'Towards a new paradigm in hydrology'. Water for the Future: Hydrology in Perspective. , Rome, Italy: IAHS Publication.

Beven, K. J. (2016) 'Facets of uncertainty: Epistemic error, non-stationarity, likelihood, hypothesis testing, and communication', Hydrological Sciences Journal, (9), pp. 1652-1665.

Cao, C., Carroll, S. M. and Michalakis, S. (2017) 'Space from Hilbert space: Recovering geometry from bulk entanglement', Physical Review D, 95(2), pp. 024031.

Clark, M. P., Nijssen, B., Lundquist, J. D., Kavetski, D., Rupp, D. E., Woods, R. A., Freer, J. E., Gutmann, E. D., Wood, A. W. and Gochis, D. J. (2015) 'A unified approach for process‐based hydrologic modeling: 2. Model implementation and case studies', Water Resources Research, 51(4), pp. 2515-2542.

Dooge, J. C. I. (1986) 'Looking for hydrologic laws', Water Resources Research, 22(9S), pp. 46S-58S.

Gelman, A. and Shalizi, C. R. (2013) 'Philosophy and the practice of Bayesian statistics', British Journal of Mathematical and Statistical Psychology, 66(1), pp. 8-38.

McDonnell, J. J., Sivapalan, M., Vaché, K., Dunn, S., Grant, G., Haggerty, R., Hinz, C., Hooper, R., Kirchner, J. and Roderick, M. L. (2007) 'Moving beyond heterogeneity and process complexity: A new vision for watershed hydrology', Water Resources Research, 43(7).

Montanari, A. and Koutsoyiannis, D. (2012) 'A blueprint for process-based modeling of uncertain hydrological systems', Water Resources Research, 48(9), pp. n/a-n/a.

Nearing, G. S. and Gupta, H. V. (2017) 'Information vs. uncertainty as a foundation for a science of environmental modeling', https://arxiv.org/abs/1704.07512.

---

[1]It’s the kind of paper that can be enjoyed with a beer.

[2]Dooge (1986) gave a somewhat more technical discussion of this same problem.

[3]e.g., https://www.quantamagazine.org/supersymmetry-bet-settled-with-cognac-20160822