Local weather  
Brought to you by
 IF - IG 
International Interdisciplinary Center For
Information and Fact Checking about
Info-Gap Decision Theory

Local time
Home About News IGDT The Campaign Myths & Facts Reviews FAQs FUQs Resources Contact Us
The rise and rise of voodoo decision making

Reviews of publications on Info-Gap decision theory (IGDT)

Review 6-2022 (Posted: January 28, 2022; Last update: January 28, 2022)

Reference Yang Liu, Penghao Wang, Melissa L. Thomas, Dan Zheng & Simon J. McKirdy. Cost‐effective surveillance of invasive species using info‐gap theory. Scientific Reports, 11:22828.
Publication typePeer-reviewed journal article.
Year of publication2021
Downloads https://doi.org/10.1038/s41598-021-02299-8, open access.
Abstract Invasive species can lead to community‐level damage to the invaded ecosystem and extinction of native species. Most surveillance systems for the detection of invasive species are developed based on expert assessment, inherently coming with a level of uncertainty. In this research, info‐gap decision theory (IGDT) is applied to model and manage such uncertainty. Surveillance of the Asian House Gecko, Hemidactylus frenatus Duméril and Bibron, 1836 on Barrow Island, is used as a case study. Our research provides a novel method for applying IGDT to determine the population threshold (K ) so that the decision can be robust to the deep uncertainty present in model parameters. We further robust‐optimize surveillance costs rather than minimize surveillance costs. We demonstrate that increasing the population threshold for detection increases both robustness to the errors in the model parameter estimates, and opportuneness to lower surveillance costs than the accepted maximum budget. This paper provides guidance for decision makers to balance robustness and required surveillance expenditure. IGDT offers a novel method to model and manage the uncertainty prevalent in biodiversity conservation practices and modelling. The method outlined here can be used to design robust surveillance systems for invasive species in a wider context, and to better tackle uncertainty in protection of biodiversity and native species in a cost‐effective manner.
ReviewerMoshe Sniedovich
IF-IG perspective This article takes us back to 2006 and the launch of the old campaign to contain the spread of IGDT in Australia. Indeed, it plays an improtant role in my decision at the end of 2021 to revive the campaign.

This very short review focuses on the article's assessment of the relationship between Wald's maximin paradigm and IGDT. But first, a brief examination of a number of other points

  1. The article emphasizes that the uncertainty under consideration is deep, or severe. Yet, it ignores well documented peer-reviewed assessments that IGDT is unsuitable for the treatment of uncertainties of this type.

  2. The reference to Johnson and Geldner (2019) on page 1 is odd. It reads:
    Various methods to support decisions in face of Knightian uncertainty have been developed12,13.
    where Ref. 12 is Johnson and Geldner (2019). Here we read this assessment of IGDT (color is used for emphasis):
    It does, however, have one major limitation: It considers only local uncertainty in the neighborhood of the best-estimate state of the world. Thus, it is most useful where a reasonable best estimate exists and there is strong reason to believe increasingly large deviations from the best estimate are decreasingly likely (Hayes et al. 2013). While this does not require the uncertainty to be well parameterized, it is a strong assumption nonetheless (Taleb 2005). If the future SOW is no more likely to be within an arbitrary neighborhood of the best estimate than far away from it, info-gap may lead to endorsement of strategies vulnerable to high regret.
    Johnson and Geldner (2019, p. 31)

    The whole point is of course is that in the framework of deep uncertainty one cannot assume that " ... a reasonable best estimate exists and there is strong reason to believe increasingly large deviations from the best estimate are decreasingly likely ...

    The bottom line is then this
    The treatment of deep uncertainty is challenging precisely because we cannot assume that " ... reasonable best estimate exists and there is strong reason to believe increasingly large deviations from the best estimate are decreasingly likely ...

    Reminder:
    According to Ben-Haim (2001, 2006, 2010) the uncertainty postulated by IGDT is severe in the sense that

    • The uncertainty space is large and diverse. It is often unbounded.
    • The best estimate is poor, it is a (educated?) guess, can even be a wild guess.
    • The uncertainty is probability, likelihood, plausibility, chance, belief -- FREE!

    And now to the Elephant in the Room (see Sniedovich (2014) "The elephant in the rhetoric on info-gap decision theory", Ecological Applications, 24(1), 229-233. https://esajournals.onlinelibrary.wiley.com/doi/abs/10.1890/13-1096.1)

    From the very start, Ben-Haim (2001, 2006, 2016) was adamant that IGDT was, at the time, a new decision theory, one that was radically different from all existing theories for decision making under uncertainty. In particular, the IGDT literature is filled with assertions and claims that IGDT is not a maximin theory, namely that its core models are not maximin models.

    On the other hands, the literature offers peer-reviewed articles that prove, formally and rigorously, that IGDT is a maximin theory, in fact a 'simple' one. So, it is perfectly legitimate to raise the following question:

    What exactly is the relationship between IGDT and Wald's famous Maximin paradigm?

    Note that the article under review repeats the standard answer given to this question by proponents of IGDT. This is what we find on page 1 of the article:

    Info-gap decision theory (IGDT) is a non-probabilistic theory that has been used in a range of areas (e.g. engineering model-testing, financial risk assessment and marine protection policy decisions9). Such robust decision making methods are often desirable in ecological systems characterized by Knightian uncertainty10. Knightian uncertainty is characterized by high knowledge-deficiency of the probability or frequency of outcomes, as opposed to the risk that can be measured probabilistically11. Various methods to support decisions in face of Knightian uncertainty have been developed12,13. Wald’s maximin to ameliorate worst outcome has been used as one of the primary methods. However, it is also known for excessive conservatism and pessimistic14. IGDT offers an alternative of Wald’s maximin to quantify the confidence in realising specified aspirations but enable a balance between them as a robust-satisfying method15.

    To put it mildly, this assessment is not just factually wrong. It implies that IGDT models can do things that Wald maximin models can't, while the truth is that Wald's maximin paradigm is much more powerful and versatile than the IGDT paradigm. Whatever the IGDT core robustness models can do, Wald's maximin paradigm can do. The converse is not true: there are many things that maximin models can do that are beyond the capabilities of IGDT.

    In short, the article's assessment of IGDT is based on a serious lack of awareness and appreciation of the versatility of Wald's maximin paradigm as far as modeling is concerned. From the perspective of deep uncertainty, the most important thing to observe is that the core IGDT models are inherently models of local analysis, whereas Wald-type maximin paradigm offers models that can be local or global depending on how the modeler specifies the uncertainty sets.

    To conclude:

    • The approach proposed in the article is unsuitable for the treatment of severe (deep) uncertainty of the type stipulated by IGDT as it is described in Ben-Haim (2001, 2006, 2010).
    • The article's assessment of the capabilities and limitations of Wald-type maximin models exhibits a lack of awareness and appreciations of the modeling prowess of Wald's maximin paradigm.
    • The article's assessment of the relationship between IGDT and Wald's maximin paradigm is factually wrong, in fact almost as wrong as it can be: contrary to the article's claim, IGDT cannot possibly offer an alternative to Wald's maximin paradigm for the simple reason that IGDT's core models are Wald-type maximin/minimax models. It is the other way around: Wald's maximin paradigm offers a much more powerful and versatile alternatives to IGDT's core models.
    • The article fails to mention the important fact that IGDT's robustness model is in fact a reinvention of the well known and well established concept that is known universally as Radius of Stability (circa 1962).
    • The article does not provide a balanced assessment of IGDT, as it ignores well documented peer-reviewed criticism of the theory that are very relevant to the material discussed in the article.

    Summary and conclusions

    The authors of the article are encouraged to read Review 2-2022 where they will find formal rigorous analysis showing that

    • IGDT's robustness and robust-satisficing decision models are simple Wald's type maximin models.
    • IGDT's opportuneness and opportune-satisficing decision models are simple Minimin models.
    • IGDT's flagship concept, namely IGDT robustness is a reinvention of the well known concept Radius of Stability (circa 1962).
    • IGDT is unsuitable for the treatment of deep uncertainty.
    Australian perspective:  
    Readers interested in the history of IGDT in Australia are encouraged to read Sniedovich's (2011) research report entitled Info-gap decision theory: a perspective from the Land of the Black Swan, as well as the history of the campaign launched at the end of 2006 to contain the spread of IGDT in Australia.

    It is a pity that the article seems to be unaware of the fact there has been a lot of research work in Australia on IGDT and its role and place in decision making under severe uncertainty.

    Bibliography and links

    Articles/chapters

    Research Reports

    Links

Disclaimer: This site, its contents and style, are the responsibility of its owner the author and do not represent the views, policies or opinions of the organizations he is associated/affiliated with.