This is a summary of the following new paper:
Constantinou
AC, Yet B, Fenton N, Neil M, Marsh W "Value of Information analysis
for interventional and counterfactual Bayesian networks in forensic
medical sciences". Artif Intell Med. 2015 Sep 8 doi:10.1016/j.artmed.2015.09.002. The full pre-publication version can be found here.
Most
decision support models in the medical domain provide a prediction
about a single key unknown variable, such as whether a patient
exhibiting certain symptoms is likely to have (or develop) a particular
disease.
However we seek to enhance decision analysis
by determining whether a decision based on such a prediction could be
subject to amendments on the basis of some incomplete information within
the model, and whether it would be worthwhile for the decision maker to
seek further information prior to the decision. In particular we wish
to incorporate
interventional actions and
counterfactual analysis, where:
- An interventional action is one that can be performed to
manipulate the effect of some desirable future outcome. In medical
decision analysis, an intervention is typically represented by some
treatment, which can affect a patient’s health outcome.
- Counterfactual analysis enables decision makers to compare
the observed results in the real world to those of a hypothetical world;
what actually happened and what would have happened under some
different scenario.
The method we use is based on the underlying principle of
Value of Information.
This is a technique initially proposed in economics for the purposes of
determining the amount a decision maker would be willing to pay for
further information that is currently unknown within the model.
The type of predictive decision support models to which our work applies are Bayesian networks. These
are
graphical models which represent the causal or influential
relationships between a set of variables and which provide probabilities
for each unknown variable.
The method is applied to
two real-world Bayesian network models that were previously developed
for decision support in forensic medical sciences. In these models a
decision maker (such as a probation officer or a clinician) has to
determine whether to release a prisoner/patient based on the probability
of the (unknown) hypothesis variable: “individual violently reoffends
after release”. Prior to deciding on release, the decision maker has the
option to simulate various interventions to determine whether an
individual’s risk of violence can be managed to acceptable levels.
Additionally, the decision maker may have the option to gather further
information about the individual. It is possible that knowing one or
more of these unobserved factors may lead to a different decision about
release.
We used the method to examine the average
information gain; that is, what we learn about the importance of the
factors that remain unknown within the model. Based on six different
sets of experiments with various assumptions we show that:
- the average relative percentage gain in terms of Value of Information
ranged between 11.45% and 59.91% (where a gain of X% indicates an
expected X% relative reduction of the risk of violent reoffence);
- the potential amendments in Decision Making, as a result of
the expected information gain, ranged from 0% to 86.8% (where an
amendment of X% indicates that X% of the initial decisions are expected
to have been altered).
The key concept of the method is that if we had known that the
individual was, for example, a substance misuser, we would have arranged
for a suitable treatment; whereas without having information about
substance misuse it is impossible to arrange such a treatment and, thus,
we risk not treating the individual in the case where he or she is a
substance misuser.
The method becomes useful for
decision makers, not only when decision making is subject to amendments
on the basis of some unknown risk factors, but also when it is not.
Knowing that a decision outcome is independent of one or more unknown
risk factors saves us from seeking information about that particular set
of risk factors.
This summary can also be found on the Atlas of Science
