6.2.1. Introduction
The diagnosis of antisocial personality disorder, like some other mental disorders, is associated with an increased risk of offending behaviour, including violence. However, antisocial personality disorder is a very broad diagnostic category (see DSM-IV; APA, 1994), even when compared with other diagnoses in mental health. It encompasses people who never commit offences as well as a minority who commit the most serious crimes, with a great range in between. As a result the diagnosis alone is of little value as an indicator of violence risk.
The clinical assessment of violence risk in antisocial personality disorder is more problematic than in some other mental disorders, such as schizophrenia, because antisocial personality disorder lacks unequivocal symptoms such as delusions and hallucinations. The clinical interview and mental state examination are therefore less reliable as a means of assessing the severity of the disorder. Some patients may be both persuasive and deceptive, making a clinical interview a poor guide to the severity of the disorder and its associated risks. Therefore much effort has been expended on the development and evaluation of tools that may assist in the assessment of violence risk. Any measure that discriminates between degrees of severity of antisocial personality disorder is likely to be of assistance in risk assessment; the Psychopathy Checklist (Hare, 1980; Hart, 1998a, 1998b) is therefore one of the most useful instruments in this field.
The statistical evaluation of risk assessment tools
Risk assessment is concerned with probability, therefore it lends itself to a statistical approach comparing prediction and outcome. In order to evaluate risk assessment tools it is necessary to appraise the extent to which they maximise the detection of violent outcomes (true positives) while minimising the number of false alarms (false positives). sets out the model for the possible outcomes of violence risk prediction.
Possible outcomes of violence risk prediction.
In this model the quality of the test or tool is judged by two main criteria:
Sensitivity is defined as the proportion of the violent outcome group scoring positive for predicted violence on the risk assessment instrument, that is, sensitivity = TP/(TP + FN).
Specificity is defined as the proportion of the non-violent outcome group scoring in the predicted non-violence group on the risk assessment instrument, that is, specificity = TN/(FP + TN).
There is a trade-off between these measures. As the test or tool is made less stringent by lowering the cut-off score it picks up more true positives (sensitivity rises) but it also picks up more false positives (specificity falls). The ideal is to maximise sensitivity while keeping specificity high.
To illustrate this: from a population in which the point prevalence rate of depression is 10% (that is, 10% of the population has depression at any one time), 1000 women are given a test with 90% sensitivity and 85% specificity. It is known that 100 women in this population have depression, but the test detects only 90 (true positives), leaving 10 undetected (false negatives). It is also known that 900 women do not have depression, and the test correctly identifies 765 of these (true negatives), but classifies 135 incorrectly as having depression (false positives). The positive predictive value of the test (the number correctly identified as having depression as a proportion of positive tests) is 40% (90/90 + 135), and the negative predictive value (the number correctly identified as not having depression as a proportion of negative tests) is 98% (765/765 + 10). Therefore, in this example, a positive test result is correct in only 40% of cases, while a negative result can be relied upon in 98% of cases.
The qualities of a particular tool are summarised in a receiver operator characteristic (ROC) curve, which plots sensitivity (expressed as %) against (100% - specificity) (see ).
A test with perfect discrimination would have a ROC curve that passed through the top left hand corner; that is, it would have 100% specificity and pick up all true positives with no false positives. In reality that is never achieved, but the area under the curve (AUC) measures how close the tool achieves the ideal. A perfect test would have an AUC of 1 and anything above 0.5 is better than chance.
The AUC is the preferred statistic for evaluating risk assessment tools and is the most common metric used in such studies (Mossman, 1994). Its main advantage, in comparison with the other statistics, is that such estimates appear not to be affected by the base rate of the phenomenon under consideration, which in this case is violence (see Mossman, 1994). For these reasons, the review below uses AUC to compare tools used for violence risk assessment.
Statistical prediction and healthcare
While the AUC is used because it is generally agreed to be the best available statistic (Mossman, 1994), practitioners should be wary of the uncritical application of statistical approaches to risk assessment and management in a health setting. The main problems are set out below.
Statistics take no account of the values that are central to healthcare: The AUC statistic is concerned with maximising the number of right decisions. As violence is relatively unusual in mental health populations, Monahan (1981) pointed out that the best way to be right most of the time is to predict that no patients will be violent. That course of action is unacceptable because errors in medicine come with values attached and their values are not equal. The consequences of failing to predict an act of serious violence (a false negative) are very different from the consequences of wrongly predicting violence (a false positive). Fulford and colleagues (2006) have written extensively on the importance of values in mental health; for the purposes of this discussion the crucial point is that the statistics cannot be considered in isolation.
The apparent value of a risk prediction instrument will be determined to a large extent by the population to which it is applied: Gordon (1977) observed that many risk assessments are tested in prisoner populations where there are high baseline levels of violence risk. The same is true of many of the studies summarised below. In these circumstances it is perhaps remarkable that these instruments are able to achieve a reasonable level of discrimination. Clinicians who work with a more average group of patients may therefore reasonably expect that a standardised assessment may be even more effective in identifying patients who have a high violence risk. This principle leads to a paradox. Standardised risk assessments are most widely used in forensic populations where most patients will have an increased violence risk, meaning that fine discrimination between degrees of risk is more difficult. In a general psychiatry population, where most patients have a lower level of risk, standardised instruments ought to be of more value in identifying the small number who present a high risk.
Even the best instruments have high rates of error when applied to individuals: Sensitivity, specificity and the AUC are population or group measures, but there are much greater uncertainties associated with individual prediction. In part this limitation is intrinsic to the statistical method; just because an individual has most attributes of a group does not mean he or she has all of them, even though those attributes generally go together.
Violence risk prediction is different because the reality is ambiguous and it is also subject to change. All the evidence concerning a particular individual may indicate an extremely high risk of violence but it counts for nothing if the potential perpetrator meets with an accident or dies of natural causes on his or her way to committing an act of violence. More realistically, a medical intervention or supervision on probation can turn a true positive into a false positive, by preventing an act of violence.
Violence risk is multifaceted rather than unitary: A comprehensive assessment of violence risk includes qualitative and descriptive elements. For example, it may specify the likely victim or class of victim (for example, women and children), the type of violence (for example, sexual versus non-sexual, predatory versus impulsive), the severity (for example, use of weapons, whether the violent act is life-threatening, and so on) and the frequency and probability of violence. Statements of probability will usually be conditional on, for example, availability of alcohol and involvement in destabilising relationships. Different considerations apply to the management of, for example, low frequency but life-threatening predatory violence on the one hand and frequent, impulsive, and less serious violence on the other. It is impossible to encapsulate this complexity within a unitary statistical measure. In clinical practice a good risk assessment is not a statement of probability but a comprehensive description of many different aspects.
6.2.2. Current practice
It is generally accepted that the best way of assessing violence risk in mental health settings is through structured clinical judgement (Monahan et al., 2001). The alternative methods are unstructured clinical judgement and actuarial measures. Unstructured clinical judgement relies on the skills of the individual clinician and has no rules beyond the basic rules of clinical practice. The clinician is free to take into account any information they see fit, and they can use their discretion to arrive at a judgement of violence risk.
The unstructured clinical approach is widely used but it is becoming difficult to defend. Although it can work reasonably well it depends on individual skill, experience and thoroughness. Practice varies between individuals and, because there is no structure or standard, it is virtually impossible to give explicit training or to raise standards. Decisions lack transparency so it is difficult to guard against bias and to guarantee non-discriminatory practice. Communication is compromised because there is no common language or agreed set of variables.
In a reaction against the clinical method, the actuarial approach specifies the information to be collected and how it is to be analysed in order to arrive at a decision. The exercise of clinical discretion is explicitly forbidden in order to exclude bias. This approach is derived from the insurance industry and it is surprisingly effective in predicting violence at the population level.
However, actuarial methods are less useful or appropriate in a clinical setting because the focus is on the individual patient. When applied to individuals, actuarial or standardised measures will often be inaccurate because they ignore idiosyncratic features, including both protective and aggravating factors. For example, morbid jealousy may be associated with a very high risk of violence even in the absence of other actuarial risk factors. Conversely, the onset of incapacitating physical illness may lower violence risk even when all the actuarial indicators are present.
In principle there is also an objection to relying on actuarial measures in clinical settings. They treat the individual as nothing more than a representative of a class of people, all of whose characteristics are assumed to be identical. It could be argued that such measures rely on the same logic as prejudice and are therefore incompatible with the value placed by health services on individual formulation and needs assessment.
Despite these reservations, actuarial assessments such as the Violence Risk Assessment Guide (VRAG; Quinsey et al., 1998), the Sex Offender Risk Assessment Guide (Quinsey et al., 1998), and Static-99 (Hanson & Thornton, 1999) are widely used by forensic mental health services. They should not be used as stand-alone measures of risk but will often form part of a comprehensive assessment. When used in that way they become incorporated into the exercise of structured clinical judgement.
Structured clinical judgement combines the positive aspects of the actuarial and clinical approaches. There is a mandatory requirement to collect standardised information, but the clinician is free to interpret that information in the light of all that is known about the individual case. There is some standardisation and transparency while clinicians retain the freedom to take into account any and all available information before reaching a decision.
The most widely used instrument in the field of structured clinical judgement is the Historical, Clinical, Risk Management-20 (HCR-20; Webster et al., 1997) which involves the collection of 20 items (see Section 6.2.5). It then requires consideration of any items that may be specific to the particular case, before requiring clinical teams to construct risk management scenarios. Each scenario considers a possible violent outcome, along with warning signs and factors that make it more or less likely, leading to a plan for managing those risk factors.
Despite the importance given to clinical discretion, this method is based on standardised measures of risk. It requires that clinical decisions are informed by such measures rather than determined by them but it still raises questions about the accuracy of the tools used for violence risk prediction. The next section considers the extent to which such measures are successful in predicting violence risk in populations of people with antisocial personality disorder.
6.2.3. Definition and aim of topic of review
Risk assessment tools are defined in the review as validated psychometric instruments that are used to predict violence and/or offending. The review was limited to assessment tools that in the view of the GDG were likely to be used in UK clinical practice. They included the Psychopathy Checklist in its full (PCL-R; Hare et al., 1991) and screening versions (PCL-SV; Hart et al., 1999), HCR-20 (Webster et al., 1997), VRAG (Quinsey et al., 1998), Level of Supervision Inventory (LSI; Andrews & Bonta, 1995), Offender Group Reconviction Scale (OGRS; Copas & Marshall, 1998), and Risk Assessment Management and Audit Systems (RAMAS; O’Rourke & Hammond, 2000).
GRADE profiles could not be generated because the guidance and software on grading reviews of such studies are at a preliminary stage. Therefore quality assessments for each individual study were provided in the evidence summary tables. The following review assesses predictive validity. It does not replicate the clinical use of these tools nor does it imply they should be used for risk assessment in a clinical setting. In some cases the tools were not designed or intended for risk prediction but that should not be an obstacle to their statistical evaluation.
6.2.5. Studies considered
The review team conducted a new systematic search for observational studies that assessed the risk of antisocial behaviour, focusing on violence and/or offending (see Appendix 8).
Broad inclusion criteria were adopted because there was initial interest in the capacity of the scale to predict violence/offending behaviour not exclusive to antisocial personality disorder. The interventions consisted of risk assessment tools seeking to predict violent and/or offending behaviour at either the group or individual level using outcomes such as sensitivity, specificity, the AUC, PPV and NPV. The primary outcome measure examined was AUC with values of 0.6 to 0.8 indicating a moderate level of prediction, 0.8 to 0.9 a high level of prediction and values greater than 0.9 indicating a very high level of prediction.
Trials consisting of 30% or more of participants with schizophrenia or psychoses were excluded from the analysis.
Twenty studies met the inclusion criteria set by the GDG. Of these, 19 were published in peer-reviewed journals between 1991 and 2007. One further study was a publication from the Ministry of Justice (Coid et al., 2007). In addition, 38 studies were excluded from the analysis. The most common reason for exclusion was not providing relevant data that met the criteria of the review.
Of the 19 included studies, five assessed the HCR-20, 15 the Psychopathy PCL-R, three the PCL-SV, eight the VRAG, three the LSI and one the OGRS. No studies on RAMAS met the eligibility criteria of the review.
Historical, Clinical, Risk Management-20 (HCR-20)
The HCR-20 (Webster et al., 1997) takes a structured clinical assessment approach to risk assessment. This scale consists of 20 items on historical, clinical and risk management issues. The ten historical items include previous violence, substance misuse problems, major mental illness, psychopathy and personality disorder. The five clinical items are concerned with lack of insight, negative attitudes, active symptoms of mental illness, impulsivity and unresponsiveness to treatment. The five risk management items include feasibility of plans, exposure to destabilisers (destabilising influences that may be general or specific to the individual), lack of personal support, non-compliance with remediation attempts and stress.
The HCR-20 is an aid to clinical management of violence risk in individuals. Some aspects of it, namely the formulation of risk scenarios, make sense only in an individual clinical context and are not amenable to statistical evaluation as predictors of risk. Nevertheless the HCR-20 has at its core 20 items said to correlate strongly with violence risk. It is both valid and essential to examine the predictive value of those items, while recognising that it is an artificial exercise not intended to represent clinical use of the tool. The HCR-20 requires the 20 items to be used as the basis for a formulation and risk management plan which goes beyond simple, actuarial prediction. Even so, if the 20 items had no predictive value it would be impossible to justify their inclusion in preference to any other collection of items.
Five studies were identified that met the eligibility criteria of the review. A summary of the study information and data for each of these studies is provided in .
Study information and data on the HCR-20.
Most studies reported data on the area under the curve (AUC). Only Grann and colleagues (2000) provided additional information on sensitivity and specificity. Mean follow-up period ranged from 2 to 10 years.
AUC statistics ranged from 0.6 to 0.8 in most studies indicating that the HCR-20 was moderately predictive of violence and/or offending. A pooled estimate was obtained from studies (Dahle, 2006; Grann et al., 2000; Warren et al., 2005; Morrissey et al., 2007) providing extractable data (AUC = 0.68; 0.65, 0.71). Almost all studies individually found AUC values to be statistically significant; only Warren and colleagues (2005) reported consistent evidence of no effect. This may be explained by the sample consisting only of women; most other studies included samples that were either exclusively or predominantly male. Serious violence is relatively unusual in women and may be associated with different causal factors than those that operate in men.
Clinical use of the HCR-20 allows for the inclusion of idiosyncratic risk items that may increase its predictive power. This flexibility means the HCR-20 can include clinical consideration of risks arising from (for example) sexual offending, stalking, morbid jealousy or dysfunctional intimate relationships even though it does not lend itself to statistical evaluation in these areas. In short it is argued by its proponents that the HCR-20 (and other structured clinical systems of risk management) has greater clinical utility than is reflected in a statistical analysis of group prediction. While the methodological challenges are considerable it seemed to the GDG that such a claim could be tested empirically. No evidence is available at present.
Psychopathy Checklist
Psychopathy is more or less synonymous with the categories of antisocial personality disorder in DSM-IV and with dissocial personality in ICD-10 (Maden, 2007). The Psychopathy Checklist Revised (PCL-R; Hare, 1991) is a measure of psychopathy rather than risk but it has been shown to correlate highly with violence risk in many situations and it is widely used in violence risk assessment as a measure of severity for antisocial personality disorder. In fact the PCL-R is one of the most widely researched of all violence risk assessment tools. This scale consists of 20 items providing a score from 0 to 40. A more recent screening version (PCL-SV) has also been developed based on only 12 items providing a score from 0 to 24 (Hart et al., 1999). Although the PCL-SV is less widely researched than the PCL-R it too has an established correlation with violence risk. In the MacArthur study of violence in general psychiatric patients the PCL-SV was the single best predictor of subsequent violence (Monahan et al., 2001). Both versions can be scored based on case notes alone, with an optional interview for additional information. Psychopathy is generally defined as a score of 30 or above in North America and 25 or above in Europe (Maden, 2007).
Fifteen studies were identified that met the eligibility criteria of the review. A summary of the study information and data for each of these studies is provided in . Most studies were of the PCL-R, but three (Edens et al., 2004; Urbaniok, 2007; Walters et al., 2007) were of the PCL-SV.
Study information and data on the PCL-R and PCL-SV.
Follow-up ranged from 2 to 32 years. As with the HCR-20, most studies reported an AUC ranging from 0.60–0.80 suggesting the PCL-R and PCL-SV versions are moderately predictive of violence and/or offending. Only three studies (Morrissey et al., 2007; Walters & Mandell, 2007; Warren et al., 2005) reported non-significant AUC statistics. Pooled estimates of AUC values for the PCL-R (Dahle, 2006; Grann et al., 1999; Warren et al., 2005) and PCL-SV (Urbaniok et al., 2002; Walters & Mandell, 2007) were calculated from studies that provided extractable data. It appears that the PCL-R (AUC = 0.69; 0.67, 0.70) predicted violence or offending slightly better than PCL-SV (AUC = 0.58; 0.54, 0.63).
The non-significant findings may partly be explained by the populations in these studies. As discussed above, Warren and colleagues (2005) comprised an exclusively female population within a high secure prison in the US. Similarly, Morrissey and colleagues (2007) differed from other studies in focusing on a sample of people with intellectual disability. Finally, Walters and colleagues (2003) focused on disciplinary violations whereas most other studies reported recidivism rates.
Violence Risk Assessment Guide (VRAG)
The VRAG (Quinsey et al., 1998) takes an actuarial approach to risk assessment. The 12 items were derived from a study of 600 male patients released from a high secure hospital in Canada as the highest predictors of violence at 7 years’ follow-up. These items include PCL-R score, problems at junior school, alcohol misuse, age, personality disorder and so on. The main criticism of VRAG is its lack of face validity, that is, three items scored by VRAG as being associated with reduced risk (having a diagnosis of schizophrenia, extent of victim injury and female victim) appear to contradict clinical judgement and the wider literature (Maden, 2007).
Eight studies were identified that met the eligibility criteria of the review. A summary of the study information and data for each of these studies is provided in .
Study information and data on the VRAG.
AUC values once more ranged from 0.60–0.80 indicating a moderately accurate prediction for the risk of violence and/or offending. A pooled estimate was obtained from studies (Grann et al., 2000; Harris et al., 2003) providing extractable data (AUC = 0.65; 0.55, 0.77).
Offender Group Reconviction Scale (OGRS)
OGRS (Copas & Marshall, 1988) is another actuarial instrument that focuses on the prediction of offending at the group level for offenders in England and Wales. It has five static factors: age, sex, number of previous convictions, number of custodial sentences under 21 years of age, and seriousness of the index offence.
One study was identified that met the eligibility criteria of the review. A summary of the study information and data for the included study is provided in . Three studies were excluded because they consisted of samples with greater than 30% of participants having a diagnosis of schizophrenia.
Study information and data on the OGRS.
The AUC ranged from 0.69 to 0.72 indicating a moderately accurate prediction. However, the data were too sparse to be able to draw conclusions on the efficacy of this assessment tool for the target population of this review.
Level of Service Inventory (LSI)
The LSI (Andrews & Bonta, 1995) is another actuarial instrument designed to predict re-offending and the need for probation supervision. The LSI consists of 54 items and 10 subscales using both static (for example, age and previous conviction) and dynamic factors (for example, alcohol misuse and accommodation problems) to predict re-offending.
Three studies were identified that met the eligibility criteria of the review; all were focused on predicting criminal convictions either generally or more specifically of violent recidivism. A summary of the study information and data for each of these studies is provided in .
Study information and data for LSI.
As with the previous instruments the AUC values ranged from 0.60 to 0.80; all were statistically significant and indicated moderate predictive validity. However, it was not possible to pool the AUC values because of a lack of extractable data—only Dahle (2006) provided sufficient detail.
6.2.7. From evidence to recommendations
All of the risk assessment tools included in the review appeared to predict risk moderately well and there did not appear to be clear evidence to distinguish these measures in their level of prediction. Therefore the GDG concluded that the use of a structured instrument would be beneficial as a supplement to a structured clinical assessment. It was also noted that these measures should be provided by staff with sufficient expertise (for example, working in tertiary services) and already be familiar in UK clinical practice (for example, the PCL-R, PCL-SV and HCR-20).
In addition, for secondary services, where there may not be the resources to conduct assessments using such instruments, the GDG felt it would be important for staff to record detailed histories of previous violence and other risk factors.
Finally, in the event that a violence risk assessment may be required in primary care, the GDG concluded that a history of previous violence should be taken and referral to specialist services should be considered.
