A systematic review and meta-analysis of 57 studies scrutinizing the practice of standardized diagnostic interviews (SDIs) used in mental health assessments is raising serious questions about these questionnaires\u2019 reliability, thanks to efforts led by psychological researchers at McMaster University in Canada.<\/p>\n
Critics of current mental health practices say that mental health practitioners are working in the shadow of the 1980 edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM III). This revision\u2014as one medical journal essay<\/a> put it back in 2012\u2014sought to paper over \u201csectarian discord among proponents of psychodynamic, behavioral, and neurobiologic explanations of mental illness\u201d by reducing all psychological disorders down to digestible checklists of signs and symptoms.<\/p>\n The new deep-dive meta-analysis, which covered SDIs conducted with over 8,000 adults from 26 countries, found that these interviews turned out to be not particularly consistent across cases where the same patient was assessed via two separate SDIs even just days apart. More concerning still, the reliability between tests and retests appeared to vary significantly depending on which mental health issue was being investigated.<\/p>\n \u201cIf we give the same interview to the same person twice, we would like to think the interview would produce the same result, but that\u2019s not always the case,\u201d the study\u2019s senior author Laura Duncan, an assistant professor at McMaster\u2019s Department of Psychiatry and Behavioural Neurosciences, said<\/a> in a statement.<\/p>\n Duncan, whose prior work as a doctoral candidate critically examined<\/a> health research methodology, added: \u201cOur findings show that these interviews are not as reliable or consistent as many people believe.\u201d<\/p>\n Eye of the beholder <\/p>\n Duncan and her team, which included researchers from the University of Copenhagen in Denmark and the University of Massachusetts\u2019 Chan Medical School in Boston, noted that these interviews were often more reliable for substance use disorders than for mental health issues.<\/p>\n \u201cThese differences suggest that structured interviews work better for conditions with clearer behaviours or timelines than for disorders that rely heavily on personal experiences and interpretation,\u201d Duncan said.<\/p>\n Of the 57 studies pooled together for this meta-analysis, 46 were ultimately selected for detailed cross comparison, using a statistical method called Cohen\u2019s kappa<\/a> to compensate for unknown sloppiness in the interview data.<\/p>\n Pairs of SDI tests and retests on various individual patients coping with substance use disorders were found to be consistent in roughly 72% of cases, while SDI retests for mental disorders only matched their original test about 65% of the time. Reliability was basically 50\/50 with SDIs testing for non-affective psychoses (a category which includes schizophrenia spectrum disorders); those retests came to the same conclusion in only 55% of cases. Retests for bipolar disorders, in contrast, matched the first SDI result in about 74% of the cases reviewed.<\/p>\n One of the most consistent diagnoses studied, they found, were tests for opioid addiction which was consistent 81% of the time.<\/p>\n A world of difference <\/p>\n Duncan and her colleagues acknowledged \u201cseveral limitations\u201d to their systematic review, including an \u201cincomplete reporting of data,\u201d which they wrote included missing information on certain variables that might have impacted each of the 46 studies\u2019 data. Nevertheless, their conclusions were stark.<\/p>\n