Psychosomatics 48:38-45, February 2007
doi: 10.1176/appi.psy.48.1.38
© 2007 Academy of Psychosomatic Medicine
Short-Term Stability of Diagnoses of Major and Minor Depression in Older Medical Inpatients
Martin G. Cole, M.D., FRCP(C),
Jane McCusker, M.D., Dr.P.H.,
Carole Dufouil, Ph.D.,
Antonio Ciampi, Ph.D., and
Eric Belzile, M.Sc.
Received August 24, 2005; revised November 30, 2005; accepted December 5, 2005. From the Dept. of Psychiatry, St. Mary’s Hospital, and McGill Univ., Montreal, Québec, Canada. Send correspondence and reprint requests to Martin G. Cole, M.D., FRCPC, Dept. of Psychiatry, St. Mary’s Hospital, 3830 Lacombe Avenue, Montreal, Québec, Canada H3T 1M5. e-mail: martin.cole{at}ssss.gouv.qc.ca
|
ABSTRACT
|
The authors sought to determine the short-term stability of diagnoses of major and minor depression in older medical inpatients. The investigation was a test–retest design involving comparisons between different pairs of interviewers who assessed patients independently on medical wards of two acute-care, university-affiliated hospitals by use of the Diagnostic Interview Schedule (DIS) at intervals between 1 and 51 days. Patients were 380 cognitively-intact patients age  65, assessed twice with the DIS. Core depressive symptoms (depressed mood, loss of interest or pleasure) were more stable than the other depressive symptoms. The presence of moderate-to-severe depressive symptoms, moderate-to-severe disability, and depression symptoms of 6 months’ duration were associated with a higher short-term stability of diagnosis. Age, gender, language, education, rater, interval between assessments, baseline number of medications, medical comorbidity, severity of medical illness, acute physiology score, and cognitive functioning were not associated with short-term depression stability. Stability of diagnosis may be improved by emphasis on core depressive symptoms or the presence of more severe depressive symptoms, moderate-to-severe disability, and duration of symptoms 6 months.
|
INTRODUCTION
|
Reliability of psychiatric diagnosis refers to the extent to which independent diagnostic assessments yield the same results.1 Usually, estimates of reliability are based on comparisons of diagnoses made by two or more raters at a single interview or at separate interviews a few days apart. Four sources of variation may affect the reliability of psychiatric diagnoses: 1) variation in the interviewer’s style; 2) variation in the patient’s story; 3) variation in diagnostic criteria; and 4) true changes in the patient’s symptoms over time. Important components of reliability include interrater reliability (IRR) and test–retest reliability (or short-term stability).2
For years, achieving adequate IRR of psychiatric diagnoses was considered to be a hopeless undertaking. Then in the 1960s and 1970s, studies demonstrated that good IRR could be achieved in younger, physically healthy adults by use of specially trained interviewers, standardized interview schedules, and explicit diagnostic criteria.3–6 In univariate analysis, patients’ age and language, duration of symptoms, presence of mild cases, and interviewers’ experience were reported to affect levels of IRR.7
Nonetheless, the short-term stability of diagnoses of depressive disorders in older medical inpatients remains a thorny clinical problem. As in younger adults, levels of depressive symptoms may change over time, even during the course of one episode, resulting in a striking number of changes in diagnoses.8,9 Also, many of the symptoms of DSM-defined depressive disorders (e.g., loss of appetite or loss of energy) may be a direct consequence of medical illnesses or the medications used to treat them, and the levels of these medical symptoms may change over time, as well. Consequently, whether or not an older medical inpatient meets criteria for a depressive disorder may depend in part on the point in time at which they are examined.
Two studies have yielded information about the short-term stability of depression diagnoses in this population. One study10 examined 54 older male medical-surgical inpatients (9 with major depression). The result of an assessment conducted by a lay interviewer using the Composite International Diagnostic Interview (CIDI)11 was compared with a gold-standard Longitudinal, Expert, All Data (LEAD) diagnosis.12 The LEAD diagnosis was based on assessment by a clinician using the Structured Clinical Interview for DSM (SCID) within 2 days of the lay assessment,13 and then review of the SCID assessment and any longitudinal information by a physician panel in order to reach a consensus diagnosis. Reported kappas ( ) for a current diagnosis of major depression (by DSM-III-R criteria) ranged from 0.43 to 0.53.
A second study14 examined 43 older male medical inpatients (18 with major depression). The results of an assessment by a geriatric-medicine fellow (using the DIS) were compared with a clinical assessment by a psychiatrist that was conducted a mean of 4 days later. The reported value for major depression (by DSM-III-R-criteria) was 0.4.
These two studies have five limitations. First, the number of subjects was small in both studies. Second, all subjects were men. Third, both studies involved different diagnostic instruments and were primarily studies of the validity of diagnosis. Fourth, both studies examined the diagnosis of major depression only. Fifth, neither study identified factors independently related to disagreements in diagnosis.
Because short-term stability of diagnoses has implications for clinical practice and research, the primary objectives of the study were 1) to determine the short-term stability of major and minor depression diagnoses in a large sample of older male and female medical inpatients; 2) to determine the independent influence of patient demographics (age, gender, language, education), rater, interval between assessments, number of medications, severity and duration of depressive symptoms, severity of medical illness, medical comorbidity, premorbid disability, acute physiology score, and cognitive functioning on the changes in diagnoses.
|
METHOD
|
Study Population
This study was conducted at two university-affiliated, acute-care hospitals in Montreal, Canada, as part of studies of the treatment and prognosis of depression in older medical inpatients. Recruitment was conducted by study personnel with clinical experience in the detection of depression in older patients. The sampling frame consisted of consecutive admissions of patients age 65-and-over from the emergency room to the medical services. The following patients were excluded: admissions to palliative care units (because of expected survival of less than 6 weeks); those who did not speak or understand English or French or were unable to communicate; and those who lived off the island of Montreal. Patients admitted to the intensive-care or cardiac-monitoring units were assessed after transfer to a medical ward. Eligible patients were screened by one of two trained research assistants (RAs; a nurse or an occupational therapist) using the Short Portable Mental Status Questionnaire;15 those with 5 or more errors (indicating moderate-to-severe cognitive impairment) were excluded. Patients were then assessed for depression by one of the two trained RAs using the DIS.5 All depressed patients and a systematic sample of non-depressed patients were invited to participate in the study. As soon as possible after recruitment, patients were interviewed by one of two different trained RAs (psychologists), blind to the results of the first assessment.
The study protocol was approved by the Research Ethics Committees of both hospitals. One hospital required written consent for screening; the other hospital required verbal consent using a standard form signed by the RA. Patients who were severely depressed (by clinical criteria), psychotic, or suicidal were referred to a specialist in geriatric psychiatry or a geriatrician.
Measures
Diagnostic Interview Schedule (DIS)
The depressive disorders section of the DIS5 was administered twice to all subjects, at T1 and at T2. On both occasions, only current symptoms with a duration of at least 2 weeks counted toward a diagnosis. Patients were classified as having major depression, minor depression, or no depression according to DSM-IV criteria.16 Diagnosis of depressive disorders in older medical inpatients may be problematic because many of the symptoms and signs of depression are mimicked by physical illness (e.g., loss of appetite and energy). Consequently, there are at least six different approaches to using DSM-IV criteria in this population.17 In the present study, symptoms were counted toward the diagnosis of major or minor depression with the "inclusive" approach (i.e., symptoms were counted toward the diagnosis regardless of the symptoms’ origins, whether physical illness or depression). The IRR (with a psychiatrist) was checked periodically (see section on IRR of simultaneous measures).
Duration of Depression
An estimate of the duration of the current depressive episode was based on responses to DIS questions about the duration of core depressive symptoms.
Hamilton Rating Scale for Depression (Ham-D)
The Ham-D (21-item version)18 is the most widely used interviewer rating scale for monitoring depressive symptoms and signs in treatment-outcome studies of depression. Items are rated from 0 to 4, with higher scores indicating more pathology. A total score 13 indicates clinical depression. Interrater reliability (r) was >0.75.18
Cognitive Functioning
Cognitive functioning was measured with the Mini-Mental State Exam (MMSE).19 Scores range from 30 (no impairment) to 0 (maximum impairment); a score of <24 indicates probable cognitive impairment.
Premorbid Disability
The Older Americans Research and Service (OARS) Activities of Daily Living (ADL) scale includes 7 items that assess basic or physical ADL (BADL) and 7 that assess instrumental ADL (IADL), each assessed on a 3-point scale: completely independent, partially dependent, and completely dependent.20 Patients were asked about their premorbid functioning (2 weeks before admission). Premorbid disability was classified as mild (IADL impairment only), moderate (impairment in 1–3 BADL), or severe (impaired in 4+ BADL).
Comorbidity
The Charlson Comorbidity Index (CCI) is a weighted scale that takes into account both the number and severity of comorbid conditions.21 The CCI was derived from a chart review of diagnoses present at or during the 2 years before the time of study enrollment.
Clinical Severity of Illness
The clinical severity of medical illness was assessed at screening.22 The score ranges from 1 (not ill) to 9 (terminal).
Acute Physiology Score (APS; APACHE II)
This is a composite index measuring physiological stability.23 We used the Acute Physiology Score only (without the Glasgow Coma Scale). The score was coded with data from the hospital chart and laboratory databases, using measures made on or just before the date of enrollment.
Number of Medications
The total number of medications patients were taking at admission was determined by chart review.
Other Variables
Other variables included sociodemographic data (age, gender, native language, years of education) and number of days between T1 and T2.
Interrater Reliability of Simultaneous Measures
The interrater reliability of simultaneous depression measures was assessed in a convenience sample of 28 patients at intervals throughout the study period, using independent simultaneous ratings of the same patient by two or more raters. During these sessions, an RA interviewed the patient and was observed by the study psychiatrist and one or more other raters, who made independent ratings.
Statistical Methods
The degree of agreement between the two raters at the initial (T1) and second assessment (T2) on binary judgments—namely, presence or absence of symptoms and diagnosis of major or minor depression—was evaluated by calculating the coefficients.24 Since the DIS is an ordinal scale, the degree of agreement of the two raters on this assessment was evaluated by calculating the weighted coefficient.24
The associations between patient and rater characteristics and disagreement in detecting depressive symptoms and diagnostic category were studied by multivariable logistic regression. For each symptom and each diagnostic category (major and minor depression), patients were divided into two groups, according to whether or not research personnel had detected the symptom or the diagnostic category at the baseline assessment. Then, for each symptom and diagnostic category and for each group of patients, a logistic-regression model was constructed from the data, the binary response being defined as disagreement between T1 and T2 (1: disagreement; 0: agreement).
All models were based on covariates that might influence interview performance or stability of depression symptoms; namely: age, gender, language, education, cognitive functioning, rater, physical health status, medications, and duration and severity of depression symptoms. Because the interval between T1 and T2 assessments varied between 1 and 51 days, we decided to take advantage of this variability to examine the effect of time-interval on short-term stability; thus, the time-interval (in three categories: 0–3 days, 4–14 days, or 15+ days) was included as a covariate in the analysis. Finally, we repeated the analysis including only those subjects with an interval of 4–14 days between T1 and T2 (N=125).
|
RESULTS
|
The characteristics of patients enrolled in the study are presented in Figure 1 and Table 1. Most were moderate-to-severely disabled women. Their mean age was 79.2 years. The prevalence rates of major and minor depression (on the basis of the first assessment) in this population were 18.1% and 10.3%, respectively.
For the simultaneous assessments, the coefficient for a diagnosis of major depression versus minor or no depression was 0.78 (95% confidence interval [CI]: 0.52–1.00), and for a diagnosis of either major or minor versus no depression was 0.61 (0.35–0.87). The IRR of the other measures was assessed in 17 patients, with ICC values of 0.93.
According to the first assessment (Table 2), 156 patients had major depression, 89 had minor depression, and 135 had no depression; according to the second assessment, 164 patients had major depression, 70 had minor depression, and 146 had no depression. Overall weighted was 0.50 (95% CI: 0.42–0.57); unweighted was 0.48 (95% CI: 0.39–0.57) for major depression and 0.17 (95%CI: 0.06–0.28) for minor depression. Notably, 90% of patients (141/156) diagnosed with major depression at T1 had a diagnosis of a depressive disorder at T2; 72% (112/156) had a diagnosis of major depression, and 18% (29/156) had a diagnosis of minor depression. Sixty-two percent of patients (55/89) diagnosed with minor depression at T1 had a diagnosis of depressive disorder at T2; 31% (28/89) had a diagnosis of major depression, and 31% (27/89) had a diagnosis of minor depression.
The prevalence of symptoms (unweighted at T1; Table 3) ranged from 21% to 69%. Percent agreement ranged from 61% to 78%; values ranged from 0.22 to 0.53. Notably, the value for at least one of the two core depression symptoms (depressed mood or loss of interest or pleasure) was 0.53 (95% CI: 0.44–0.63) whereas the value for any four other depression symptoms was 0.39 (95% CI: 0.30–0.48).
The odds ratios (ORs) of non-identical symptoms and diagnosis between assessments by covariate when the symptoms or diagnosis were present at T1 and the ORs of non-identical symptoms and diagnosis between assessments by covariate when the symptom or diagnosis was not present at T1 were most often statistically significant (although in opposite directions) for at least moderately-severe depression (Ham-D18). To a less frequent extent, the ORs were statistically significant for moderate-to-severe disability and duration of symptoms 6 months. None of the demographic factors, measures of physical illness and medications, specific rater, or interval between assessments were related consistently to changes in symptoms or diagnoses.
Some associations differed by specific depressive disorder. For major depression, more severe depression (Ham-D 18), moderate-to-severe disability and longer duration of symptoms were significantly associated with a decreased probability of change in diagnosis between assessments. For minor depression, greater comorbidity was significantly associated with an increased probability of change in diagnosis. When we repeated the analysis including only subjects with an interval of 4 to 14 days between T1 and T2 (N=125), the results were similar to the results described above.
|
DISCUSSION
|
We proposed to determine the short-term stability of diagnoses of major and minor depression and its associated factors in older medical inpatients. The value for major depression was moderate (0.48)25 and similar to values reported for older, medically ill populations,10,14 despite differences in study designs. The value for minor depression was low (0.17). The value for any depressive disorder was also moderate, (0.50), even though 90% and 62% of patients diagnosed with major or minor depression, respectively, at T1 were diagnosed with a depressive disorder at T2. The value for the presence of at least one of the two core symptoms of depression (depressed mood or loss of interest or pleasure) was 0.53, versus 0.39 for other depression symptoms. The presence of at least moderately-severe depression symptoms and, to a lesser extent, moderate-to-severe disability and duration of symptoms of 6 months or more were associated with higher short-term stability.
Four potential sources of variation may have influenced short-term stability in this study: differences in interviewer style, changes in the patient’s story, and true changes in depression symptoms or differences in medical symptoms. The use of a structured interview schedule (the DIS), well-trained interviewers, and the relatively high values achieved during independent simultaneous ratings of the same patient by two or more raters throughout the study suggest that the first two sources of variation were not major influences. It seems more likely that short-term stability was affected by true changes in symptoms over time, especially other depression symptoms, many of which may have been affected by changes in the patient’s medical condition.
These findings have clinical and research implications. First, a diagnosis of major depression (using the inclusive approach) appears to be moderately stable and may justify initiation of treatment or enrollment in clinical studies. Second, a diagnosis of minor depression appears to be much less stable, and may indicate a need to reassess the patient before making a decision about treatment or study enrollment. Finally, emphasis on the presence of more severe symptoms, moderate-to-severe disability, duration of symptoms of 6 months or more, or the presence of core symptoms (depressed mood, loss of interest or pleasure), rather than the presence of other depression symptoms, may lead to more stable diagnoses of depressive disorders in this population.
Future research in this area should be of two types: First, studies should investigate approaches to improving the short-term stability of diagnoses of major and minor depression in this population. Although there are different approaches to the application of DSM criteria,17,26 the results do not suggest that these other approaches would lead to more stable diagnoses. Perhaps the use of a depression severity (i.e., Ham-D 18) and disability threshold, consideration of symptom duration, and emphasis on the presence of core symptoms would improve stability. Second, longitudinal studies should investigate the prognostic significance of the changes in diagnosis.
This study has three strengths: First, the number of subjects with major or minor depression was relatively large. Second, the test–retest design involved comparisons between different pairs of interviewers who assessed patients independently at different points in time. Of possible designs, the test–retest design provides the closest approximation to the actual clinical situation. Third, the multivariate analysis included many clinical and design variables to determine those that independently influenced short-term stability.
This study has seven potential limitations. First, we used the "inclusive" approach to the application of DSM-IV criteria; however, this approach is the most reliable method of assessing symptoms in this population,14 and the best overall approach when the longitudinal course of symptoms is taken into account.17 Second, we could not determine whether the changes in other depression symptoms were due to changes in physical or psychological status because we did not collect multiple longitudinal and independent measures of the nature and severity of medical illnesses. Third, there was no a-priori definition of the T1–T2 interval because this study was a secondary analysis of data from a longitudinal study in which the two assessment points were designed to be as close as possible; however, the time interval (as a covariate) was not related to stability, and the results of the analysis that included only subjects with a T1–T2 interval of 4 to 14 days were similar to the main analysis. Fourth, the design of the study could have been improved by scheduling independent assessments by two pairs of RAs. Such a design would have allowed closer examination of variability related to interviewers; however, this was not feasible in our setting. Fifth, different pairs of RAs were used at T1 and T2; ideally, RAs should have been randomly allocated to complete the first or second assessment, but this was not possible for logistical reasons. Sixth, concerns may be raised about possible (upward) biases in the calculations of values because we enriched the analysis sample with cases of major and minor depression; however, we repeated the calculation, adjusting for the base rates of these two disorders in our population, and found only negligible changes in the values. Finally, interpretation of the value is complicated. Although there are guidelines to determine when the coefficient is acceptable,24,25 the meaning of the coefficient depends on the number of "easy" and "difficult-to-diagnose" cases included in the study sample, the base rate(s) of the disorder(s), and the sensitivity and specificity of the instrument/criteria in the population studied. We have estimates of base rates of the disorders in our sample, but ease of diagnosis and sensitivity and specificity in this population would be difficult to determine.
To conclude, a diagnosis of major depression in older medical inpatients appears to be moderately stable; a diagnosis of minor depression appears to have low short-term stability. Short-term stability may be improved by emphasis on the presence of the core depression symptoms (depressed mood, loss of interest or pleasure), more severe depression symptoms, moderate-to-severe disability, and duration of symptoms of 6 months or more.
|
ACKNOWLEDGMENTS
|
This study was funded by the Canadian Institutes of Health Research Grants MCT-15476 and MOP-42529.
|
REFERENCES
|
- Grove WM, Andreasen NC, McDonald-Scott P, et al: Reliability studies of psychiatric diagnosis: theory and Practice. Arch Gen Psychiatry 1981; 38:408–413[Abstract]
- Streiner DL, Norman GR: Health Measurement Scales. A Practical Guide to Their Development and Use, 2nd Edition. New York, Oxford University Press, 1995, pp 28-53
- Wing JK, Birley JLT, Cooper JE, et al: Reliability of a procedure for measuring and classifying present psychiatric state. Br J Psychiatry 1967; 113:499–515[Abstract/Free Full Text]
- Spitzer RL, Endicott J, Robins E: Research diagnostic criteria: rationale and reliability. Arch Gen Psychiatry 1978; 35:773–782[Abstract]
- Robins LN, Helzer JE, Croughan J, et al: National Institute of Mental Health Diagnostic Interview Schedule: its history, characteristics, and validity. Arch Gen Psychiatry 1981; 38:381–389[Abstract]
- Burnam A, Karno M, Hough RL, et al: The Spanish Diagnostic Interview Schedule. Arch Gen Psychiatry 1983; 40:1189–1196[Abstract]
- Robins LN, Helzer JE, Ratcliff KS, et al: Validity of the Diagnostic Interview Schedule, Version II: DSM-III diagnoses. Psychol Med 1982; 12:855–870[Medline]
- Angst J, Merikangas K: The depressive spectrum: diagnostic classification and course. J Affect Disord 1997; 45:31–40[CrossRef][Medline]
- Judd LL, Akiskal HS, Maser JD, et al: A prospective, 12-year study of subsyndromal and syndromal depressive symptoms in unipolar major depressive disorders. Arch Gen Psychiatry 1998; 55:694–700[Abstract/Free Full Text]
- Booth BM, Kirchner JE, Hamilton G, et al: Diagnosing depression in the medically ill: validity of a lay-administered structured diagnostic interview. J Psychiatr Res 1998; 32:353–360[CrossRef][Medline]
- Robins LN, Wing J, Wittchen HU, et al: The Composite International Diagnostic Interview. Arch Gen Psychiatry 1988; 45:1069–1077[Abstract]
- Spitzer RL: Psychiatric diagnosis: are clinicians still necessary? Compr Psychiatry 1983; 24:399–411[CrossRef][Medline]
- Spitzer RL, Williams JBW, Gibbon M, et al: The Structured Clinical Interview for DSM-III-R (SCID) I: history, rationale, and description. Arch Gen Psychiatry 1992; 49:624–629[Abstract]
- Koenig HG, Goli V, Shelp F, et al: Major depression and the NIMH Diagnostic Interview Schedule: validation in medically ill hospitalized patients. Int J Psychiatry Med 1989; 19:123–132[Medline]
- Pfeiffer E: A short portable mental status questionnaire for the assessment of organic brain deficit in elderly patients. J Am Geriatr Soc 1975; 23:433–441[Medline]
- American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders, 4th Edition. Washington, DC, American Psychiatric Association, 1994
- Koenig HG, George LK, Petersen BL, et al: Depression in medically ill, hospitalized older adults: prevalence, characteristics, and course of symptoms according to six diagnostic schemes. Am J Psychiatry 1997; 154:1376–1383[Abstract]
- Hamilton M: Development of a rating scale for primary depressive illness. Br J Soc Clin Psychol 1967; 6:278–296[Medline]
- Folstein MF, Folstein SE, McHugh PR: "Mini-Mental State:" a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975; 12:189–198[CrossRef][Medline]
- Fillenbaum GG: Multidimensional functional assessment of older adults: The Duke Older Americans Resources and Services Procedures. Mahwah, NJ, Lawrence Erlbaum Associates, Inc., 1988
- Charlson ME, Pompei P, Ales KL, et al: A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987; 40:373–383[CrossRef][Medline]
- Charlson ME, Sax FL, MacKenzie R, et al: Assessing illness severity: does clinical judgment work? J Chronic Dis 1986; 39:439–452[CrossRef][Medline]
- Knaus W, Draper E, Wagner D, et al: APACHE II: a severity-of-disease classification system. Crit Care Med 1985; 13:818–829[Medline]
- Fleiss J: The measurement of interrater agreement, in Statistical Methods for Rates and Proportions, 2nd Edition. New York, Wiley, 1981, p 321
- McGinn T, Wyer PC, Newman TB, et al: Tips for learners of evidence-based medicine, 3: measures of observer variability (kappa statistic). CMAJ 2004; 171:1369–1373[Free Full Text]
- Cohen-Cole SA, Stoudemire A: Major depression and physical illness: special considerations in diagnosis and biologic treatment. Psychiatr Clin North Am 1987; 10:1–17[Medline]
Get information about faster international access.
Privacy Policy
Copyright © 2007
Academy of Psychosomatic Medicine.
All rights reserved.
Home
| Search
| Current Issue
| Past Issues
| Subscribe
| All APPI Journals
| Help
| Contact Us
|
TOP
ABSTRACT
INTRODUCTION
