Psychosomatics 48:338-347, July-August 2007
doi: 10.1176/appi.psy.48.4.338
© 2007 Academy of Psychosomatic Medicine
Recognition of Depression in Medical Patients With Heart Failure
Harold G. Koenig, M.D.
Received January 23, 2006; revised April 28, 2006; accepted May 3, 2006. From the Duke University Medical Center and GRECC VA Medical Center. Send correspondence and reprint requests to Dr. Koenig, Box 3400, Duke University Medical Center, Durham, NC 27710. e-mail: koenig{at}geri.duke.edu
© 2007 The Academy of Psychosomatic Medicine
|
ABSTRACT
|
The author examined physician and patient factors related to recognition of depression in depressed medical patients. Medical inpatients over age 50 were systematically identified with depressive disorder (N=1,000). Medical physicians (N=422) treating these patients were asked whether they believed patients had depression warranting specific treatment. Frequency of seeing and treating older depressed patients and attitudes toward treatment effectiveness were key factors related to physicians’ recognition of depression. Patient factors were younger age, white race, female gender, and persistence of depression after discharge. Although physicians’ intuition about depression course was often correct, persistent depression was not recognized in nearly 40% of patients.
|
INTRODUCTION
|
For patients with congestive heart failure (CHF), depression accounts for nearly $5 billion of the $20 billion total treatment costs,1 and severely depressed CHF patients show a fourfold increase in mortality.2 Approximately 14 to 20 million Americans have chronic pulmonary disease (CPD), which is the fourth leading cause of death in the United States,3 and death rates are likewise increased by over threefold among depressed patients.4 Studies using structured psychiatric interviews have reported that depressive disorder is present in 36%–59% of medical inpatients with CHF (16%–22% with minor and 20%–37% with major depression). Depression rates in patients with CPD are also high (7%–57%).5 This is particularly true for hospitalized CPD patients, where the rate of depressive disorder is close to 60% (unpublished data). Thus, a significant proportion of hospitalized patients with CHF/CPD (CHF and/or CPD) have depressive disorders that interfere with functioning, quality of life, and medical outcomes. Underrecognition of depression is widespread among older medical patients in general and CHF/CPD patients in particular6 (over 90% in one study7).
Physician and patient characteristics that might help explain this are poorly understood. A MEDLINE review of the psychiatric and primary-care literature between 1968 and 2005 found only three studies that addressed physician factors.8–10 None of these studies were conducted in hospitalized patients or those with a specific medical disorder, nor did they examine the effects of physician characteristics on treatment of individual patients or have information on the future course of depression. In a recent large study of depressed CHF/CPD inpatients systematically identified by use of a structured psychiatric interview, the authors documented a low rate of depression treatment (under 50%) by physicians and infrequent referral for psychiatric consultation (less than 10%).11,12 In the present report from this study, we hypothesize that there are physician factors (demographic and attitude) and patient factors (demographic and health characteristics), besides severity or type of depression, that influence physicians’ recognition of depression in patients with CHF/CPD.
|
METHOD
|
This report is part of a larger project ("parent" study) whose primary aim was to examine the predictors and course of depression in CHF/CPD patients. A secondary question involved predictors of physician recognition of depression. Patients in the parent study were consecutively-admitted patients age 50 or over with diagnoses of CHF/CPD admitted to Duke University Medical Center (DUMC: >1,000 beds) or three nearby community hospitals (102–369 beds). Trained psychiatric research nurses identified depressive disorder by use of the Structured Clinical Interview for DSM–IV (SCID–I/NP, Version 2.013). Severity of depression was assessed with the 17-item Hamilton Rating Scale for Depression (Ham-D).14 Previous episodes of depression similar to the current one were determined. We assessed severity of CHF/CPD with the Dyspnea subscale of the Chronic Heart Failure/Chronic Respiratory Disease Questionnaire.15 Overall severity of medical illness (all disorders) was measured with the clinician-rated Cumulative Illness Rating Scale (CIRS).16 Research nurses followed up patients at 6 weeks by telephone and at 12 weeks in person. The Longitudinal Interval Follow-up Evaluation (LIFE–II)17 was used to collect information retrospectively on course of depressive disorder since the last contact and allowed weekly psychiatric ratings (PSR) from 1 ("usual self") to 6 (definite criteria for severe depression). Patients were categorized as remitted if they had PSR ratings of 1 or 2 for 2 consecutive weeks.18 After a depression patient was identified, research nurses contacted the treating hospital physician (HP) to notify him/her that their patient met research criteria for a current depressive disorder and then asked the HP to complete two questionnaires: Questionnaire A (Q-A), which asked about the HP’s personal demographics, attitudes, and behaviors related to their usual treatment of older depressed patients, and Questionnaire B (Q-B), which asked questions related to this particular patient’s depression (see Appendix 1). Question #1 of Q-B asked, "Do you believe that Mr./Ms. X has a depression warranting some type of specific treatment?"
Six weeks after the baseline evaluation, patients were contacted and asked whether they had seen their primary-care physician (PCP) at least once since hospital discharge and asked for permission to contact that PCP; if the patient had not seen their PCP by 6 weeks, then permission was sought at the 12-week follow-up. Once patient permission was obtained, the PCP was sent two questionnaires in the mail: Q-A (same questionnaire as the HP, above, received) and Questionnaire C (Q-C), which contained Question #1 of Q-B along with others (see Appendix 1). For this report, the only information relevant from Q-B and Q-C is whether the HP or PCP believed the patient had depression. In some cases, the HP was also the PCP for a patient; in that case, Question #1 was answered only on Q-B. If the physician had several patients enrolled in the depression study, he or she was asked to fill out a separate Q-B and/or Q-C for each patient. Physicians were paid $20 for each questionnaire completed.
Statistical Analysis
Three categories of variables: 1) physician characteristics; 2) physician behaviors/attitudes; and 3) patient characteristics/outcome (see Table 1) were examined as predictors of physicians’ recognition of depression (Question #1). Since each physician could fill out multiple questionnaires, depending on how many of their patients were enrolled in the study, the MIXED procedure in SAS was used, with Physician ID in the class statement and Patient in the repeated statement.19 In bivariate analyses, only a single predictor was entered into the model with the dependent variable (Table 2 and Table 3). For multivariate analyses within categories, variables from the bivariate analyses in each category significant at the p  0.05 were entered into the model, and nonsignificant variables (p <0.05) were eliminated using a backward stepwise procedure. For multivariate analyses across categories, significant variables from the within-category multivariate models were entered into a final model, and nonsignificant variables were eliminated, as above. The significance criterion was set at p=0.05 and was not adjusted for multiple comparisons because of the exploratory nature of this research.
|
RESULTS
|
Patients
A total of 1,000 patients (see Table 1 for patient characteristics) were identified with depressive disorder. Depression diagnosis required impairment of psychological, social, or occupational functioning. Follow-up data on depression course were obtained on 86.5% of all patients.
Physicians
A total of 422 physicians (HP and PCP) returned Q-A with usable data (see Table 1 for physician characteristics). The response rate was 80% for HPs but only 50% for PCPs (see below). Note that physicians were required to complete Q-A before filling out Q-B or Q-C; 14% of physicians (N=59) were both the HP and the PCP, so they filled out both Q-B and Q-C (answering Question #1 on Q-B only). Physicians’ usual treatment of depression (Q-A) in older patients and their attitudes toward treatment have been reported elsewhere,20 but are summarized below.
Behaviors and Attitudes About one-third (37%) indicated that they saw between 5 and 9 depressed older patients per month, and an additional 44% said that they saw 10 or more. Questions were asked about the average number (per month) of older depressed patients who were started on antidepressants (52% started 2), were referred for counseling/psychotherapy (86% referred 2), and were referred to psychiatrists (89% referred 2), whether they thought these treatments were effective (70%–83% said they did not help a lot), and reasons why they might not treat older patients with depression (62%: patient resistant to treatment; 61%: medical issues too pressing; 56%: unsure of depression diagnosis, etc.).
Recognition of Depression After being told that their patient met research criteria for depressive disorder, physicians were asked whether they believed the patient had depression warranting treatment (Question #1). Belief that the patient was depressed was predicted by several physician and patient characteristics. Results are reported first for HPs who treated patients during their hospital stay and then for PCPs who treated patients after discharge.
For hospital physicians (HPs), a total of 286 answered Question #1 of Q-B for 801 of 1,000 patients (80% response rate). The number of HPs was less than 1,000 because many HPs treated more than one patient in the study (1 to 26 patients, especially for community physicians). In 511 of 801 patients (63.8%), HPs indicated that they believed depression warranting treatment was present. Bivariate analyses (Table 2) indicated that physicians’ characteristics related to recognition of depression were older age, internal-medicine specialty, training (for attending and community physicians), seeing fewer patients over age 55, and seeing 10 or more older depressed patients per month. Recognition of depression was also more common among physicians who felt that antidepressants helped patients a lot, those more certain of the depression diagnosis, and, curiously, physicians indicating that patients could not afford treatment.
Patient characteristics favoring HP recognition of depression were younger age, white race, female gender, being hospitalized in the community (versus DUMC), having more severe medical illness (CIRS), having major or more severe depression, and not remitting during follow-up.
Multivariate analyses within categories indicated that HPs’ recognition of depression was more common in physicians with fewer older persons in their practice and, not surprisingly, those who reported seeing more depressed older persons per month. Likewise, those who thought antidepressants helped patients a lot and those who were more certain of the depression diagnosis were more likely to believe the patient was depressed.
Patient characteristics predictive of physicians’ recognition of depression were younger age, white race, female gender, severity of medical illness, and depression not remitting during follow-up. In the final multivariate model, the only physician characteristics predicting depression-recognition were seeing 10 or more depressed older patients per month and believing that antidepressants helped patients a lot. Patient characteristics that predicted HPs’ recognition were younger age, white race, female gender, and depression not remitting during follow-up. Despite the fact that physicians usually recognized patients with a depression that would have a chronic course, there were many exceptions. On one hand, numerous patients in whom they did not recognize depression had not remitted at 12-week follow-up (38.8%; N=100), and most of these patients (71.7%) did not receive any treatment for depression during their hospitalization. On the other hand, many patients whom physicians thought were depressed ultimately remitted from depression (40.6%; N=179), the majority of these receiving no treatment (58.7%). Thus, physician accuracy in recognition of depression, based on the eventual course of depression, was good but not exemplary.
For primary-care physicians (PCPs), a total of 197 completed Q-C for 416 patients, with each PCP completing Q-C for 1 to 22 patients. After excluding Q-C for 77 patients where the PCP was also the HP and excluding 54 patients where the PCP did not answer Question #1 on Q-C, this left 285 patients for whom 178 PCPs answered Question #1. Although the response rate by PCPs to Q-C was low, it was higher than 29% (285/1,000) for the following reasons: 1) 14% of patients were lost to follow-up by Week 6 and an additional 6% by Week 12, so that 20% of patients were not available to give permission to contact their PCP; 2) 8% (77/1,000) were excluded because their PCP was also the HP; and 3) an estimated 15% had not seen their PCP by the time of the 6- or 12-week follow-up or refused to consent to have their PCP contacted. This reduced the number of questionnaires sent to PCPs to 573 of 1,000 patients, giving an estimated response rate of 50% (285 of 573). The 285 patients with completed PCP questionnaires were compared with the 715 patients who were either ineligible (PCP was never contacted to fill out questionnaire or PCP was same as HP and not included) or whose PCP did not return the questionnaire or answer Question #1.
This analysis revealed that there was no significant difference in age, race, gender, or diagnosis of major versus minor depression. There was a small difference (all p <0.05) favoring patients with completed PCP questionnaires in terms of greater education (11.9 versus 11.2 years), diagnosis of CHF (49% versus 42%), and less severe medical illness (12.0 versus 12.6).
Also, a larger difference was found in type of hospital where patient was admitted. Among patients whose PCPs completed questionnaires, 69.8% were admitted to DUMC, as compared with 30.2% of patients admitted to community hospitals (p <0.0001). Thus, results here likely generalize more to PCPs with patients hospitalized at academic medical centers like DUMC than to community hospitals. PCPs believed that 180 of 285 patients (63.2%) had a depression warranting specific treatment.
Several physician and patient characteristics predicted depression recognition. Bivariate analyses (Table 3) indicated that recognition was more common by female physicians. Recognition was also more common among physicians who started more than two older patients on antidepressants per month and who were more certain about the effectiveness of treatments. Patient characteristics favoring PCP recognition of depression were white race, CPD, history of depression, major depression (versus minor), more severe depression, and depression not remitting during follow-up. Multivariate analyses within categories indicated that besides being female, physicians more likely to recognize depression were those who started two or more older depressed patients per month on antidepressants and were more certain of treatment effectiveness.
Patient characteristics most strongly associated with PCP recognition of depression were white race, a history of depression, and depression less likely to remit during follow-up. The final multivariate model indicated that PCPs who started two or more older patients on antidepressants per month and were more certain of treatment effectiveness were more likely to recognize depression. The only patient characteristics independently related to PCP recognition were white race and depression not remitting during follow-up. PCP and HP recognition of a treatable depression was a strong predictor of whether the patient actually received treatment (either antidepressants or psychotherapy) during hospitalization (F[511]=42.8; p<0.0001, for HPs and F[121]=45.3; p<0.0001, for PCPs).
|
DISCUSSION
|
This study examines how the characteristics of physicians and patients influence physician belief that a patient has depression warranting specific treatment. The uniqueness of this study is the large number of physicians, the large number of depressed patients with a specific medical illness (CHF/CPD), the detailed information on both groups (including course of depression after discharge), and the ability to link physicians’ characteristics and attitudes to specific patients. Patients were diagnosed with current depressive disorder by use of a structure psychiatric interview during their hospital stay, and physicians were asked whether they believed the patient was depressed only after they were informed that the patient met criteria for a depressive disorder. Although physicians agreed in two-thirds of cases, fewer patients than this received treatment or had psychiatric consultations, even those with severe (major) depression. One would think that HPs who treated the patients in the hospital when diagnosed with depression would be more likely to agree with depression diagnoses than would PCPs who saw patients several weeks after discharge (sometimes after patients had remitted from depression). However, this was not the case; HPs and PCPs agreed with the depression diagnosis almost equally (63.8% versus 63.2%). HPs most likely to believe that the patient had a depression warranting treatment were those who reported seeing more depressed older patients per month and who felt that antidepressants helped a lot (i.e., had more experience with geriatric depression and believed in the efficacy of treatment). PCPs’ recognition of depression was also predicted by number of older depressed patients started on antidepressants and belief in the effectiveness of treatments. If a physician regularly identifies older patients with depression and believes that there is treatment that can help these patients, then they will probably be more alert for this diagnosis.
Whether patients had CHF or CPD did not influence either HP or PCP depression recognition. As noted earlier, study after study has reported a high rate of depression in older medical inpatients, and there is evidence that treatment with antidepressants and/or counseling can be helpful.21–23 Failure to recognize and treat depression, then, is a problem. Physicians acknowledged that they were often unsure of the diagnosis or the efficacy of treatment, and those with less training (house staff) were less likely to believe that patients were depressed (at least in the bivariate analysis). Interestingly, belief that patients were resistant to treatment or that they did not have enough time to treat depression (the most common reasons given for failure to treat) had no impact whatsoever on the belief that a particular patient was depressed.
Patient characteristics were also relevant to physicians’ belief in the presence of depression, especially for the HP. Physicians were more likely to diagnose depression if the patient was younger, white, or female, independent of severity, type, or course of depression. Failure to recognize depression in older patients, black patients, or men may indicate the effects of bias, stereotyping, or the value placed on the quality of life for these patients. Some physicians may think that older patients should be depressed, that black patients cannot afford or need treatment, or that women are more emotional and prone to depression than men. These attitudes may blind the physician to the presence of depression. The authors found that, for both HPs and PCPs, the strongest predictor of depression recognition was depression that did not remit during follow-up. PCPs likely saw patients after many of them had remitted from depression, so this finding in PCPs is not surprising. However, this was also true for HPs, who seemed to intuitively know which depressed patients would get better and which would not, and they were often accurate in this assessment. This intuition, however, was by no means 100%; in fact, nearly 40% of patients whom HPs believed were not depressed continued to have significant symptoms throughout the 12-week follow-up (72% without treatment during hospitalization), and over 40% of those whom HPs believed were depressed ultimately remitted during follow-up (59% without treatment).
Little is known about factors that influence medical physicians’ belief that a medical patient is depressed and warrants treatment, and, until now, nothing was known about this in older medical patients with CHF/CPD. In a study of 55 physicians treating 83 medical outpatients diagnosed with depression by the Diagnostic Interview Schedule, Robbins and colleagues8 found that physicians more sensitive to nonverbal expressions of emotion and those who tended to blame patients less for exaggerating or prolonging their depression were more accurate in recognizing depression.
In a study of primary-care physicians in the United States and Canada, Main and colleagues9 found that clinician training in depression, beliefs about the discomfort associated with having depression, and self-efficacy in diagnosing and treating depression determined whether they thought depression was an important primary-care concern; this, however, was not examined in relationship to individual patient treatment. Similarly, in the present study, self-efficacy in diagnosis of depression (at least in bivariate analyses) for HPs and belief in the effectiveness of treatments for PCPs were associated with recognition of depression. Finally, Sliman and colleagues10 examined recognition of depression by internal-medicine residents in medical outpatients identified as depressed by the Beck Depression Inventory. Residents’ assessment of patients’ depression status was poor; however, they were more likely to rate patients as depressed if they had a previous psychiatric history. No specific physician characteristic (years of training or gender) predicted accuracy in depression assessment.
In the present study, we also found that interns, residents, or fellows were less likely to recognize depression diagnoses than were attending and seasoned physicians in community hospitals. This raises concern about the training of medical physicians, and whether it sufficiently emphasizes the recognition and treatment of depression in older patients. Like Sliman and colleagues,10 the author found, at least among PCPs, that a patient’s history of psychiatric problems—depression in particular—was a predictor of physicians’ recognition of depression. PCPs probably knew these patients better than HPs did, and if patients had a history of depression, then the PCP would be more likely to acknowledge that it may still be a problem.
Strengths and Limitations
The low response rate by PCPs (50%) may have affected rates of PCP recognition of depression and therefore makes comparison between PCPs and HPs difficult. Also, PCPs were more likely to respond if they were DUMC (versus community) physicians, thereby affecting the generalizability of results to community settings. Finally, a significant percentage of patients remitted from depression by the time they saw their PCP (29% by 6 weeks and 50% by 12 weeks), further affecting recognition rates by PCPs. Other limitations apply to the study as a whole. First, the large proportion of physicians in training at DUMC may have affected the generalizability of results to more seasoned clinicians, who were more likely to recognize depression. Second, it may not be entirely accurate that this study measures only physicians’ recognition of depression, since nurses told them that patients were depressed; rather, the authors measured their agreement with the research diagnoses. Third, some patients may have refused or postponed treatment offered in the acute hospital setting, thus affecting treatment rates. Fourth, medical symptoms may have been mislabeled as symptoms of depression in some patients, affecting the validity of depression diagnoses; all patients, however, had to have at least 2 weeks of depressed mood or pervasive loss of interest, which are not likely to be confused with medical illness.
Study strengths include the large number of patients and physicians, the 80% response rate for HPs, the use of a structured psychiatric interview (SCID) to diagnose depression, the 86% patient follow-up with the LIFE, and the ability to connect physicians’ characteristics with beliefs about depression in individual patients.
|
CONCLUSION
|
When told that their patient met criteria for a depressive disorder, only two-thirds of physicians agreed with the diagnosis and need for treatment. Although physicians were often quite accurate in assessment (based on remission of depression after hospitalization), nearly 40% of the time they were wrong, and these patients continued to suffer from depression for months after discharge (most without treatment). For physicians managing patients in the hospital, there are patient characteristics (age, gender, race) and physicians’ attitudes toward treatment that influence whether or not they believe that patients have depressive disorder warranting treatment, and these factors are independent of the type of depressive disorder, severity of depression, or the course of depression after discharge. This may be partially the result of bias or stereotyping. Medical training programs should emphasize the recognition and treatment of depression in older patients with CHF/CPD and the necessity for referral if patients are not responding to treatment.
|
ACKNOWLEDGMENTS
|
Funding was provided by NIMH Grant R01-MH57662.
|
REFERENCES
|
- Sullivan M, Simon G, Spertus J, et al: Depression-related costs in heart failure care. Arch Intern Med 2002; 162:1860–1866[Abstract/Free Full Text]
- Murberg TA, Bru E, Svebak S, et al: Depressed mood and subjective health symptoms as predictors of mortality in patients with congestive heart failure: a two-year follow-up study. Int J Psychiatry Med 1999; 29:311–326[CrossRef][Medline]
- Tiep BL: Disease management of COPD with pulmonary rehabilitation. Chest 1997; 112:1630–1656[Medline]
- Almagro P, Calbo E, Ochoa de Echaguen A, et al: Mortality after hospitalization for COPD. Chest 2002; 121:1441–1448[CrossRef][Medline]
- Mikkelsen RL, Middelboe T, Pisinger C, et al: Anxiety and depression in patients with chronic obstructive pulmonary disease (COPD): a review. Nord J Psychiatry 2004; 58:65–70[CrossRef][Medline]
- Kunik ME, Roundy K, Veazey C, et al: Surprisingly high prevalence of anxiety and depression in chronic breathing disorders. Chest 2005; 127:1205–1211[CrossRef][Medline]
- Rapp SR, Parisi SA, Walsh DA, et al: Detecting depression in elderly medical inpatients. J Consult Clin Psychol 1988; 56:509–513[CrossRef][Medline]
- Robbins JM, Kirmayer LJ, Cathebras P, et al: Physician characteristics and the recognition of depression and anxiety in primary care. Med Care 1994; 32:795–812[Medline]
- Main DS, Lutz LJ, Barrett JE, et al: The role of primary care clinician attitudes, beliefs, and training in the diagnosis and treatment of depression: a report from the Ambulatory Sentinel Practice Network, Inc. Arch Fam Med 1993; 2:1061–1066[Abstract/Free Full Text]
- Sliman RJ, Donohue TA, Jarjoura D, et al: Recognition of depression by internal-medicine residents. J Community Health 1992; 17:143–152[CrossRef][Medline]
- Koenig HG: Depression outcome in medical inpatients with congestive heart failure. Arch Intern Med 2006; 166:991–996[Abstract/Free Full Text]
- Koenig HG: Depression outcome in medical inpatients with chronic pulmonary disease. Am J Geriatr Psychiatry 2006; 14:211–219[CrossRef][Medline]
- First MB, Spitzer RL, Gibbon M, et al: Structured Clinical Interview for DSM-IV Axis I Disorders, Non-Patient Edition (SCID-I/NP, Version 2.0). New York, Biometrics Research Department, N.Y. State Psychiatric Institute, 1996
- Hamilton M: Development of a rating scale for primary depressive illness. Br J Soc Clin Psychol 1967; 6:278–296[Medline]
- Guyatt GH, Nogradi S, Halcrow S, et al: Development and testing of a new measure of health status for clinical trials in heart failure. J Gen Intern Med 1989; 42:101–107
- Linn B, Linn M, Gurel L: Cumulative Illness Rating Scale. J Am Geriatr Soc 1968; 16:622–626[Medline]
- Keller M, Lavori P, Friedman B, et al: The longitudinal interval follow-up evaluation: a comprehensive method for assessing outcome in prospective longitudinal studies. Arch Gen Psychiatry 1987; 44:540–548[Abstract/Free Full Text]
- Frank E, Prien RF, Jarrett RB, et al: Conceptualization and rationale for consensus definitions of terms in major depressive disorder: remission, recovery, relapse, and recurrence. Arch Gen Psychiatry 1991; 48:851–855[Abstract/Free Full Text]
- SAS/STAT Software Changes and Enhancements Through Release 6.11; SAS/STAT Software Changes and Enhancements for Release 6.12, Cary, NC, SAS Institute, Inc., 1996
- Koenig HG: Physician attitudes toward treatment of depression in older medical inpatients. Aging Ment Health 2006; 11:197–204[CrossRef]
- Harpole LH, Williams JW Jr, Olsen MK, et al: Improving depression outcomes in older adults with comorbid medical illness. Gen Hosp Psychiatry 2005; 27:4–12[CrossRef][Medline]
- Simon GE, Ludman EJ, Tutty S, et al: Telephone psychotherapy and telephone care management for primary-care patients starting antidepressant treatment: a randomized, controlled trial. JAMA 2004; 292:935–942[Abstract/Free Full Text]
- Thomas L, Mulsant BH, Solano FX, et al: Response speed and rate of remission in primary and specialty care of elderly patients with depression. Am J Geriatr Psychiatry 2002; 10:583–591[CrossRef][Medline]
Get information about faster international access.
Privacy Policy
Copyright © 2007
Academy of Psychosomatic Medicine.
All rights reserved.
Home
| Search
| Current Issue
| Past Issues
| Subscribe
| All APPI Journals
| Help
| Contact Us
|
TOP
ABSTRACT
INTRODUCTION
