​​The Diagnostic and Statistical Manual of Mental Disorders by Thomas A. Widiger

Review – DSM-IV Sourcebook
Volume 1
by Thomas A. Widiger, Allen J. Francis, Harold Alan Pincus (editors)
American Psychiatric Press, 1994
Review by Jeffrey S. Poland, Ph.D.
Apr 7th 2001 (Volume 5, Issue 14)

The deep cultural entrenchment of the DSM system of classification is based to a significant extent upon its claim to scientific credibility, a claim which has been repeatedly made and nurtured by its developers. The system is alleged to be grounded on a massive scientific research effort which has informed, and continues to inform, the processes of development which led to DSM-III (1980) and to its successors, DSM-III-R (1986) and DSM-IV (1994). These same processes are now in place for the production of DSM-V, expected to be published sometime in the middle of the current decade. As a consequence of this scientific research framework and the decision making processes informed by it, the DSM is now presented by its developers as a system of classification which rests upon an empirical basis and whose development is appropriately and rigorously responsive to evolving science. As we shall see below, this image of the DSM is somewhat overstated.

Building upon the system presented in DSM-III-R, the process by which decisions were made regarding production of DSM-IV was a complex hierarchical process that proceeded roughly as follows. Final decisional authority for revision of DSM-III-R rested with the Board of Trustees of the American Psychiatric Association, although the final decisions were informed by the APA membership and by recommendations of the Task Force assigned the job of proposing and attempting to resolve issues concerning: the addition or deletion of diagnostic categories and sub-categories, the development or modification of diagnostic criteria associated with existing or new categories, the development or modification of text associated with existing or new categories, and any other issues associated with the production of the DSM-IV manual. The Task Force, appointed by the Board, was composed of 27 individuals responsible for oversight of the development process and for making final recommendations to the Board. Many members of the Task Force also functioned as chairs or co-chairs of the 13 work groups, each responsible for the study of a specific set of mental disorders. The work groups were charged with identifying issues and possible options for resolution concerning their assigned disorders, developing an empirical basis for deciding among the options, and making recommendations regarding each of the issues. The work groups were also provided with general guidelines regarding how they were to proceed with, and report upon, their various tasks. In addition, they worked in concert with a large number of advisors, each of whom was invited to participate in the process because of their possession of relevant expertise regarding particular mental disorders or types of issues.

The apparently overriding and frequently rehearsed charges to the groups were to pursue their deliberations in an open-minded fashion, to base their deliberations and proposed changes as much as possible on the “best data available,” to maximize congruence with the ICD-10 as much as possible (in order to establish international consistency regarding psychiatric diagnostic practices), and to take a relatively conservative approach to revision (in order to preserve stability of criteria across editions of the DSM), and, hence, to propose changes only when there was a solid basis for doing so. The combination of these ideals and the impressively long list of distinguished consultants contributes substantially to the prima facie scientific legitimacy of the development process.

The DSM-IV Sourcebook is a four volume set of papers which are authored by participants in the development processes leading to the publication of DSM-IV and which are supposed to provide a detailed presentation of the empirical evidence and associated rationales for the many decisions involved in the production of DSM-IV. Volumes 1-3 present the literature reviews conducted by the 13 work groups. In these volumes, the critical issues addressed by the groups are formulated, the relevant scientific research based upon the literature reviews is identified, the bearing of this research on resolution of the issues is discussed, and tentative recommendations concerning possible revisions or the necessity of further research are made. This initial review process culminated in the production of the DSM-IV Options Book which was a compendium of the various recommendations for change resulting from the literature reviews and which was then widely circulated for the purpose of eliciting as much critical feedback as possible. Volume 4 of the Sourcebook presents the results of 40 data re-analyses (i.e., new analyses of existing data sets) which were proposed for the purpose of providing additional data relevant to issues not resolved by the literature reviews. Further, this volume also presents the results of 12 Field Trials conducted for the purpose of eliciting data and feedback on the potential clinical impact of changes in the DSM and on the reliability and performance characteristics of the criteria. The Field Trials were also conducted with an eye to collecting additional data to help resolve issues for which existing data were not sufficient. Lastly, volume 4 contains the final reports of each of the work groups outlining their final recommendations regarding proposed revisions. Thus, the DSM Sourcebook provides a unique glimpse of certain aspects of the DSM-IV development process, with specific emphasis on the empirical foundation and rationales for the recommendations which were made. As such, the Sourcebook may provide important insights into the character and quality of the decisions that led to the production of DSM-IV, and hence into the scientific credibility of contemporary psychiatric classification.

This last point is exceedingly important because the development of the various recent DSMs (III, III-R, IV) has taken place in an atmosphere of continuing controversy and discontent with the DSM system of classification. There are many dimensions of criticism and complaint; some concern the practical utility of the system while others bear directly upon its foundational assumptions and scientific credibility. Criticisms include: (1) continued objections that the concept of mental disorder lacks cogency and that a medical model for understanding and responding to many areas of human suffering and disability is inappropriate, (2) allegations that the methodological commitments of the approach to classification (e.g., commitment to a categorical and “atheoretical” approach) are misguided, (3) contentions that the DSM development process lacks objectivity and scientific rigor, (4) enduring charges that DSM criteria and categories have severe problems and lack reliability and validity, and (5) widespread concerns that DSM diagnostic categories lack clinical, research, and educational utility and that they are misused in a variety of contexts.

So, it is in the dual context of this broad ranging discontent with the philosophical, scientific, and practical value of the DSM combined with its deep cultural entrenchment supported by claims of scientific credibility, that I place the present review. Because of the controversial character of this context and the unique glimpse the Sourcebook provides into the DSM-IV development process, one can and should study the DSM Sourcebook with at least two questions in mind: (1) How well does the process, as reflected in the Sourcebook, live up to the ideals, aspirations, and claims of the developers?, and (2) How responsive is the process, as reflected in the Sourcebook, to the deeper concerns about the utility, fundamental commitments, and scientific credibility of the DSM-IV system of classification? In this review, I focus upon Volume 1 of the Sourcebook; subsequent reviews will be devoted to the other volumes.

Volume 1 is organized in terms of five main sections each focusing on one major thematic grouping of mental disorders as follows: Substance-Related Disorders; Delerium, Dementia, and Amnestic and Other Cognitive Disorders; Schizophrenia and Other Psychotic Disorders; Medication-Induced Movement Disorders; Sleep Disorders. Each section consists of an introductory overview chapter, discussing the main issues addressed by the work group and summarizing the findings and deliberations of the work group, and a set of detailed chapters each of which is devoted to specific issues concerning the mental disorders assigned to the group. The format of each of the 45 substantive chapters is the same, consisting of separate sections for the following: Statement of the Issues, Significance of the Issues, Methods, Results, Discussion, and Recommendations.

To begin, nowhere in these 45 chapters is serious attention given to the sorts of foundational issues of concern to the critics of the DSM (e.g., concept of mental disorder, commitment to an atheoretical categorical approach, appropriateness of a medical model, etc.) It is a serious shortcoming of the process of development to ignore or give short shrift to these matters in light of the broad ranging dissatisfaction with the DSM in clinical practice, the serious challenges which have been mounted to the commitments of the approach, and the necessity of secure foundations to the ultimate success of any approach to classification. Perhaps, Volumes 2-4 will contain such discussions, but it is more likely that such discussions will be left to future iterations of the DSM development process (e.g., in the future a Work Group should be assigned the task of studying foundational issues.)

Turning to the deliberations reported in Volume 1, we find that many were significantly influenced by the ideal of promoting coherence with the ICD-10 (in order to maximize international agreement on criteria and categories of mental disorder.) Although sensible from a certain point of view, this raises concerns about whether attempts to promote such coherence might continue the process, clearly visible in the development of DSM-III, of basing categories and criteria on “expert consensus” rather than empirical evidence and theories. To a number of critics, the source of many of the problems of the DSM is an overriding concern with achieving consensus in the absence of constraints based on scientific research. If, as appears to be the case, good quality evidence concerning construct and predictive validity of DSM (and ICD-10) categories is in short supply, existing problems will in all likelihood persist.

Indeed, the deepest problem with DSM categories in general is their lack of demonstrated construct and predictive validity. Volume 1 of the Sourcebook provides little evidence that such issues were seriously addressed, let alone the issue of whether an adequate research base exists to validate the categories. It is simply not the case, for example, that systematic reviews of treatment efficacy research were conducted for each of the disorders, nor were systematic studies of construct validity conducted for each DSM-IV category. For the most part, categories in DSM-III-R, which were included in that edition without adequate evidence of validity or clinical utility, were largely retained in DSM-IV without any new or satisfactory evidence vindicating their original inclusion in the official diagnostic classificatory system. Although there are some discussions of validity issues in Volume 1 of the Sourcebook, these discussions are neither systematic nor deep. The inertial forces operating within the DSM development process are quite powerful and they all but guarantee the retention of categories of mental disorder that were not introduced on the basis of demonstrated validity and have not been vindicated in this regard by subsequent research.

It is somewhat puzzling that a process of development, widely advertised as bringing to bear scientific research on the classification of mental disorder, would give so little attention to the central research issues for any system of classification, and hence utilize so few pertinent empirical constraints on the development of the DSM system. Perhaps part of a solution can be found in the conservative stance adopted by the developers: only those issues for which there was a significant research literature to make meaningful discussion possible were considered and changes were made only when there was a substantial basis for doing so. This stance had the dual effect of narrowing the scope of the process and of making significant changes difficult to make. As a consequence, the vast majority of categories in DSM-III-R were retained in DSM-IV with at most mild revisions (there were, of course, some additions, deletions, and revisions of categories, despite the fact that those categories have yet to be validated by scientific research.) The rationale for this conservative approach is that (1) validation requires substantial time for building an appropriate research base, (2) we are still at an immature stage of research (i.e., there is not a lot of research sufficient for establishing validity of the categories), and (3) significant changes would violate the integrity of the gradually evolving research base thereby impeding the validation effort. However, the bottom line is that DSM-IV categories, at least those studied in Volume 1 of the Sourcebook, have not been validated by scientific research. And yet they continue to inform, not only current and future scientific research, but also numerous cultural practices (e.g., clinical, legal, educational, health care). Some critics of the DSM believe that this is a wasteful and harmful prospect requiring serious and immediate attention.

With respect to how well the deliberations concerning the issues addressed in Volume 1 lived up to the ideals and claims made about the development process (e.g., open minded and balanced treatment of issues, changes made only when a significant basis exists, basing decisions as much as possible on the best available research), the work reported in Volume 1 was quite uneven and there are several sources of concern about the rigor and empirical character of the process. First, the quality of the research base is questionable: the studies considered often date back decades, the methodological quality of the studies is too frequently not addressed, the comparability of the studies is in doubt (e.g., due to different conceptualizations of the disorders), and the relevance of the studies to the questions addressed is not always clear. Second, the poverty of the research base becomes evident when it is seen that numerous issues cannot be meaningfully addressed at all, let alone resolved, because of insufficient evidence (this is frequently acknowledged by the contributors to the Sourcebook). Third, empirical evidence is not used in many decisions; rather, loosely constrained speculations regarding concepts, coherence with other systems, face validity, and consensus of the field are taken to suffice for making a decision. Fourth, even when empirical evidence is employed, it is often not clearly relevant, or, even when relevant, the decisions are only loosely connected to the evidence cited. Finally, it should be noted that, given the lack of demonstrated construct validity of the categories, many of the issues discussed are potentially quite meaningless (e.g., deliberations about accuracy of diagnosis).

Thus, as evidenced by Volume 1, the process of DSM-IV development very imperfectly lives up to the ideals, aspirations, and claims made by the developers, and it neither seriously addresses nor allays the concerns of the critics of the DSM approach to classification. As a consequence, the idea that the process exhibited in this volume places the development of the DSM classification system on a solid empirical basis cannot be seriously entertained. The real and very significant value of the Sourcebook is that it provides us with a good look at how decisions about the DSM-IV were made. Unfortunately, far from establishing the scientific credibility of the DSM, it helps us to see more accurately just how far we have to go before a credible system will be achieved.

Jeffrey Poland, University of Nebraska-Lincoln