Chen Hirschheim 2004

Blackwell Science, LtdOxford, UKISJInformation Systems Journal1350-1917Blackwell Publishing Ltd, 2004143197235Original ArticleExamination of
IS researchW Chen & R Hirschheim
Info Systems J (2004) 14, 197235
197
A paradigmatic and methodological examination of information systems research from 1991 to 2001
WenShin Chen* & Rudy Hirschheim
*Bauer College of Business, University of Houston, Houston, TX 77204-6282, USA, and Ourso College of Business Administration, Louisiana State University, Baton Rouge, LA 70803, USA, email: rudy@lsu.edu
Abstract. The eld of information systems (IS) has evolved for more than three decades. Although many schools of thought have emerged and even become well established, few historical analyses of research paradigms and methodologies have been undertaken. One of the rare exceptions is Orlikowski & Baroudi (1991). Yet, the IS research community has evolved substantially since 1991 in many aspects. A variety of journal outlets have emerged and become well established. More attention has been paid to paradigmatic and methodological issues. Political and professional contexts have also changed noticeably. Therefore, it should be an opportune time for the eld to ask: What changes are manifested in journal publications? Is the eld making progress regarding pluralism in IS research? How will the elds publications practices change in the future? The purpose of this paper is to investigate these questions and, in turn, reect on the paradigmatic and methodological progress made since 1991. We examined 1893 articles published in eight major IS publication outlets between 1991 and 2001. Our ndings suggest that the long-term endeavours of interpretivist researchers might need to continue because the paradigmatic progress appears somewhat inconsequential; positivist research still dominates 81% of published empirical research. In particular, US journals, as opposed to European journals, tend to be more positivist, quantitative, cross-sectional and survey oriented. With respect to research design, survey research is still the most widely used method (41%), although case studies have gained substantial recognition (36%). Further, the increase of qualitative research (30%), empirical studies (61%) and longitudinal cases (33%) at the expense of laboratory experiments (18%) might suggest that IS researchers have become more interested in obtaining scientic knowledge in real world settings. In summary, we suggest that the eld has been dominated by the positivist paradigm, despite calls to the contrary. Indeed, if the eld was to truly embrace pluralism, it would have to nd ways to fundamentally change the publication practices of the journal system, including the current tenure and promotion system, which pose considerable obstacles for the acceptance of alternative paradigms.
2004 Blackwell Publishing Ltd
198
W Chen & R Hirschheim
Keywords: IS interpretivism
research,
paradigms,
research
methodology,
positivism,
INTRODUCTION
There appears a growing interest as to how the information systems (IS) research community evolves with respect to philosophical and methodological assumptions. Considerable attention has been paid to the issues of research diversity and methodological pluralism during the past two decades (e.g. Hirschheim & Klein, 1989; 1992; Galliers, 1991; Klein et al., 1991; Olaisen, 1991; Walsham, 1995a; Hirschheim et al., 1996; Wicks & Freeman, 1998; Cook & Brown, 1999). Specically, alternative paradigms such as interpretivism have emerged in the eld (Walsham, 1995b) and have become more widely accepted even in traditionally positivistoriented journals such as MIS Quarterly (MISQ) (Trauth & Jessup, 2000 ). Qualitative research methods, as opposed to the dominant quantitative methods, have also become more popular in the eld (Lee, 1989; Walsham, 1995a; Silverman, 1998). Yet, the belief that alternative paradigmatic research exists within the IS eld remains largely anecdotal and conceptual (Hassard, 1988; Hirschheim & Klein, 1989; Klein et al., 1991; Olaisen, 1991). Little effort has been made to empirically examine the philosophical and methodological assumptions underlying IS journal publications. This is not to say that there are no visible examples of alternative paradigmatic publications adopting Burrell & Morgans (1979) four paradigms (functionalism, interpretivism, radical structuralism and radical humanism); their differences with respect to ontological and epistemological assumptions have been widely discussed in IS literature (cf. Hirschheim & Klein, 1989; Olaisen, 1991; Hirschheim et al., 1995; Walsham, 1995a). While conceptual discussion brings attention to the issues surrounding alternative paradigms, an empirical examination would help to reveal how such attention is reected in journal publications. After all, to discuss is one thing, to reect on real research practice is another. As such, we extend the analysis of Orlikowski & Baroudi (1991) which is one of the rare empirical examinations of IS researchs paradigmatic and methodological base. Orlikowski and Baroudi examined 155 articles published between 1985 and 1989 in MISQ, Communications of the ACM (CACM), Management Science and Proceedings of the International Conferences on Information Systems (ICIS). Their ndings indicated that the positivist paradigm overwhelmingly dominated the IS research community (96.8%) whereas little attention was paid to the interpretive paradigm (3.2%) and no empirical research work was done using a critical paradigm (0%). With such an extreme positivist dominance, it was not surprising that methods such as survey research (49.1%) and laboratory experiments (27.1%) that aim for deductive, hypothetical reasoning and generalization, were the dominant research designs. Such paradigmatic and methodological skewing in a research community might not be healthy. Orlikowski & Baroudi (1991), thus, articulated the essence of each paradigm and specied the appropriateness of employing other research methodologies to better investigate research questions of interest. While the positivist paradigm could aim for replicability and gen-
2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235
Examination of IS research
199
eralizability, the interpretivist paradigm could enhance the in-depth understanding of the phenomenon examined. Each paradigm provides unique attributes for different purposes of scientic inquiry; the same argument could also be applied to each methodology. The issue, thus, should not be simply about paradigm wars (Tashakkori & Teddlie, 1998) but about the research questions investigated. Alternative paradigms or methodologies such as interpretivism and qualitative methods should be welcomed and encouraged because they provide different dimensions for research investigation that the positivist paradigm and survey methods would not be able to accomplish. Although Orlikowski & Baroudi (1991) brought attention to the paradigmatic and methodological issues of IS research to the community, their examination was conducted more than a decade ago. Much has changed in the IS research community since 1990. First, many new journals have been established to provide more research outlets and different perspectives. For example, Information Systems Research (ISR) launched its debut issue in 1990 and became one of the top journals in the eld. Accounting, Management, and Information Technology (AMIT) (Information and Organization after 2001), Information Systems Journal (ISJ) and European Journal of Information Systems (EJIS) all started in 1991 and became some of the major research outlets, especially in Europe. The recently established electronic journals, Communications of the AIS and Journal of the AIS, are only a few years old and just nding their respective niches. Second, more attention has been paid to paradigmatic and methodological pluralism. Landry & Banville (1992), for example, highlighted the issue of the IS eld being more of a fragmented adhocracy than a unied discipline. Hirschheim & Klein (2003) took this viewpoint one step further to explain why the fragmented adhocracy occurred and how the IS community could possibly build a body of knowledge which might help establish the identity of IS as a discipline. In addition, alternative research paradigms have been considered more acceptable in the eld. For example, Wicks & Freeman (1998) challenged the traditional dominant positivist position; Walsham (1995a; 1995b) indicated the emergence of interpretivism; Klein et al. (1991) advocated the necessity of pluralism for any serious IS research agenda. Other discussions in relation to paradigmatic or methodological issues in recent years can also be found in Cook & Brown (1999), Farhoomand & Drury (1999), Goles & Hirschheim (2000), Hirschheim & Klein (1992), Iivari & Hirschheim (1996), Iivari et al. (1998), Silverman (1998) and Trauth & Jessup (2000). Furthermore, the political and professional context has also changed signicantly since 1990. For example, the mainstream journal, MISQ, publicly announced its acceptance of alternative research approaches in 1993. Allen Lee, a qualitative researcher and pluralist, was later appointed as its editor-in-chief. His effort to change the political context and advocate research pluralism should not go unnoticed. The debate between Benbasat & Weber (1996) and Robey (1996) indicates the variety of interests in the eld, even among some of the most well known researchers. While Benbasat and Weber argue for a unied discipline, Robey advocated the necessity of research diversity and suggested specic ways to achieve it. Their argument should also provoke deeper discussion in the community as to how the eld should be established. Moreover, a special issue of MISQ in March 1999 gathered a group of well-known
200
researchers (Applegate & King, 1999; Benbasat & Zmud, 1999; Davenport & Markus, 1999; Lee, 1999; Lyytinen, 1999; Markus & Lee, 1999) to debate the issue of rigour vs. relevance. Communications of the AIS also dedicated an entire issue to a similar discussion in March 2001. All of this suggests that it has been an exciting and progressive period for the IS research community since 1990. The communitys interest in paradigmatic and methodological issues has widened considerably. With an expanded variety of recently established journal outlets, alternative voices should be expected to be heard. As such, it would be reasonable to expect that these changes of paradigmatic and methodological understanding would be reected in journal publications. Our research purpose, thus, is to uncover whether the journal publications do indeed reect the changes occurring since 1990. To this end, we undertook an empirical analysis of 1893 articles published in eight major IS journals from 1991 to 2001. The selection of journals four European and four US and the period of investigation should help us understand the progress that has been made in recent years, including: between the contrast emergingthe traditionally dominant paradigm, positivismin(Orlikowski & Baroudi, 1991), and an paradigm, interpretivism (Walsham, 1995b), each journal; a cross-continental comparison between European and North American journals, as con ventional wisdom has long perceived that European journals are more receptive to interpretive
approaches while North American journals tend to be positivist; and four areas of comparison of methodology: empirical vs. non-empirical, qualitative vs. quantitative, cross-sectional vs. longitudinal, and the comparison of ve research designs (survey, laboratory experiment, eld experiment, case study and action research). Such an empirical examination would not just continue the research tradition of Orlikowski & Baroudi (1991) but also extend the empirical understanding of the current state of the IS discipline. Conceptual arguments, regardless of how convincing they might appear, would not reect the state of concurrent research practice in the journal publications. The empirical result of the study could, thus, help us to understand better how the IS research community has evolved with respect to its paradigmatic and methodological assumptions. In order to achieve this goal, the paper is organized as follows. First, we articulate our classication of paradigms and methodologies. Then, our research procedure is described especially as it relates to the journals chosen for study and the paradigms and methodologies used in the analysis. The section that follows outlines the ndings of the empirical examination of 1893 articles. The discussion of the results and the implication of the analysis are then presented in relation to our research questions. Finally, we present some concluding remarks, which reect upon our history, and we also offer some suggestions for future research endeavours.
PARADIGMATIC CLASSIFICATIONS
Considerable variation exists when it comes to classifying paradigms. For instance, Landry & Banville (1992) categorize researchers into three groups: (1) mainstream navigators who
201
inherit a logical positivism tradition; (2) unity advocates who favour methodological monism; and (3) knights of change who advocate methodological pluralism. This categorization broadly differentiates those who endorse one single paradigm (mainstream navigators and unity advocates) and those who advocate methodological pluralism (knights of change). Similar to Burrell and Morgans typology, Landry and Banvilles grouping differentiates functionalists (mainstream navigators and unity advocates) from non-functionalists (interpretivist, radical humanist and radical structuralist). According to Hirschheim & Klein (1992), neohumanism and radical structuralism have not been well developed in the IS research community.1 Orlikowski & Baroudi (1991) and Goles & Hirschheim (2000) indicate that positivism dominates IS research while other paradigms are relatively small in number. In fact, the only real alternative paradigm observable in any numbers in IS research is interpretivism (Walsham, 1995a; 1995b; Nandhakumar & Jones, 1997; Trauth & Jessup, 2000). Thus, the paradigmatic comparison in our analysis will simply focus on positivism vs. interpretivism. The major differences between positivism and interpretivism concerning research are threefold. Ontologically, positivists believe that reality exists objectively and independently from human experiences while interpretivists emphasize the subjective meaning of the reality that is constructed and reconstructed through a human and social interaction process (Burrell & Morgan, 1979). Epistemologically, positivists are concerned with the hypotheticdeductive testability of theories. Scientic knowledge should allow verication or falsication and seek generalizable results. As such, a causal relationship is usually presented and a tight coupling among explanation, prediction and control is expected (Orlikowski & Baroudi, 1991). Interpretivists, by contrast, assume that scientic knowledge should be obtained not through hypothetic-deductive reasoning but through the understanding of human and social interaction by which the subjective meaning of the reality is constructed (Walsham, 1995a). Methodologically, positivists contend that, to test hypothetic-deductive theory, research should take a value-free position and employ objective measurement to collect research evidence. A quantitative method such as the survey is a typical positivist instrument. Interpretivists, on the other hand, argue that to understand the meaning embedded in human and social interaction, researchers need to engage in the social setting investigated and learn how the interaction takes place from the participants perspective. Field studies that engage researchers in the real social setting would be more appropriate for generating interpretive knowledge (Orlikowski & Baroudi, 1991). In brief, positivist research is mostly depicted through: (1) the formulation of hypotheses, models, or causal relationships among constructs; (2) the use of quantitative methods, although not always necessary, that test theories or hypotheses; and (3) researchers objective, value-free interpretation. A classic example of positivist research listed in Hirschheim & Klein (1992) is the Dickson et al. (1977) discussion of the Minnesota experiments. Interpretivist studies, in contrast, could be observed through: (1) evidence from a non-deterministic (free
1
Alternative paradigmatic classication could be found in Chua (1986), in which the three basic paradigms are positivism, interpretivism and critical theory (Hirschheim & Klein, 2003). Critical theory, however, is not yet well established in the IS eld (Orlikowski & Baroudi, 1991).
202
will) perspective; (2) researchers engagement in the specic social and cultural setting investigated; and (3) an analysis based on participants viewpoints (Orlikowski & Baroudi, 1991; Walsham, 1995a). Examples of interpretive studies that include these attributes are Orlikowski (1993) and Orlikowski & Robey (1991).
METHODOLOGICAL CLASSIFICATIONS
Various schemes have been applied among researchers to classify research methodology. For example, Cash & Nunamaker (1989; 1990; 1991) compare qualitative research methods to experimental and survey research methods. They discovered that only 17% of the papers included in their analysis employed qualitative research methods. Alavi et al. (1989) analyzed empirical and non-empirical articles and found that empirical articles constitute only 46.5% of publications in the period between 1968 and 1988. They further divide empirical studies into eight categorizes: laboratory experiment, eld experiment, eld study, case study, survey, development of MIS instruments, ex post descriptions and others. Orlikowski & Baroudi (1991) surveyed 155 articles in four major North American journals. The major categories in their nding are surveys (49.1%), laboratory experiments (27.1%), case studies (13.5%), mixed methods (3.2%), instrument development (2.6%), protocol analysis (1.3%) and action research (0.6%). Galliers & Land (1987) and Galliers (1991) created a taxonomy for IS research approaches within which 10 modes were discussed: theorem proof, laboratory experiment, eld experiment, case study, survey, forecasting and future research, simulation and game/role playing, subjective/argumentative, descriptive/interpretive and action research. One of the more recent examinations is by Farhoomand & Drury (1999) in which the methodology of the studies they analyzed were categorized as non-empirical (39%), survey (32%), case study (17%), laboratory experiment (10%) and eld experiment (2%). Among these different taxonomies the most consistent comparisons are: empirical vs. nonempirical (Alavi et al., 1989) and quantitative vs. qualitative (Cash & Nunamaker, 1989). Detailed examinations of research designs appear diverse and ambiguous. For example, although Galliers (1991) included more research methodologies than others, forecasting/future research, simulation and game/role playing, and theorem/proof, never gained much attention in the IS community. Additionally, although Alavi et al. (1989) distinguish between case study and eld study, this distinction is not well articulated. Most case studies are conducted in the eld; hence, the attempt to separate eld study from case study appears unnecessary. Our methodological classication will primarily follow the classication of Orlikowski & Baroudi (1991) for two reasons. First, Orlikowski & Baroudi (1991) are concerned not just with methodology but also with paradigmatic trend. Thus, with a similar purpose, both studies could provide a better comparison if the same classication is employed. Second, their sample was collected between 1985 and 1989 from four major US research outlets, while ours was collected between 1991 and 2001 from eight major journals. A better reection, as to how the eld has progressed, could be made if the same classication is used. However, three categories have been modied for the following reasons:
203
1 Mixed method is not a clear category on their list. We compare mixed method with quantitative and qualitative methods. Articles that use more than one research design (e.g. laboratory experiment, case study) will be counted in each category and will also be considered as mixed method. 2 Protocol analysis and instrument development are more practitioner oriented. As this is not the main interest of our research, we place them in the others category. 3 With respect to research evolving over time, we distinguish between repeated measure design and cross-sectional, multiple snapshots. While repeated measure design involves multiple occasions, its focus is on understanding how phenomena change as time passes, which makes it more dynamic than cross-sectional, multiple snapshots.
RESEARCH PROCEDURE
Our research procedure involved two stages. The rst stage was conducted from fall 2000 to spring 2001. We collected and analysed articles published in MISQ and AMIT to capture and compare overall trends, if any, in US and European journals. Based on the results of the initial analysis, we extended article collection to an additional six research outlets and modied our categorization approach in the second stage, which was from fall 2001 to spring 2002. The comparison of research duration was extended to four categories: cross-sectional, multiple snapshots, longitudinal and repeated measure designs. We believe that this comparison presents a better picture of the different research endeavours involved. Repeated measure design is of particular interest because it emphasizes the time factor and, thus, can provide a richer understanding of the researched phenomenon. Journal representations For the purpose of illustrating the research trend in IS and comparing the traditionally dominant positivist approach (Orlikowski & Baroudi, 1991) and the emerging interpretive approach (Walsham, 1995a; 1995b), we decided to focus on mainstream journals which, we thought, would reect changes or progress in the eld. As such, we examined four mainstream US publications MISQ, ISR, Journal of Management Information Systems (JMIS) and ICIS and four main European journals2 AMIT,3 ISJ,4 Journal of Information Technology (JIT) and EJIS. The rationale behind these choices has been well explained by Walsham (1995a) and Orlikowski & Baroudi (1991). In brief, these US journals5 represent the largest forums of IS publications and arguably, the most inuential IS researchers. The
This distinction between European and US is simply based on where the journal originates, i.e. the location of the publisher. To a large extent, it also reects the composition of the editorial board (Galliers & Meadow, 2003). 3 In 2001, AMIT changed its name to Information and Organization. 4 ISJ was called Journal of Information Systems until 1997. 5 We are, of course, aware that ICIS is not a journal but its acceptance rate (traditionally around 10%) is much like the top IS journals. It also has a pre-eminent position with respect to other conferences in the scholarly community.
204
four selected European journals, on the other hand, embrace a more non-traditional, nonpositivist nature and attract IS researchers with non-traditional orientations. The contrast and comparison between these two groups should reect IS research trends and, thus, illustrate any differences within the IS research community.6 The only difference between our list and Walshams (1995a) is that we have replaced CACM with JMIS. Our reasoning is that CACM has changed so as to appeal more to general readers and members of the Association for Computing Machinery (ACM), thereby reducing the scholarly nature of its publication. Thus, the journals desire to become practical to meet the needs of the larger practitioner community makes it inappropriate for this study. Paradigmatic representations As Goles & Hirschheim (2000) indicate, Although the proportion of positivist to nonpositivist articles varies somewhat from study to study, the indisputable consensus is that positivism dominates information systems research (p. 254). Hirschheim & Klein (1992) further explain that examples of IS research in radical structuralism and neohumanism (radical humanism) are rarely found. What might gain increasing attention in IS research, according to Walsham (1995a), is interpretive approach. As such, comparing and contrasting the trend of positivist and interpretive paradigmatic approaches might better reect IS research.
Positivist The criteria for categorizing positivist articles are the indications of hypotheses, propositions, model formation, quantiable measures of variables and the inferences drawn from samples to populations (Orlikowski & Baroudi, 1991), all of which demonstrate the belief that objective data could be collected to predict the relationship among factors and to test hypotheses or theories (Walsham, 1995a).
Interpretive The criteria for selecting interpretive articles are threefold. First and foremost, articles should not involve any positivist indicators as described above, that is, no deterministic perspectives imposed by the researchers. Second, participants perspectives are taken as the primary sources of understanding and investigating the phenomena. Third, the phenomena are examined with respect to cultural or contextual circumstances (Walsham, 1995a).
6 Alavi et al. (1989) also include journals that are not in our list such as Data Base, Decision Science, Harvard Business Review, Management Science and Sloan Management Review. They are either practitioner oriented (Harvard Business Review and Sloan Management Review), not IS focus (Management Science and Decision Science) or according to Hardgrave & Walstrom (1997), not considered part of the of the top-ranking journal list (Data Base).
205
Methodological representations Our analysis of research methodology focuses on the following areas: empirical vs. nonempirical; quantitative vs. qualitative; cross-sectional vs. longitudinal; and the comparisons among ve research designs (survey, laboratory experiment, eld experiment, case study and action research). Empirical vs. non-empirical While empirical studies rely on observations and data, non-empirical studies emphasize ideas and concepts (Alavi et al., 1989). Each of which makes different contributions to the research community. While non-empirical studies help to develop concepts and build theory, empirical studies provide concrete evidence for testing theories. As a discipline becomes more mature, one might expect theory-testing to outweigh theory-building efforts. Therefore, a comparison between empirical and non-empirical studies serves as a meaningful indication of the progress made in the eld. Our criterion for categorizing studies as empirical or non-empirical is determined by whether the articles obtain real data or observations, which could be gathered through quantitative, qualitative, or a mixed approach, including archival data.7 Quantitative vs. qualitative Although qualitative research methods such as case studies have gained increasing attention (Benbasat et al., 1987; Lee, 1989; Klein et al., 1991; Trauth & OConnor, 1991; Visala, 1991; Walsham, 1995b; Darke et al., 1998; Silverman, 1998; Klein & Myers, 1999; Markus & Lee, 1999), the comparison between quantitative and qualitative research methods in IS research has been treated somewhat inadequately. While the former typically uses numerical analysis to illustrate the relationship among factors in the phenomenon studied, the latter emphasizes the description and understanding of the situation behind the factors. To make the comparison meaningful, both methods should be categorized as empirical studies. As non-empirical studies typically involve describing or arguing for theories and frameworks, they could be categorized as a qualitative research method but this would muddy the distinction of quantitative methods. As such, our criterion for categorizing research methods as quantitative or qualitative is based on whether studies use a statistical or numerical approach to collect and analyse data. There is a possibility that research could use both quantitative and qualitative methods in different stages of the study. On such occasions, we categorize them as a mixed research method. Cross-sectional vs. longitudinal As described previously, in contrast to Orlikowski & Baroudi (1991), our categorization of the duration of a research project distinguishes between repeated measure design and multiple
There are occasions when we locate articles with data/observations in the non-empirical category. In such cases, articles were merely descriptive without scholarly research questions, practitioner oriented for systems development, or mathematic simulation and analysis.
7
206
snapshots. Four categories are designated: longitudinal, repeated measure design, multiple snapshot and cross-sectional each requiring different research efforts and time commitment. We dene longitudinal as research that evolves over an uninterrupted period of time and focuses on process. Cross-sectional, on the other hand, is research that collects data through one snapshot at a particular point of time (Orlikowski & Baroudi, 1991). Multiple snapshots are also cross-sectional in nature but involve more than one single data-collection period or setting, which could mean different experiments, treatments or subjects and so on. Repeated measure designs are those that conduct studies at various time periods to examine how a phenomenon evolves at various periods of time.
Six major research designs Six research designs were identied, as per Orlikowski & Baroudi (1991): fall this category data questionnaires. Survey: Studies thatthat into involved with agather site orby means ofover a certain period of Case study: Studies are single a few sites are located in this category. The case study inquiry usually relies on multiple sources of time evidences (Yin, 1994, p. 13). Laboratory experiment: These include studies that take place within a designed, controlled environment and usually involve special treatments of different groups to contrast the precise relationships among variables (Galliers, 1991). Field experiment: As with laboratory experiments, special treatments are used to study two or more controlled groups. Field experiments differ in that they are conducted in a real-world setting. Action research: Often embodied in a case study, action research distinguishes itself in that the researchers are an integral part of the phenomenon under study. The researchers input often inuences the outcomes of the phenomenon and his/her role could change from researcher to subject (Galliers, 1991). Others: this category includes articles that are practitioner oriented (systems or tools development), non-empirical pieces, or descriptive/argumentative as noted in Gallierss (1991) classication. Research with secondary data such as public records or existing datasets is also located in this category.
The following section presents our empirical ndings. Our analysis discusses overall trends, which combines the results of all eight journals from 1991 to 2001 and a total of 1893 articles. The detailed categorization of each article is listed in the Appendix wherein readers can see how each article was located with respect to the classication criteria: (1) positivist vs. interpretive; (2) empirical vs. non-empirical; (3) quantitative vs. qualitative vs. mixed; (4) longitudinal vs. repeated measure vs. multiple snapshots vs. cross-sectional; and (5) survey vs. lab experiment vs. eld experiment vs. case study vs. action research vs. others.8 These detailed cat8 Includes subcategories of conceptual, existing (archival) data, documents/reports, meta-analysis, systems design, news data, simulation/mathematic calculation, Delphi techniques, observation and tutorials.
207
egorizations are listed by journals, and then summarized in terms of the number of articles by year and categories. More details for each journal are listed in the Appendix under the title Summary of Categorization of Journal Articles.
FINDINGS
Whilst a detailed analysis of 1893 articles might appear overwhelming, in this section we focus our analysis on three major areas that better t our research interests: (1) overall trend; (2) the cross-continental comparison; and (3) the trend of individual journals. Table 1 displays a summary of all articles as they relate to our classication criteria.
Overall trend: all journals Positivist vs. interpretive Figure 1 shows that positivist research maintains a clearly dominant position while the number of interpretive studies has increased slightly over the years. As can be seen from Table 1, the overall percentages of positivist and interpretive studies are 81% (917) and 19% (214) respectively. What is perhaps somewhat surprising is the fact that the overall trend has remained remarkably constant over the past 10 years.
Empirical vs. non-empirical Figure 2 illustrates the trend of empirical vs. non-empirical studies. In the early 1990s, nonempirical studies were in the majority in IS research. This is consistent with the nding of Alavi et al. (1989). Since 1993, however, the number of empirical-research studies has exceeded non-empirical studies and the gap has widened over the years. Moreover, among the nonempirical studies there are many articles that contain empirical data but we locate them in the non-empirical category as they are typically practitioner-oriented studies. As such, the real number of empirical-research studies, i.e. those which contain empirical data, is more than what Figure 2 illustrates. In Table 1, the overall percentages of empirical and non-empirical studies are 60% (1131) and 40% (762) respectively. This exactly mirrors the ndings of Farhoomand & Drury (1999) who examined 2098 articles published between 1985 and 1996 and noted that the breakdown between empirical and non-empirical studies was 61% and 39% respectively. Even though the journal selections are signicantly different in the two studies, the results are the same. So the dominant trend of empirical research in IS is demonstrably visible.
Quantitative vs. qualitative Figure 3 shows that quantitative research still dominates the IS discipline but the number of qualitative research papers has been increasing over the years. The gap between quantitative
208
Table 1. Summary of journals in all categories Year 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 Total Positivist 63 60 73 92 89 97 79 89 100 92 83 917 Interpretive 3 4 12 10 10 20 32 31 24 42 26 214 Empirical 66 64 85 102 99 117 111 120 124 134 109 1131 Non-empirical 82 72 64 79 76 70 66 48 72 60 73 762 Quantitative 50 49 49 73 63 65 54 59 75 70 73 680 Qualitative 6 7 22 23 29 36 47 52 34 54 31 341 Mixed 10 8 14 6 7 16 10 9 15 10 5 110 Cross-sectional 53 45 55 70 66 65 50 63 65 69 66 667
and qualitative research in 1997 and 1998 was very narrow. Subsequently, the gap has widened. Mixed method research, on the other hand, has remained steady but not substantial from 1991 to 2001. As noted in Table 1, the overall percentages of quantitative, qualitative and mixed methods are 60%, 30% and 10% respectively.
Longitudinal vs. cross-sectional Figure 4 shows that there has been a steady, continuous popular trend of cross-sectional studies although there has been a considerable growth in longitudinal studies as well. The growth of longitudinal research corresponds well with the trend toward qualitative methods, as shown in Figure 3. This should be expected because the qualitative research method usually involves case studies, which requires a long-term research process. Repeated measure design and multiple snapshots, on other hand, continue to be at and few in number, indicating a very limited growth over the decade. Figure 5 displays that the overall percentage of longitudinal, repeated measure design, multiple snapshots and cross-sectional as 33%, 4%, 4% and 59% respectively.
Major research designs Figure 6 illustrates that survey research and case studies are, by far, the most popular research designs while action research and eld experiments are rather insignicant. Laboratory experiments, on the other hand, maintain a somewhat steady position throughout the years, albeit with occasional uctuations. While the survey method remains a steady research force, case studies demonstrate a noticeable growth throughout the years. In the years 1997, 1998, 2000, the number of case studies actually exceeded that of survey research. This could be an indication that IS researchers are increasingly interested in gathering rich data through the use of the case study method.
209
longitudinal 11 12 23 22 30 40 51 52 41 56 36 374
Multiple snapshots 0 5 2 4 3 6 7 3 12 5 1 48
Repeated measures 2 2 5 6 0 6 3 2 6 4 6 42
Survey 30 38 35 52 51 44 42 41 47 35 39 454
Case study 15 16 30 25 30 42 52 56 47 53 36 402
Lab experiment 20 15 23 15 15 28 16 11 27 21 13 204
Action research 0 0 0 3 1 7 6 1 3 8 3 32
Field experiment 3 1 0 2 1 2 2 2 3 2 1 19
Others 90 81 75 94 87 87 74 64 88 85 96 921
120 100 Number of Articles 80 60 40 20 0 1991 Year 1992 1993 1994 1995
Positivist
1996
1997
Interpretive
1998
1999
2000
2001
Figure 1. Overall paradigmatic trend.
Figure 7 offers the overall percentages of survey, case study, laboratory experiment, action research and eld experiment as 41%, 36%, 18%, 3% and 2% respectively. This result differs somewhat from that of Orlikowski & Baroudi (1991) where the survey method embodied 49.1% of IS research production followed by laboratory experiments (27.1%) and case studies (13.5%). It is, however, more in line with Farhoomand & Drury (1999) where the survey, case study and laboratory experiment share was 52%, 28% and 16% respectively. This indicates that while surveys maintain their steady popular position, case studies have gained considerable interest over the decade. It must be noted, however, that research categorized as others makes up 46% of the entire research production. In another words, the IS research community still relies heavily on non-empirical studies such as conceptual/framework development.
210
160 140 120 Number of Articles 100 80 60 40 20 0 1991 Year 1992 1993 1994 1995
Empirical
1996
1997
1998
1999
2000
2001
Non-Empirical
Figure 2. Overall trend of empirical vs. non-empirical studies.
80 70 Number of Articles 60 50 40 30 20 10 0 1991 Year 2001
1992
1993
1994
1995
Quantitative
1996
1997
1998
Mixed
1999
2000
Qualitative
Figure 3. Overall trend of quantitative vs. qualitative studies.
Cross-continental comparisons US vs. European journals: positivist vs. interpretive The focus of our second analysis is on the cross-continental comparison between US and European journals. Conventional wisdom suggests that US journals are more positivist oriented and use more quantitative methods while European journals tend to be more interpretive with more qualitative methods. Here, we empirically examine whether this assumption holds true. Figure 8 illustrates that US positivist research clearly dominates with 58% while US interpretive articles account for only 7%. This clearly indicates that US research is overwhelmingly dominated by the positivist paradigm. On the other hand, the
211
80 70 60 Number of Articles 50 40 30 20 10 0 1991
1992
1993
1994
1995 Longitudinal
1996
1997 Multiple Snapshots
1998
1999
2000
2001 Year
Cross-sectional
Repeated Measures
Figure 4. Overall trend of cross-sectional vs. longitudinal studies.
Multiple Snapshots 4%
Repeated Measures 4%
Longitudinal 33%
CrossSectional 59%
Figure 5. Percentage of cross-sectional vs. longitudinal studies.
difference between European positivist (23%) and European interpretive studies (12%) is considerably less signicant. Moreover, as Figure 9 shows US positivist research not only remains a clearly dominating force but it has also maintained its predominant position over the years. The same, although to a somewhat lesser extent, can be seen in Europe where approximately two-thirds of the papers are positivist. Interpretive studies in both continents, while showing a modest increase, are still quite small in number. Interpretive researchers, expecting a more equitable publication distribution may or may not be surprised by this result.
US vs. European journals: quantitative vs. qualitative As seen in Figure 10, US quantitative research is vastly dominant (46%), but the second most popular type of publication is qualitative research in European journals (17%). It slightly edges
212
60
50
Number of Articles
40
30
20
10
0 1991
1992 Survey
1993
1994 Case Study
1995
1996
1997
1998
1999
2000
Lab Experiment
Action Research
Field Experiment
2001 Year
Figure 6. Overall trend of ve major research designs.
Lab experiment 18%
Field Action research experiment 2% 3% Survey 41%
Case study 36%

Figure 7. Percentage of ve research designs.
out qualitative research in the USA (13%) and quantitative research in European journals (14%). Mixed methods in both US and European journals are still relatively small, comprising only 6% and 4% respectively. From Figure 11, one notes that as far as the overall trend is concerned, the only method that clearly stands out throughout the years is the quantitative method in the USA. It has remained consistent over the past 11 years and even increased slightly, in recent years. The gradual increase of European qualitative research is also noteworthy. No other trend is discernible.
213
Interpretive (US) 7%
Interpretive (European) 12%
Positivist (US) 58% Positivist (European) 23%
Figure 8. Paradigmatic comparison: US vs. European journals.

80 70 60 Number of Articles 50 40 30 20 10 0 1991 Year 2001
1992
1993
1994
1995
1996
1997
1998
1999
2000
US Positivist US Interpretive
European Positivist European Interpretive
Figure 9. Paradigmatic trend: US vs. European journals.
Trends of individual journals To simplify our analysis of the individual journals, we restrict our examination to paradigmatic trend and research design. For each journal, we will present an overall paradigmatic trend and a pie chart indicating the percentage of the ve major research designs employed in the journal. The sequence of presentation is MISQ, AMIT/IO, EJIS, JIT, ISJ, ISR, JMIS and ICIS, with no particular priority.
MISQ trend The total number of MISQ articles included in our analysis is 232, broken down into 166 empirical and 66 non-empirical studies. The paradigmatic trend of MISQ shown in Figure 12 very
214
Mixed (US) 6% Qualitative (European) 17%
Mixed (European) 4% Quantitative (US) 46%
Qualitative (US) 13% Quantitative (European) 14%

Figure 10. Quantitative and qualitative percentages: US vs. European journals.
70
60
50 Number of Articles
40
30
20
10 Year 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001
0 1991
US Quantitative European Qualitative
European Quantitative US Mixed
US Qualitative European Mixed
Figure 11. Quantitative vs. qualitative trend: US vs. European journals.
closely corresponds to the overall paradigmatic trend with positivism dominating interpretivism throughout the years. Never in any given year has interpretive studies produced more than ve articles in MISQ. Occasionally (e.g. 1991, 1992 and 1995), no interpretive research appears in the journal. With respect to research design (Figure 13), survey research is still the most popular data collection method with 46% while case studies make up 34% of the research output. The other three methods appear much less signicant. Although MISQ has publicly announced
215
18 16 14 Number of Articles 12 10 8 6 4 2 0 1991 Year 1993 1995

Positivist Interpretive
1997
1999
2001
Figure 12. Paradigmatic trend of MISQ.
Action research 1% Case study 34%
Survey 46%
Field experiment 4%
Lab experiment 15%
Figure 13. Percentage of research designs in MISQ.
its interest in accepting alternative research, overall, MISQ remains a positivist (91%), quantitative (64%), cross-sectional (60%) and survey research (46%) dominated journal.
AMIT/IO trend We examined 122 articles in AMIT/IO, of which 65 were empirical and 57 non-empirical. Figure 14 depicts an interesting paradigmatic trend in AMIT/IO. The two paradigmatic lines are intertwined particularly in the mid-1990s, until interpretive studies became the mainstream in 1997. In 2001, IO published zero positivist research papers thus establishing
216
7 6 5 Number of Articles 4 3 2 1 0 1991 Year 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001
Positivist
Interpretive
Figure 14. Paradigmatic trend of AMIT/IO.
Action research 5%
Survey 14% Lab experiment 15%
Case study 64%

Figure 15. Percentage of research designs in AMIT/IO.
Field experiment 2%
interpretive research as the paradigm of choice. The overall percentage of interpretive and positivist studies in AMIT/IO are 63% and 37% respectively. Among the journals examined, AMIT/IO is the only one in which interpretive research is the mainstream research production. Figure 15 demonstrates that the most popular method in AMIT/IO is the case study making up 64% of the publications. Positivist research methods such as surveys, laboratory experiments and eld experiments have relatively marginal representation with 14%, 15% and 2% respectively. Action research appears to have gained increasing attention in this outlet with a modest total of 5% production. This should be encouraging for action researchers because most journals pay little attention to this type of research (e.g. EJIS). Clearly, this journal is receptive to alternative research approaches.
217
EJIS trend We examined 233 EJIS articles with a breakdown of 118 empirical and 115 non-empirical studies. Figure 16 indicates that EJISs paradigmatic trend looks similar to that of MISQ a clear indication of the dominance of positivist research, the past 2 years notwithstanding. In 1991, 1992, 1995 and 1996, there were no interpretive articles published in EJIS. The percentage of positivist studies in EJIS is 88%, which closely resembles that of MISQ (91%). If conventional wisdom is that European journals are more interpretive and qualitatively oriented, EJIS does not follow this trend. In terms of research design, Figure 17 shows that EJIS is clearly dominated by survey methods (51%) while action research (2%) and eld experiment (1%) are insignicant. Case studies also gain wide acceptance in EJIS although looking at its paradigmatic trend (Figure 16), it is clear that these are likely to be positivist case studies.
JIT trend We examined 198 articles in JIT with 120 empirical and 78 non-empirical studies. As Figure 18 indicates, our study only involves articles published from 1994 onwards. This was unfortunately due to our inability to obtain the earlier issues. We note that the gap between interpretive and positivist studies has narrowed noticeably since 1996. In fact, if one excludes 1994 and 1995, the difference between the number of positivist and interpretive studies is, at best, marginal. The percentages of positivist and interpretive research in JIT are 60% and 40% respectively. With respect to research design, Figure 19 reveals that JIT is receptive to qualitative research as case studies and action research share relatively high percentages of 52% and 7% respectively. Only 39% of the articles used survey methods, a somewhat low percentage relative to other outlets.
14 12 10 8 6 4 2 0 1991 Year 2001
Number of Articles
1992
1993
1994
1995
1996
1997
1998
1999
2000
Positivist
Interpretive
Figure 16. Paradigmatic trend of EJIS.
218
Survey 51%
Field experiment 1%
Lab experiment 6%
Figure 17. Percentage of research designs in EJIS.
16 14 12 Number of Articles 10 8 6 4 2 0 1991 Year 1992 1993 1994 1995 Positivist 1996 1997 Interpretive 1998 1999 2000 2001
Figure 18. Paradigmatic trend of JIT.
ISJ trend We examined 172 articles in ISJ, of which 90 were empirical and 82 non-empirical. Figure 20 demonstrates an interesting paradigmatic pattern of ISJ. Both lines of the paradigmatic trends punctuate around each other, whereas interpretive studies occasionally exceed positivist ones (e.g. 1997 and 2000). The number of interpretive studies has also shown, by and large, an increasing trend through the years. The overall percentages of positivist and interpretive studies in ISJ are 60% and 40% respectively. The percentage of research designs in ISJ also appears noticeably different. While case studies dominate the journal, a relatively high number of action research papers (12%) indicates its traditional European qualitative orientation (Figure 21). Survey research, comprising only 23%, is the lowest of any examined outlet.
219
Action research 7% Survey 39%
Case study 52% Field experiment 0%
Lab experiment 2%
Figure 19. Percentage of research designs in JIT.
10 9 8 Number of Articles 7 6 5 4 3 2 1 0 1991 Year 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001
Positivist
Interpretive
Figure 20. Paradigmatic trend of ISJ.
ISR trend Out of the 213 ISR articles examined, 108 were empirical and 105 were non-empirical. Figure 22 indicates that interpretive studies in ISR are virtually non-existent. In 6 out of 11 years, ISR published no interpretive studies. The gap between positivist and interpretive research has, if anything, increased over the years. The total percentages of positivist and interpretive studies in ISR are 94% and 6% respectively. In the 11-year period, ISR produced only seven empirical, interpretive studies. Quantitative studies make up 87% while crosssectional studies are similarly high at 74%. Like MISQ, ISR is positivist, quantitative and crosssectional oriented. Figure 23 further indicates that ISR is a traditional positivist outlet where
220
Action research 12%
Survey 23%
Case study 50%

Figure 21. Percentage of research designs in ISJ.
Field experiment 1%
Lab experiment 14%
14 12 10 8 6 4 2 0 1991 Year 1992 1993 1994 1995 1996 Positivist 1997 1998 1999 2000 2001
Number of Articles
Interpretive
Figure 22. Paradigmatic trend of ISR.
laboratory experiments (45%) and survey research methods (38%) dominate more than 80% of the published research.
JMIS trend We examined 370 JMIS articles comprising 234 empirical and 136 non-empirical studies. Figure 24 illustrates that the paradigmatic trend of JMIS also shows the extremely dominating position of positivist research throughout the years. As in the case of ISR, the overall percentages of positivist and interpretive studies in JMIS are 94% and 6% respectively. With respect to research design, Figure 25 demonstrates that surveys are the preferred instrument (50%) while qualitative approaches comprise less than one fourth of research publications (22%). If
221
Field experiment 3%
Case study 14%
Action research 0% Survey 38%
Lab experiment 45%

Figure 23. Percentage of research designs in ISR.
30
25
Number of Articles
20
15
10
5 Year 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001
0 1991
Positivist
Interpretive
Figure 24. Paradigmatic trend of JMIS.
traditional wisdom considers US journals to be more positivist and quantitative, then MISQ, ISR and JMIS certainly bear this out.
ICIS trend We examined 351 ICIS articles comprising 230 empirical and 121 non-empirical studies. Similar to the other US outlets, ICIS also strongly tends to be positivist (83%). As illustrated by Figure 26, positivist research clearly dominates interpretive studies with the number of its publications remaining high. Indeed, the gap was largest in 2001. With respect to research design, ICIS papers appear to be more equally divided among surveys (39%), case studies (36%) and laboratory experiments (22%) (Figure 27).
222
Field experiment 3% Survey 50% Lab experiment 25%

Figure 25. Percentage of research designs in JMIS.
30
25 Number of Articles
20
15
10
5 Year 1991 1992 1993 1994 1995 Positivist 1996 1997 Interpretive 1998 1999 2000 2001
Figure 26. Paradigmatic trend of ICIS.
DISCUSSION
The empirical evidence gathered from 1893 articles in eight leading IS publication outlets between 1991 and 2001 demonstrates that even after years of advocacy of paradigmatic pluralism, not much has really changed. The gap between positivist and interpretive research is still signicant. The increase of interpretive studies is still marginal and a considerable discrepancy can be easily found in MISQ, ISR, JMIS, ICIS and EJIS. In other words, ve out of eight journals are dominated by the positivist approach. This is even more apparent in ISR and JMIS where positivist research comprises 94% of the publications. The only journal in which
223
Action research 3% Case study 36% Survey 39%
Field experiment 0%
Lab experiment 22%
Figure 27. Percentage of research designs in ICIS.
interpretive research exceeds positivist research is AMIT/IO. This exception might be due to the editorial policy advocated in its debut issue: The journal is open to all forms of scholarship and analysis concerning information systems . . . . No reference discipline, research method or theoretical perspective will be preferred over another. The journal is open to established topics and approaches . . . as well as to more openly interpretive and critical analysis. (Boland & OLeary, 1991, p. 1) With respect to the cross-continental comparison, US journals tend to be dominated by the positivist approach. Despite calls for research methodological pluralism and the appointment of a sympathetic editor-in-chief of MISQ, no signicant change is reected in the mainstream research publications. A recent paper by Galliers & Meadow (2003) offers a possible explanation. While examining articles published in two mainstream North American (ISR and MISQ) and two European journals (ISJ and JSIS) from 1994 to 2000, Galliers and Meadow discovered that the ISR and MISQ editorial board members were primarily North American researchers (between 75% and 87.5% for ISR and between 51.2% and 86.8% for MISQ). These journals also tended to publish articles written by North American authors (74% for ISR and 83% for MISQ). A similar situation was found in European journals where the editorial board members were largely European as were the papers published. Galliers and Meadow concluded that the IS eld maintained a strong home grown perspective. While these results may be of little surprise to some, the research community might wish to assess what progress, if any, has been made over the last decade towards pluralism. The analysis of Orlikowski & Baroudi (1991) noted the overwhelming dominance of positivism (96.8%). Our analysis shows that the positivist paradigm still accounts for 81% of all empirical publications, and almost 86% of the US journal empirical publications (661 out of 738 articles). Clearly, paradigmatic progress over the past 11 years has been marginal. If diversity is to be encouraged to avoid an unnecessarily restrictive and insular perspective for studying IS phenomena (Orlikowski & Baroudi, 1991; Robey, 1996; Hirschheim & Klein,
224
2002a), IS researchers need to ask: What can the IS research community do to accept alternative research perspectives? Consistent results could also be observed in the comparison between quantitative and qualitative research and between longitudinal and cross-sectional studies. It is quite noticeable that quantitative and cross-sectional studies enjoyed more popular positions throughout the decade. This might be due to the publish or perish research publication games (Walsham, 1995a). Quantitative research usually employs survey questionnaires or laboratory experiments. Researchers time commitment and engagement in the research process are, relatively speaking, minimal. As pressures converge for tenure evaluation, researchers will predictably choose an easier approach to ensure that their research works are published in time for the promotion. Quantitative and/or cross-sectional studies, therefore, appear to be a better solution for that purpose. Another interesting observation is illustrated by the comparison between empirical and non-empirical studies. Our analysis demonstrates that IS researchers have increased their use of empirical data collection. Descriptive, conceptual and analytical types of research are still popular but are in decline (Alavi et al., 1989). Such a result is consistent with Farhoomand & Drury (1999) in which 61% of research is empirical in nature. This might be due to long-term advocacy and discussion of rigour and relevance. For example, Communications of AIS devoted an entire issue to the subject in March 2001 while MISQ did something similar in March 1999. As a eld matures, theoretical and conceptual developments become less appealing. Empirical studies, in contrast, become more popular because of the need for theory testing and practical relevance. The growing number of empirical studies might be an indication that the IS eld has become more mature and is continuing to evolve. With respect to research design, our ndings show some change. Orlikowski & Baroudi (1991) noted that the most frequently adopted methods were surveys (49.1%), laboratory experiments (27.1%) and case studies (13.5%). Our analysis, in contrast, demonstrates that while survey (41%) is still the most widely used research design, the number of case studies (36%) has increased substantially. Farhoomand & Drury (1999) found a similar situation with case studies comprising 28% and surveys 52% of all empirical publications. Such results might indicate that researchers are not just engaging in more empirical studies but rather paying more attention to how the phenomenon evolves. [This is based on the belief that the case study, normally conducted through an uninterrupted period, might better reveal the detailed process of the phenomenon investigated and/or even be incorporated into positivist, theorytesting research (Lee, 1991; Sarker & Lee, 1998).]
Implications Our analysis indicates that whilst there has been some paradigmatic change in the IS research community since 1990, this change has not signicantly manifested itself in journal publications. In most research journals, positivism maintains its prevailing, dominant position. This could have implications with respect to pluralists and general researchers.
225
First, if the eld were to become a pluralist discipline (Klein et al., 1991; Robey, 1996), the long-term advocacy for pluralism would need continuous and stronger endeavours. It is reasonable to claim, a single research perspective for studying information systems phenomena is unnecessarily restrictive (Orlikowski & Baroudi, 1991, p. 1). It could also be true that there are signs that interpretivism is gaining ground (Walsham, 1995b, p. 376) and There has been an increase in recent years in the number of in-depth case studies which focus on human actions and interpretations surrounding the development and use of computer-based information systems (IS) (Walsham, 1995a, p. 74). The empirical evidence shown in this study, however, suggests that while the emergence of interpretivism in IS research and the wider use of case studies might be true, their inuences are still rather insignicant, particularly given that case studies have been widely used in a positivist fashion (e.g. Swanson, 1992; Jones, 1994; Kirsch & Cummings, 1996; Nelson et al., 1997; Reeves-Ellington & Anderson, 1997). As Landry & Banville (1992) claimed: it is relatively easy to get in, but getting in the eld is quite afar from exerting a meaningful and lasting inuence (p. 86). Although plentiful activities have evolved in the IS community since 1990, to assure a long lasting inuence, pluralist researchers will need a stronger commitment and continuous effort. Particular attention could be paid to the publication system and the tenure and promotion system in the eld. Traditional wisdom has long recognized that positivist research is more easily accepted because its research tradition has been more successfully established (Hirschheim & Klein, 2002a). Positivist research is generally considered more rigorous, scientic, generalizable and reliable, and therefore it is encouraged and widely accepted (Tornatzky & Klein, 1982). As such, it is safer for authors to stick to positivist orthodoxy (Walsham, 1995b, p. 391). Walsham (1995b) further indicates: [P]ositivist research is typically less time consuming than interpretive methods such as ethnography, certainly in elapsed time terms, and thus authors are likely to choose the former in response to our publish or perish age. (p. 391) While under the pressure of tenure and promotion, researchers would tend to choose a less time consuming approach positivism to avoid perishing. This might help explain why interpretive inuences are still marginal and likely to continue to be this way. As such, for a pluralist research tradition to be established, both the publication and tenure and promotion systems might need some modications. The reason that most researchers employ a less timeconsuming approach is because for tenure and promotion, the quantity of publications counts. The less time consuming the approach, the more productions are possible; the more productions, the better the chance for tenure and promotion. While the tenure and promotion system values the quality of publications as well, it is difcult to get tenure at a good school without suitable numbers of publications. In the existing system, researchers are more concerned about how to get a research project published rather than asking signicant research questions. Many research papers fail to address meaningful issues as they are too difcult and take too long to publish. On the publication system side, pluralists need to develop ways of educating journal editors so that they would be more receptive to alternative research approaches. First, editors should
226
move beyond conservatism and parochialism (Galliers & Meadow, 2003, p. 6) and be more receptive to authors from other continents. Failing to do so could indicate that the development of the discipline itself could well be retarded by our failure to appreciate the work of colleagues elsewhere in the world . . . journal editors may well be the unwitting mouthpieces of unwitting researchers, publishing results of partial, culturally biased accounts (Galliers & Meadow, 2003, p. 6). In addition, editors traditionally play the role of gatekeeper rather than facilitator to ensure the rigour of research publications. However, such a view of rigour typically means applying a positivist approach and thus excludes other scholarly research (Hirschheim & Klein, 2002a). As Hirschheim and Klein argue, there are many scholarly vehicles for knowledge creation and they need to be recognized as rigorous as long as they employ sound forms of reasoning and evidence giving (p. 13). While positivist research might serve the purpose of generalizability, interpretivist studies aim for an in-depth understanding of meaning embedded in the phenomenon investigated (Orlikowski & Baroudi, 1991). They provide an alternative dimension for scientic inquiries and should supplement rather than exclude each other. In other words, the editors evaluation of a research article should not be based upon the approach employed, but on the reasoning and evidence provided. Editors should not reject interpretive research simply because there is no objective criterion for the evaluation. Following Habermass notion of the force of the better argument, Hirschheim & Klein (2002a) suggest three aspects for the objective evaluation of research: intelligibility, novelty and believability. Intelligibility relates to the question how well the research approach and results are comprehensible, i.e. how closely others can follow them with similar qualication. Novelty can be judged in at least three ways: (1) by the amount of new insight added; (2) by the signicance of the research reported in terms of the implications it has for seeing important matters in a new light and/or provide a new way of thinking about the phenomenon under study; (3) by the completeness and coherence of the research report(s) . . . Believability, on the other hand, relates to how well the research arguments make sense in light of our total knowledge. The key question for believability is how well the research in method and results ts with other ideas and arguments that are taken for granted within the current stage of knowledge (p. 13). These criteria could then help journal editors to evaluate research employing alternative approaches, more reasonably. If the tenure and promotion system would better incorporate the quality of articles into its evaluation criteria, it would encourage researchers to be more committed to conduct research projects that require in-depth, long-term endeavour. If the publication system could be more receptive to alternative research approaches and evaluate the research based on reasoning and argument instead of the traditional form of rigourgeneralizability, it could encourage researchers to ask different research questions requiring alternative research approaches. For a pluralist research tradition to be established, the collaboration of these two systems would then be essential. Third, our analysis could help general researchers better allocate their research endeavours in terms of their publication outcomes. For quantitative researchers, better publication outlets appear to be MISQ, ISR, EJIS, ICIS and JMIS. The survey method is prevalent in almost every journal but particularly dominates the research in these ve outlets. For qualitative researchers,
227
on the other hand, the best publication opportunities are found in Information and Organization, JIT and ISJ although MISQ, ICIS and EJIS appear increasingly receptive to alternative methods such as the case study. There are three implications of these ndings. First, for authors, recognizing the political and professional preferences of each journal could help them allocate their resources with respect to the reviewing and publication process. In responding to the publish or perish game, authors could then better ensure their survival chances. Second, for members of tenure and promotion committees, a variety of journal outlets such as Information and Organization, JIT and ISJ, among others that publish high quality research using alternative approaches, should be recognized and taken into consideration in tenure and promotion evaluations. Third, for publishers, an attempt to enhance the availability and visibility of alternative journals is critical. For example, Information and Organization is not yet electronically available which reduces its exposure to the general research community. It is, therefore, not surprising that Information and Organization, JIT and ISJ do not appear on the journal-ranking list of ISWorld. High quality research that involves a long-term commitment and is published in these journals appears not to be seriously recognized and appreciated. This may lead researchers to shun alternative research approaches because such research is not recognized in the tenure and promotion system. Clearly, this serves to dampen any drive toward pluralism in the IS eld.
Limitations While this paper provides a reection of IS research, some limitations need to be noted to avoid possible misinterpretation. First, our goal has been reection rather than generalization. For researchers who are interested in generalizing results, the scope of data collection would need to be broadened. Management Science and Organization Science, for example, are not included in our survey due to their non-IS orientation. Their inclusion might, however, have some effect on the eld. The most cited articles between 1986 and 1995, according to Walstrom & Leonard (2000), include many inuential works that are published in those journals: DeSanctis & Gallupe (1987), Daft & Lengel (1986), Swanson (1974) in Management Science; and DeSanctis & Poole (1994) in Organization Science. Therefore, for the purpose of generalization, a broader survey that includes a greater variety of journals should be encouraged. Second, while categorizing the paradigmatic position of articles, we adopted the approach of Walsham (1995a) and Orlikowski & Baroudi (1991). As Walsham (1995a) indicated, even within the interpretive paradigm, there is no homogeneous school of thought with respect to knowledge claims. Different interpretive researchers might apply different categorizing schemes with different scaling justications, resulting in different outcomes. Given this potential subtle bias, we can only claim that our ndings serve as a reection of IS research based on the Walsham (1995a) and Orlikowski & Baroudi (1991) interpretive school of thought. Finally, our analysis is based on quantity evidence. One may argue that the quantity of articles does not accurately reect their inuence. Alternative approaches, such as citation analysis, that emphasize quality should also be examined. If so, we would then conduct an analysis
228
based on the results of Walstrom & Leonard (2000). However, none of the most cited articles on their list was published between 1991 and 1995, which is the overlapping period of their and our data collection. Therefore, it would not appropriately reect the inuence of articles published between 1991 and 2001. A parallel analysis of classic articles published between 1991 and 2001 might be needed to better illustrate the inuence of research, with respect to quality.
CONCLUSION
Although many schools of thought have emerged and even become well established, reective studies that trace such historical trends appears inadequate. Hirschheim & Klein (2002b) once stated: We are not sure whether the eld considers itself too young to need such a reection or whether there simply are not enough old timers around who could provide such a view. Whatever the case, we believe this to be a serious shortcoming of the IS discipline. This paper, therefore, aims to reduce such a deciency. We have rst stated the needs of conducting such a historical reective piece, described some prior works related to paradigmatic and methodological issues, and explained our rationale of classifying paradigms and methodology. Our intent is to continue the research tradition of Orlikowski & Baroudi (1991) and reect on the progress made in the IS discipline since 1990. We have argued and illustrated that many activities have evolved in the IS research community since the paper of Orlikowski & Baroudi (1991) was published. We then ask, Are those changes reected in journal publications? Our empirical analysis of 1893 articles reveals a result similar to that of Orlikowski & Baroudi (1991), particularly in relation to the paradigmatic trend. The inuence of long-term advocacy of paradigmatic pluralism, thus, appears negligible. In other words, changes that have occurred and efforts that have been devoted since 1990 still have not manifested themselves in journal publications. This could be good news for mainstream navigators (traditional positivists) and unity advocates (Landry & Banville, 1992) because a more unied, positivist tradition may have led to a better-established IS eld thus far. Such a result might be due to the existing tenure and promotion and publication systems. The tenure system overly emphasizes the number of publications in promoting a faculty member and, thus, encourages researchers to engage in less time-consuming projects. Positivist and quantitative research, as opposed to interpretive or qualitative research, generally suits this purpose better, particularly when researchers are under tremendous publish or perish pressure. In addition, on the publication system side, journal editors and reviewers tend to be less receptive to interpretive and qualitative research because the evaluation criteria are less objective (Hirschheim & Klein, 2002a). Consequently, researchers may be more reluctant to conduct time-consuming research projects particularly in the domain of interpretivism where the reward of publication is less likely. As such, if the IS discipline believes that a monistic, Kuhnian view of IS research is inadequate (Hirschheim et al., 1996; p. 4), then a continuous commitment to paradigmatic plural-
229
ism is critical. This would likely involve the need to modify the current tenure and publication systems. While the former could consider incorporating the quality of publications into the evaluation criteria, the latter could develop a more receptive attitude toward alternative research approaches. In doing so, IS researchers would not just conceptually embrace the possibility of alternative assumptions and interests but also empirically allow the notions to be manifested in journal publications. Only when alternative approaches are empirically reected in the journal publications would the IS discipline establish itself in a pluralist research tradition that allows alternative approaches to help build its body of knowledge.
REFERENCES
Alavi, M., Carlson, P. & Brooke, G. (1989) The ecology of MIS research: a twenty year status review. In: The Proceedings of the Tenth International Conference on Information Systems, DeGross J.I., Hendersonm, J.C. & Konsynski, B.R. (eds), pp. 363375. Boston, MA. Applegate, L.M. & King, J.L. (1999) Rigor and relevance: careers on the line. MIS Quarterly, 23, 1718. Benbasat, I., Goldstein, D.K. & Mead, M. (1987) The case research strategy in studies of information systems. MIS Quarterly, September, 369386. Benbasat, I. & Weber, R. (1996) Research commentary: rethinking diversity in information systems research. Information Systems Research, 7, 389399. Benbasat, I. & Zmud, R.W. (1999) Empirical research in information systems: the practice of relevance. MIS Quarterly, 23, 316. Boland, R.J. Jr & OLeary, T. (1991) Editorial Technologies of inscribing and organizing. emerging research agendas. Accounting, Management and Information Technology, 1, 17. Burrell, G. & Morgan, G. (1979) Sociological Paradigms and Organizational Analysis. Heinemann, London, UK. Cash, J. & Nunamaker, J (1989) The Information Systems Research Challenge, Vol. I: Qualitative Research Methods. Harvard Business School Press, Boston, MA, USA. Cash, J. & Nunamaker, J (1990) The Information Systems Research Challenge, Vol. II: Experimental Research Method. Harvard Business School Press, Boston, MA, USA. Cash, J. & Nunamaker, J. (1991) The Information Systems Research Challenge, Vol. III: Survey Research Method. Harvard Business School Press, Boston, MA, USA. Chua, W.F. (1986) Radical developments in accounting thought. The Accounting Review, 61, 601632. Cook, S.D.N. & Brown, J.S. (1999) Bridging epistemologies: the generative dance between organizational knowledge and organizational knowing. Organization Science, 10, 381400. Daft, R.L. & Lengel, R.H. (1986) Organizational information requirements media richness and structural design. Management Science, 20, 178188. Darke, P., Shanks, G. & Broadbent, M. (1998) Successfully completing case study research: combining rigour, relevance and pragmatism. Information Systems Journal, 8, 273289. Davenport, T.H. & Markus, M.L. (1999) Rigor vs. relevance revisited: response to Benbasat and Zmud. MIS Quarterly, 23, 1923. DeSanctis, G. & Gallupe, R.B. (1987) A foundation for the study of group decision support systems. Management Science, 33, 589609. DeSanctis, G. & Poole, M.S. (1994) Capturing the complexity in advanced technology use: adaptive structuration theory. Organization Science, 5, 121147. Dickson, G., Senn, J. & Chervany, N. (1977) Research in management information systems: the Minnesota experiments. Management Science, 23, 913923. Farhoomand, A.F. & Drury, D.H. (1999) A historiographical examination of information systems. Communications of the Association for Information Systems, 1, 127. Galliers, R.D. (1991) Choosing appropriate information systems research approaches: a revised taxonomy. In: Information Systems Research: Contemporary Approaches and Emergent Traditions, Nissen, H.-E., Klein, H.K. & Hirschheim, R. (eds), pp. 327345. Elsevier Science Publishers, North Holland. Galliers, R.D. & Land, F. (1987) Choosing appropriate information systems research methodologies. Communications of the ACM, 30, 900902. Galliers, R.D. & Meadow, M. (2003) A discipline divided: globalization and parochialism in information systems research. Working Paper.
230
Goles, T. & Hirschheim, R. (2000) The paradigm IS dead, the paradigm is dead . . . long live the paradigm: the legacy of Burrell and Morgan. Omega, 28, 249268. Hardgrave, B.C. & Walstrom, K.A. (1997) Forums for MIS scholars. Communications of ACM, 40, 119124. Hassard, J. (1988) Overcoming hermeticism in organization theory: an alternative to paradigm incommensurability. Human Relations, 41, 247259. Hirschheim, R. & Klein, H.K. (1989) Four paradigms of information systems development. Communications of the ACM, 32, 11991216. Hirschheim, R. & Klein, H.K. (1992) Paradigmatic inuences on information systems development methodologies: evolution and conceptual advances. Advances in Computers, 34, 293392. Hirschheim, R. & Klein, H.K. (2002a) Information systems research in the making: a critical reection on the current state of the IS discipline. Working paper. Hirschheim, R. & Klein, H.K. (2002b) The rise and fall of information systems research: implications for the future of the discipline. Working paper. Hirschheim, R. & Klein, H.K. (2003) Crisis in the information systems eld? A critical reection on the state of the discipline. Journal of the Association for Information Systems, 4, 237293. Hirschheim, R., Klein, H.K. & Lyytinen, K. (1995) Information systems development and data modeling: conceptual and philosophical foundations. Cambridge University Press, Cambridge. Hirschheim, R., Klein, H.K. & Lyytinen, K. (1996) Exploring the intellectual structures of information systems development: a social action theoretic analysis. Accounting, Management and Information Technology, 6, 164. Iivari, J. & Hirschheim, R. (1992) An assumption analysis of emerging schools of information systems development. In: Proceedings of the Third Australian Conference in Information Systems, Wollongong, Australia, October 1992, pp. 115137. Iivari, J. & Hirschheim, R. (1996) Analyzing information systems development: a comparison and analysis of eight IS development approaches. Information Systems, 21, 551575. Iivari, J., Hirschheim, R. & Klein, H.K. (1998) A paradigmatic analysis contrasting information systems development approaches and methodologies. Information Systems Research, 9, 164193. Kirsch, L.J. & Cummings, L.L. (1996) Contextual inuences on self-control of is professionals engaged in systems development. Accounting, Management and Information Technology, 6, 191219.
Klein, H.K., Hirschheim, R. & Nissen, H.-E. (1991) A pluralist perspective of the information systems research arena. In: Information Systems Research: Contemporary Approaches and Emergent Trends, Nissen, H.-E., Klein, H.K. & Hirschheim, R (eds), pp. 120. Elsevier Science Publishers, Amsterdam, North Holland. Klein, H.K. & Myers, M.D. (1999) A set of principles for conducting and evaluating interpretive eld studies in information systems. MIS Quarterly, 23, 6794. Landry, M. & Banville, C. (1992) A disciplined methodological pluralism of MIS research. Accounting, Management and Information Technology, 2, 7797. Lee, A.S. (1989) A scientic methodology for MIS case studies. MIS Quarterly, 13, 3350. Lee, A.S. (1991) Integrating positivist and interpretive approaches to organizational research. Organization Science, 2, 342365. Lee, A.S. (1999) Rigor and relevance in MIS research: beyond the approach of positivism alone. MIS Quarterly, 23, 2934. Lyytinen, K. (1999) Empirical research in information systems: on the relevance of practice in thinking of is research. MIS Quarterly, 23, 2528. Markus, M.L. & Lee, A.S. (1999) Special issue on intensive research in information systems: using qualitative, interpretive, and case methods to study information technology forward. MIS Quarterly, 23, 3538. Nandhakumar, J. & Jones, M. (1997) Too close for comfort? Distance and engagement in interpretive information systems research. Information Systems Journal, 7, 109131. Nelson, H.J., Irwin, G. & Monarchi, D.E. (1997) Journeys up the mountain: different paths to learning objectoriented programming. Accounting, Management and Information Technology, 7, 5385. Olaisen, J. (1991) Pluralism or positivistic trivialism: important trends in contemporary philosophy of science. In: Information Systems Research: Contemporary Approaches and Emergent Traditions, Nissen, H.-E., Klein, H.K. & Hirschheim, R (eds), pp. 235265. Elsevier Science Publishers, North Holland. Orlikowski, W.J. (1993) CASE tools as organizational change: investigating incremental and radical changes in systems development. MIS Quarterly, 17, 309340. Orlikowski, W. & Baroudi, J.J. (1991) Studying information technology in organizations: research approaches and assumptions. Information Systems Research, 2, 128. Orlikowski, W.J. & Robey, D. (1991) Information technology and the structuring of organizations. Information Systems Research, 2, 143169.
231
Reeves-Ellington, R.H. & Anderson, A. (1997) The ethnology of information: cultural learning through cooperative action research in a multinational rm. Accounting, Management and Information Technology, 7, 139168. Robey, D. (1996) Research commentary: diversity in information systems research: threat, promise, and responsibility. Information Systems Research, 7, 400 408. Sarker, S. & Lee, A.S. (1998) Using a positivist case research methodology to test a theory about IT-enabled business process redesign. In: The Proceedings of the Nineteenth International Conference on Information Systems, Hirschheim, R., Newman, M. & DeGross, J. (eds), pp. 237252. Helsinki, Finland. Silverman, D. (1998) Qualitative research: meanings or practices? Information Systems Journal, 8, 320. Swanson, E.B. (1974) Management information systems: appreciation and involvement. Management Science, 20, 178188. Swanson, E.B. (1992) Information accessibility reconsidered. Accounting, Management and Information Technology, 2, 183196. Tashakkori, A. & Teddlie, C. (1998) Mixed Methodology: Combining Qualitative and Quantitative Approaches. Sage Publishers, London, UK. Tornatzky, L.G. & Klein, K. (1982) Innovation characteristics and innovation implementation: a meta-analysis of ndings. IEEE Transactions on Engineering Management, 29, 2845. Trauth, E.M. & Jessup, L.M. (2000) Understanding computer-mediated discussions: positivist and interpretive analyses of group support system use. MIS Quarterly, 24, 4379. Trauth, E.M. & OConnor, B. (1991) A study of the interaction between information technology society: an illustration of combined qualitative research methods. In: Information Systems Research: Contemporary Approaches and Emergent Traditions, Nissen, H.-E.H., Klein, K. & Hirschheim, R. (eds), pp. 131144. Elsevier Science Publishers, North Holland. Visala, S. (1991) Broadening the empirical framework of information systems research. In: Information Systems Research: Contemporary Approaches and Emergent Traditions, Nissen, H.-E., Klein, H.K. & Hirschheim, R. (eds), pp. 341364. Elsevier Science Publisher, North Holland. Walsham, G. (1995a) Interpretive case studies in IS research: nature and method. European Journal of Information Systems, 4, 7481.
Walsham, G. (1995b) The emergence of interpretivism in IS research. Information Systems Research, 6, 376 394. Walstrom, K.A. & Leonard, L.N.K. (2000) Citation classics from the information systems literature. Information & Management, 38, 5972. Wicks, A.C. & Freeman, R.E. (1998) Organization studies and the new pragmatism: positivism, anti-positivism, and the search for ethics. Organization Science, 9, 123 140. Yin, R.K. (1994) Case Study Research: Design and Methods, 2nd edn. SAGE Publications, Thousand Oaks, CA.
Biographies
WenShin Chen is a doctoral student at the University of Houston. He received an MBA in information systems from the University of Houston, an MS in business education from the University of Wisconsin, and a BS in industrial engineering from Tunghai University in Taiwan. His current research interests include the epistemology/methodology of IS research, mobile commerce, managerial issues of emerging technologies, cultural and sociological interpretation of information technology, organizational issues of the ubiquitous work environment, e-learning and distance education and action research in information systems. Rudy Hirschheim is the Ourso Family Distinguished Professor of Information Systems in the Information Systems and Decision Sciences Department of Louisiana State University. He was previously the Tenneco/Chase International Professor of Information Systems at the University of Houston. He has held visiting professorial positions at Monash University, University of New South Wales and the University of Bayreuth. He has previously been on the faculties of Templeton College, Oxford and the London School of Economics. He has also worked as a Senior Consultant with the National Computing Centre in Manchester, England. He received his PhD in Information Systems from the University of London. He and Richard Boland are the Consulting Editors of the John Wiley Series in Information Systems. He is on the editorial boards of Information and Organization, Information Systems Journal, Journal of Information Technology, Journal of Strategic Information Systems and Journal of the Association for Information Systems. He was also VP of Publications for the Association for Information Systems.
232
APPENDIX A: SUMMARY OF CATEGORIZATION OF JOURNAL ARTICLES
Year MISQ 12 9 4 7 6 4 9 4 7 3 3 68 AMIT/IO 5 9 3 6 6 3 5 7 4 3 6 57 18 41 6 42 6 1 1 5 2 1 1 1 2 3 1 0 1 2 5 4 5 1 3 4 4 7 5 0 1 0 0 1 3 1 0 0 0 0 1 3 3 4 6 3 4 4 4 7 3 0 0 1 0 0 1 0 0 2 0 2 0 0 0 0 1 1 0 0 0 0 0 2 1 1 6 2 0 0 1 2 1 1 0 15 0 0 0 1 2 3 0 1 1 1 0 9 1 1 3 1 1 1 0 0 2 0 0 10 0 1 0 0 0 0 0 0 0 0 0 1 1 3 4 3 6 4 3 4 4 6 5 43 0 0 0 1 0 0 1 0 0 1 0 3 106 43 17 51 10 6 99 77 26 6 58 13 12 10 10 11 8 5 7 8 11 11 1 3 5 2 3 5 4 8 3 8 1 0 2 3 1 1 2 1 1 4 1 1 2 4 7 2 2 7 5 8 6 6 2 1 0 0 1 0 1 0 1 2 2 2 0 2 1 0 0 0 1 0 2 0 0 11 11 10 10 13 7 4 7 5 12 9 8 12 6 8 7 5 4 8 7 6 6 0 2 7 1 2 3 1 1 5 3 1 2 0 0 2 0 0 1 0 0 0 1 1 6 7 2 5 6 5 8 8 8 2 0 0 0 0 0 1 0 0 1 0 0 2
PosiIntertivist pretivist Survey
Empirical
Non Empirical
Quantitative
QualiLongitative Mixed tudinal
Repeated Multiple Measure Snapshots
Cross sectional
Lab Ex.
Field Ex.
Case Action study research
Others
1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001
14 17 16 12 15 12 8 15 14 16 12
0 0 2 1 0 3 2 1 1 4 1
14 17 18 13 15 15 10 16 15 20 13
15 12 5 10 9 6 9 6 8 6 7 93
Total
151
15
166
1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001
1 0 5 3 3 4 1 2 3 2 0
1 4 5 3 4 1 4 4 4 6 5
2 4 10 6 7 5 5 6 7 8 5
5 10 6 6 6 3 6 8 4 3 6 63
Total
24
41
65
Year EJIS 17 8 13 11 12 12 11 6 9 9 7 115 JIT 63 39 16 41 3 1 73 65 8 1 51 2 6 3 4 9 7 3 10 8 6 5 2 0 0 4 4 0 4 1 7 4 6 9 1 0 2 1 1 4 1 2 4 0 0 0 0 2 4 1 6 1 7 6 5 9 1 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 6 3 7 9 7 5 11 10 7 6 2 5 3 4 9 8 5 10 9 8 3 1 1 0 0 1 0 2 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 0 1 0 6 5 0 7 2 9 8 4 9 0 0 0 0 0 0 0 0 0 2 0 19 8 14 12 10 14 11 7 10 10 8 123
PosiIntertivist pretivist Survey Others
Empirical
Non Empirical
Quantitative
Cross sectional
Lab Ex.
Field Ex.
1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001
7 3 9 12 8 11 11 12 11 8 5
0 0 1 2 0 0 1 5 3 3 6
7 3 10 14 8 11 12 17 14 11 11
2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235 13 10 13 6 5 17 6 8 78 43 63 14 64 0 4 6 7 5 5 6 5 3 6 4 8 10 10 10 6 11 4 1 0 2 2 2 4 3 0 4 8 9 10 10 8 12 3 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 7 7 7 6 7 6 5 7 52 6 6 6 6 8 6 6 5 49 1 0 0 0 0 1 0 0 2 0 0 0 0 0 0 0 0 0 4 8 10 10 11 9 11 4 67 1 0 2 2 0 1 2 1 9 15 12 14 7 6 19 6 10 89
Total
97
21
118
1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001
10 14 9 9 12 9 9 7
1 1 8 8 6 6 8 3
11 15 17 17 18 15 17 10
Total
79
41
120
233
234
APPENDIX A
cont.
Non Empirical Survey ISJ 14 11 13 5 7 9 4 7 5 2 5 82 ISR 6 7 8 13 10 13 11 8 6 11 12 105 94 8 6 10 10 5 8 3 7 5 10 7 12 13 13 11 0 0 0 0 1 3 2 1 1 0 0 1 1 2 0 0 1 1 0 0 0 0 0 1 2 0 1 3 1 0 0 1 1 0 0 0 0 0 3 2 1 1 1 2 0 0 0 0 0 2 1 1 3 1 0 8 6 8 3 7 5 6 6 11 10 10 8 80 1 3 3 4 2 2 5 4 6 1 4 35 4 4 2 3 2 8 2 3 4 6 3 41 1 0 0 0 0 0 0 0 1 1 0 3 1 1 1 0 0 4 4 1 1 0 0 13 0 0 0 0 0 0 0 0 0 0 0 0 6 10 9 13 12 14 12 13 9 16 16 130 32 48 10 52 0 3 35 20 12 1 43 1 2 1 6 5 3 1 0 3 5 5 2 1 1 4 3 4 10 7 5 8 3 0 1 0 0 0 0 1 2 3 1 2 2 2 1 3 3 3 11 8 6 8 5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 2 0 1 2 1 6 5 4 1 1 5 4 5 0 2 0 2 3 2 1 0 4 4 2 1 0 1 1 2 2 0 1 2 1 1 0 0 0 0 1 0 0 0 0 0 0 2 2 1 3 2 2 9 7 6 5 4 0 0 0 1 0 1 2 1 1 3 1 10 14 12 13 8 7 10 4 7 6 3 8 92 Quantitative QualiLongitative Mixed tudinal Repeated Multiple Measure Snapshots Cross sectional Lab Ex. Field Ex. Case Action study research Others
Year
PosiIntertivist pretivist
Empirical
1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001
3 4 1 9 6 5 4 4 8 6 6
0 0 1 1 2 2 8 5 3 8 4
3 4 2 10 8 7 12 9 11 14 10
Total
56
34
90
1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001
6 9 5 7 5 11 9 12 13 13 11
0 0 0 0 1 3 1 1 1 0 0
6 9 5 7 6 14 10 13 14 13 11
Total
101
108
Year JMIS 14 15 13 17 12 8 13 6 11 10 17 136 ICIS 14 13 10 7 13 8 7 5 13 16 15 121 144 64 22 66 6 17 13 4 12 19 11 12 8 6 18 15 26 1 1 4 3 4 9 13 10 5 9 5 4 3 4 3 2 1 0 1 0 4 0 4 2 4 3 4 7 10 9 5 11 7 0 0 2 4 0 0 0 0 0 0 0 0 1 1 2 1 2 3 1 3 2 1 14 5 13 16 12 13 8 7 15 15 23 141 7 6 9 12 9 6 3 4 7 7 13 83 7 2 6 5 4 5 2 0 7 4 4 46 0 0 0 0 0 0 0 0 1 0 0 1 4 4 6 6 3 6 12 10 5 13 7 76 0 0 0 0 1 3 1 0 0 0 1 6 16 13 13 11 16 13 10 9 16 24 21 162 180 35 19 48 7 7 172 116 59 6 51 0 11 19 14 14 16 23 17 18 19 17 12 1 0 3 2 5 0 4 5 6 5 4 4 0 3 0 2 3 3 1 0 1 2 2 0 4 2 5 2 9 6 6 6 6 0 2 1 1 0 1 1 0 0 1 0 0 2 0 0 1 0 1 0 3 0 0 14 15 15 13 17 23 13 18 16 16 12 9 12 13 10 14 15 13 7 8 7 8 6 6 4 2 4 7 10 5 5 6 4 0 0 0 0 0 2 1 2 1 0 0 5 0 5 2 6 3 7 6 6 6 5 0 0 0 0 0 0 0 0 0 0 0 15 16 15 19 15 13 15 8 16 17 20 169
PosiIntertivist pretivist Survey Others
Empirical
Non Empirical
Quantitative
Cross sectional
Lab Ex.
Field Ex.
1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001
15 19 19 16 22 26 23 21 23 18 16
1 0 1 0 1 0 1 3 2 5 2
16 19 20 16 23 26 24 24 25 23 18
Total
218
16
234
1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001
17 8 18 23 16 19 14 11 19 20 26
1 0 2 2 1 3 7 6 4 8 5
18 8 20 25 17 22 21 17 23 28 31
Total
191
39
230
235

Chen Hirschheim 2004

Загружено:

Сведения о документе

Исходное описание:

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Chen Hirschheim 2004

Загружено:

Авторское право:

Доступные форматы

Blackwell Science, LtdOxford, UKISJInformation Systems Journal1350-1917Blackwell Publishing Ltd, 2004143197235Original ArticleExamination of

IS researchW Chen & R Hirschheim

Info Systems J (2004) 14, 197235

2004 Blackwell Publishing Ltd

W Chen & R Hirschheim

2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235

2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235

W Chen & R Hirschheim

2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235

2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235

W Chen & R Hirschheim

2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235

2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235

W Chen & R Hirschheim

2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235

2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235

W Chen & R Hirschheim

2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235

2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235

W Chen & R Hirschheim

2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235

Case study 15 16 30 25 30 42 52 56 47 53 36 402

Lab experiment 20 15 23 15 15 28 16 11 27 21 13 204

Figure 1. Overall paradigmatic trend.

2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235

W Chen & R Hirschheim

Figure 2. Overall trend of empirical vs. non-empirical studies.

80 70 Number of Articles 60 50 40 30 20 10 0 1991 Year 2001

Figure 3. Overall trend of quantitative vs. qualitative studies.

2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235

80 70 60 Number of Articles 50 40 30 20 10 0 1991

1997 Multiple Snapshots

Figure 4. Overall trend of cross-sectional vs. longitudinal studies.

Figure 5. Percentage of cross-sectional vs. longitudinal studies.

2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235

W Chen & R Hirschheim

1994 Case Study

Figure 6. Overall trend of ve major research designs.

Lab experiment 18%

Field Action research experiment 2% 3% Survey 41%

Case study 36%

2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235

Interpretive (European) 12%

Positivist (US) 58% Positivist (European) 23%

Figure 8. Paradigmatic comparison: US vs. European journals.

European Positivist European Interpretive

Figure 9. Paradigmatic trend: US vs. European journals.

2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235

W Chen & R Hirschheim

Mixed (US) 6% Qualitative (European) 17%

Mixed (European) 4% Quantitative (US) 46%

Qualitative (US) 13% Quantitative (European) 14%

US Quantitative European Qualitative

European Quantitative US Mixed

US Qualitative European Mixed

Figure 11. Quantitative vs. qualitative trend: US vs. European journals.

2004 Blackwell Publishing Ltd, Information Systems Journal 14, 197235

18 16 14 Number of Articles 12 10 8 6 4 2 0 1991 Year 1993 1995

Figure 12. Paradigmatic trend of MISQ.

Action research 1% Case study 34%

Lab experiment 15%

Figure 13. Percentage of research designs in MISQ.