Control Variable Use and

Reporting in Macro and Micro

Management Research

Organizational Research Methods

15(1) 57-74
The Author(s) 2012
Reprints and permission:
sagepub.com/journalsPermissions.nav
DOI: 10.1177/1094428110397773
http://orm.sagepub.com

Guclu Atinc1, Marcia J. Simmering1, and

Mark J. Kroll2

Abstract
This study explores the use of control variables in management research, as reflected in both macro
and micro management studies published in four leading management journals. Based on a review of
812 empirical articles published from 2005 to 2009a much larger sample than was employed by
earlier studies of control variablesthe authors make several important observations. One key
finding is that, given it is not at all uncommon for the control variables included in studies to account
for more variance than the main effects, it is surprising how infrequently adequate justification for
inclusion is provided. In addition, even when justification is provided, often no expectation of the
nature of the relationship between control and dependent variables is offered. The authors also
make several recommendations for both authors and reviewers. The most important may be to
avoid simple mimicry of others and think more deeply about the theoretical foundation for the
control variables included in empirical studies.
Keywords
research design, quantitative research, survey research
In management research, scholars attempt to use research design and statistical analysis to
facilitate causal inference (Cook & Campbell, 1979). In the broadest sense, explanatory research
can be conceived of as an attempt to explain variability of the phenomena of interest (the dependent variable) by attributing it to its presumed causes (the independent variables) (Pedhazur &
Schmelkin, 1991, p. 212). A central element in empirical research is control, in which researchers
attempt to exert control in research designs in order to rule out threats to valid inferences (Cook &
Campbell, 1979). Not only do researchers want to determine whether the focal independent variables have the relationship that is hypothesized (Pedhazur & Schmelkin, 1991), but they must
minimize the possibility of confounded results that limit the explanatory power of the model
(Kish, 1959; Pedhazur & Schmelkin, 1991).

1
2

Department of Management and Information Systems, Louisiana Tech University, Ruston, LA, USA
School of Business, University of Texas at Brownsville, Brownsville, TX, USA

Corresponding Author:
Guclu Atinc, Department of Management and Information Systems, PO Box 10318, Ruston, LA, 71272
Email: gma005@latech.edu

Downloaded from orm.sagepub.com at UNIV AUTONOMA METROPOLITANA on March 3, 2015

58

Organizational Research Methods 15(1)

Kish (1959) developed a typology of sources of variation: (a) explanatory variables, (b) extraneous
variables that are controlled, (c) extraneous uncontrolled variables that are confounded with explanatory variables, and (d) extraneous uncontrolled variables that are treated as randomized error. In this
article, we address extraneous variables that are controlled. These variables may be controlled though
experimental design with manipulation, elimination or inclusion, and randomization (Pedhazur &
Schmelkin, 1991). Alternately, the researcher can statistically control for the effects of the variable
on the dependent variable. Such a variable is extraneous; that is, it is not a focal variable of the study.
It may be called a covariate, confounding variable, or nuisance variable.
Pedhazur and Schmelkin (1991) distinguish between elimination and inclusion of control
variables. Using elimination as a control technique involves holding some of the variables constant.
For example, studying only African Americans provides insight into how this particular racial group
might behave in the study context. Alternately, if people of more than one race are included in a
sample, with race measured and included as a control variable, the potential effect of race on the
dependent variable can be studied. This latter approach is the inclusion technique, and by using
it, the researcher has an opportunity to account for variance caused by race. Another form of the
inclusion technique involves using a matched-group design in which research entities (e.g., people,
organizations) that vary in terms of independent and dependent variables are matched based on a
covariate (Breaugh & Arnold, 2007). Continuing with the example above, this would mean that a
researcher would collect data from both African American participants and participants of other
races, matching them on other covariates (e.g., age, job title, etc.). In the current study, we consider
only the statistical inclusion method rather than the matched-group design or exclusion method. That
is, we investigate variables that are entered as extraneous variables with a suspected relationship
with the dependent variable.
In the simplest terms, a typical regression formula is composed of dependent and independent
variables and error terms. If a researcher decides to test a hypothesis using regression techniques,
he or she considers various independent variables relating to proposed dependent variables. Also,
if the author hypothesizes the effect of a moderating variable, he or she creates an interaction term.
In such a regression equation, the researcher also might want to account for any confounding effects
using control variables. In a sense, control variables are just like the components of the main or interaction effects and have the potential to relate to the dependent variable just like their neighbors on
the predictor side of the equation. However, the control variables or covariates are typically entered
into the hierarchical regression before other independent variables in order to determine their explanatory power exclusive of the independent variables (Tabachnik & Fidell, 2001).
For example, firm size might be a very important determinant of CEO compensation, even though
the researchers aim is to explain the impact of firm performance on compensation levels. Although
a CEO may be rewarded based on firm performance, firm size might also be related to CEO rewards.
To determine whether a statistical relationship between CEO compensation and firm performance
exists, the researcher can control for firm size in order to isolate the explanatory power of firm performance in such a model. In other words, the researcher adds firm size into the model so as to rule it
out as an alternate explanation for the findings. Yet, how such a control variable is reported in an
empirical study, to some extent, dictates its value. For instance, if the rationale for including the control variable is inadequate, if no information is provided as to how it is measured, or if no discussion
of its relationship with the dependent variable is presented, then the empirical evidence of its relation
to the dependent variable cannot be fully appreciated (Breaugh, 2008).
In this study, we analyze the degree to which macro (i.e., strategic management) and micro (i.e.,
organizational behavior and human resources) management studies properly use control variables
and adequately report their use. We first review best practices regarding control variable use and
reporting, drawing from prior studies such as Beckers (2005) and critiques such as those by Breaugh
(2006, 2008), and Spector and Brannick (2011). Next, we argue that control variable use and

Downloaded from orm.sagepub.com at UNIV AUTONOMA METROPOLITANA on March 3, 2015

Atinc et al.

59

reporting in management research is primarily driven by isomorphism. Finally, we present an

analysis of control variable use in 812 empirical articles published in four top management journals
in a 5-year span. We draw conclusions based on this data analysis and provide recommendations for
authors and reviewers regarding control variables. In our analysis, we compare control variable use
in macro versus micro management studies.

Best Practices for Control Variable Use and Reporting

Prior works have lamented the misuse of control variables in the social sciences. In particular,
Becker (2005) provided an enlightening review of the use of control variables in management
research. In his qualitative analysis, Becker argues that control variables are as important as predictor and response variables for the following reasons: (a) Improper inclusion of control variables
might produce false results; (b) in order to produce effective replications of studies, readers should
be able to determine which control variables were used and in what manner; and (c) a control variable in one study might be a predictor, mediator, interaction term, or even a response variable in
another study. After an empirical analysis of 60 studies, Becker developed recommendations for the
use of control variables in management research.
Becker (2005) recommends that authors provide readers with the reasoning behind the inclusion
of a control variable; specifically, he suggests using at least a brief explanation for why each
[control variable] was selected (p. 285). However, Becker also indicates that evidence from other
studies, or citations that provide such evidence, are preferable to a brief explanation. Further,
Spector and Brannick (2011) offer the purification principle, in which they argue that researchers
often introduce control variables without a proper basis for inclusion, under the assumption that they
will in some way purify results and expose true relationships. In addition, researchers may include
certain control variables simply because other researchers did so, without regard to the role that
they might play in relation to substantive variables (Meehl, 1971; Spector & Brannick, 2011). Thus,
researchers should provide theoretical evidence of the need for control variables and not include
such variables without thought or explanation.
A fully developed basis for inclusion of a control variable is critical because, in essence, a control
variable is simply a third variable considered in a relationship between independent and dependent
variables. Such a variable may act as a confound, a moderator, or a suppressor (MacKinnon, Krull, &
Lockwood, 2000; Spector & Brannick, 2011). For instance, if researchers do not provide an adequate
theoretical justification for including firm performance as a control variable, rather than as a
suppressor or moderator, then empirical evidence of a relation to the dependent variable cannot
be fully understood (Breaugh, 2008; Schwab, 1999). As MacKinnon et al. (2000) note, confounding
variables, mediating variables, and suppressor variables can all be estimated with the same statistical
methods, and it is only in their conceptual presentation that they truly differ.
Related to the need to offer a theoretical basis for inclusion for each control variable is the importance of indicating the nature of the relationship expected between the control variables and dependent variable. Breaugh (2008) argues that simply noting that a control variable may be correlated
with the dependent variable does not provide adequate information. He notes that there may be
instances in which the sign of the zero-order correlation between a control variable and predictor
variable may be opposite of the regression weights between the same variables. This can cause confusion in interpretation; thus, careful consideration of the conceptual relationship between control
and substantive variables is important.
Control variable reporting should not be confined to the method section of an article alone.
Indeed, to truly understand the role that confounding variables play in an empirical relationship,
it is necessary that researchers address control variables in their hypotheses, results, and discussion.
Because control variables fundamentally alter the empirical relationship between an independent

Downloaded from orm.sagepub.com at UNIV AUTONOMA METROPOLITANA on March 3, 2015

60

Organizational Research Methods 15(1)

and a dependent variable by removing variance associated with the control variable, the resulting
relationship is actually with the residual of the dependent variable (Breaugh, 2006, 2008), and this
limitation in interpretation should be noted in results and discussion sections.
Becker (2005) provides other recommendations based on his analysis of management studies.
Specifically, as noted above, researchers should provide a citation in support of each control variable
included in an article. Also, researchers should specify their measures, giving appropriate detail
about scales, dummy coding, and so on. Finally, Becker suggests that control variable information
should be presented in tables of descriptive statistics (means and standard deviations) and
correlations.

Use of Control Variables as Isomorphism

The ubiquity of control variables in management research, especially the frequency with which some
are used (such as firm size in macro papers) struck us as being something akin to an isomorphism.
Recall DiMaggio and Powells (1983) definition of isomorphism as a constraining process that forces
one unit in a population to resemble other units that face the same set of environmental conditions
(p. 149). They further envisioned a population being contained within an institutional field, which
may be conceptualized as a community of organizations and individuals that participate in common
sense-making and value systems and that are subject to commonly recognized control mechanisms
(Scott & Meyer, 1994). We would suggest that the management academy, its various institutions
(e.g., key editorial boards), and the members of those institutions constitute such a field. More accurately, we may belong to subfields (i.e., macro and micro management academies).
As is the case with various institutional fields, our managerial academic subfields are likely regulated by isomorphism related to research design. DiMaggio and Powell (1983) argued that isomorphism is the result of three mechanisms: coercion, mimicry, and normative standards. We anticipate
that all three forces are at work in the management research fields. For instance, as most members
of the management academy aspire to publish, we look to earlier articles published in the journals we
seek to publish in and may indeed feel compelled to both mimic the research designs previously published and feel coerced to include control variables that members of those editorial boards specify,
even though the case for such inclusion may not be compelling. In addition, inclusion of some control variables, such as firm size in macro papers and respondent sex in micro papers, have become so
commonplace that their inclusion may have become normative in terms of reviewer expectations.
We therefore hope that by stepping back for a moment and examining macro and micro management
research at a higher level of abstraction, we might observe patterns in research work that reflects
either best practices or mimicry and nods to convention.

Micro Versus Macro Research

Although the main intent of this article is to evaluate the degree to which researchers appropriately
use and present statistical controls in empirical articles, based on best practices recommended by
scholars, we further examine the degree to which there are differences in micro and macro management research in terms of their treatments of control variables. Strategic management studies have
improved rapidly in their attention to methodological concerns (Ketchen, Boyd, & Bergh, 2008), yet
there has not yet been a systematic investigation of the degree to which control variables are appropriately used in these subfields. Although these fields are related, they differ in terms of the types of
data typically used (e.g., primary for micro and secondary for macro), the number of control variables used, and the amount of variance explained by data. For instance, the use of archival data
in many macro studies suggests that control variables are likely to be readily available and as a result
more often used, as opposed to most primary studies, in which researchers may have limited time

Downloaded from orm.sagepub.com at UNIV AUTONOMA METROPOLITANA on March 3, 2015

Atinc et al.

61

and survey space to acquire data. Because the field of macro management is younger, there may be
different institutional forces that influence it. Thus, another goal of this article is to shed light on
current practices related to control variables across the management discipline and determine how
well they conform to best practices, with the potential for one area to inform the other in terms of
control variable use and reporting.

Method
Article Selection and Coding
To assess the degree to which researchers in management are appropriately presenting information
on and using control variables, we coded all empirical articles for a 5-year period from four top management journals. Our sample consisted of articles from the Academy of Management Journal
(AMJ), Journal of Applied Psychology (JAP), Journal of Management (JOM), and Strategic
Management Journal (SMJ) from 2005 to 2009. We chose these journals because they present
high-quality work in micro management (JAP), macro management (SMJ), or both (AMJ and JOM),
and we believed that this sample provides a high degree of generalizability for our findings. Further,
we chose this time period in an attempt to account for the most recent recommendations regarding
best practices for control variable use and presentation. Because we were interested in only those
articles that made use of control variables, we did not evaluate editorials, research forums, nonempirical review pieces, meta-analyses, and articles that contained only data simulations or methodological demonstrations. From the remaining articles, we determined whether or not the authors used
control variables; if they did not, we did no further coding of the article. Articles that included multiple studies in which the same method was used (and thus the same control variables were used and
described in the same way) were coded as a single article. An example of this would be an article that
stated, The independent variables, control variables, and dependent variables in Study 2 were measured as indicated in Study 1. Conversely, in articles that included multiple different studies (such
as those that included different variables and tested different hypotheses), we coded each study separately in a different row on the spreadsheet. The resulting population of articles that were considered for inclusion in the current study included 1,199 articles. Out of these, 387 articles did not use
any control variables, so the remaining 812 articles constitute our sample. The list of articles
included in our analyses is available upon request from the first author.
Based on our review of the control variables literature, we created a coding scheme that captures
important use and reporting issues for control variables. Using the coding categories created, the first
and second authors of the current article met to discuss the coding of four selected articles (one from
each of the four journals). Based on this initial effort, coding categories were refined. Then, the first
and second authors independently coded four more articles (again, one from each of the selected
journals) and then met to assess the level of agreement across coding categories. Agreement was
92%, and after discussion to further refine coding where discrepancies occurred, these two authors
coded the remaining articles independently. The first author coded articles in SMJ and AMJ, and the
second author coded articles in JAP and JOM. After the coding was completed, these two authors
coded a random sample of 25 articles from the other authors set independently. For these 50 articles
our agreement levels across each dimension ranged from 80% to 100%, with an average of more
than 90%, leading us to believe that the coding methods of the two authors were consistent. The specific level of agreement for each dimension is listed in the Measures section.

Measures
All articles in the sample were coded on the following dimensions, which captured basic information
about the journals and articles and also addressed best practices described above.

Downloaded from orm.sagepub.com at UNIV AUTONOMA METROPOLITANA on March 3, 2015

62

Organizational Research Methods 15(1)

Journal. Dummy codes were used to account for each of the four journals in the sample.
Publication year. The publication year (2005-2009) was recorded for each article.
Macro versus micro. Articles were coded 0 for macro studies (those with organizational-level or
industry-level outcomes) and 1 for micro studies (those with individual-level or group-level outcomes). Thus, studies that included data from multiple levels were categorized based on the level
of analysis of the dependent variable.
Separate section for control variables. A separate section for control variables may indicate more
detailed attention to these variables. Thus, we coded this variable as 1 if there was a separate heading
or subheading for control variables (or nuisance variables or covariates) and 0 if there was not. Interrater reliability on a subsample of articles was 98%.
Data source for independent and dependent variables. Using dummy codes, we recorded whether a
studys data were primary, secondary, or both, and interrater agreement on a subsample of articles
was 98%. Primary data are those that are collected by the researchers themselves, such as with a
survey (Vogt, 1993). Secondary data were not collected by the researchers themselves but are available from another source such as a database (e.g., Compustat) or company records (Vogt, 1993).
Nonperceptual control variable. If the study used at least one control variable that was nonperceptual, the code assigned to this variable was 1. Our agreement on coding (as indicated in a subsample)
was 98%. Examples of nonperceptual controls include personal characteristics such as age and sex
or firm characteristics such as industry or firm size. A study that utilized no nonperceptual control
variables was coded 0. Nonperceptual controls are often proxy variables, which are variables that are
used to substitute for theoretically specified variables or are surrogates for variables that were not
measured (Pedhazur & Schmelkin, 1991). A proxy variable may not adequately capture the true
nature of the attribute that it is intended to represent and thus may cause misinterpretation of study
results (Pedhazur & Schmelkin, 1991; Spector & Brannick, 2011).
Perceptual control variable. If the study used at least one perceptual control variable, a code of 1 was
assigned. A perceptual control variable is one about which a respondent or the study authors make a
judgment or rating. For instance, if researchers controlled for positive affect, this would be perceptual. If the study used no perceptual controls, this was coded 0. Interrater reliability on a subsample
of articles was 91%.
Perceptual control variables corrected for unreliability. Breaugh (2006) argues that if perceptual control variables are used, they should be corrected for unreliability before they are included in substantive analyses. If no correction is made, then researchers may not be fully removing the extraneous
variance in the control variable. Thus, if authors did correct for unreliability of perceptual control
variables, this variable was coded 1. If there was no correction, this variable was coded 0. Interrater
agreement on the sample of 50 articles was 100%.
Basis for inclusion. As suggested by Becker (2005), authors should provide a basis for inclusion for
each control variable. Thus, we assigned the following codes based on increasing sophistication of
authors basis for inclusion. If no rationale was provided, we assigned a value of 0. An example of
this would be, We included age and gender as control variables, with nothing more being said. We
assigned a value of 1 to this variable if authors argued that other researchers use of the control variable prompted their own inclusion of it. For example, Bolino and Turnley (2005) write, Some

Downloaded from orm.sagepub.com at UNIV AUTONOMA METROPOLITANA on March 3, 2015

Atinc et al.

63

researchers have argued that negative affectivity should be included as a control variable in studies
of stresses and strains (e.g., Burke, Brief, & George, 1993) (p. 743). A second example is, On the
basis of prior research, we controlled gender (Golden, Veiga, & Dino, 2008, p. 1416). A code of 2
was assigned to articles in which the authors argued that the control variables might be related to the
independent variable, dependent variable, or both, but without a specific theoretical argument or
explicit prior empirical findings. Thus, the following would be coded as such: Furthermore, we
include the strategic importance of the outsourced activity as a control variable, given that activities
that are considered of strategic importance are less likely to be outsourced (Verwaal, Commandeur,
& Verbeke, 2009, p. 432). Finally, a code of 3 was assigned to articles in which a theoretical basis for
including a control variable was provided or because of prior empirical evidence. An example of the
former would be,
As resource dependence theory (Pfeffer & Salancik, 1978) suggests, dependence imbalance
leads to asymmetrical power relations such that power accrues to the less dependent party.
Hence, we also controlled for interunit difference in knowledge dependence for every pair
of units. (Wong, Ho, & Lee, 2008, p. 137)
An example of the latter, from Keller (2006, p. 205), reads, Actual team size . . . was included as a
control variable because prior findings have shown it to be related to internal team communication,
team performance, and leader behaviors (Ancona & Caldwell, 1992; Bass, 1990). These two rationales were coded the same because both were believed to be strong arguments and because both
often appeared at the same time. When different arguments were given for different control variables
included in a study, we coded the best rationale provided for any of the control variables. We did so,
as a single instance in which the authors followed best practices likely reflects their awareness of
best practices. Interrater agreement on this dimension for a subsample was 80%.
Citations. Becker (2005) recommends providing citations for the control variables used in a study.
As such, we assigned a code of 0 if no citations were provided, 1 if at least one citation was provided
for a control variable, and 2 if citations were provided for all control variables. Note that, following
Becker, we judged a citation to be provided only if it was given in support of a control variables use.
That is, if a citation was offered to indicate a data source or a method of measurement for a control
variable, this was not deemed to be a citation. Our agreement level from coding a subsample of articles for this dimension was 89%.
Clarity of measurement. As with basis for inclusion, we coded this dimension based on the best
information given for any of the control variables in the study. The clarity with which the measures
are described by authors, which Becker (2005) suggests should be improved, was coded 0 if no information was given regarding measurement, 1 if the variables used for control were named, and 2 if the
variables were named and the way in which they were measured was described. Interrater reliability
on a subsample of articles was 96%.
Prediction of a sign for relationship between control and dependent variables. We coded this variable as
0 if no sign was predicted for the controldependent variable relationship and 1 if a sign was predicted. As we did with basis for inclusion and clarity of measurement, we coded this dimension
based on the best information given for any of the control variables. Our agreement on coding
(as indicated in a subsample) was 98%.
Controldependent variable relationship sign was as predicted. If the authors of a study offered a prediction regarding the sign of the relationship between the control variables and the dependent

Downloaded from orm.sagepub.com at UNIV AUTONOMA METROPOLITANA on March 3, 2015

64

Organizational Research Methods 15(1)

variable, and that studys results supported that prediction, we coded this dimension as 1. However, if
the sign was not as predicted, this dimension was coded 0. Just as we did for prediction of a sign, we
assessed the best-case scenario for this dimension. That is, if the authors predicted the signs of relationships for three control variables, but found their predictions to be accurate only once, we coded this
dimension as 1. Interrater agreement on this dimension in our coded subsample was 100%.
Hypotheses stated in terms of control variables. If the control variables were not mentioned in the
hypotheses or if the dependent variable was not explained in terms of its residual, this variable was
coded 0. If at least one hypothesis addressed control or residual variance, this dimension was coded 1.
The agreement level between coders on a subsample of articles was 98% for this dimension.
Results section addressed control. A code of 0 was assigned to this variable if the control variables
were not discussed beyond basic mention in the results (e.g., We entered control variables in Step 1
of the regression equation). A code of 1 was assigned if the relationships between controls variables and the dependent variable were discussed. Interrater reliability for this dimension on a subsample of articles was 89%.
Discussion section addressed control. This variable was coded 0 if there was no mention of control
variables in the discussion section or coded 1 if the authors discussed findings related to control variables and/or if the control variables were discussed in terms of the nature of the findings. As an
example, Boswell and Olson-Buchanan (2007) observe,
It is important to emphasize that these results were over and above any role for family
demands as well as number of hours spent working during non-work hours more generally.
By controlling for this latter variable, we provide a particularly conservative test of the role of
[communication technologies] use after hours and specifically that there is a role for using
[communication technologies] after hours beyond simply working after hours. (p. 604)
Our agreement on coding (as indicated in a subsample) was 83%.
Controls listed in descriptives. As recommended by Becker (2005), the means and standard deviations of control variables should be presented. Thus, we coded this as 1 for articles that included
descriptive statistics for all controls and 0 for those that did not. If only some were presented or none
were presented, this dimension was assigned a code of 0. Interrater reliability for this dimension was
89% on a subset of articles.
Controls presented in correlation table. Again, Becker (2005) suggests that authors present control
variables in the correlation table. We coded this variable as 1 if the authors included all control variables in the correlation table and 0 if none or not all were included. The agreement level on a subset
of articles for this dimension was 91%.
Analyses used. The primary type of analysis used in the article was coded 1 for correlations,
ANOVAs, or t tests; 2 for regression, ANCOVA, or MANOVA; and 3 for structural equations modeling. The type of analysis is likely to dictate whether or not control variables are used. In many
cases, control variables are more likely to be used in multiple regression than with other statistical
tests. Interrater reliability for dimension on a subset of articles was 96%.
R2 of control variables and best incremental R2 with independent variables. For those studies that presented an R2 statistic for the block of control variables, we recorded the R2 for the corresponding

Downloaded from orm.sagepub.com at UNIV AUTONOMA METROPOLITANA on March 3, 2015

Atinc et al.

65

dependent variable. We then recorded the best incremental increase in R2 when the independent
variables (including interaction terms) were added. Ideally, substantive variables will account for
as much or more variance than the control variables will.

Analyses
With the exception of the R2 statistics, all of the variables in our data set are either binary or categorical variables. Thus, we report results using contingency tables and corresponding chi-square values provided by SPSS 17.0.

Results
Table 1 reports the frequencies, percentages, and Pearson chi-square values of our variables for
macro and micro management studies. It should be noted that this table not only demonstrates differences in practices with regard to use of controls in macro versus micro management research but
also shows the use of control variables in management research as a whole within the context of our
coding criteria. Further, Figure 1 illustrates the results of the content coding, comparing micro and
macro articles on most of the coded dimensions.
We first address the percentage of articles in our data set that use control variables. Among the
four journals included in our analysis for the past 5 years, we identified 813 articles that made use of
control variables. There were 387 articles that did not use control variables, which results in a control
variable use of 67.7%. Thus, the practice of using control variables in management research is very
popular and obviously warrants in-depth analysis. As can be seen in Table 1, most articles in our
sample provided a separate section for control variables, with significant differences on this dimension favoring macro studies (80.6% of macro studies, 70.4% of micro studies, w2 11.464, p <
.001). These results may be a reflection of the relative accessibility of, and objective descriptions
available for, secondary macro variables.
With regard to the use of perceptual and nonperceptual measures, the results were interesting.
Nearly all articles in our sample made use of at least one nonperceptual control variable (94.7%),
with slightly more being used in macro studies. This is consistent with increased use of secondary
data in macro studies (67.1% vs. 8.7% of micro studies). The use of perceptual control variables was
much lower, at only about 26% in the entire sample. Interestingly, none of the macro studies and
only one micro study corrected for unreliability when a perceptual measure was used as a control
variable. Thus, this was a serious limitation in the studies that made use of such control variables.
Results for basis for inclusion, which is one of the focal variables of our study, demonstrated the
poor practices in management research and also indicated some differences between the two fields.
Based on our coding, only 48% of articles provided a theoretical reason and/or prior empirical evidence for using the control variables, with a significantly higher number of macro studies doing so at
54.5% as compared with micro studies at 42.1%. Fewer studies (29.8%) offered the argument that
the control variables might be related to the dependent variable, and the least used argument for control variable inclusion was because other authors had done so (3.9%). A surprisingly large number of
studies (18.2%) provided absolutely no rationale for the control variables. As can be seen in Table 1,
across all of these categories, macro studies fared better than micro studies in terms of following best
practices for providing a basis for inclusion for control variables.
One limitation of our coding of basis for inclusion was that we coded only one control variable for
each study (i.e., the one that had the best rationale given). To address this, we conducted an in-depth
analysis of the basis for inclusion for all control variables in a subset of articles. From the articles in
the full sample, we chose a random sample of articles for which the basis for inclusion was coded 3.
Out of the 391 articles that provided a strong basis for inclusion as part of their justification for a

Downloaded from orm.sagepub.com at UNIV AUTONOMA METROPOLITANA on March 3, 2015

66

Organizational Research Methods 15(1)

Table 1. An Examination of Various Dimensions of Control Variable (CV) Use Both Across and Between
Macro and Micro Studies

Coded Variable

Value

Separate section for CV

Yes
No
Total
Pearson w2 11.464***
Primary
Secondary
Both
Total
Pearson w2 396.991***
Used at least once
None
Total
Pearson w2 23.632***
Used at least once
None
Total
Pearson w2 52.341***
Yes
No
Total
Pearson w2 0.869
No rationale
Because others did it
It might relate to dependent
variable (DV)
Theoretical reason/prior
empirical evidence
Total
Pearson w2 33.554***
No cites at all
Cite for at least one variable
Provided cites for all CVs
Total
Pearson w2 64.320***
No information at all
Named the variables
Named the variables and how
they are measured
Total
Pearson w2 25.914***
No sign predicted for CV-DV
Prediction for sign of CV-DV
for at least one variable
Total
Pearson w2 4.782*

Source of data

Nonperceptual measures
used

Perceptual measures used

If perceptual measures used,

corrected for unreliability

Basis for inclusion

Citation provided

Clarity of measure

Sign predicted

Count (%)
Within Macro
Studies

Count (%)
Within Micro
Studies

Total Count
(%) in All
Studies

312 (80.6)
75 (19.3)
387 (100.0)

299 (70.4)
126 (29.6)
425 (100.0)

611 (75.2)
201 (24.7)
812 (100.0)

67
260
60
387

369
37
19
425

436
297
79
812

(17.3)
(67.1)
(15.5)
(100.0)

(86.8)
(8.7)
(4.4)
(100.0)

(53.6)
(36.5)
(09.7)
(100.0)

382 (98.7)
5 (1.3)
387 (100.0)

387 (91.0)
38 (8.9)
425 (100.0)

769 (94.7)
43 (5.3)
812 (100.0)

55 (14.2)
332 (85.7)
387 (100.0)

155 (36.5)
270 (63.5)
425 (100.0)

210 (25.8)
602 (74.1)
812 (100.0)

0 (0.0)
368 (100.0)
368 (100.0)

1 (0.3)
423 (99.7)
424 (100.0)

1 (0.2)
791 (99.8)
792 (100.0)

40 (10.3)
19 (4.9)
117 (30.2)

108 (25.4)
13 (3.1)
125 (29.4)

148 (18.2)
32 (3.9)
242 (29.8)

211 (54.5)

179 (42.1)

390 (48.0)

387 (100.0)

425 (100.0)

812 (100.0)

56
259
72
387

159
178
88
425

215
437
160
812

(14.5)
(66.9)
(18.6)
(100.0)

(37.4)
(41.9)
(20.7)
(100.0)

(26.5)
(53.8)
(19.7)
(100.0)

5 (1.3)
12 (3.1)
370 (95.6)

24 (5.6)
40 (9.4)
361 (84.9)

29 (3.6)
52 (6.4)
731 (90.0)

387 (100.0)

425 (100.0)

812 (100.0)

350 (90.4)
37 (9.6)

363 (85.4)
62 (14.6)

713 (87.8)
99 (12.2)

387 (100.0)

425 (100.0)

812 (100.0)
(continued)

Downloaded from orm.sagepub.com at UNIV AUTONOMA METROPOLITANA on March 3, 2015

Atinc et al.

67

Table 1. (continued)

Coded Variable

Value

Sign between DV-CV as

predicted

No or no prediction made
Outcome was as predicted
Total
Pearson w2 2.929
CV not mentioned in
hypotheses
CV mentioned in hypothesis
Total
Pearson w2 9.548**
Does not talk about CV at all
Talks about CV in results
section
Total
Pearson w2 38.864***
Does not talk about CV at all
Talks about CV in discussion
section
Total
Pearson w2 5.745**
No M or SD for CV
M or SD provided for CV
Total
Pearson w2 2.031
CV not in the correlation
table
CV in correlation table
Total
Pearson w2 3.861*
Correlation or t test
Regression, ANCOVA,
MANCOVA
SEM
Other
Total

Hypotheses stated in terms of

residual substantive
variables

Authors talk about the CV in

results section

Authors talk about the CV in

discussion section

Descriptive statistics provided for CV

Correlations provided for CV

Type of statistical analysis

Count (%)
Within Macro
Studies

Count (%)
Within Micro
Studies

Total Count
(%) in All
Studies

360 (93.0)
27 (7.0)
387 (100.0)

380 (89.6)
44 (10.4)
424 (100.0)

740 (91.2)
71 (8.8)
811 (100.0)

365 (94.3)

417 (98.3)

782 (96.4)

22 (5.7)
387 (100.0)

7 (1.7)
424 (100.0)

29 (3.6)
811 (100.0)

108 (27.9)
279 (72.1)

209 (49.3)
215 (50.7)

317 (39.1)
494 (60.9)

387 (100.0)

424 (100.0)

811 (100.0)

268 (69.4)
118 (30.6)

326 (76.9)
98 (23.1)

594 (73.3)
216 (26.7)

386 (100.0)

424 (100.0)

810 (100.0)

46 (11.9)
341 (88.1)
387 (100.0)

65 (15.3)
359 (84.7)
424 (100.0)

111 (13.7)
700 (86.3)
811 (100.0)

32 (8.3)

53 (12.5)

85 (10.5)

355 (91.7)
387 (100.0)

371 (87.5)
424 (100.0)

726 (89.5)
811 (100.0)

0 (0.0)
357 (93.2)

6 (1.4)
355 (84.1)

6 (0.7)
712 (88.4)

26 (6.8)
0 (0.0)
383 (100.0)

50 (11.8)
11 (2.6)
422 (100.0)

76 (9.4)
11 (1.4)
805 (100.0)

* p < .05.
** p < .01.
*** p < .001.

control variable, we randomly chose 40 articles and coded each control variable on more detailed
dimensions. In this subsample, 23 articles were micro and used an average of 4.48 (SD 2.63) control variables, and 17 were macro and used an average of 10.24 (SD 7.45) control variables. This
difference in the number of control variables used between macro and micro studies is likely due to
the greater availability of these variables in secondary data and the larger degrees of freedom
afforded by larger sample sizes often seen in macro studies.
Of the 273 control variables included in this subsample, 224 (82.0%) of the individual control
variables were justified using a strong basis for inclusion (a logical explanation or reference to prior
research); for micro articles this frequency was 76 of 104 (73.0%), and for macro articles this frequency was 148 of 169 (87.6%). To further explore the quality of the basis for inclusion, in this

Downloaded from orm.sagepub.com at UNIV AUTONOMA METROPOLITANA on March 3, 2015

68

Organizational Research Methods 15(1)

100.0%
90.0%
80.0%
70.0%
60.0%
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%

Correlaons provided

CV menoned in results
secon
CV menoned in discussion
secon
Descripve stascs
provided

CV menoned in hypothesis

Outcome was as predicted

Sign predicted

Clarity of measure

Citaon provided

It might related to DV

Theorecal reason/Prior
empirical evidence

Because others did it

No raonale

Corrected for unreliability

Perceptual measures used

Source of data (Both)

Nonperceptual measures
used

Source of data (Secondary)

Source of data (Primary)

Separate secon for CV

Macro
Micro

Figure 1. Percentage of micro and macro articles demonstrating best practices in control variable reporting
and use
Note: CV control variable; DV dependent variable.

subsample of articles we assessed whether the justification was based on a finding in prior research
and, if so, whether there was a direction of relationship provided (e.g., firm size positively related to
compensation). For all 273 control variables, 131 (47.9%) were not supported by mention of prior
research, 99 (36.3%) were supported by prior research without a direction of relationship given,
and 43 (15.7%) referenced prior research and noted a direction of relationship. For macro studies,
56.2% of control variables were not justified by mention of prior research, 34.3% were justified by
mention of prior research with no indication of direction of finding, and only 9.5% were justified
by mention of prior research with an indication of direction of relationship. Justifications for micro
studies were slightly better, with 34.6% justified with no prior research, 39.4% with prior research
without direction of relationship, and 26.0% justified with prior research with an indication of
direction of relationship.
Finally, in this subsample, we assessed how often logic and theory were used to provide a basis
for inclusion for control variables. In these articles, 138 (50.5%) control variables were justified
with logic, and the basis for inclusion of 8 (2.9%) control variables referenced a theory. These
findings differed across macro and micro studies, with logic used for a basis for inclusion for
71.0% of macro control variables and only 17.3% of micro control variables. The micro studies
did not have any control variables that were supported with an actual theory, but 8 (4.7%) control
variables in the macro articles did.
In addition to the criteria of basis for inclusion, it is useful to review the results for the variable for
citation provided. Slightly more than half of the articles provided a citation for at least one variable,
but fewer (19.7%) provided citations for all variables. Again, macro studies had a slightly better
record of following these best practices. Use of best practices was much better for the variable for
clarity of measurement, as 90% of all studies in our sample named the variables and described how
they were measured. Although both fields were fairly successful on this dimension, the difference is
once again in favor of macro studies (w2 25.914, p < .001), which may be due to the relative ease of
describing the objective measures often used in macro studies versus the less concrete variables used
in micro studies.

Downloaded from orm.sagepub.com at UNIV AUTONOMA METROPOLITANA on March 3, 2015

Atinc et al.

69

When considering best practices in regard to the prediction of the sign between the control
variables and the dependent variables and whether hypotheses were stated in terms of residual substantive variable, the results were disappointing. Very few of the studies provided any predictions
concerning the relationships between the control variables and the dependent variables (87.8%),
although micro studies fared slightly better in terms of making predictions (14.6% vs. 9.6% for
macro, w2 4.782, p < .05). Further, when a sign was predicted, it was rare that the outcome was
as predicted. Best practices for hypotheses were also ignored in many of the studies. Only 3.6% of
the articles in our sample had hypotheses that were stated in terms of the residual substantive variable. A slight difference favored macro articles (w2 9.548, p < .01).
We also coded articles as to whether control variables were mentioned in the results and the discussion sections and whether control variable descriptive statistics and correlations appeared in the
article. Our coding shows that authors in both fields often talk about the control variables in their
results sections (60.9%) but mention control variables in their discussion sections much less often
(26.7%). In both of these categories, the difference between the two fields was significant and
favored macro studies. Most authors followed best practices for including control variable information in the descriptive statistics, and the difference between the fields was not statistically significant. However, although most articles reported correlations for control variables (89.5%), macro
studies did so slightly more often than micro studies (91.7% and 87.5%, respectively; w2
3.861, p < .05). We also conducted a more detailed coding of the results sections of articles in our
subsample of 40 articles. Specifically, we sought to determine whether authors acknowledged the
shared variance between predictor variables and control variables, since a high shared variance will
create a residual predictor that may not be conceptually similar to the original predictor (Breaugh,
2008). In this sample of 40 articles, only 1 article directly addressed the notion of shared variance or
residual predictor.
Regression, ANCOVA, and MANCOVA were the most often used statistical techniques in our sample (88.4%), and this allowed us to explore the amount of variance explained by the control variables. Of
the 812 articles in our sample, 261 reported the R2 values explained by the control variables and the incremental change in R2 when the focal variables were introduced in the models. Although the percentage
seems surprisingly small, many macro studies used logistic regression and therefore reported Wald and
log-likelihood values instead of R2 values. When both of the fields are considered, average R2 reported
for control variables is .145, with a maximum reported value .90 and a minimum that is only slightly
above zero. The incremental change in R2 due to main and interaction effects was .1763 on average, with
a maximum value of .803. Looking at only micro studies, the average R2 was .1192 (with a maximum of
.90) for control variables, and the corresponding average incremental change was .1843 (with a maximum of .7710). Finally, the average variance explained by control variables in macro studies was .203
(with a maximum of .86), whereas the average incremental variance explained by adding the focal variables in the analysis was .1584 (with a maximum of .803). Thus, on average, the amount of variance
explained by control variables in macro studies was much higher than that in micro studies, and it was
even higher than the average additional incremental variance explained by the focal variables. In summary, these average values demonstrate that control variables play a very important role in statistical
analyses, yet little attention is paid to them by management researchers.

Discussion
In this study we sought to answer two questions: (a) To what extent do management researchers follow best practices regarding the use and reporting of control variables? and (b) What differences
occur between micro and macro research with regard to the use of control variables? Our results
show that more than two thirds of the articles published during the past 5 years in these four journals
utilized control variables, yet many of these articles did not follow best practices recommended in

Downloaded from orm.sagepub.com at UNIV AUTONOMA METROPOLITANA on March 3, 2015

70

Organizational Research Methods 15(1)

the field (e.g., Becker, 2005). The major weaknesses appear to be that researchers generally do not
provide a proper basis for inclusion for their control variables. The coding of all articles and the indepth coding of a subset of 40 articles indicated that the strongest basis for inclusion could have been
better. More authors should emphasize the function of the control variable in relation to the substantive variables, offering logic and theory to support control variable inclusion (Meehl, 1971; Spector
& Brannick, 2011). Note that even when authors do provide a theoretical rationale or cite prior
empirical evidence as a basis for inclusion of control variables, they still often fail to provide an
expectation of the nature of the relationship between control and dependent variables. Further, the
role of control variables is not adequately considered in the hypotheses or in the results or discussion
sections. Finally, a finding that warrants further attention is that control variables often account for
more variance than the main effects in the studies in our sample. This clearly indicates that better
attention to such variables is warranted.
We found some differences between micro and macro management research in regard to control
variable use and reporting, and much of this may be due to the difference in type of data used (primary vs. secondary) and type of control variable (perceptual vs. nonperceptual). Regardless of any
differences between these two, however, there remains significant room for improvement across the
board for following best practices in regard to control variables.
As we observed at the outset, the employment of control variables appears to have become, to a
certain extent, an isomorphism. In macro studies, control variables such as firm size often appear to
be included as much out of mimicry and normative expectations as compelling deliberative need.
The same appears to be true for micro papers, with variables such as sex or age included with little
justification. One wonders if such variables are included because they may truly confound the relationships between the dependent and independent variables or because editorial boards are accustomed to seeing them included. Given the frequently scant justification provided for their
inclusion, the latter may often be the case.
Note that one of the main sources for understanding best practices for control variable use and
reporting has been Beckers (2005) article, primarily because it offers such detailed prescriptions
for researchers. In our sample of journal articles from 2005 to 2009, we were not able to directly
assess the degree to which Beckers work may have influenced researchers. With publication lag,
it is possible that many of the articles published in this time period were not able to benefit from
Beckers recommendations. However, a cited reference search using Web of Science (Thompson
Reuters) conducted on October 2, 2010, indicated that 49 published articles have cited Becker
(2005), 4 of which were in our sample (all in JAP). This search also indicated that, with the exclusion
of 2010, Becker has been cited an increasing number of times as each year has passed (2, 6, 13, 16,
and 12 times in years 2006, 2007, 2008, 2009, and 2010, respectively). In addition, Breaugh (2006,
2008) provided critiques and recommendations regarding control variables in management research,
and a cited reference search indicated that these articles had been cited several times. Breaughs
2006 article was cited once in 2008 and four times in 2009, and Breaughs 2008 article was cited
twice in 2009 and once in 2010. Thus, it appears that information on best practices regarding the
use and reporting of control variables is being more widely read; yet, the findings in the current article indicate that these best practices are not always being implemented.

Recommendations for Researchers

The findings from our content coding of control variable use and reporting in four top journals in
a 5-year period indicate that a weakness of authors is providing a proper basis for inclusion of
control variables. Fewer than half of the studies provided the proper rationale, a theoretical reason, or a citation of a specific finding in the literature for even one of their control variables.
Thus, our first recommendation is one that has been made by others (i.e., Breaugh, 2006; Meehl,

Downloaded from orm.sagepub.com at UNIV AUTONOMA METROPOLITANA on March 3, 2015

Atinc et al.

71

1971; Spector & Brannick, 2011)that authors provide this level of detail when justifying the
inclusion of control variables. Specifically, this means demonstrating how control variables are
related to study variables, that is, indicating whether there is a causal relationship or only a correlation between the control and a study variable. In addition, authors should specify a direction
of relationship between these variables. Further, we recommend, when possible, the citation of
empirical relationships that have been found in meta-analyses to support control variable use.
If meta-analytic results are not available, then authors should be explicit about the relationship
found in prior research (i.e., whether it is a correlation, a finding from longitudinal research, etc.).
If no prior empirical research guides control variable inclusion, researchers should go beyond a
logical explanation of a purported relationship and rely on theory to justify the inclusion of a
control variable. Such proper justification should avoid problems with confusing control variables
with suppressor or moderator variables.
Our second recommendation is that researchers predict the sign of relationships between control
variables and dependent variables when there is theory or prior empirical findings that indicate a
consistent direction of relationship. This encourages the more thoughtful inclusion of control variables and also helps increase the likelihood that authors will provide a proper basis for inclusion.
Further, authors should present the results of the relationship between control and dependent variables not only in tables but in the text of the article as well. As noted by Breaugh (2008), interpretation problems may occur when zero-order correlations and regression weights for controlpredictor
relationships are opposite of one another. Such unexpected findings may indicate a suppression
effect; thus, just as authors would explain such findings in analyses of substantive variables, so
should they explain them when they involve controls.
Finally, we recommend that authors present the amount of variance explained by the set of control variables in the dependent variables in the tables or text. A surprising number of studies in our
sample did not provide this information, and thus, readers are unable to compare the amount of variance explained by control versus substantive variables. In our data set, the amount of variance
explained by control variables, on average, was not much smaller than the amount explained by substantive variables. In some studies, the amount of variance explained by control variables was larger
than that explained by substantive variables. In this circumstance, we recommend that authors consider whether or not it makes sense to examine the control variable as a substantive variable. Related
to this point is that, depending on the relationship between a control variable and a predictor variable, there may be so much shared variance between the two that the original predictor is not meaningfully represented (Breaugh, 2006, 2008). Thus, reporting the relationship between the control
variables and substantive variables in a study can indicate the degree to which predictor variables
share variance with controls. As a concluding recommendation on this topic, we suggest that authors
consider the practical significance of their findings in relation to control variables. That is, if a
dependent variable can be adequately quantified (e.g., dollars of profit), what actual incremental
change in the outcome is accounted for by control and substantive variables?

Recommendations for Reviewers and Editors

As we argue above, it is possible that many authors use and report control variables in the way that
they do because of isomorphism. Although authors should have the primary responsibility for
explaining a control variables inclusion and properly presenting relevant information, reviewers
and editors of journals have powerful roles as well. Reviewers and editors need to go beyond simply
making sure authors have rounded up the usual suspects in terms of control variables. They need
to expect and request a better basis for inclusion, proper citation, a prediction of sign of relationship,
and presentation of the results of control variables. In addition, reviewers may need to consider

Downloaded from orm.sagepub.com at UNIV AUTONOMA METROPOLITANA on March 3, 2015

72

Organizational Research Methods 15(1)

challenging the inclusion and rationale for the inclusion of control variables in a manner similar to
that perhaps previously reserved for substantive variables.

Limitations and Future Research

As with any study, ours suffers from certain limitations. Although we coded whether or not
articles in our data set provided an adequate basis for the inclusion of control variables, we did
not evaluate the strength of these explanations. Further, we coded only the best basis for inclusion presented for any of the control variables, which may not represent all control variables in
each article. Related to this, our coding of only one variable may have favored macro articles
because they generally use more control variables than do micro articles (as can be seen in our
subsample of 40 articles). Thus, macro articles may have fared better in some of these comparisons simply because there were more control variables for which best practices could be
assessed. However, our coding of all control variables in the subset of 40 articles indicates that
the basis for inclusion within each article is likely to be consistent; thus, our coding of the best
basis for inclusion for any control variable in our full sample probably captured a fairly accurate picture of control variable use in those articles. If this was not the case in the full sample,
our coding of only the best basis for inclusion still bolsters our recommendations. Our results
provide a best-case scenario because many of the studies that provided a strong basis for inclusion for one control variable may not have done so for the other control variables; our advice to
improve in this area may be warranted in more cases than our data indicate.
Another limitation of this study is that we did not code whether or not the control variables used
by authors were proxy variables (e.g., demographic characteristics). Although we have no doubt
that these variables create confounds and perhaps should be treated as nuisance variables, the
rationale for their inclusion is often weak (Spector & Brannick, 2011). As Breaugh (2006) argues,
proxy variables like gender are intended to capture variance related to the experience of being
male or female that likely differs, not the actual chromosomal difference between the two. He further warns against using proxies because of the unknown relations of such variables with the
underlying causal variables. In the current study, it is precisely because of the weak basis for inclusion in many articles that we could not accurately determine whether control variables were
proxies or not.
The findings in the present study may be extended by future research in various ways. First, quantitative methods such as data simulation may be used to calculate the impact of control variables on
dependent variables, specifically identifying the variance explained in regression models with and
without the presence of control variables. Furthermore, in the present study we were not able to evaluate the impact of control variables in structural models. As path analysis and structural equations
modeling are getting more popular in the analysis of both primary and secondary data, use of control
variables in structural models needs further attention. Third, future researchers may concentrate on
the most popular control variables (i.e., firm size in macro management research and age in micro
management research) and assess their use in their respective fields. Fourth, one of the limitations of
the current study is our sole concentration on management journals. Although our findings are likely
generalizable to other fields, they might be verified and extended by looking at the practices in other
organizational research areas. Finally, our sample studies were gathered from only four journals in
management. We adopted that practice to minimize the potential journal effects, but it may be interesting to investigate the use of reporting of control variables in other journals to assess any journal
effects (e.g., quality).
Despite the above limitations and the need for more research in this area, this study provides a
highly generalizable evaluation of control variable use and reporting. Our findings indicate that
in both micro and macro management research, greater attention should be paid to offering an

Downloaded from orm.sagepub.com at UNIV AUTONOMA METROPOLITANA on March 3, 2015

Atinc et al.

73

adequate basis for inclusion of control variables and explanation of the findings related to those
control variables.
Acknowledgments
A previous version of this article was presented at the 2008 Southern Management Association meeting in
St. Pete Beach, Florida. We would like to thank action editor Donald Bergh and three anonymous reviewers
for their insightful and supportive comments on the manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the authorship and/or publication of this
article.

Funding
The author(s) received no financial support for the research and/or authorship of this article.

References
Ancona, D. G., & Caldwell, D. F. (1992). Demography and design: Predictors of new product team
performance. Organization Science, 3, 321-341.
Bass, B. M. (1990). Bass and Stogdills handbook of leadership: Theory, research, and managerial applications
(3rd ed.). New York, NY: Free Press.
Becker, T. E. (2005). Potential problems in the statistical control of variables in organizational research:
A qualitative analysis with recommendations. Organizational Research Methods, 8, 274-289.
Bolino, M. C., & Turnley, W. H. (2005). The personal costs of citizenship behavior: The relationship between
individual initiative and role overload, job stress, and workfamily conflict. Journal of Applied Psychology,
90, 740-748.
Boswell, W. R., & Olson-Buchanan, J. B. (2007). The use of communication technologies after hours: The role
of work attitudes and worklife conflict. Journal of Management, 33, 592-610.
Breaugh, J. A. (2006). Rethinking the control of nuisance variables in theory testing. Journal of Business and
Psychology, 20, 429-443.
Breaugh, J. A. (2008). Important considerations in using statistical procedures to control for nuisance variables
in non-experimental studies. Human Resource Management Review, 18, 282-293.
Breaugh, J. A., & Arnold, J. (2007). Controlling nuisance variables by using a matched-groups design.
Organizational Research Methods, 10, 523-541.
Burke, M. J., Brief, A. P., & George, J. M. (1993). The role of negative affectivity in understanding relations
between self-reports of stressors and strains: A comment on the applied psychology literature. Journal of
Applied Psychology, 78, 402-412.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings.
Boston, MA: Houghton Mifflin.
DiMaggio, P. J., & Powell, W. W. 1983. The iron cage revisited: Institutional isomorphism and collective
rationality in organizational fields. American Sociological Review, 48, 147-160.
Golden, T. D., Veiga, J. F., & Dino, R. N. (2008). The impact of professional isolation on teleworker job performance and turnover intentions: Does time spent teleworking, interacting face-to-face, or having access to
communication-enhancing technology matter? Journal of Applied Psychology, 93, 1412-1421.
Keller, R. T. (2006). Transformational leadership, initiating structure, and substitutes for leadership: A longitudinal study of research and development project team performance. Journal of Applied Psychology,
91, 202-210.
Ketchen, D., Boyd, B., & Bergh, D. (2008). Research methodology in strategic management: Past accomplishments and future challenges. Organizational Research Methods, 11, 643-658.
Kish, L. (1959). Some statistical problems in research design. American Sociological Review, 24, 328-338.

Downloaded from orm.sagepub.com at UNIV AUTONOMA METROPOLITANA on March 3, 2015

74

Organizational Research Methods 15(1)

MacKinnon, D. P., Krull, J. L., & Lockwood, C. M. (2000). Equivalence of the mediation, confounding and
suppression effect. Prevention Science, 1, 173-181.
Meehl, P. E. (1971). High school yearbooks: A reply to Schwarz. Journal of Abnormal Psychology, 77, 143-148.
Pedhazur, E. J., & Schmelkin, L. P. (1991). Measurement, design, and analysis: An integrated approach.
Hillsdale, NJ: Lawrence Erlbaum.
Pfeffer, J., & Salancik, G. R. (1978). The external control of organizations. New York, NY: Harper and Row.
Schwab, D. P. (1999). Research methods for organizational studies. Mahwah, NJ: Lawrence Erlbaum.
Scott, W. R., & Meyer, J. W. 1994. Institutional environments and organizations: Structural complexity and
individualism. Thousand Oaks, CA: Sage.
Spector, P. A., & Brannick, M. T. (2011). Methodological urban legends: The misuse of statistical control
variables. Organizational Research Methods, 14, 287-305.
Tabachnik, B. G., & Fidell, L. S. (2001). Using multivariate statistics (4th ed.). Needham Heights, MA: Allyn
& Bacon.
Verwaal, E., Commandeur, H., & Verbeke, W. (2009). Value creation and value claiming in strategic outsourcing decisions: A resource contingency perspective. Journal of Management, 35, 420-444.
Vogt, W. P. (1993). The dictionary of statistics and methodology: A nontechnical guide for the social sciences.
Newbury Park, CA: Sage.
Wong, S., Ho, V. T., & Lee, C. H. (2008). A power perspective to interunit knowledge transfer: Linking knowledge attributes to unit power and the transfer of knowledge. Journal of Management, 34, 127-150.

Bios
Guclu Atinc is a doctoral student in the Management and Information Systems Department of Louisiana Tech
University. His current research interests are in the area of upper echelons and boards of directors in young
entrepreneurial firms and on the impact of environment in strategic decision making.
Marcia J. Simmering is an associate professor of management and holds the McCallister/Humana Foundation
endowed chair in the Management and Information Systems Department of Louisiana Tech University. Her
research, which primarily addresses compensation, training, and research methods, has appeared in journals
such as Academy of Management Journal, Journal of Applied Psychology, and Organizational Research
Methods.
Mark J. Kroll is the Dean of the College of Business at the University of Texas at Brownsville. His research
interests are primarily in the areas of corporate governance and mergers and acquisitions, and he has published
in journals such as Academy of Management Journal, Academy of Management Review, and Strategic
Management Journal.

Downloaded from orm.sagepub.com at UNIV AUTONOMA METROPOLITANA on March 3, 2015