San Diego County Building & Construction Trades Council, AFL-CIO

Phony study on cost of labor agreements in construction debunked An association of anti-union contractors is set to release a report that attempts to discredit Project Labor Agreements (PLAs) but instead confirms previous research documenting that PLAs do not create additional cost for school construction. The authors, funded by the Associated Builders and Contractors (ABC), do not accurately represent the factors that contribute to school construction costs, and have a biased sample that does not represent California school projects. The inferences of the study to all school PLAs are not statistically valid. Dr. Dale Belman of Michigan State University's School of Labor & Industrial Relations, a wellknown expert in the field of economics and Project Labor Agreements, who was frequently cited in the study, reviewed the study and wrote to the authors: "Although your study has several serious statistical issues, at the end of the day, your results are basically consistent with those presented in my article on PLAs and Massachusetts school construction costs. The take-away from your results can be summarized as follows: When appropriate controls are included for differences in the characteristics of schools built including school type and location, building specifications, materials used etc., there is no statistical evidence that PLA schools are more costly compared to non PLA schools." SPURIOUS CAUSATION From data obtained through Public Records Act requests, the authors found that actual school construction costs were higher than what contractors bid we estimate an average of 20% higher. This was because of change orders, where contractors charged schools more than the bid amount, sometimes for faulty work. Since this result was embarrassing to the sponsoring contractors, the authors downplayed it and cherry-picked only those variables that suited their desired narrative. The study has omitted significant factors that influence construction costs from the econometric model. Here are some examples of omissions: 1. Wages: The ABC study considers wage rates as a neutral factor, implying that wages are not included in describing construction costs. However, California is a large state with a significant variation in prevailing wages under Labor Code Section 1720 et seq, between regions. Urban areas like Los Angeles have higher prevailing wages, as well as a

1|Sa n Die go Buil ding & Co nstr uct ion T rades Counc il

higher number of PLA projects. For example, the basic hourly wage for a wireman electrician is $32.85 in Fresno and $36.65 in San Diego, under California prevailing wage law. In Del Norte County, the same classification of worker, whether union or nonunion, is paid $46 an hour on school construction projects . 1 Charter cities like San Francisco conduct their own prevailing wage surveys, which are independent of state surveys. This variation alone can explain a significant part of the fluctuation in costs. 2. School location: The ABC study ignores local building conditions, such as land costs, weather, materials supply and fuel costs, and requirements from local building codes. These factors account for significant variation in school building construction costs in California.2 According to Reed Construction Data, the construction costs for an elementary school in San Francisco in 2009 were $189.33 per square foot, which was 18% higher than the cost of school construction in San Diego, which was $160.21 per square foot. 3. Development Regulations: School buildings in the 2000s were required by law to be safer and greener than a decade ago. Yet, the study freezes the regulatory framework over a large period in time from 1996 to 2008, as if state and local regulations were uniform over 12 years. This is a fundamentally flawed assumption, as there were significant changes in the California Environmental Quality Act (CEQA), Public Contract Code and California Building Code during this period. For example, soil contamination is a major factor adding time and expense to new school projects. Significant revisions in CEQA statute and case-law have driven repeated re-siting of school projects recently, which has nothing to do with PLAs. In addition, California schools have to adhere to Title 24 (Field Act) requirements on seismic standards. A recent Division of State Architect study showed that Field Act compliance alone added about 4 percent to the cost of a new school, even though compliance with Field Act is spotty. 3 In statistics, a spurious relationship is when two events that have no relationship are wrongly inferred to have a causal relationship. For example, on a hot day there are a lot of ice-cream sales, as well as power outages. To infer that ice-cream sales cause power outages is a false interpretation of the data. In reality, complex projects in urban areas are more expensive, and also more likely to benefit from PLAs. Therefore, to infer that PLAs cause increased costs is spurious.
1

Journeyman prevailing wage determinations, effective 1-2011. California Department of Industrial Relations. http://www.dir.ca.gov/DLSR/PWD/index.htm

Jeffrey M. Vincent and Deborah McKoy, Center for Cities and Schools, University of California-Berkeley. 2008. The Complex and Multi-Faceted Nature of School Construction Costs: Factors Effecting California. http://citiesandschools.berkeley.edu/reports/K-12_CA_Construction_Report.pdf 3 A California Watch investigation uncovered that the State Architect under Governor Pete Wilson was not implementing Field Act regulations on school construction during his tenure. http://californiawatch.org/k-12/laxoversight-school-construction-raises-doubts-about-earthquake-safety-9537 This lack of regulatory oversight left many school construction projects built without the required seismic safety standards.

2|Sa n Die go Buil ding & Co nstr uct ion T rades Counc il

SAMPLE BIAS Since the study was sponsored by an organization that openly wants to ban PLAs, there is a valid reason to scrutinize the projects sampled, and responsive school districts for expectation bias and sample bias. In addition, there is no way of knowing whether respondents were primed by the sponsors in follow-up calls to provide negative reports on PLA projects, since the phone script is not published. In a typical statistical analysis, there is a random draw of equally probable projects from a comparable universe. This did not happen with this analysis. Here is how the sample is biased: 1. Apples, oranges and watermelons: This report does not provide an apples-to-apples comparison. Newer projects were built under different state regulations than older ones. With the exception of Los Angeles, all the PLAs in the ABC study were approved in the 2000s, which implies that the earlier time-frame mainly non-PLA projects built under different conditions. San Diego Unified PLA was approved in 2009, which is after the timeline of the study, so it is strange that it is even included. 2. The Caveman Effect: Looking for paintings only in caves will lead one to conclude that all paintings are in caves. The Public Records Act requests ostensibly got a 50% response. However, the PLAs received are from only 9 out of 29 jurisdictions (31%) that have adopted such agreements. This indicates that there may have been a bias in follow-up, with certain jurisdictions emphasized over others. Almost 1,000 nonPLA projects were analyzed, and only 65 PLA projects. Whether the sample size of PLA projects is large enough to draw generalized conclusions about all PLA projects in California is questionable. 3. Self-selection Bias: School districts, like LAUSD, that have actually tracked detailed information on costs are more likely to respond than others. There are two scenarios for districts that chose not to respond that would bias the results: a. Those that had a PLA, but did not want public exposure: this is especially the case if the person inquiring about PLAs was politically involved in a public fashion, or associated with an organization that was openly hostile to PLAs, as was the case with this study. b. Those that did not have a PLA, and did not want to expose costly projects. Since the authors openly advocate for efficiency, it is extremely likely that school districts that had expensive non-PLA projects would hesitate to respond. They were also likely to drop out (attrition bias) even if they responded initially, after they computed their costs and found them to be excessive. The only way to know whether there was a selection bias would be to calibrate the variables in the sample with the values statewide. However, the authors present no evidence to suggest that the projects sampled actually represent the cross-section of characteristics in school construction statewide. Since the published sources from McGraw-Hill only cover bid values,

3|Sa n Die go Buil ding & Co nstr uct ion T rades Counc il

and not change orders, it is impossible to verify the authors claims on actual costs. The authors own analysis of 65 matched schools with and without a PLA failed to find any statistically significant relationship. Hence the study has no external validity. INCONGRUENCY WITH EMPERICAL DATA PLAs have been used increasingly in the private sector by major businesses like Toyota and WalMart, with a solid record of completions on-time and on-budget, with quality construction. PLAs also have an established record of significant value to public agencies, the community and taxpayers. For example, the San Diego Unified School Districts Project Stabilization Agreement a PLA negotiated in 2009 has been highly successful despite the economically-challenging environment. To date, the district has saved 40% of the budgeted costs on the career technical education facilities built under the agreement a savings already in excess of $8 million. The ABC-funded report researchers continue their claims for increased cost related to PLAs by citing anti-PLA lobbyist who say "the requirements imposed by PLAs discourage non-union contractors from bidding on projects and subcontractors from participating." Once again, the report does not investigate or comment further on this accusation. However, if any research would have been included on this subject they would have found that this argument does not have any basis for support. In fact, on the largest project mentioned in the ABC-funded report, the LAUSD Bond Funded Construction Program, there were over 4,100 general contractors and subcontractors that successfully bid and were awarded projects. Of that number over 55% were non-union contractors. Many continued to bid, and were awarded numerous projects, after their first experience performing a contract under a Project Labor Agreement. POLITICALLY MOTIVATED STUDY The ABC is currently collecting signatures for a ballot measure to ban PLAs within the cities of San Diego and Sacramento, in California. That effort has come under increased scrutiny, as San Diego CityBeat exposed that the paid signature gathers were using misleading and false statements about the initiative. This ABC-funded study is politically motivated to further mislead the public about the value of PLAs.

San Diego County Building & Construction Trades Council response to ABC funded report on California school Construction Tom Lemmon: (619) 521-2914 ATTACHMENT: Letter from Professor Dale Belman, Michigan State University to study authors, July 18, 2011.

4|Sa n Die go Buil ding & Co nstr uct ion T rades Counc il

July 18, 2011 Mr. Vince Vasquez, Mr. W. Erik Bruvold and Dr. Dale Glaser National University System, Institute for Policy Studies 11355 North Torrey Pines Road La Jolla, California 92037 Dear Mr. Vasquez, Mr. Bruvold and Dr. Glaser: I have read your study, Measuring the Cost of Project Labor Agreements on School Construction in California, with great interest. As you know, I have researched and published studies on project labor agreements (PLAs) and school construction costs including Project Labor Agreements Effect on School Construction Costs in Massachusetts, Industrial Relations 49,no. 1 (2010) and remain interested in all new research in this area. I have reviewed your work closely to assure that I understood your data, model and methods. I find that your studys conclusion is not supported by your research; that you have overlooked important factors that affect costs, and that you have misinterpreted and drawn erroneous conclusions from my work; mistakes that I hope you will want to correct. Correctly interpreted, your results are basically consistent with those presented in my article on PLAs and Massachusetts school construction costs. The take-away from your results can be summarized as follows: When appropriate controls are included for differences in the characteristics of schools built including school type and location, building specifications, materials used etc., there is no statistical evidence that PLA schools are more costly compared to non PLA schools. Since you may not realize that this is what your research results mean, let me provide some detail: As I have shown in the past, it is challenging to separate out the effect of PLAs on school construction costs from the effect of the specific characteristics of particular projects. Because different schools can have very different construction specifications and can differ in other ways, it is critical to separate out any PLA effect from the effect of building characteristics. If the effect of school characteristics is not distinguished from any possible PLA effect, there will be omitted variable bias which potentially completely invalidate the research results. o Because of the set of factors used to explain school construction costs is so limited(you have only six explanatory factors other than PLA) there is considerable risk of omitted variable bias. For example, although all workers on school construction projects are paid prevailing wage, the prevailing wage varies systematically by region. If high prevailing wage regions, such as the Los Angeles area, are more likely to use PLAs, your PLA variable will be biased upward by the omission of a control for differences in prevailing wages between regions. A control, such as the prevailing wage for a benchmark trade such as carpenter, at the time the school was constructed, would likely have been sufficient to remove the bias. Similarly, if PLAs are used for schools built to higher seismic

standards, and controls for differences in construction are not incorporated into the model, the measure of the cost effect of PLAs will be biased upward.1 o There is considerable evidence in your study of omitted variable bias in the estimates provided in chart 6. The estimates with a sample of PLA and non-PLA schools which are matched on their characteristics better control for the characteristics of schools and of the construction environment than other estimates in the report. On page 15, you write: In our second phase, we analyzed the matched set of 130 projects (incorporating a propensity weight covariate) using the ordinary least squares method. We found that PLAs were not statistically significant. Similar results were found when the propensity score was omitted from the model. The implications from this are clear, but downplayed in the report: when the model better controls for differences in characteristics between PLA and non-PLA schools, PLAs do not affect school construction costs. This section also indicates that, parallel to my work, there are statistically meaningful differences between PLA and nonPLA schools and that the majority of schools built without PLAs are unlike schools built with PLAs. These differences suggest that PLAs are, as they should be, used on challenging projects rather than plain vanilla schools.

The estimates in Chart 7 also indicate that your study suffers from omitted variable bias. Similar to my work, you find that, when controls for construction in a large urban district are included in the model, the PLA variable is no longer statistically significant. The district in question, LAUSD, builds to higher seismic standards than other school districts and is more likely to build multi-story steel structures which differ considerably from typical schools. When a control for construction by the LAUSD school district is included in your model (Chart 7), the PLA variable becomes small in magnitude and is far from statistical significance. Again, this is consistent with omitted variable bias.

I am concerned that your results do not provide apples-to-apples comparisons. For example, when you estimate a model which excludes LAUSD schools, you change the explanatory variables in the specification without explanation. As a result, the reader doesnt know whether the seeming positive
The problem of confusing the effect of PLAs on costs with the higher costs of project on which PLAs are used can be illustrated with a hypothetical example. A mile of above-ground light-rail track in Los Angeles costs about $80 million. A mile of below ground track, with tunneling and other challenges, costs about $400 million. The end result of both projects is a mile of track. If, because of the challenges of the below ground project, a PLA is used on the below ground segments, and if we dont allow for the differences between above and below ground construction, we will erroneously conclude that PLAs raise the cost of construction by several orders of magnitude. Our PLA measure is capturing not only any effects PLAs may have on construction costs, but also the higher costs of building below ground. If we include measures of factors related to the added costs of below ground construction to our model, it may be possible to get an accurate measure of the effect of PLAs on cost. Given that architects and engineers use more than 100 characteristics in developing cost estimates for schools, it is doubtful that the six which are used in the Chart 6 and 7 models are sufficient to guard against omitted variable bias.
1

effect of the PLA variable in this sample is indeed a positive effect, or the result of changing the specification when you switch your data sample. To avoid concern about manipulating your results, you need to use the same model when testing for PLA cost effects on data using LAUSD data and when excluding that data. Another apples-to-oranges comparison in your research of the mixing of rehab, renovation and remodeling data in with new construction. This is a bad idea simply because the specific needs of individual renovation projects can vary so widely. One school might just need a roof while another might require a rebuild to meet earthquake seismic standards. The two schools could have exactly the same square foot size and greatly different square foot renovation costs. These kinds of projects should not be lumped together, much less thrown in with new construction. A cleaner data set would have used new construction only to avoid apples-to-oranges comparison mistakes. But at the very least, you should have had a control variable in your model indicating whether the project was a renovation project and what type of renovation was done.

Some of the results reported in Chart 6 are not sensible. An implication of your model is that a very large school will cost nothing. While this might be viewed as good news for financially stressed school boards, it is obviously the result of a misspecified variable. Similarly, your model indicates that school costs rise without limit by $7.50 per square foot each year. It is not credible that school construction costs will rise by $75 per square foot between now and 2021, or by $300 per square foot by 2051 simply because of the passage of time. This is obviously wrong and again, suggests serious misspecification. Another apples-to-oranges mistake in your work is failing to use clustered errors to allow for common factors affecting school costs among schools in a single school system. School district construction policies can be very different across school districts. As noted previously, the LAUSD builds to different seismic standards than many other school districts and this certainly affects both costs and the error term of schools built in the LAUSD. This is a technical point but an important one: errors in your estimates will not be independent across observations and your estimates of statistical significance will be wrong. The large effect of controlling for the LAUSD on the estimates strongly suggests that observations are not independent. There are also some errors with respect to my work. The 2005 paper is an early version of the article which, having gone through peer review, appeared in the January 2010 issue of Industrial Relations. The 2010 work builds on the prior working paper and extends that work, it would be most appropriate to use only the 2010 version. Also, your chart 4 took the estimate of the PLA effect from a model which I was using to demonstrate the effects of under specification on estimates of PLA effects. My final conclusion, based on the whole of my work, was that there was considerable evidence that PLAs did not affect school costs, but that it was difficult to separate out the effects of PLAs from the effects of characteristics which cause PLAs to be used in school construction. All said, I was pleased to review your report and find that, similar to my work, it supports the view that PLAs do not affect the cost of construction of schools. I doubt that is the conclusion which you intended, but it is clearly there in your results. To the degree you disagree with this, the appropriate forum for deciding the merit of your work would be a peer reviewed journal. This is the accepted avenue for the evaluation of research as it provides review by disinterested experts on the subject and methodology. I would suggest you try Industrial Relations, The Industrial and Labor Relations Review or Economic Inquiry as these are journals which are well respected and are likely to be interested in this issue.

I am interested in taking a closer look at the data and would be most grateful if you would share your data with me; I would be happy to provide you with the data my colleagues and I collected from Massachusetts. Yours,

Dale Belman
Dale Belman Professor School of Human Resources and Labor Relations Michigan State University 517-927-9244 drdale@msu.edu