To: From: Date:

Elizabeth Pauley and Nahir Torres, The Boston Foundation Oscar Cerna (Research Associate) and Genevieve Orr (Research Assistant), MDRC October 15, 2010

Subject: Massachusetts Achieving the Dream Phase 1 Findings and Recommendations

Overall summary of the project This memorandum describes the progress made across four Massachusetts community colleges that joined the Achieving the Dream (AtD) initiative in 2007. The Phase 1 (JanuaryOctober 2010) evaluation of this project provides information to funders, colleges, and other initiative stakeholders about how far these colleges have progressed to date in implementing AtDs process for increasing student success through institutional improvement. In addition, it includes information about the state policy work being conducted in Massachusetts, including efforts to strengthen ties between community colleges activities and actions at the state policy level. Finally, the memorandum concludes with a set of recommendations about how leaders and funders might further support colleges efforts to improve student success. Phase 1 Summary of Findings Each of the four Massachusetts AtD colleges entered into the initiative with strong institutional leadership and active administrators already committed to improving student outcomes. Participation in the initiative has provided colleges with an organizational framework that helps them translate this commitment into institutional action. The four colleges have increased their capacity to use and report on longitudinal student outcomes data, as well as data disaggregated across race/ethnicity and income groups. The colleges have also utilized more informational technology resources to facilitate increased data usage and distribution. This elevated demand for data reflects a growing awareness of the value of data to inform decision making, but in some cases, it has also placed increased strain on institutional research personnel at colleges that operate with limited staffing.

The colleges have made progress engaging more faculty and staff members in their AtD activities, especially by inviting them to participate in special taskforces and committees on campus. Each of the four colleges has also made concerted efforts to involve external stakeholders from their local communities in their student success agendas, either by collaborating on strategic initiatives or by soliciting external input on colleges institutional plans. However, like other community colleges, some continue to face challenges involving substantial proportions of their faculty and staff, particularly part-time/adjunct instructorswho represent a significant percentage of the faculty at each collegein their AtD efforts. The four colleges have designed and piloted strategies that focus primarily on improving the success of their developmental education students. In general, they have begun by piloting and evaluating their strategies on a small scale, as recommended by the AtD model for institutional improvement (see below). Some have proceeded to modify or scale up their strategies based on evidence of their success while others remain in the planning and piloting stages at this point in the initiative. The Massachusetts AtD colleges have begun taking important steps to institutionalize their progress under AtD, with several commenting on the important role that their coaches and data facilitators played in this work. Most have improved their strategic planning and budgeting processes and organized active committee structures to monitor their AtD work, though some work still remains in these areas. The colleges also noted funding obstacles to scaling up and sustaining promising practices. Despite initial delays in establishing a functional workplan, the Massachusetts AtD State Policy Team has become increasingly focused and agenda-driven in its work. One of its most recent accomplishments was the successful roll-out of a convening of the fifteen Massachusetts community colleges to discuss developmental education policies and practices. This convening represented an important step towards aligning efforts across colleges and policy-makers in the state.

Methodology From March through August of 2010, two MDRC researchers conducted field research and qualitative data analyses to assess the progress that the four Massachusetts AtD colleges made in implementing the initiatives tenets of institutional improvement to improve student success. Specifically, the researchers sought to identify the following: 1. What progress have the colleges made in building a culture of evidence, and what factors may be helping or hindering their progress? 2. What strategies have colleges put into place to improve student outcomes? 3. What policy work has been accomplished at the state level to facilitate colleges efforts? 4. What recommendations can be made to the funders and leadership of AtD in Massachusetts to further support the promising work that has already begun in the state? The analyses, findings, and feedback highlighted in this memorandum are based primarily on two-day site visits to each of the four colleges, which consisted of interviews and focus groups with college presidents, administrators, key faculty and staff involved with the initiative, and in one case, an AtD coach and data facilitator. Interviewees were asked to discuss their involvement in AtD and describe the changes that have resulted at their institutions, as well as any setbacks they have experienced. MDRC researchers also reviewed colleges annual reports to the initiative, as well as reports from colleges AtD coaches and data facilitators, all of which provided context for the data collected during their site visits. In addition to this site-level work, the research team held phone interviews with various members of the Massachusetts State Policy Team. It should be noted here that this study of the Massachusetts AtD colleges was not designed as a direct comparison to evaluations of AtD colleges from Rounds 11 and 32 of the initiative. This is due both to the difficulty of making such comparisons to a cohort of only four colleges, and to the desire to preserve anonymity for the colleges involved. Nonetheless, MDRC researchers have reviewed research on the Round 1 and Round 3 AtD colleges as a general reference for assessing the Massachusetts colleges progress implementing the goals and objectives of the initiative.
1 2

Brock et al. (2007). Jenkins et al. (2009).

Understanding Colleges Progress Using Achieving the Dreams Five-Step Process for Institutional Improvement Upon joining AtD, participating colleges are expected to undertake a five-step process of institutional change in which they: 1) commit to improving student outcomes and reducing achievement gaps; 2) prioritize actions based on student outcomes data and other evidence of barriers to achievement; 3) engage faculty, staff, and other internal and external stakeholders in developing priority strategies for improvement; 4) implement and evaluate strategies; and 5) institutionalize practices and policies that show evidence of improving student success.3 The expectation is that this five-step process will help colleges to develop a greater focus on student success and evidence-based practices, which will ultimately lead to better student outcomes and reduced achievement gaps across student subgroups. The diagram in Exhibit 1 depicts this continuous model for institutional improvement. The interview protocols and analytical tools for this evaluation, as well as the themes for this Phase 1 memorandum, were designed to assess how well the four Massachusetts AtD colleges implemented each of these five steps for institutional improvement. MDRC has produced individual progress reports for each of the colleges, which were shared separately with the institutions. However, this memorandum considers the colleges progress as a collective group in order to preserve the anonymity of our interview and focus group participants.

Phase 1 Findings Overall, the four Massachusetts AtD colleges have made significant progress toward making the policy and programmatic changes that promote a culture of evidence on their campuses. As members of the fourth cohort of AtD, representatives from the Massachusetts colleges have acknowledged the benefit of belonging to a growing network of AtD colleges, many of which have shared the lessons they learned during their work in the initiative. They also emphasized the value of the expert guidance they received from their AtD coaches and data facilitators (see Step 5 below). Moreover, the funding from the initiative has allowed the colleges to begin investing in some of the resources and personnel needed to make the desired changes in
3

Achieving the Dream (2009).

Exhibit 1 Achieving the Dream Five-Step Process for Increasing Student Success through Institutional Improvement

SOURCE: Field Guide for Improving Student Success (2009), from the Achieving the Dream: Community Colleges Count initiative, p 11.

their institutional processes and programming. Below is a step-by-step assessment of the colleges collective progress implementing AtDs improvement model. Each section provides an overview of the initiatives expectations for colleges under each step, followed by themes and evidence of colleges progress towards meeting these expectations.

Step #1: Leadership Commitment Step 1 in AtDs process of institutional transformation calls for senior leadership at the colleges to make a commitment to student success and develop an agenda for this work, demonstrating their willingness to change policies, programs, and resource allocations to improve student outcomes. The initiative recommends that leaders organize a core team comprised of administrators, faculty, and staff to spearhead and manage these efforts. These colleges already demonstrated strong leadership commitment to student success prior to their participation in AtD, but the initiative provided them with a clearer organizational framework for institutional action. Upon joining the initiative, the senior leadership at each of the four colleges had already established their commitment to supporting active student success agendas. Building upon this foundation, the initiative gave the college leaders a framework to further define and focus their commitment in order to make targeted changes in programming and practice. These changes included new approaches to incorporating data into institutional decision-making processes, engaging more mid-level administrators in these processes, and implementing new classroom instruction practices and campus services. College presidents have been active leaders and supporters of AtD. While some college presidents in previous Rounds remained removed from the AtD activities on the ground at their institutions, all four of the Massachusetts college presidents have been actively engaged in AtD planning since the beginning of the initiative. Since all of the presidents had established institutional goals centered on student success before joining AtD, they found the initiative to be a good opportunity to further bolster their college leadership goals. Each Massachusetts college president led his or her AtD core team early on in the initiative, and some continue to participate in ongoing meetings. They have also provided regular updates to their boards of trustees about their goals and activities under AtD, presented on student- and institution-level data to campus

and external audiences, and referenced data regularly when making decisions about policies and practices on their campuses. Each president has also served a one-year term as a member of the Massachusetts AtD State Policy Committeea team created to improve state policy related to community colleges and to improve student outcomes (see more below). Each has also sought to strengthen their ties with community members by meeting with local business leaders, K-12 school leaders, and community-based organizations to communicate their institutions goals and progress toward improving student success. Most senior and mid-level administrators have also played key roles in the initiative. Most of the vice presidents and deans with whom we spoke have served as key contributors to AtD at their institutions, including writing their colleges AtD proposals, overseeing strategy development plans, and identifying key performance indicators by which to monitor their work under the initiative. At most of the colleges, senior administrators have later passed the torch of AtD leadership to mid-level administrators, such as deans and department heads, in order to extend the notion of shared governance in making decisions about policy or practice that affect the entire campus. One dean noted, I think that to some extent, one of the benefits of that approach is thatas youre trying to grow it and bring more people on boardit doesnt get viewed as a top-down initiative. The AtD core teams at the Massachusetts colleges are well-structured, inclusive, and agenda-driven. All four core teams have met consistently throughout the initiative to implement new reforms, approve major decisions made by special taskforces and committees, and oversee certain aspects of program management. With one exception, each core team includes at least ten members from across the college. Furthermore, all four colleges have designated competent and well-respected AtD coordinators to serve as their core team leaders. Senior leaders at these colleges were intentional about selecting core team leaders whom they felt could garner broad campus buy-in to the initiative while bridging across different constituencies. One core team member described the team as a model of central leadership driven also by faculty and staff. Similarly, another interviewee commented on the value of the core team to bring together separate departments and programs and areas of the college to review data and make decisions together.

Step #2: Use of Data to Prioritize Actions Step 2 in AtDs model for institutional improvement calls for participating colleges to use data to prioritize areas for reform, which are then used to inform colleges development of programs and practices that promote student success. This step includes collecting and analyzing longitudinal student cohort data and disaggregated demographic data in order to identify achievement gaps across different student groups. To guide this work, an institutional AtD data team is created to oversee the data and evaluation tasks undertaken as part of the initiative. All four colleges have demonstrated improvements in their use of data over the course of the initiative, though the nature of these improvements has varied based on colleges data capacities when they began AtD. Two of the colleges that entered the initiative with relatively low data capacities have used AtD as an impetus to secure new institutional research (IR) personnel who can fulfill the initiatives expectations for data usage. One hired its IR director right at the start of the initiative to bring order to what one interviewee described as a disorganized data dump, while another college replaced its former IR director with a datasavvy research analyst, who now leads its data team and strategy evaluations. The other two colleges, which entered the initiative with experienced IR leaders, used their participation in AtD to further streamline their already well-developed data analyses. For instance, these colleges trained more administrators and faculty leaders in data collection, analysis, and reporting, thereby spreading data tasks and reducing the heavy workloads of their institutional research directors. Despite these advances, institutional researchers at three colleges continue to experience strain as their limited personnel juggle growing reporting and analytical responsibilities. These colleges are still looking for new approaches to prioritize their data tasks and further delegate data responsibilities. For example, some hope to reduce the heavy workloads of their IR staff by training more administrators and faculty across the college to participate in data analysis activities. Findings from colleges disaggregation of data further highlighted the necessity to help student subgroups that face the highest risks of failing or dropping out. Since they began AtD in 2007, the Massachusetts colleges have increasingly made use of disaggregated

data, and especially data disaggregated by race and by developmental education placement. Interviewees across the colleges cited concerns about achievement gaps, which persist for some groups even as their enrollment numbers grow. All four colleges have recognized the need to promote student success across increasingly diverse student bodiesespecially among Hispanics, English Language Learners, immigrant populations, and/or older students, all of whom tend to be more likely to place into developmental education and less likely to persist in college. While all of the colleges have looked at data disaggregated by race, gender, and age, some have also considered other variables in their data mining, including socioeconomic backgrounds, enrollment status, and financial aid status. One administrator noted, Having AtD really made us focus on all of these different groups and cohorts. I think thats the key thing its the focus. We were doing it [before], but we werent really focused on it like we are now. Changes to developmental education policies and practices have emerged as a particularly important area for reform. The data analyses that colleges conducted during their AtD planning year and early implementation years prompted many researchers and administrators to reconsider their policies in developmental education. For example, some deans and department heads have reviewed cohort data on developmental education students in order to reassess the types of courses they offer, examining which have the highest success and pass rates and which are in greatest demand by students. Colleges policy changes have ranged from the specific, such as adding reading, writing, and math prerequisites to certain introductory college courses, to the broad, such as conducting a comprehensive review of the entire core curriculum in order to make math and English course sequences more coherent and manageable. Colleges have found new ways to manage and distribute their data by bolstering their information technology (IT) systems. Three of the four colleges reported notable changes in IT as they have improved the efficiency with which student data are reported across their institutions. These colleges have incorporated (or are in the process of incorporating) new software to streamline access to data such that increasing numbers of deans, department heads, and faculty can directly pull data and generate their own reports. Some have also designed websites and web portals to disseminate data with greater frequently to broader segments of the campus community, including posting real-time outcomes data disaggregated by various demographic categories. Two of these sites also post data reports that can be accessed by college

employees and students. While these colleges have already made noteworthy progress revamping their IT systems, some have experienced delays rolling out new hardware and software systems that fit their specific data needs. Furthermore, these colleges still have work ahead as they train additional faculty, staff, and administrators to use their data analysis and reporting software in meaningful and relevant ways.

Step #3: Engaging Stakeholders Step 3 in AtDs model for institutional improvement calls for college administrators to encourage the larger campus communitywhich includes faculty, student services staff, and other internal and external stakeholdersto actively contribute to institutional decision-making by identifying priority problems and developing strategies to help remedy those problems. This engagement includes contributing to the design and implementation of new strategies or policies, assessing and deciding on changes in instruction, and improving the delivery of student services. Colleges AtD core and data teams, as well as specialized taskforces and strategy committees, have served as important mechanisms for increasing faculty and staff involvement in AtD. As mentioned in Step 1, administrators deliberately tapped key faculty and staff members to lead or participate in their AtD core and data teams, strategy development committees, and program data and evaluation efforts. Some Massachusetts college leaders have constructed cross-disciplinary committees and taskforces to review and act on particular areas of need, such as improving course completion in developmental math, or revamping a first-year seminar course to incorporate student service presentations. A small number of experienced faculty and staff manage most strategy-specific teams at all of the colleges, and three colleges reported that their strategy leaders have worked with their institutional research departments in order to plan strategy evaluations and select key data measures to assess program success. Colleges have generally increased their engagement of full-time faculty and staff, but most still struggle to involve larger proportions of their faculty and staff in their work. Part-time staff were particularly difficult to engage. Not surprisingly, in light of similar experiences at the Round 1 and Round 3 AtD colleges, broad engagement has proven to be a challenge for the Massachusetts colleges during their first three years in the initiative. Some of the reasons interviewees cited for this challenge include a general hesitancy among some faculty

10

and staff to deviate from their routine day-to-day tasks, a lack of time for or interest in new initiatives on campus, and a fear of reviewing or analyzing data. While the colleges have made some progress getting full-time faculty members to participate on committees or as strategy leaders, engaging adjuncts remains a particular challenge across all four colleges. One faculty member argued that, Those [adjuct faculty] who are on the fringes are more skeptical. Our job is to pull more of them in. Most of the colleges, however, have tried different approaches to get more adjuncts involved, such as offering them paid office hours for teaching specialized courses (such as learning communities), having them serve as additional tutors, or training them to serve as on-call advisors during peak enrollment and registration periods. All four colleges have been intentional about building stronger ties to their local communities by inviting external stakeholders to collaborate in promoting student success. Some of the Massachusetts colleges have worked during their participation in AtD to strengthen partnerships with their local K-12 school systems, such as by offering early placement testing and dual enrollment courses for high school juniors and seniors. Several interviewees across the four colleges mentioned the importance of working with their feeder high schools to better prepare students to take their college placement tests and reduce placement into developmental education courses. Some colleges also engaged other community leaders from the public and private sectors to solicit their input on how best to prepare students for different sectors of the workforce.

Step #4: Implementing, Evaluating, and Improving Strategies and Practices Step 4 in AtDs model for institutional improvement calls for colleges to design and implement strategies to improve student performance and persistence. Colleges are then expected to evaluate how these programs and policies affect student outcomes, modifying or eliminating those that do not show promise of increasing student success, while scaling up those that do. Taken together, these activities represent a continuous process of assessment and refinement for colleges AtD strategies. All four Massachusetts AtD colleges developed strategies that focused primarily on improving the success of their developmental education students. Upon entering the initiative,

11

many college representatives were already aware of the growing needs of their developmental education students. Their early data analyses under AtD reinforced their desire to focus on reforms in this area, leading them to target most of their strategies towards helping students progress through their developmental education sequences. In particular, the Massachusetts colleges focused heavily on developmental math, which has traditionally posed the greatest obstacle to student success. Three of the four colleges designed strategic interventions linking tutoring or supplemental instruction to their developmental math courses. The fourth initiated a small pilot strategy that increased the contact time for a developmental math course from three hours of class (three days per week) to five hours (five days per week) in order to allow students more time to master the material. Another area of focus at the Massachusetts colleges was increasing campus services and supports for students in developmental and gatekeeper courses. Each of the Massachusetts AtD colleges designed strategies to increase the support services offered to students in developmental and gatekeeper courses. For instance, each college had already revised or was in the process of revising its academic advising services; one college was developing a rubric to help guide faculty advisors, while others offered more intrusive case management services for struggling students. Three of the four colleges also revamped their college success or freshman seminar courses to better assist incoming students. Finally, two of the colleges sought to strengthen the ties between their academic and student service areas by having student services staff work regularly inside learning community classrooms. The Massachusetts colleges generally began their strategies with small pilots, in accordance with AtDs suggested model for strategy development; most of the colleges are still working on strategy designs that they hope to bring to scale over the next few years. As recommended by the initiative, and reinforced by coaches and data facilitators, most of the strategies undertaken by the four Massachusetts colleges began as small pilots. With the guidance of their AtD data facilitators, all of the colleges had developed plans to evaluate their strategies, and a few had begun collecting and discussing pilot data in strategy committees, as well as with their AtD coaches and data facilitators, to determine which of their strategies should be expanded, adjusted, or redesigned. These colleges drew on a combination of their AtD data teams, strategy-specific teams, researchers, and AtD coaches to develop and refine plans for evaluating each of their strategies. Some colleges were collecting and monitoring quantitative data on student outcomes, such as course

12

completion and persistence, with a couple comparing students who received an intervention to similar students who did not. Some colleges also utilized focus groups and surveys to assess qualitative measures such as student attitudes and behaviors. Strategy leaders usually worked with their campus researchers and data analysts to develop customized evaluation plans and received updates on the progress of their strategies. Step #5: Establishing a culture of continuous improvement Step 5 of AtDs institutional improvement process calls for colleges to sustain their progress by institutionalizing policies and practices that have been shown to improve student successa process known as establishing a culture of continuous improvement.4 Colleges are expected to integrate AtDs five-step improvement process into their ongoing institutional decision making. This entails revamping strategic planning and budgeting practices such that resource allocations are informed by student outcomes data, as well as creating and sustaining evidence-driven committees to monitor colleges student success agendas. Colleges participation in AtD has contributed to their strategic planning and budgeting processes over the course of the initiative. Three of the Massachusetts AtD colleges were embarking on new strategic plans early on in the initiative, which provided a timely opportunity for them to incorporate their AtD goals into their institutional plans. All of the colleges had reformed their strategic planning processes to be more focused on key student success outcomes, and some had begun to develop quantitative benchmarks to monitor their progress. For instance, administrators at one college reported that their strategic planning goals included the use of data to make institution-wide decisions, the selection of indicators to monitor changes in student outcomes, and the formulation of action plans to accomplish their goals during their planning cycle. An administrator at another college noted that his colleges new strategic plan, really looks at what we can do with the tenets of AtD to make this a strong plan for student success. Finally, a third college seized the opportunity to bring together business leaders, board members, alumni, students, and community members to develop its long-range strategic plan, which places an increased emphasis on evidence-based decision making and student success.

Achieving the Dream (2009), p. 32

13

All four colleges have also developed active committee structures to guide and sustain their progress under AtD. As mentioned earlier in this memorandum, each college has convened taskforces or teams to review student data and coordinate planning around particular activities or priority areas. Two colleges have already taken steps to ensure the sustainability of this committee work, both at the individual strategy level and at the institution-wide decision-making level, while others were developing plans for this work. For example, one college integrated its AtD data team with a preexisting institutional effectiveness taskforce to maximize efficiency and ensure that the team will continue after initiative funding ends. The responsibilities delegated to these committees vary in scope; however, they all seek to involve more college-wide representation when making institutional decisions. Colleges have consistently recognized the value of AtD coaching and data facilitation to help focus their efforts around key institutional priorities. Among the supports provided by AtD, coaching and data facilitation were repeatedly cited among interviewees as the most significant contributors to colleges progress under the initiative. Data facilitators were seen as knowledgeable not only about technical work, but also more broadly about institutional leadership and decision making. While data facilitators helped colleges design their strategy evaluation plans and develop more streamlined, user-friendly methods for reporting data, AtD coaches have helped to push, guide, and cheerlead for their colleges. One President reported that it was the coaching from the initiative, rather than the grant money, that proved most useful in advancing the colleges goals under AtD. Coaching and data facilitation have proven particularly valuable to colleges in focusing their strategic planning processes around select institutional priorities, narrowing down a large set of key performance indicators, and providing a more focused approach to define, track, and report on institutional progress. The annual AtD Strategy Institute conferences were noted as a useful venue for networking and learning about best practices. All four colleges highlighted the important role that the Strategy Institutes played in their ability to network with other institutions and share ideas about their reform plans. In the words of one administrator, The annual Strategy Institute is one of the more substantial and useful national [conferences] that Ive gone to in the years that Ive been doing this kind of business. In fact, personnel at all four colleges suggested that they would benefit from more frequent opportunities for such cross-college collaboration.

14

The Massachusetts AtD colleges have demonstrated solid progress toward institutionalizing their cultures of evidence, but they continue to face obstacles to program scale up. In spite of their accomplishments in building a culture of evidence, the Massachusetts colleges face a challenging budget environment that may threaten the sustainability and scale up of their AtD strategies. Two colleges cited the need for additional funding to support the desired improvements to their data capacities, while all four mentioned the strain of financial limitations to program scale-up, even for their most promising strategies. Securing funding to continue these strategies after the AtD grant will be an important area for future work among the AtD colleges and other stakeholders in the state of Massachusetts.

Achieving the Dream State Policy Work In addition to institution-level reform, the AtD initiative encompasses a substantial state policy component. The Massachusetts AtD State Policy Team was created to collaborate with Jobs for the Future (JFF), one of the AtD partner organizations, in an effort to improve regulation, legislation, and resource allocation for community colleges in the state. The ultimate goal of these efforts is to improve the success of underserved students by promoting systemic and programmatic reforms to better meet their needs. The teamcomprised of representatives from the states Department of Higher Education, the Community College Executive Office, the Boston Foundation and partner funders, as well as leaders from the four Massachusetts AtD collegesconvenes approximately twice each semester. Massachusetts State Policy Team experienced some initial setbacks in defining a clear direction for its work. In the words of one member, the team floundered a bit at first as it pursued a state-level agenda that was largely detached from colleges work on the ground under AtD. While the team succeeded at bringing together stakeholders from across the higher education system and laying out a broad agenda from the start, several interviewees noted the time it took to identify specific action items and designate roles for each of the team members. This difficulty is largely a reflection of the Massachusetts higher education system itself, which has historically been relatively decentralized. As such, the State Policy Team faced an ambitious challenge to improve alignment across community colleges, four-year institutions, high schools, and state policy-makers.

15

Despite initial setbacks, the State Policy Team has made important strides in becoming more focused and agenda-driven over time. By 2009, the AtD State Policy Teams Annual Workplan had laid out three main goals: (1) collaborating with AtD colleges to identify and promote statewide the best student success practices and policies;5 (2) creating a communications strategy to inform the public about community colleges work across the state; and (3) informing activities under the states new Vision Project.6 The last of these goals included an effort to extend the priority student outcomes outlined by AtDincluding success and graduation ratesto guide higher education work across the state. As it has clarified its focus, the State Policy Team has also taken early action to achieve its goals, including organizing the first all-state convening of the fifteen Massachusetts community colleges. In addition to further defining its priorities, the State Policy Team also developed several actionable steps for accomplishing these goals and has made some progress in reaching them. The most impressive of these steps was a statewide convening of all 15 Massachusetts community colleges, which was hosted jointly by the State Policy Team and JFF in June 2010. This meeting was designed as an opportunity to increase crossinstitutional collaboration in developmental education reform, as well as to allow the four Massachusetts AtD colleges to share their work with other institutions in the state. College personnel and State Policy Team members alike responded positively to this convening. One team member noted the states progress toward communicating at all levels so that were all working towards the same thing. Outside of organizing the June convening on developmental education, the State Policy Team has made important contributions to other statewide efforts, including the creation of a new transfer policy.7 JFF has played a crucial role in helping the State Policy Team to identify and organize around key priorities for the state. Members of the State Policy Team consistently emphasized the important role that JFFs guidance and policy expertise played in helping shape their focus. For instance, one interviewee commented that JFF led us to certain discussions and
5 6

Achieving the Dream Massachusetts State Policy Team (2009), p. 3. The Vision Project is an initiative, operated through the Massachusetts Department of Higher Education, to promote unity and accountability across the states higher education system. http://www.mass.edu/currentinit/visionproject.asp 7 The MassTransfer policy, which includes a portable general education transfer block, tighter transfer agreements, and a website for students and families with information about the transfer process, will go into effect in Fall 2010.

16

helped us to determine where we should be at while another emphasized the value of JFFs technical/logistical perspective andcontent perspective. More specifically, team members noted the critical insight JFF has provided on policy work in other states, which has helped the Massachusetts Team to generate ideas and strategize about how to leverage national reforms for the benefit of the state. This guidance has helped the team to increase its focus on what one member described as the outrageous importance of developmental education in student success. The Massachusetts Policy Team has made significant advances toward promoting a coherent and collaborative state agenda for community college students success; however, state-level policy change is rarely fast or easy, and much remains to be done. It will be important for the team to build upon its accomplishments to date by establishing clear follow-up plans and action items for future work. Given such ambitious and far-reaching goals, as well as the manifold job responsibilities of its members, the team might benefit from holding more frequent meetings to identify next steps and deadlines.

Conclusion MDRCs analysis of the four Massachusetts collegeswhich joined AtD as part of the Round 4 cohort in 2007suggests that they have made more consistent early progress in institutionalizing a culture of evidence than had the Round 1 and Round 3 colleges at comparable points in their implementation grants. Anchored by strong executive leaders, all of whom demonstrated a commitment to improving student success before joining AtD, the four Massachusetts colleges entered the initiative well-equipped to make data-informed institutional changes. These leaders have been instrumental in aligning their preexisting institutional goals for student success with the initiatives expectations for institutional improvement, as well as in articulating these goals across their campuses. While these colleges were already moving in a positive direction before joining AtD, their participation in the initiative further bolstered their progress. In particular, college leaders have benefited from the insights and lessons learned in prior rounds of the initiative, which have been imparted to them by coaches and data facilitators who have experience working with earlier cohorts of AtD colleges.

17

Across the Massachusetts colleges, AtD has also provided the impetus to make data more accessible and meaningful at the institutional level, as colleges apply evidence from their data analyses to effect changes in programs and services. Though most of these colleges still have work ahead of them to use data broadly and systematically in their daily practice, they have taken important steps to develop the personnel and systems they need to integrate data use into their decision making processes. In addition, growing numbers of faculty and staff have stepped up as new campus leaders by participating at the decision-making table and leading the charge to build stronger programs that promote student success; nonetheless, some segments of the campus community remain on the periphery of this work. AtD has also played a key role in helping at least two of the colleges to focus their longterm strategic planning and budgeting processes on prioritizing key performance indicators, broadened participation, and data-driven decision making. Externally, all four colleges have demonstrated a strong commitment to reach out to their local communities in an effort to develop cross-sector goals and strengthen partnerships to support student success. The Massachusetts State Policy Team initially took more time than expected to solidify its objectives and action plans, particularly with respect to aligning its work with activities at the college level. More recently, however, the team has clarified its direction and priorities for improving the states public higher education system. It took a significant step with its state-wide convening on developmental education to promote dialogue between institutions and state-level personnel.

Recommendations and Suggestions for the Funders and for the Initiative Based on our site visits and our review of institutional documents on data and programs, MDRC would like to make the following recommendations to the initiative leaders and funders about how to further support the work of the Massachusetts AtD colleges: Provide more opportunities to share knowledge and create partnerships across colleges. Interviewees across the four colleges reported that they would appreciate more frequent opportunities to learn from one anothers experiences, just as they have benefited from the experiences of the colleges that joined the initiative in earlier rounds. They wanted to be aware of promising practices and lessons learned at other colleges in the state, as well as state policies that might affect their own institutional plans. They also hoped to collaborate more 18

closely with the other community colleges in Massachusetts which, though not participating in the initiative, might still possess valuable information about promising practices, resources, and lessons learned. Provide college leaders with increased guidance about how to fund and sustain emerging and promising practices. College administrators could benefit from greater assistance from the initiative leadership and funders as they look for opportunities to compete for additional funding and develop innovative ideas for maximizing in-house resources. More opportunities are needed for the colleges to brainstorm with experts about potential opportunities to secure sustainable resources for the promising work they have already accomplished. Furthermore, some college leaders were particularly interested in learning about ways to maintain their strong relationships with their coaches and data facilitators after the grant period ends. Involve college leaders more consistently in the work of the State Policy Team. Although the college presidents have each had an opportunity to serve on the Massachusetts AtD State Policy Team, other college representatives noted that they would like to be more involved in, or at least more aware of, the nature of the State Policy Teams efforts. The state-wide convening this past summer was a good start, but college leaders would like to know more about how their institutional work serves the goals of the State Policy Team, and vice-versa. One approach might be to have the Massachusetts State Policy Team or the Boston Foundation provide additional forums for colleges to come together to share ideas about the ways in which college practices can inform the higher education state policy agenda. Allow future AtD colleges to provide input on the types of coaches and data facilitators they need to support their specific goals. Most of the Massachusetts AtD colleges found their relationships with their AtD coaches and data facilitators to be very positive and productive. However, one college needed to ask for a new AtD coach early on in the initiative, when its leadership felt that the coach originally assigned did not meet the colleges particular needs. If the initiative continues to provide colleges with coaching and data facilitation, then it might solicit more input from college leaders about what particular knowledge or skill sets they value most in a potential coach or data facilitator.

19

Send more foundation and AtD representatives to visit the colleges in order to increase communication about the work being done at different levels of the initiative. College leaders would like to invite the funders and initiative leaders to make more frequent visits to their campuses to observe first-hand the progress they have made towards building a culture of evidence. List of References Achieving the Dream. 2009. Field Guide for Improving Student Success. Chapel Hill: MDC, Inc. Achieving the Dream Massachusetts State Policy Team. 2009. Achieving the Dream Massachusetts State Policy Workplan: FY 2009-2010. Unpublished. Brock, Thomas, Davis Jenkins, Todd Ellwein, Jennifer Miller, Susan Gooden, Kasey Martin, Casey MacGregor, and Michael Pih with Bethany Miller and Christian Geckeler. 2007. Building a Culture of Evidence for Community College Student Success: Early Progress in the Achieving the Dream Initiative. New York: MDRC. Jenkins, Davis, Todd Ellwein, John Wachen, Monica Reid Kerrigan and Sung-Woo Cho. 2009. Achieving the Dream Colleges in Pennsylvania and Washington State: Early Progress Toward Building a Culture of Evidence. New York: Community College Research Center, Columbia University, and MDRC.

20