CCI participants have come to recognize the mismatch between the evaluation needs of CCIs and prevailing approaches of the evaluation field. Traditional approaches to evaluation are rarely designed to capture the breadth and complexity of factors operating at the neighborhood level. In recent years, evaluators have begun to experiment with a variety of evaluation approaches that will enhance the fields' ability to learn from and to judge the effectiveness of CCIs. The readings in this section examine the challenges of evaluating CCIs and introduce some of the innovative approaches to evaluation that are being tested by the CCI field.

	A Framework for Evaluating Systems Initiatives, Coffman, Julia., Build Initiative., August, 2007. This publication provides strategies and key considerations for evaluating “systems initiatives,” meaning initiatives that aim to influence, reform, or build institutions related to healthcare, education, childhood, or other public services. The findings are based on a meeting of evaluators sponsored by the Build Initiative, a multi-site early childhood development systems initiative. The report emphasizes the importance of a “theory of change” in systems initiative evaluation. A theory of change maps the relationship between an initiative’s goals, the programmatic interventions it employs to achieve those goals, and the outcomes that would demonstrate that the goal had been achieved. The report provides a menu of five different theories of change, focusing on systems context, components, connections, infrastructure, and scale. It goes on to describe how to evaluate initiatives that fall within these theories of change, focusing on two primary questions: “did the initiative do what it said it would do?” and “did the initiative produce the expected results?” The report then applies the suggested framework to an evaluation of the Build Initiative itself. It concludes with a set of principles for systems initiative evaluation, including: “clarify the evaluation’s audience and intended use for the evaluation’s findings,” and “identify an appropriate level of methodological rigor.” (Topics: Evaluation, Planning)
	Applying a Theory of Change Approach to the Evaluation of Comprehensive Community Initiatives: Progress, Prospects, and Problems, Connell, James P. and Anne Kubisch, New Approaches to Evaluating Community Initiatives: Theory, Measurement, and Analysis, The Aspen Institute, 1998 This paper, part of the second volume produced by The Aspen Institute on evaluating CCIs, attempts to take the theory of change approach and apply it specifically to the circumstances of CCIs. The authors define a theory of change approach to CCI evaluation and describe steps stakeholders can follow to carry one out. The description is followed by reflections on the constraints and promise of the approach, including its capacity to reinforce the basic principles of CCIs and contribute to a knowledge base that can inform future neighborhood-based interventions. (Topics: Evaluation)
	Challenges of Measurement in Community Change Initiatives, Gambone, Michelle Alberti, New Approaches to Evaluating Community Initiatives: Theory, Measurement, and Analysis, The Aspen Institute, 1998 This chapter builds on a meeting attended by evaluators and researchers convened by the Aspen Institute Roundtable on CCIs to identify the specific measurement-related challenges facing those evaluators of CCIs and to organize measures of community-level change. By distilling the discussions that emerged from the meeting, Gambone offers a framework for thinking through the various CCI dimensions that need measuring, covering context, progress, and early, interim, and long-term outcomes. (Topics: Evaluation)
	Core Issues in Comprehensive Community-Building Initiatives, Stone, Rebecca, ed., Chapin Hall Center for Children, 1996 A seminal contribution to the field of comprehensive community initiative research, this book represents a compilation of essays and discussions surrounding community-building initiatives. The essays focus on eight core issues at the heart of CCIs: theories of neighborhood change, exploring visions of 'built' community, the role of the sponsor in CCIs, neighborhood governance, evaluation and CCIs, the economic context for community building, building capacity, and community organizing. Response essays, which are included in this volume, challenge readers to question their assumptions, to take risks, and to be forthright about the challenges, uncertainty, and the potential of comprehensive community building. The tone of the volume is set by their willingness to break free of more traditional modes of discussion and by the honesty and forthrightness of their convictions. With eight essays and thirty-two responses, this volume encompasses a broad range of views and experiences that characterize the field of community-building. Central to, if not explicit in, these discussions is the issue of power. Whether the subject is "empowerment" of local actors, distribution of resources within community, barriers to effective community collaboration, control of the initiative process, or the legitimacy of outside sponsorship in community-based initiatives, power and the influence of resources underlie almost every practical issue as well as the "tensions" found within most initiatives. Ordering info: http://www.chapin.uchicago.edu/ProjectsGuide/PublicationsOrder.html. (Topics: Understanding CCIs, Neighborhood-based change, Community Building, Participation, Capacity building, Community Building Governance, Evaluation, Funding/Financing, Governance, Organizing, Economic Development)
	Establishing a Causality in Evaluations of Comprehensive Community Initiatives, Granger, Robert C., New Approaches to Evaluating Community Initiatives: Theory, Measurement, and Analysis, The Aspen Institute, 1998 Turning to the thorny issue of attributing causality--a problem that dominates evaluators' discussions about CCIs--Granger suggests several strategies evaluators can use to strengthen the case that neighborhood outcomes can be attributed convincingly to CCIs. Beginning with a brief discussion of the theory of change approach, the author then submits the approach to a "test" for evaluations--that is whether the evaluation is responsive to the needs of stakeholders and produces credible and generalizable results. The paper concludes that theory-based approaches can help on these counts if evaluators attend to the need for sufficiently credible counterfactuals (control groups or comparison communities) at all stages of their work. (Topics: Evaluation)
	Evaluating Community Development Programs: Problems and Prospects, Rossi, Peter H., in Urban Problems and Community Development, Ronald F. Ferguson and William T. Dickens (eds.), Brookings Institution Press, 1999 Social policy evaluation has evolved from a primary concern with impacts and effectiveness toward a growing interest in diagnostic evaluation that aims to improve program design and implementation. Community development efforts that are complex and multifaceted pose special challenges for both impact and diagnostic evaluations. Impact evaluations require comparing a group of people who have experienced an intervention with a group whose experience approximates what would have happened in the absence of intervention. In evaluating community development, selecting such groups is difficult but, Rossi argues, not impossible. He also notes that diagnostic evaluations might produce more useful results if methods were more refined and standardized. He proposes several projects for improving the future of community development evaluation. Ordering Info: http://www.brook.edu/press/books/urbancom.htm. (Topics: Evaluation, Community Development)
	Evaluation of Comprehensive Community-Building Initiatives, Brown, Prudence, in Core Issues in Comprehensive Community-Building Initiatives, edited by Rebecca Stone, Chapin Hall Center for Children This essay brings into focus the ways evaluation of CCIs has become the arena in which many of the issues, expectations, and frustrations of initiatives get played out. Because of its inextricable link to public perception, future funding, and even internal feelings of optimism, "evaluation" of any program is often unpopular with participants and others invested in the success of the endeavor. Outside evaluators, prized for their objectivity by funders and policymakers, may be viewed with skepticism by program participants and staff for the same reason: an outsider's objectivity can be equated with ignorance of what's "really" happening, and thus dismissed as not reflective of the truth. Evaluations of CCIs are further complicated by a number of other issues peculiar to the nature of CCIs: the complexity of factors at work in neighborhoods; the nonlinear progression of community change; human factors that are difficult to capture with traditional evaluation techniques; and increasing demands on evaluators themselves to serve as interpreters or facilitators within CCIs, possibly compromising the "objectivity" so valued in traditional evaluation. Thus, CCIs pose a challenge to evaluators to rethink such issues as who an evaluation should serve, whether objectivity comes at the expense of more important facets of evaluation, and whose interests are represented by different approaches to learning. Ordering info: http://www.chapin.uchicago.edu/ProjectsGuide/PublicationsOrder.html. (Topics: Evaluation)
	Finding Out What Works: Understanding Complex, Community-Based Initiatives., Coote, Anna, Jessica Allen, and David Woodhead., King’s Fund., November 2004. This report critiques evaluation of community-based initiatives that address health inequality and social exclusion, focusing on six case studies from the United States and the United Kingdom: in the U.K., Health Communities Collaboratives, Employment Retention and Advancement Demonstration Project, and the Social Action Research Project; and in the U.S., Plain Talk, California Works for Better Health, and the East Tennessee Foundation Peer to Peer Learning Project. Through interviews with local practitioners and government funders, the report documents key problems in the assessment of complex-community based initiatives, such as: the tension between “evidenced-based” practice and local empowerment; the lack of rigorous evaluation techniques for complex initiatives; the competing demands on evaluation from researchers as compared to civil servants or politicians; and the political, institutional, and organizational pressures that impair knowledge-building and peer learning. The report recommends improving the quality of evaluation by encouraging more detailed research and deliberation on the use of varied kinds of evidence, greater inclusion of practitioners in the evaluation process, and strengthening the peer learning and knowledge-building capacity within government. (Topics: Evaluation, Case Studies)
	How Can Theory-Based Evaluation Make Greater Headway, Weiss, Carol H., Evaluation Review, Vol. 21, No. 4, Sage Publications, 1997 The idea of theory-based evaluation is plausible and cogent, and it promises to bring greater explanatory power to evaluation. However, problems beset its use, including inadequate theories about pathways to desired outcomes in many program areas, confusion between theories of implementation and theories of programmatic action, difficulties in eliciting or constructing usable theories, measurement error, complexities in analysis and others. This article explores these problems, describes the nature of potential benefits, and suggests that the benefits are significant enough to warrant continued effort to overcome the obstacles and advance the feasibility of theory-based evaluation. This journal article can be accessed through ProQuest Direct at http://www.umi.com/proquest/. (Topics: Evaluation)
	Introduction: New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts, Kubisch, Anne C., Carol H. Weiss, Lisbeth B. Schorr, James P. Connell, New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts, The Aspen Institute, 1995 The introduction to this first volume published by The Aspen Institute on CCI evaluations outlines some of the key issues and challenges associated with the evaluation of CCIs and provides an overview of the papers commissioned for this volume. The first paper puts today's CCIs and the problems of their evaluation in historical context by reviewing the experience of the juvenile delinquency programs of the 1950s, the Gray Areas, Community Action and Model Cities programs, and the community development corporation movement. The next two papers focus on evaluation problems that emerge as a result of the complex design of CCIs and suggest new ways of approaching the task. The following two papers address methodological problems associated with CCI evaluation, while the final paper looks at new roles that evaluators can take with respect to CCIs. (Topics: Evaluation)
	IX: Measuring Community Development Impact: New Technology Tools for Participatory Evaluation. Papers of the Summit on Technology and Community Development., Grieve, Margaret., Living Cities., November 2002. This paper describes participatory evaluation technology for measuring the impact of community development initiatives. It highlights the Success Measures Data System, a web-based impact measurement tool developed by the McAuley Institute, the Development Leadership Network and the Fannie Mae Foundation. The Success Measures Project enables community development practitioners and community stakeholders to select relevant outcome indicators from a pre-set list to create a desired “benefits picture” for their initiative. Data can then be entered and organized according to these indicators, and then aggregated and compared with data from other initiatives in order to assess national trends in community development. In addition, the paper documents programs that focus on improving information systems and lists tools that facilitate data collection and interpretation. Grieve argues that the failure of community development organizations to affectively measure, catalogue, and disseminate data about their programs has limited their impact as well as their funding prospects. She concludes by noting potential benefits of the Success Measure Data System, and other participatory evaluation technologies, for the community development field, such as cheaper, higher quality evaluations, common indicators that allow for more meaningful programmatic comparisons, and greater communications and knowledge-building between organizations. (Topics: Evaluation, Technology)
	Measures for Community Research, Aspen Institute Roundtable on Comprehensive Community Initiatives for Children and Families The Measures for Community Research data base is a collection of measures used to evaluate outcomes viewed as important by Comprehensive Community Initiatives (CCIs), public policy makers, program funders and experts in relevant research fields. The data base includes descriptions of primary data collection instruments, such as survey instruments, interview protocols, and self-assessment guides. (Topics: Evaluation, Evaluators)
	Nothing As Practical as Good Theory: Exploring Theory-Based Evaluation for Comprehensive Community Initiatives for Children and Families, Weiss, Carol Hirschon, The Aspen Institute, 1995 In this paper, part of the first volume published by the Aspen Institute on evaluating CCIs, Weiss advances a mode of evaluation which is based on the "theories of change" that underlie an initiative and discusses how such an approach would serve the multiple purposes of evaluation. Weiss challenges CCI designers to be specific about theories of change and suggests that doing so would improve their overall evaluation plans while also providing guidelines for data collection and analysis. (Topics: Evaluation)
	Principles for Evaluating Comprehensive Community Initiatives, David Chavis, Kien Lee, Elizabeth Jones, Association for the Study and Development of Community, June 2001 This document describes principles for evaluating CCIs. These principles also promote the use of evaluation to build community capacity. The purpose of these principles is to provide evaluators and others with practical guidance on how to approach the evaluation of CCIs. These principles also raise important issues to consider during the implementation of an evaluation. Another equally important purpose is to begin a learned discussion among a larger group of evaluators on the methods used to evaluate CCIs. The discussion of each principle was developed to provide some initial advice on how to conduct the evaluation of CCIs with the hope that readers can build upon the ideas in their own work and share them with others. (Topics: Evaluation)
	Problems in the Evaluation of Community-Wide Initiatives, Hollister, Robinson G. and Jennifer Hill, New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts, The Aspen Institute, 1995 In this paper, part of the first volume published by the Aspen Insitute on evaluating CCIs, Hollister and Hill focus on the absence of control groups or comparison communities (or counterfactuals) for CCI evaluation purposes and discuss the problems that arise as a result. The paper explores efforts to establish a counterfactual and the problems of establishing comparison groups against which to judge the effects of an intervention. Concluding that none of these alternative approaches serve as an adequate counterfactual "primarily because individuals and communities are changing all the time with respect to the measured outcome even in the absence of any intentional intervention," Hollister and Hill examine steps that can be taken to facilitate the development of better methods of CCI evaluation. (Topics: Evaluation)
	The Evaluation Exchange, Harvard Family Research Project Harvard Family Research Project's evaluation periodical, The Evaluation Exchange, addresses current issues facing program evaluators of all levels, with articles written by the most prominent evaluators in the field. Designed as an ongoing discussion among evaluators, program practitioners, funders, and policymakers, The Evaluation Exchange highlights innovative methods and approaches to evaluation, emerging trends in evaluation practice, and practical applications of evaluation theory. (Topics: Evaluation, Journals)
	The Role of the Evaluator in Comprehensive Community Initiatives, Brown, Prudence, New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Analysis, The Aspen Institute, 1995 Brown's paper, part of the first volume published by the Aspen Insitute on evaluating CCIs, examines the new roles that evaluators are taking with respect to CCIs, particularly those that engage the evaluator in the initiative more than has traditionally been the case. The paper begins by reviewing the CCI characteristics that create special challenges for the evaluator, the existing social science context in which evaluations are being developed, and the different purposes and audiences for such evaluations. This is followed by sections on the current status of evaluations in the field and on the different options and strategies-both their limits and possibilities-open to evaluators. The paper ends with some thoughts on how evaluators can maximize the learning opportunities CCIs present through increased innovation and experimentation and through the opportunity to structure disciplined cross-site learning. (Topics: Evaluation)