Foundations of Empowerment Evaluation

Rating:
Summary: Empowerment Evaluation Need Empowerment
Review: As a social worker committed to empowerment of oppressed peoples, I come to David Fetterman's "Foundations of empowerment evaluation" favorably predisposed. This favorable disposition is reinforced when Fetterman writes that empowerment evaluation has usually "focused on traditionally disenfranchised, oppressed, and economically impoverished populations" (p. 114). He could just as well be talking about the focus of social work practice.

I also bring to Fetterman's book a growing discomfort with traditional, empiricist research and program evaluation. My colleagues have been working for several years with different Native American tribes and have learned first-hand why research is so often considered a dirty word in Indain Country. At best, positivist research and evaluation are usually irrelevant to the experience and concerns of Native peoples, at worst it contributes to their continued subjugation by reinforcing negative stereotypes. I therefore come to Fetterman's book with a keen interest in research and evaluation which in itself is a benefit to the participants.

Empowerment evaluation, as articulated by Fetterman, is located within "the larger context of emancipatory research" (Fetterman, 2001, p. 110) and specifically action research. Empowerment evaluation differs from action research, however, because it is "explicitly driven by the concept of self-determination" and collaboration at every level of the agency, program, or community being evaluated (p. 11). Within that framework, empowerment evaluation helps people analyze their programs using self-evaluation and self-reflection. In contrast with some program assessments conducted by outside evaluators, empowerment evaluation is not an endpoint but in integral part of program development and improvement (p. 3). As a result, empowerment evaluation "ideally helps organizations internalize evaluation principles and practices, making evaluation an integral part of program planning" (p. 35). Not surprisingly, Fetterman equates using empowerment evaluation with teaching participants to fish.

The methodology of empowerment evaluation is deceptively simple. First, gather everyone involved in the program together, from clients to workers, to administrators, to funders. The empowerment evaluator then acts as a coach or facilitator, who leads the group through three activities: 1) explicate the mission of vision of the program; 2) take stock of each program activity, rating how well each is doing; and 3) plan for the future by setting goals, strategies for reaching the goals, and documentation to be used to track progress. Because empowerment evaluation is ideally an ongoing process, the first time through these steps forms the baseline for monitoring future progress.

Methodologically, the obvious potential weakness of empowerment evaluation is the possibility of generating self-serving, 'sugar-coated' results (p. 5-6). "An outside evaluator charged with monitoring the progress can help keep the effort credible, useful, and on track, providing additional rigor, reality checks, and quality controls throughout the evaluation" (Fetterman, 2001, p. 6). This task is operationalized during the ranking phase, for example, when the empowerment evaluator questions each individual's ranking of each activity, asking for specific documentation of why the rating is no higher and no lower than they have indicated. Additional rigor is gained, according to Fetterman, by indicating that people can change their ratings at any time. Mostly, however, Fetterman argues that trustworthiness flows from the nature of the group (the insistence on democratic participation of clients, workers, and administrators), and the group process itself which he claims comes to conclusions which are more realistic than the conclusions of external evaluators (pp. 32-33).

Unfortunately Fetterman's organization and writing of this volume does little to support his claim that empowerment evaluation is rigorous and inherently trustworthy. In terms of organization, note that the two quotes I used to locate empowerment evaluation within research and evaluation traditions were separated by 100 pages. Repeatedly, ideas and concepts are partially developed in one place, only to be modified or more fully defined/described much later. In many ways, this review has made Fetterman's book more comprehensible than it is. For example, I stated previously that when participants rank program activities they must provide documentation. Fetterman, however, discusses ranking on page 29, but waits 60 pages (almost half the book) to mention that "documentation is always required to support a high or low rating about a program assessment, and is scrutinized." (p. 93).

In addition, Fetterman's writing often confuses or obscures important points. The trustworthiness of empowerment evaluation rests in large part on including the perspective of every stakeholder and participant in the program, from clients to administrators. Fetterman, unfortunately, uses the term program participants variously, mostly to refer to paid employees, but occasionally also to include clients (e.g. pp. 3, 34, &146). As a result we are never sure who he is talking about participating in what aspect of the evaluation. Similarly, Fetterman refers to "outside evaluators" both as the 'coaches' who help participants throughout the process and, in other places, as people not otherwise involved in any aspect of the evaluation. Consequently when Fetterman writes that "An outside evaluator charged ... can help keep the effort credible, useful, and on track" (2001, p. 6), I have no idea who is responsible for this crucial methodological function.

Finally, Fetterman states that the trustworthiness and utility of empowerment evaluation flow, in large measure, from use of baseline data and repeated empowerment evaluation sessions. Though Fetterman devotes more than a third of his book to case examples, every one of those examples is of an initial evaluation, never a follow-up session. Because Fetterman (belatedly) admits that empowerment evaluation "takes more time than most traditional evaluations (p. 145), I am left wondering if follow up is ever conducted.

The disorganization of the book unfortunately contributes to an impression that empowerment evaluation is methodologically less rigorous than I believe it really is. Fetterman thus undermines the very evaluation methodology which he passionately believes in and espouses.

Rating:
Summary: Empowerment Evaluation Need Empowerment
Review: As a social worker committed to empowerment of oppressed peoples, I come to David Fetterman's "Foundations of empowerment evaluation" favorably predisposed. This favorable disposition is reinforced when Fetterman writes that empowerment evaluation has usually "focused on traditionally disenfranchised, oppressed, and economically impoverished populations" (p. 114). He could just as well be talking about the focus of social work practice.

I also bring to Fetterman's book a growing discomfort with traditional, empiricist research and program evaluation. My colleagues have been working for several years with different Native American tribes and have learned first-hand why research is so often considered a dirty word in Indain Country. At best, positivist research and evaluation are usually irrelevant to the experience and concerns of Native peoples, at worst it contributes to their continued subjugation by reinforcing negative stereotypes. I therefore come to Fetterman's book with a keen interest in research and evaluation which in itself is a benefit to the participants.

Empowerment evaluation, as articulated by Fetterman, is located within "the larger context of emancipatory research" (Fetterman, 2001, p. 110) and specifically action research. Empowerment evaluation differs from action research, however, because it is "explicitly driven by the concept of self-determination" and collaboration at every level of the agency, program, or community being evaluated (p. 11). Within that framework, empowerment evaluation helps people analyze their programs using self-evaluation and self-reflection. In contrast with some program assessments conducted by outside evaluators, empowerment evaluation is not an endpoint but in integral part of program development and improvement (p. 3). As a result, empowerment evaluation "ideally helps organizations internalize evaluation principles and practices, making evaluation an integral part of program planning" (p. 35). Not surprisingly, Fetterman equates using empowerment evaluation with teaching participants to fish.

The methodology of empowerment evaluation is deceptively simple. First, gather everyone involved in the program together, from clients to workers, to administrators, to funders. The empowerment evaluator then acts as a coach or facilitator, who leads the group through three activities: 1) explicate the mission of vision of the program; 2) take stock of each program activity, rating how well each is doing; and 3) plan for the future by setting goals, strategies for reaching the goals, and documentation to be used to track progress. Because empowerment evaluation is ideally an ongoing process, the first time through these steps forms the baseline for monitoring future progress.

Methodologically, the obvious potential weakness of empowerment evaluation is the possibility of generating self-serving, `sugar-coated' results (p. 5-6). "An outside evaluator charged with monitoring the progress can help keep the effort credible, useful, and on track, providing additional rigor, reality checks, and quality controls throughout the evaluation" (Fetterman, 2001, p. 6). This task is operationalized during the ranking phase, for example, when the empowerment evaluator questions each individual's ranking of each activity, asking for specific documentation of why the rating is no higher and no lower than they have indicated. Additional rigor is gained, according to Fetterman, by indicating that people can change their ratings at any time. Mostly, however, Fetterman argues that trustworthiness flows from the nature of the group (the insistence on democratic participation of clients, workers, and administrators), and the group process itself which he claims comes to conclusions which are more realistic than the conclusions of external evaluators (pp. 32-33).

Unfortunately Fetterman's organization and writing of this volume does little to support his claim that empowerment evaluation is rigorous and inherently trustworthy. In terms of organization, note that the two quotes I used to locate empowerment evaluation within research and evaluation traditions were separated by 100 pages. Repeatedly, ideas and concepts are partially developed in one place, only to be modified or more fully defined/described much later. In many ways, this review has made Fetterman's book more comprehensible than it is. For example, I stated previously that when participants rank program activities they must provide documentation. Fetterman, however, discusses ranking on page 29, but waits 60 pages (almost half the book) to mention that "documentation is always required to support a high or low rating about a program assessment, and is scrutinized." (p. 93).

In addition, Fetterman's writing often confuses or obscures important points. The trustworthiness of empowerment evaluation rests in large part on including the perspective of every stakeholder and participant in the program, from clients to administrators. Fetterman, unfortunately, uses the term program participants variously, mostly to refer to paid employees, but occasionally also to include clients (e.g. pp. 3, 34, &146). As a result we are never sure who he is talking about participating in what aspect of the evaluation. Similarly, Fetterman refers to "outside evaluators" both as the `coaches' who help participants throughout the process and, in other places, as people not otherwise involved in any aspect of the evaluation. Consequently when Fetterman writes that "An outside evaluator charged ... can help keep the effort credible, useful, and on track" (2001, p. 6), I have no idea who is responsible for this crucial methodological function.

Finally, Fetterman states that the trustworthiness and utility of empowerment evaluation flow, in large measure, from use of baseline data and repeated empowerment evaluation sessions. Though Fetterman devotes more than a third of his book to case examples, every one of those examples is of an initial evaluation, never a follow-up session. Because Fetterman (belatedly) admits that empowerment evaluation "takes more time than most traditional evaluations (p. 145), I am left wondering if follow up is ever conducted.

The disorganization of the book unfortunately contributes to an impression that empowerment evaluation is methodologically less rigorous than I believe it really is. Fetterman thus undermines the very evaluation methodology which he passionately believes in and espouses.