September 8, 2022
The following story appeared in the Sisters of Charity Foundation of Cleveland's 2021 annual report: 25 Years of Partnership and Authenticity.
Asking the Right Questions: Equity & Partnership in Evaluation
In 2021, SOCF Cleveland worked to shift our knowledge, evaluation and learning practices to be in service of equity and better reflect the context of the communities we aim to uplift.
Christine Baker, Ph.D, the foundation’s senior director of evaluation and learning, sought to embrace a more complex understanding of how these efforts furthered our value of justice and supported our mission. By advancing the Equitable Evaluation Framework™, we committed to aligning our evaluation practices with the structural conditions and context shaping the daily experiences of the populations we aim to serve.
“Historically at foundations, program staff or leadership define what success looks like, whether it’s who is making the ‘best’ use of foundation resources or who is making choices that align with that foundation’s
values, as examples,” Baker said. “This perpetuates the power dynamic between funder and grantee, and often doesn’t account for patterns of oppression and systemic discrimination.”
We do our best work when we are among community leaders, those families, young people or Catholic sisters we seek to support. We hear their questions, concerns and desires, and we realized they don’t always match the approaches we used, the data we collected or the results we assumed we would find.
Through critical conversations and learning about how the EEF™ can be integrated into our practices, three entry points have surfaced:
- Our evaluative work must account for the cultural contexts and structural conditions that surround our interventions, initiatives and partnerships.
- We can only define success when we define it with those affected.
- We use approaches and methods that result in different forms of knowledge.
FINDING SUCCESS IN FAMILY PARTNERS
Family Partners is an initiative that utilizes a network of experts in early childhood development, family health and social connections to build relationships with families in Cleveland’s Central neighborhood, affirm their strengths and areas for growth, and help them build the support systems necessary for positive, stable outcomes for children and caregivers alike. Several partners comprise the initiative, and each has specific roles related to education of children and caregivers, as well as behavioral health and wellness outcomes. A key element of the Family Partners model is to incorporate resident and family voices into the growth of the program.
In 2019, ahead of the Family Partners launch, staff developed the Family Partners evaluation plan using evidence-based indicators, outcomes and data collection tools aligned with the research-informed Protective Factors framework.
When Ohio went into COVID-19 lockdown in March 2020, the program staff changed the program delivery model, canceling many in-person activities and shifting to virtual home-visits and therapy sessions. Richaun Bunton, managing director, Cleveland Central Promise Neighborhood, led the development of a Family Advisory Council to ensure family and caregiver voices could continue to inform program adjustments. The staff and families adjusted to a different program experience quickly, but the effects were significant.
The pandemic laid bare the impact of historical and structural racism on living and working conditions, stress and access to health care for communities of color. These disparities affected the Central Neighborhood and the families in the program, all of whom are black or brown. Through the Family Advisory Council, parents and caregivers emerged as community leaders even amid unprecedented neighborhood strife. Their leadership changed SOCF’s thinking on how to measure many of the Family Partners outcomes.
For example, in terms of child academics, success was previously defined solely by scores on state-mandated assessments in math, reading, social-emotional and non-cognitive development. However, families recognized that they had better control over the educational factors in the home rather than individual academic outcomes.
Instead of only looking to assessment scores, families on the advisory council agreed that they defined success as:
- Working hard
- Completing tasks or work they start
- They (and their children) feel they have well-established networks of resources and support.
"This program helped me become a better me, a better parent," one Family Advisory Council member said. "This program has changed my life for the better."
We now use qualitative data from caregivers describing these factors along with assessment scores and results to more holistically measure child academic growth.
Families shared that the realities of meeting their daily basic needs during the ongoing pandemic limited their capacity to engage in long-term program commitments.
This changed our perspective on how we should collectively provide educational, emotional and community resources. The family and community safety net must be strengthened before any educational shifts are possible. The partners are now working toward building an infrastructure for crisis management and resource distribution to ensure family needs are met.
Our original evaluation framework included an outcome measuring strengthened family relationships. Families indicated the programming shift from in-person to virtual, including a private Facebook page and virtual activities like family pizza-making nights and self-care workshops, enabled them to continue to reach out to one another to share resources and offer support during a time of increased social isolation. Based on this, we chose to adapt the original staff-designed family relationship outcome to include measuring the benefit of peer-to-peer connections between program participants.
Over the three years of evaluating Family Partners, SOCF Cleveland has been able to shift thinking to respond to needs we would not have recognized without the leadership and partnership of caregivers.
This creates deep connections and trusting relationships, which are essential to the well-being of each and every family.