A provider, a researcher, and a new way of improving lives.
Here's what they had to say.
Heather Reynolds: When I became CEO of CCFW more than a decade ago, I read a newspaper article about a CEO retiring from a local homeless shelter. In the article, he shared that after more than two decades of work with the homeless, he thought they were not any better off than the day he had started. Last year, when our organization set a goal of moving 10,000 families out of poverty over the next decade, I was asked if that goal scared me. It does. What scares me even more is the idea that I would be quoted in the newspaper sharing similar sentiments.
The destination matters, and if the journey is what gets you there, then you had better believe that the journey matters, too. A huge part of the journey at CCFW is to invest in research so we can get to our end goal--our destination--of ending poverty one family at a time. We have upped the bar on what ending poverty means--it means families making a living wage, having three months of savings, and being free of debt and government assistance.
We decided to make a change: no more band aids or repeat customers, no more quantifying output goals that only counted the number of people served. We decided we were going to double down on things that we know work with families and shed ourselves of programs and services that did not.
And that requires practitioners and researchers coming together to find out what really works to end poverty.
James Sullivan: Enter the Wilson Sheehan Lab for Economic Opportunities (LEO)--a premier national poverty research lab housed in the Department of Economics at the University of Notre Dame. We match top researchers with social services providers to conduct impact evaluations that identify the innovative, effective, and scalable programs and policies that support self-sufficiency. LEO's research is conducted by Notre Dame faculty, along with an interdisciplinary network of scholars from across the country, with expertise in designing and evaluating the impact of domestic programs aimed at reducing poverty and improving lives. William Evans and I co-founded LEO in 2012 and were quickly introduced by a national partner, Catholic Charities USA, to the interesting work and leadership at CCFW.
HR: We were asked in those early days, again and again, "Are you sure you want to be told what you are doing doesn't work?" Our response--"bring it on."
For example, we know that one of the keys to ending poverty is helping people find living-wage jobs, and one of the keys to getting a living-wage job is a certificate or associate's degree in a growing industry in our local market. But we also know that less than 20 percent of students who start community college actually graduate, even though a degree is a surefire way to break the cycle of poverty.
JS: That's right. Previous research tells us there are four main reasons why community college students drop out: cost, not being prepared for the academic rigors, social and institutional obstacles like not knowing how to access financial aid or settle on a degree plan, and personal obstacles not related to school--life just getting in the way.
Research and services largely focus on the first two. But much less attention has been given to personal obstacles and social and institutional obstacles.
HR: And this would be our sweet spot. As we designed the rollout of our new Stay the Course program, LEO worked with us to embed a randomized control trial (RCT) evaluation in order to rigorously measure the impact of the program and really understand the cause-and-effect mechanisms of the program. Together, we are learning if and how case management makes a difference for low-income students to persist in school and graduate, moving them forward on their path out of poverty.
Stay the Course students are paired with a Navigator--a case manager who walks with them for up to three years of their college career, helping them traverse the school system and overcome the obstacles that normally derail their education. Support may initially be securing housing to avoid homelessness for a family unit, or help enrolling in classes for someone who has never had a family member attend college, or funding a car repair to get that twenty-something single man to class for his exam, or help getting back on track when a class is failed because a single working mom could not keep up when her child got ill. This kind of support--the financial, emotional, tangible support of having a case manager work alongside these clients--this is what creates success.
Since we started Stay the Course more than three years ago, we've served about 400 students and have expanded from one campus to two.
JS: We will release a report on the effect of Stay the Course on student academic outcomes later this summer. The results thus far are quite promising, both for persistence in school and for degree completion.
HR: And now, we plan to replicate Stay the Course in 6-10 locations around the country to demonstrate that our intervention works outside of Fort Worth. We are scaling up next year to add three additional community colleges with plans to add additional sites in the years ahead.
JS: The story of Stay the Course is an important one for students and for evidence-based policy and practice. CCFW saw a need in the clients they were serving that attended community college. They designed a program, drawing on their own expertise in case management, and applied specifically to the nonacademic needs of this population. LEO worked with CCFW to evaluate Stay the Course to provide both continuous feedback to the program managers and staff and to determine, independently and rigorously, the impact of the program on the students it serves. The replication of Stay the Course represents the next stage in creating, evaluating, and scaling evidence-based programs and policies. This effort will inform not only the work of the communities where Stay the Course becomes active, but more generally, national and state policy on community college persistence and completion.
HR: Right now, the social services industry concerns me. So much of what we do is based on funding and the anecdotal story of someone's success. But anecdotes are not evidence. I cringe when I'm approached with the sentiments from a colleague in the industry who says, "I don't think it is ethical for you to conduct research and have people in a control group when they desperately need the services you offer."
Not one nonprofit I know can serve everyone. So why not at least use our "no" as an opportunity to better our services?
JS: A control/comparison group does not mean denying services--often it means providing one group "the status quo" and providing another group a new/bold/enhanced service that has not yet been tested. A comparison group allows you to determine if it was the program itself that helped your clients achieve their goal--in this case to complete college.
Sometimes we get questions about the ethics of conducting research in this way. To be clear, this only works because systems and agencies are already constrained--by funding, staffing, space, and mission--by whom they can and cannot serve. Furthermore, the research we do is always reviewed by Notre Dame's Institutional Review Board (IRB) to ensure clients are properly aware of their participation in research, and that we, the researchers, and the agencies are appropriately using the information and data gained from the research to inform practice and improve understanding of a given field of research.
RCTs are more familiar in the medical field--drug companies run trials to test new products as a part of standard practice before the Federal Drug Administration (FDA) will approve a drug for a given ailment. Without FDA approval, medicines are not allowed to be used, because they are not proven to work. We do not currently have a similar agency or entity that requires and regulates poverty solutions.
HR: If we would never accept approval without RCT in the health care industry, why is it okay for those we serve in poverty? Why is it okay for our nation, our taxpayers, to not even know if their dollars work? Why are we on our ethical high horse all the while providing services without evidence of whether they really make a difference or worse, actually harm the underprivileged?
It is scary to test what you are doing. Every time we know we are getting a new report from LEO, I think we all hold our collective breath with concern and anticipation for what the results will say. We want to get it right and it is hard to be willing to accept the hard truth--that sometimes what you do does not work.
JS: Our goal at LEO is to use the tools of analysis we have to benefit frontline providers and agency leaders as they develop and run programs that truly impact their clients--help them secure a job, move them through school, improve housing stability, move them to self-sufficiency. We know that the best way to measure cause-and-effect of a program is to carefully create a comparison group so that the differences we might find between the people being served and those not being served by a new program are clearly attributable to the program. We can also help by measuring the cost-benefit of the program so that where money is tight and each dollar counts, providers can make informed decisions about which programs do the most for the best value.
HR: Like LEO, we are committed to cracking the code on how to end poverty. So, are we sure we want to be told what we are doing does not work? Yes, if it doesn't work, we want to know. The stakes are too high. Bring it on.
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||Bring It On|
|Publication:||Policy & Practice|
|Date:||Aug 1, 2017|
|Previous Article:||Tapping into the potential of public health and social services partnerships: a framework to improve outcomes for disadvantaged workers.|
|Next Article:||Using behavioral economics to move the needle on maternal and child health.|