What Do College Accountability Metrics Really Tell Us Anyway?

We’re not being very thoughtful about how we judge how much good a college is doing — for students or for society. Then, two community-college leaders join our advisory board. And we tell you why we believe in this.


Graduation Rates Tell Us What Exactly?

Imagine your classic vision of an automobile factory. I’m picturing Henry Ford’s assembly line in the first decades of the 20th century. The managers have a goal for the number of cars coming out the end of the line. They can work backwards to figure out all the parts they need, the workers required, the speed of the machinery. In the view of organizational theorists, this a “tightly coupled system.” The people running the factory have a clear picture of the outputs and what it takes to get there.

Now imagine a big public bureaucracy like, say, a university. What exactly are the technical outputs? How should they be measured? And what do the managers even know?

Christina Ciocca Eller, a recent Columbia University graduate and now a new assistant professor at Harvard University, published a working paper earlier this year that wrestles with those questions and her answer is a little depressing: “We haven’t been terribly thoughtful about what it means to have a ‘good’ college.”

The measures we tend to use to judge a college’s outputs don’t always do a very good job, she says, of telling us much about the direct role colleges play in producing them.Unlike that tightly coupled assembly line, Eller argues, the university is a “superficially coupled system.” In this understanding, the college’s “technical output” is not a new automobile but its graduation rate. And our focus on accountability means college leaders pay a lot of attention to that rate. But that metric alone, she argues, may often reflect more the student’s attributes and experiences before college than anything the specific college did.

Eller’s research digs into a treasure trove of student data from a large public higher-ed system. She uses several statistical strategies to account for student differences and comes to the conclusion that colleges that appear to be doing a better job because they have a better graduation rate may actually be having little or no positive effect.

And that’s why she maintains that the accountability metrics so far have “backfired.” Colleges end up spending time and attention on broad, general measures, Eller says, rather than on those capable of conveying college effectiveness.But she also sounds bullish on the broader role of college in society. She just wants to make sure we’re capturing that:

“What else should we think about measuring for the sake of actually improving what ends up happening to students when they walk onto the campus for the first time? That’s the bigger issue,” she told me. “This is not just an issue about how we want to improve BA completion rates. This is an issue about thinking more broadly about what college means and what it does for people in this country. What kind of health impacts does it have? What kind of impact does it have on marital rates and marital stability? On community participation? On voting?”

There’s a ton more to unpack in Eller’s paper (it just won an award at this month’s American Sociological Association meeting). Here’s the link again: Superficially Coupled Systems: The Organizational Production of Inequality in Higher EducationSome additional Reading:

Scott Smallwood

Community Colleges and New Advisors

We’re excited to have the leaders of two prominent community colleges join us as Open Campus advisors. Eduardo Padrón, the retiring president of Miami Dade College, and Maria Harper-Marinick, the chancellor of Maricopa Community Colleges, are new additions to our advisory board.

Community colleges, of course, are central to conversations about how higher education can shape neighborhoods, towns, and cities. And how well those institutions perform can have big effects on social mobility and economic opportunity in an entire region. More than two in five of the nation’s undergraduates are community-college students, and most of them stay very close to home. The median community-college student travels just eight miles to get to campus.

Spotlight

Farewell, Madam President: Diana Natalicio retires after 31 years as UT-El Paso’s leaderNatalicio, 79, steps down this week as UTEP’s president after becoming the first woman to hold the position in 1988. (The Texas Tribune)

Articles overstate millennials’ loss of interest in going to college Doug Lederman at Inside Higher Ed digs into the survey behind the alarming New York Post headline: “Half of Young Americans Say College Is No Longer Necessary.” As Doug says, “Trouble is, that’s not at all what the survey found.”(Inside Higher Ed)

Op-Ed: Why California needs to institute a ‘student borrower bill of rights’Setting clear rules of the road for student loan servicers will help ensure that borrowers are treated fairly and can access the rights they have been promised. (Los Angeles Times)

Small colleges hope university status will boost their fortunesCan a one-word tweak spell the difference between a bustling campus and shuttered classrooms? An increasing number of higher education institutions in Massachusetts are betting that dropping “college” from their name and transforming themselves into universities will be the key to their long-term survival at a volatile time for small, private schools. (Boston Globe)

Why We’re Doing This

When we first got going on this project, we wrote about why we believe more local reporting on higher ed matters. If you missed our post, you can check it out here.

We see public confidence in colleges falling and local journalism crumbling. We want to help strengthen both.

The bottom line is, we believe more thoughtful, aggressive coverage of all parts of the American higher-education system — on the ground, in cities and states — can help decrease public cynicism. College, like journalism, is a public good that needs to be fostered and held up to high standards. Its best chance to reclaim the confidence of more Americans is through greater transparency and accountability.

Follow Along and Stay in Touch

Our Open Campus Back-to-School Road Trip starts next Wednesday. (We’d be lying if we said we’ve already packed our bags; neither of us tends to plan that far ahead.) We’ll be on the road from August 21 to August 28. You can follow our journey here.

Keep the feedback coming! We love hearing your stories, tips, and ideas. You can reach us both at info@opencampusmedia.org.

One more request: please share our weekly newsletter with other people you think would be interested. They can sign up here.

You May Also Like