Earlier this year, the Carnegie Classification of Institutions of Higher Education came out with a new classification, focused on colleges’ low-income student enrollments and whether their students wind up in well-paying jobs. Released in April, the Student Access and Earnings Classification marks a shift in Carnegie’s approach, years in the making, to assess institutions based on student success. The move was designed in part to acknowledge institutions, such as community colleges, that don’t spend heavily on research or confer doctorates but drive economic opportunity for students.

But not everyone is pleased with the new classification and the way the results shook out. Community college leaders are raising concerns that the methodology isn’t well-suited for their institutions and paints community colleges, particularly in high-cost-of-living areas, in a negative light by labeling some “lower access” or “lower earnings.” In a letter obtained by Inside Higher Ed, the National Council of State Directors of Community Colleges demanded Carnegie remove the classification from its website and rework the methodology or take community colleges out of it. A second letter from Community College Association Executives called for the same.

“We appreciate the ambition of the new classification and share the aspiration that higher education should be assessed more by outcomes than by reputation alone,” wrote Brian Durham, chair of the NCSDCC, in his letter. “But we also fear the potential unintended consequences of reinforcing inequality in institutional prestige, rather than shining a light on the mobility that is central to all our institutions’ missions … To leave this information in the public domain in its current form is potentially harmful to students, families, and our institutions.”

Mushtaq Gunja, executive director of the Carnegie Classification Systems and senior vice president of the American Council on Education, said he’s open to feedback and debate, and future refinements, but at the end of the day, “we stand by the data.”

“We have confidence in this methodology, we have confidence in this data and we won’t be pulling it down,” he said.

Measuring Competitive Wages

The new classification assesses colleges based on two measures: access and earnings. It defines access as whether institutions are enrolling student bodies representative of the geographic areas they serve, in terms of low-income and underrepresented students. Earnings refers to whether students who’ve left the college earn competitive wages. To measure that, Carnegie compares students’ median earnings eight years after attending a college to the earnings of working-age people in the same geographic area with a high school diploma or higher.

Community college leaders, first and foremost, take issue with how the classification categorizes colleges as “high earnings” or “low earnings.” Community colleges serve local students, so the classification compares their students’ earnings with median earnings in their metropolitan areas. But four-year institutions draw students from multiple regions, so the classification compares their students’ earnings to a composite of the median earnings from multiple states. Community college advocates say that’s bad news for community colleges in high-cost-of-living areas like San Francisco or Boston. The wages in these areas skew higher, so institutions have a higher threshold to hit if they want to be classified as “high earnings” compared to their neighboring four-year institutions, which pull students from states with lower average wages.

For example, the letter from NCSDCC pointed out that students at Middlesex Community College in Massachusetts would have to achieve median earnings of $51,301 to avoid a “lower earnings” label. But Harvard University, 12 miles away, needs median earnings of just $39,534 to steer clear of that category because its student population comes from all over the world, most of which has lower median earnings than the Boston area.

That Boston community college graduates would need to outearn Harvard graduates to avoid a negative rating is “just absurd, and it doesn’t pass the smell test,” said Nate Mackinnon, executive director of the Massachusetts Association of Community Colleges. “That’s a penalty to any community college that happens to be in a high-cost-of-living area. This is not the right yardstick to be measuring public community colleges by.”

Out of 910 community colleges, upward of 200 were designated as “lower earnings” in this year’s Student Access and Earnings Classification.

Community colleges are being “slapped with a negative label that really isn’t fair and reflective of … the uplift that they are providing to their graduates,” Mackinnon said. “How is this a fair measurement system, and is it telling institutions and the public what they really need to know in making decisions about where they seek higher education?”

Gunja pushed back, noting that Carnegie’s site deliberately discourages the comparison of two-year and four-year institutions in its visualization tools by not showing results for the two sectors together. He also argued that community college students are competing for jobs locally, so it makes sense to look at their earnings in relation to median earnings in the local labor market.

“The question that we were asking is, how well do students who attend these schools—community colleges, public regionals, private institutions— [do] in the job market, in the places where those students are going to be working?” The classification compares community college students to “the set of people that they’re competing against,” he said, so “if your institution’s earnings are below the 22- to 40-year-old cohort, high school plus, that means that the earnings are less than the job market.”

As a result, in highly educated areas like Boston, the job market may be “tougher” for associate degree earners, Gunja acknowledged. But that doesn’t mean “we are saying that those institutions are not worthwhile,” he said. He believes the data does raise questions about how community colleges in such markets can make their students more competitive, whether by upping transfer rates or offering new kinds of associate degrees.

Gunja also noted that Carnegie didn’t measure institutions’ return on investment, but if it had, “the community college sector as a whole would undoubtedly look better” because of its affordability.

The classification “doesn’t measure everything, and we’re fully transparent about what is measured and what is not,” he said.

Defining Access

The other point of contention for community college leaders is the classification’s access metric. Carnegie defines access based on the number of Pell Grant recipients and underrepresented students colleges enroll relative to the demographics of their surrounding communities. Community college leaders argue their students notoriously fill out the Free Application for Federal Student Aid at lower rates than their university counterparts.

As a result, some low-income students aren’t represented in those Pell Grant numbers, including undocumented students, students confused or intimidated by the FAFSA, and students who attend colleges with free tuition programs who don’t bother to fill out the form, said Larry Galizio, president and CEO of the Community College League of California. He worries Carnegie’s definition of access makes it look like community colleges serve fewer low-income students than they do. Over all, he’s concerned that putting community colleges and four-year institutions in the same classification, and breaking them up into the same categories, is like “comparing apples and oranges.”

The classification deems 79 community colleges “lower access,” which Mackinnon described as “mind-boggling” and “honestly just preposterous.”

“Community colleges are open access—they accept anyone who’s graduated high school into their institutions—by their very nature,” he said. “That is part of the mission and the cloth that community colleges are cut from.”

He and other community college leaders worry that the classification’s “lower access” and “lower earnings” labels make it sound as if these institutions are low value and could deter students from attending their local community colleges or sway business leaders and state lawmakers not to partner with them.

“In its current form, [the classification] simply needs to come down and be reworked,” Mackinnon said.

Gunja emphasized that 92 percent of all community colleges fall into the higher-access category, and 95 community colleges boasted both higher access and higher earnings. He added that a lower-access label isn’t “necessarily an indictment of the institution”; it could indicate contextual factors, like low FAFSA completion rates or high school graduation rates in their surrounding communities—or, for whatever reason, that the institutions aren’t reaching everyone.

In either case, “being able to note that, and being able to study it, is probably good for the higher ed ecosystem as a whole,” Gunja said.

He acknowledged the word “access” can be used in different ways.

But the classification is asking “specific questions”—and “they’re not the only questions that one could ask,” he said.

He added that community colleges are “woefully understudied” and left out of other classifications and rankings like U.S. News & World Report’s. Carnegie didn’t want to make the mistake of ignoring an “incredibly vital, thriving, important part of the sector.”

But classifying them is “a little bit tricky,” because few have done it at all, let alone well, he said. He suspects community colleges, like other institutions, are experiencing some of the discomfort that comes with being analyzed and categorized in such a public way for the first time. He said he knows colleges and universities use the Carnegie classifications as a marker of prestige, whether or not that’s how the classifications’ stewards view their own work.

Over all, he believes debate about the classification is “healthy.”

“This is part of an ongoing dialogue with the field,” he said.