Redistributing Power in Program Assessment

Talk: 4Cs, 2016

With luck, I’ll manage in this talk to link assessment, writing, and social class, which have been the major areas of research in my career. I know that when one teases out one element of de-privileging, one risks ignoring others. For a full treatment, one has to consider all categories of social discrimination, but in this brief discussion of an alternative model of writing assessment, I’ll be pointing primarily to social class, which as a consequence of other structures of inequality unfortunately includes race, gender, and sexual preference. We know that straight white males, of which I am one, get the best of everything.

I’m working from a series of assumptions common to stratification studies, the dominant one of which is that social institutions, like colleges and universities, function in part (and perhaps primarily) to reproduce the social structure in which they are embedded. In the United States, that means reproducing structures of inequality. Bourdieu calls these structuring structures. Althusser calls them ideological state apparatuses. They work underground, disguising themselves, allowing the structuring organism to reproduce by putting the blame elsewhere. Burton Clark’s discussion of the cooling-off effect of two-year colleges is one notable example. The use of grades is another. And as we are collectively arguing here, so are assessments, particularly those that pretend, like grades, to be objective or as rich white males like to think, “fair,” giving everyone a more or less equal chance of snatching the ring

One of the more interesting books I have lately read is Jeff Schmidt’s Disciplined Minds. Schmidt’s basic argument is that the more educated you are, the dumber you get. Put more palatably: the more you are subjected to a major indoctrinating social institution, i.e., school, the more you concede to the indoctrination.

Looked at from one lens, a dominant function of schools is to train citizens to be good little boys and girls who believe we have the greatest country on earth, that inequality is the price one pays for freedom, that competition contributes to excellence, that the Bell Curve is natural. One could go on. There is the chance that by reading and thinking too much, some people like Schmidt might slip through the fence, but the chances are slim. It should perhaps go without saying that each of us has been indoctrinated into thinking we are thinking for ourselves, or to use a current self-damning code phrase—thinking critically.  

Let me turn to assessment. First, let me note, as Ed White, Norbert Elliot, and I have argued in Very Like a Whale, assessments come in genres: any specific articulation is a response to what Caroline Miller has called a recurring rhetorical situation. I will be describing here our alternative program assessment after interrogating the social reproduction function of placement assessment. My theme is that at Drexel, we are at least gesturing toward a redistribution of power by directly assessing the writing program rather than indirectly assessing the program by assessing student writing performances. By assessing ourselves through assessing others, we have the shoe on the wrong foot. Rather than helping students with their writing, we are perpetuating a structure of control and social discrimination.

Although I have played my part in developing placement assessments, I want to challenge their consequential validity. Actually, if you believe that school should perpetuate inequality (which is probably a closet belief of many educators, particularly middle- and upper-middle-class white males whose parents are doctors, lawyers, or professors), then both local and national placement assessments are doing their jobs. Through these assessments, we are perpetuating social stratification. Placement assessments for writing classes are based on white, middle-class language as the norm. If you grow up speaking it, you’re in. If you don’t, you have to make-over your given language and adopt the discourse of the enemy culture—the one that has kept speakers of your home culture down.

The underlying logic of placement assessment is a thinly disguised way of naturalizing inequality: In “What is Placement?”, Roger Gilles and Daniel Royer have argued that effective teaching is based on a homogeneous class—most students being at the same writing ability level—whatever that is. The stratifying effect is obvious—as is the data showing that students placed in the “lower” levels disproportionately never make it “up.” The imperative of stratified writing classes is predicated on the logic of grades. If you threw grades out the window, you would find it easy to teach students of any level in the same classroom.  

That unexamined need to “grade” students, to rank their writing ability level, lies behind the homogeneous dream.

The fetichization of argument, the genre in which we primarily assess writing, also perpetuates inequality. Like language conventions, argument is taught differently in different social classes. 

Bourdieu documents this differentiation extensively; Annette Lareau in Unequal Childhoods likewise documents the different approaches to argument in the different social classes. I also wrote a chapter in Going North, Thinking West on the relationship between argument and social class. In composition studies, privileging argument is also linked to social class—an attempt in the late forties to distance ourselves from “composition” and hook our stars to “rhetoric” to make ourselves equal to our literary brothers and sisters.

Let me make one more claim about the social reproductive function of placement assessment—indeed of any kind of assessment based on student writing as performance, the kind of writing in which someone asks students to pretend they are in this or that rhetorical situation, writing to these or those readers, for this or that purpose. Admittedly, in some sense, we are assessing our students’ abilities to write and by implication our abilities to teach, but even in our most sophisticated writing assessments, we are assessing an incredibly narrow strand of what we mean by “writing”; in addition, we are assessing our students’ abilities to perform, to fake it, as if it were real.

Unless we are intending to assess our students’ abilities to perform writing rather than really write, we are including in our assessment a construct irrelevant variable and through that inclusion promoting an unintended consequence—i.e., reproducing social stratification because like argument, the ability to perform, to be on stage, is social-class related. Lareau gives us several vignettes of middle-middle and upper-middle class children being taught how to perform in contrast to the working-class ethos of being only yourself; admittedly, I am ignoring the post-modern interrogation of identity construction, an interrogation that might be unintentionally reflexive.

How we assess plays back into what we teach—and what our students learn. If we assess “performance,” then that’s what we’ll teach and that’s what our students will learn. But many of us at Drexel believe that “performing” writing is not real writing. It may be school writing, but that’s not writing as exploring, expressing, or communicating--as I hope I am doing here. Although school writing may be the kind of writing being graded in other classes, we have imagined at Drexel our task not as one of teaching students how to perform in other classes;

rather, we’re teaching writing for life.

We want to have our students take writing with them after they graduate. We want writing to enrich the rest of their personal, civic, and professional lives. And we believe that kind of writing is writing to communicate, to enact a real function rather than writing to perform so that the writer’s writing ability can be graded.

Consequently, we make writing as communication the focus of our program assessment. I have to wrap this up, so I’ll describe our project briefly. Rather than have students give a writing performance, we ask them to communicate with us, making our program the object of inquiry. At the beginning of their first quarter, we have them describe themselves as writers and their relationship with writing. At the beginning of each subsequent quarter, we distributive to all first-year students a survey with scaled and open-ended questions so they can tell us how well they liked or didn’t like their previous course, what they did or didn’t like about it, and what we might do to improve it. We send these requests out through the university and receive about 250 responses out of approximately 3000 students.

Our director of assessment and I code these responses and write a report of our analyses, which we distribute in the second and third quarters to all students and teachers.

We encourage teachers to discuss these reports with their students. To close the circle, we run workshops with our teachers at the beginning of the new year, making these results and representative student comments organized around themes, like creativity, topic choice, or voice, the subjects of discussion. We talk about how we can improve our program structure and teaching.

This is a skeletal description of what we do. Because we are teaching writing for life, we want students to walk away from our classes with a positive attitude toward writing. That’s why we care about their experiences in our classes. That’s why we ask them to tell us how we are doing. We are, we believe, modeling the kind of writing we want to teach: real writing, writing as communication, not performance. You might say we’re assessing our performance, not theirs.