Redistributing Power in
Program Assessment
Talk: 4Cs, 2016
With luck, I’ll manage in this talk to link assessment,
writing, and social class, which have been the major areas of research in my
career. I know that when one teases out one element of de-privileging, one risks
ignoring others. For a full treatment, one has to consider all categories of
social discrimination, but in this brief discussion of an alternative model of
writing assessment, I’ll be pointing primarily to social class, which as a
consequence of other structures of inequality unfortunately includes race,
gender, and sexual preference. We know that straight white males, of which I am
one, get the best of everything.
I’m working from a series of assumptions common to stratification
studies, the dominant one of which is that social institutions, like colleges
and universities, function in part (and perhaps primarily) to reproduce the
social structure in which they are embedded. In the United States, that means reproducing
structures of inequality. Bourdieu calls these structuring structures. Althusser
calls them ideological state apparatuses. They work underground, disguising
themselves, allowing the structuring organism to reproduce by putting the blame
elsewhere. Burton Clark’s discussion of the cooling-off effect of two-year
colleges is one notable example. The use of grades is another. And as we are
collectively arguing here, so are assessments, particularly those that pretend,
like grades, to be objective or as rich white males like to think, “fair,”
giving everyone a more or less equal chance of snatching the ring
One of the more interesting books I have lately read is Jeff Schmidt’s Disciplined Minds. Schmidt’s basic
argument is that the more educated you are, the dumber you get. Put more
palatably: the more you are subjected to a major indoctrinating social institution,
i.e., school, the more you concede to the indoctrination. 
Looked at from one lens, a
dominant function of schools is to train citizens to be good little boys and
girls who believe we have the greatest country on earth, that inequality is the
price one pays for freedom, that competition contributes to excellence, that
the Bell Curve is natural. One could go on. There is the chance that by reading
and thinking too much, some people like Schmidt might slip through the fence,
but the chances are slim. It should perhaps go without saying that each of us has been indoctrinated into thinking we are thinking for
ourselves, or to use a current self-damning code phrase—thinking critically.
Let me turn to assessment. First, let me note, as Ed White, Norbert
Elliot, and I have argued in Very Like a
Whale, assessments come in genres: any specific articulation is a response
to what Caroline Miller has called a recurring rhetorical situation. I will be
describing here our alternative program assessment after interrogating the
social reproduction function of placement assessment. My theme is that at
Drexel, we are at least gesturing toward a redistribution of power by directly
assessing the writing program rather than indirectly assessing the program by
assessing student writing performances. By assessing ourselves through
assessing others, we have the shoe on the wrong foot. Rather than helping
students with their writing, we are perpetuating a structure of control and
social discrimination.
Although I have played my
part in developing placement assessments, I want to challenge their
consequential validity. Actually, if you believe that school should perpetuate
inequality (which is probably a closet belief of many educators, particularly
middle- and upper-middle-class white males whose parents are doctors, lawyers,
or professors), then both local and national placement assessments are doing
their jobs. Through these assessments, we are perpetuating social
stratification. Placement assessments for writing classes are based on white,
middle-class language as the norm. If you grow up speaking it, you’re in. If
you don’t, you have to make-over your given language and adopt the discourse of
the enemy culture—the one that has kept speakers of your home culture down.
The underlying logic of placement assessment is a thinly disguised way
of naturalizing inequality: In “What is Placement?”, Roger Gilles and Daniel
Royer have argued that effective teaching is based on a homogeneous class—most
students being at the same writing ability level—whatever that is. The
stratifying effect is obvious—as is the data showing that students placed in the “lower” levels
disproportionately never make it “up.” The imperative of stratified writing
classes is predicated on the logic of grades. If you threw grades out the
window, you would find it easy to teach students of any level in the same
classroom.
That unexamined need to
“grade” students, to rank their writing ability level, lies behind the
homogeneous dream.
The fetichization of argument,
the genre in which we primarily assess writing, also perpetuates inequality. Like
language conventions, argument is taught differently in different social classes.
Bourdieu documents this differentiation
extensively; Annette Lareau in Unequal
Childhoods likewise documents the different approaches to argument in the
different social classes. I also wrote a chapter in Going North, Thinking West on the relationship between argument and
social class. In composition studies, privileging argument is also linked to
social class—an attempt in the late forties to distance ourselves from
“composition” and hook our stars to “rhetoric” to make ourselves equal to our
literary brothers and sisters.
Let me make one more claim about the social reproductive function of
placement assessment—indeed of any kind of assessment based on student writing
as performance, the kind of writing in which someone asks students to pretend
they are in this or that rhetorical situation, writing to these or those readers,
for this or that purpose. Admittedly, in some sense, we are assessing our
students’ abilities to write and by implication our abilities to teach, but
even in our most sophisticated writing assessments, we are assessing an
incredibly narrow strand of what we mean by “writing”; in addition, we are
assessing our students’ abilities to perform, to fake it, as if it were real.
Unless we are intending to
assess our students’ abilities to perform writing rather than really write, we
are including in our assessment a construct irrelevant variable and through
that inclusion promoting an unintended consequence—i.e., reproducing social stratification
because like argument, the ability to perform, to be on stage, is social-class
related. Lareau gives us several vignettes of middle-middle and upper-middle
class children being taught how to perform in contrast to the working-class
ethos of being only yourself; admittedly, I am ignoring the post-modern
interrogation of identity construction, an interrogation that might be
unintentionally reflexive.
How we assess plays back into
what we teach—and what our students learn. If we assess “performance,” then
that’s what we’ll teach and that’s what our students will learn. But many of us
at Drexel believe that “performing” writing is not real writing. It may be
school writing, but that’s not writing as exploring, expressing, or
communicating--as I hope I am doing here. Although school writing may be the kind of writing being graded
in other classes, we have imagined at Drexel our task not as one of teaching
students how to perform in other
classes;
rather, we’re teaching
writing for life.
We want to have our students
take writing with them after they graduate. We want writing to enrich the rest
of their personal, civic, and professional lives. And we believe that kind of
writing is writing to communicate, to enact a real function rather than writing
to perform so that the writer’s writing ability can be graded.
Consequently, we make writing as communication the focus of our program
assessment. I have to wrap this up, so I’ll describe our project briefly.
Rather than have students give a writing performance, we ask them to
communicate with us, making our program the object of inquiry. At the beginning
of their first quarter, we have them describe themselves as writers and their
relationship with writing. At the beginning of each subsequent quarter, we distributive
to all first-year students a survey with scaled and open-ended questions so
they can tell us how well they liked or didn’t like their previous course, what
they did or didn’t like about it, and what we might do to improve it. We send
these requests out through the university and receive about 250 responses out of
approximately 3000 students.
Our director of assessment
and I code these responses and write a report of our analyses, which we distribute
in the second and third quarters to all students and teachers.
We encourage teachers to
discuss these reports with their students. To close the circle, we run
workshops with our teachers at the beginning of the new year, making these
results and representative student comments organized around themes, like
creativity, topic choice, or voice, the subjects of discussion. We talk about
how we can improve our program structure and teaching.
This is a skeletal description of what we do. Because we are teaching
writing for life, we want students to walk away from our classes with a
positive attitude toward writing. That’s why we care about their experiences in
our classes. That’s why we ask them to tell us how we are doing. We are, we
believe, modeling the kind of writing we want to teach: real writing, writing
as communication, not performance. You might say we’re assessing our
performance, not theirs.
