By Paul A. Djupe
In a recent post, I reviewed results from a survey of liberal arts college (LAC) faculty, including Denison, about the amount of public engagement (PE) activities they do, the value they see in PE, and the potential limits they would impose on the form of PE — whether it should involve political advocacy and its relationship to research. The upshot is that we already do a fair amount of this sort of work, we think it has promise, we don’t want limits, we don’t want mandates, and there may be some disagreement on its tether to a specific research project. Anyone familiar with the faculty species would find none of this surprising. So, now the 1-2% (that’s a reference to raises) question: If we’re doing this sort of activity, how should it count toward tenure, merit raises, and promotion (PMT)?
Is it Earth, Wind, or Fire?
Not just because we’re academics, we need to figure out just what PE is. This is a pretty loaded concern as you’ll soon see. Normally, what we do falls into one of three buckets — it’s either teaching, research, or service. PE, however, doesn’t fit cleanly into any of these. Take this blog, for instance. Its very existence probably best qualifies as service that I give to the campus community. But it features original research that I (and mostly students) have done both inside and outside of the classroom. Of course none of it is peer reviewed in the classic sense, so it does not meet the traditional bar, though it is valid (imo) research activity. However, I also engage in a good bit of teaching linked to 127; I typically supervise a student doing an independent study writing for 127, I work closely with other students writing for 127 voluntarily, and the content of the blog is a teaching exercise for the Denison public about the value of social science research methods and data analysis.
This is probably why the account offered by Ernest Boyer (1997), a strong advocate for counting PE, could be considered “hyphenated.” In his view, “scholarly service activities” are those in which scholars engage their specialized knowledge and skills in addressing relevant social needs. Service-research, in some applications, perhaps service-teaching in others, may better describe the quite varied forms of PE out in nature, but it does not make the bean counters happy.
Most of the controversy over PE involves what counts as research, since one of the more difficult buckets for faculty to fill is the research expectation. Under traditional norms for how research is evaluated, PE is a tough sell. If it is tied to a research project (e.g., it’s becoming more common for journals in political science to require a short blog post when articles are accepted), then it’s not new knowledge but may be synergistic. If it’s not linked to peer reviewed work, then it’s unclear if it is of high quality or, put differently, there is no agreed-upon standard by which it should be judged. Boyer argues that ipso facto PE “is serious, demanding work, requiring the rigor—and the accountability—traditionally associated with research activities” (1997: 22). Absent peer review, though, it is not quite clear to whom faculty are accountable or who would vouch for the rigor involved. Activist researchers, a slice of the PE community, therefore often push one step farther to suggest that the only valid “peers” to evaluate their work (housing advocacy, for instance) are those who are being served — they will know if the work was valuable and rigorous or not. Clearly, this is a profound challenge to traditional academic norms.
OK, let’s see what the data say. We took a few tacks at this question and I’ll follow them here.  Some of these analyses involved the entire dataset of 7 LACs to have enough statistical power to slice and dice the data by division of the college, for instance. Some are just Denison faculty data. I noted which I used below.
Tack 1 – Personal and Perceived Institutional Views
First, we asked faculty if they would personally count each activity in PMT decisions. The results, sorted by division of the college, are below in Figure 1. The activities closest to traditional teaching, research and service are clear yes’s (e.g., art installations, public lectures, and publishing in print magazines). There are a lot of activities that average “not sure” – a finding which will pop up again shortly. And there really is just one clear no — protesting shouldn’t count. Note, as well, the persistent divide between the college divisions. Art and Humanities faculty are almost always more in favor of these activities than those in the Sciences. An interesting reversal can be found on government advising, where the humanists are the most opposed and social scientists most in favor (though, admittedly, none of these differences are large).
Figure 1 – Support for Counting Each Public Engagement Activity by College Division (total sample, not just Denison)
One reason why so many were unsure is because they have very few institutional cues to base their opinions on, as Figure 2 shows. For all but the slam dunks on either end (yes or no), 20-30 percent of faculty just “don’t know” what their institutions would do. That the rest disagree in about equal proportions suggests they are just not getting clear cues from institutional sources (and/or faculty aren’t talking to each other yet). To validate this view, there’s a very strong correlation between personal views and perceptions of the stance of the institution — we’re projecting our wishes onto the college.
Figure 2 – Perceived Institutional Support for Counting Public Engagement Activities (just Denison)
Tack 2 – Experimental Evidence
We also aimed to gain a behavioral measure (bestowing reward) of public engagement through the use of a survey experiment regarding a fictional candidate up for tenure. Our goal in composing this candidate’s dossier was to pitch a marginal candidate so that any boost from public engagement would not be limited by ceiling or floor effects. The candidate in the control condition, either Mary or Martin Jones (we varied the candidate’s gender),
is an assistant professor of Sociology at your college who is up for tenure. Jones has been rated a competent teacher by students and peers, which is an improvement from when she was first hired. Since her appointment, she has presented several papers at regional sociology conferences, amounting to 1 every year and a half. She has published one of those papers—a solo authored article in a mid-tier, peer reviewed journal (impact factor = .73). Jones has served as advisor to several student groups and has served on one university-wide committee.
We then varied the type of public engagement activity that Professor Jones had performed – one was writing for Huffington Post, one was delivering a high profile public lecture, and one was service learning — and asked respondents to evaluate whether their particular institution’s faculty status committee (or its equivalent) would view such activities favorably for tenure and in what category (scholarship, teaching, or service) they would count. [For the wording of the PE activity descriptions see note 2]. While only 20 percent would award Jones tenure in the control condition (without PE), that proportion just about doubles when Jones’ public engagements are included (all have significant and positive effects). This shows good evidence that public engagement is considered positively and would be counted by faculty. Importantly, faculty did not shift their evaluation of the candidate based on their gender — Mary and Martin got the same scores.
Figure 3 – Effects of Public Engagement Activities on the Respondent’s Tenure Vote (all schools)
Note: Comparing any two confidence intervals (the capped lines) is the equivalent of a 95% difference of means test. The dots are scores of the various schools in the sample. The bars are sample average scores in those conditions.
Notably, faculty respondents were not as sanguine about how Jones would be treated by the institution. The likelihood that tenure would be awarded rose significantly for the op-ed writing Jones, marginally increased for the public lecturer Jones, and did not increase from the control condition for the service-learning candidate. Clearly, there is some disjuncture here about perceptions about their colleages and what they would do and what they value themselves that could probably be resolved with open dialogue.
Since we had a “real example” of public engagement as represented by Jones’s candidacy, we asked faculty who said they would count it how they would count that activity (they could check multiple categories). Moreover, we can also see if those judgments vary by Jones’ gender. Figure 4 shows this evidence. In all but one case (counting service learning as service), the male candidate gets indistinguishable scores from the female candidate. Generally, few see these activities as research or teaching and most (60-70+%) see them as service.
Figure 4 – How Are the Public Engagement Treatments Counted? (By Candidate Gender)
Note: The figure shows the proportions counting the news, lecture, and service treatments as research, teaching, or service, by gender of the candidate. For example, the first couplet of bars shows that when given the “news” treatment, nearly 40% of respondents counted it as “research” when conducted by the female candidate for tenure (black bar) compared to the 30% who rated it as research for the male candidate (gray bar). This difference was not statistically significant (p = .30).
Just how much “hyphenation” there was is useful to see in Figure 5. Among those who counted PE in their tenure decision about Jones, most considered PE as service. But substantial numbers (32%) checked multiple categories, with the most common as research-service (14%), followed by the triple research-teaching-service (9%). Thus, we are left with about twenty percent of faculty who said they did not count Jones’ PE activity, about a quarter of faculty who would count Jones’ PE in multiple categories, and the remaining slim majority that would count it in a single category (mostly in service).
Figure 5 – The Degree of Double or Triple Counting PE (aka “Hyphenation”)
I started doing PE work a few years ago across a number of fronts that has engaged my teaching, research, and service. I’ll put a list in Note 3 in case you’re curious. Clearly, the reason why we are in academia is that we believe it has substantial public value. While that does not necessarily mandate that we engage the public directly, the kind of work I do easily lends itself to public engagement, I saw very positive benefits to my research agenda and visibility, I saw that it offered students yet another outlet for their growing talents, and I found it really enjoyable to do. That’s a lot of win.
From my experience, then, you might think that I would want to “count” it all. While you would not be wrong, I would advocate for a pretty significant discount on certain kinds of these activities. I really enjoy writing posts about the research I and others have done. These “synergistic” activities pay real dividends for citation counts, general awareness of the literature, and my ability to think more broadly about how what I study might connect with the public. But they don’t necessarily produce new knowledge, though some of them do. The fact that I started two blogs was an important service that enables more widely distributed teaching and research activities. The research I do there is mostly new, but it’s not peer reviewed. In fact, I go to those outlets in order to avoid having to jump through all of the documentation hoops that would convince my peers (that’s one reason why writing those pieces is so enjoyable!).
I am a big fan of an institution encouraging its citizenry to engage in behaviors that are beneficial to them. PE is a great example of a set of activities that *can* be beneficial to faculty as well as to the college (under certain perhaps controversial conditions). And I see Denison offering forms of support for faculty to try new things and get advice about how to start. What I don’t see Denison doing is offering any sense of counting.
If PE is going to count and most faculty value it as service, then PE should be able to replace some of the on-campus organizational involvement we have now. Otherwise, the only way for PE to count is in the accretion model of how we treat research, which is an activity we pursue in all of our free time. There’s a neat irony, then, that we would be engaging in PE to “save the world on our own time” (with no apologies to Stanley Fish), but it’s clearly not an ideal solution in my view. If faculty lives are zero sum, which mine is at this point, I’d much rather have some time back. Therefore, my preference would be a mix of these two approaches to valuing PE, reflecting its truly hybrid nature: some credit as service, some as research, though not as a replacement for peer-reviewed work.
Regardless of my opinions, it is clear that we could use more open conversations about how Denison is going to treat PE. We engage in a fair amount of PE already and most view it as beneficial to themselves, to the college, and to society, but there is a great amount of uncertainty about whether it is valued by the college. If we want faculty to be doing this, we should find a way to assign it some value.
1. I conducted this study with my friends and colleagues Liz Smith (Furman, Political Science) and Sean Patrick O’Rourke (formerly Furman Comm. Studies, now Sewanee) under a grant from the AALAC. Together with Luke Christie, we have an article about PE questions that is forthcoming at the Journal for Higher Education Outreach and Engagement.
The “News” Treatment added:
Jones maintains a blog with regular posts about inequality that is well subscribed; a few of her posts have been republished at Huffington Post (a well known news and commentary website) and two op-eds have appeared in major metropolitan newspapers, including one in the New York Times.
The “Lecture” Treatment added:
Jones is frequently invited to give public lectures on the topic of inequality at the local Rotary club meetings and has been invited twice to speak at a Smithsonian speaker series on culture and community which aired on C-Span. Jones has served as advisor to several student groups and has served on one university-wide committee.
The “Service” Treatment added
Jones is very involved in working closely with the community on the issue of economic inequality and her students’ work with impoverished communities as part of her Introduction to Sociology course service learning requirement have resulted in both local and national awards and accolades.
3. Most of my PE has been through blogging as of late through 127 and religioninpublic.blog, both of which I started. I have also written for 538, the Washington Post’s Monkey Cage, and Vox. Some of that writing is new research, some of it describes published work of mine. I have consulted for lawyers and other organizations doing survey work. I have given several public talks over the last few years. Lastly, I regularly teach a course that hosts a deliberative forum at Denison with 100+ participants. I’ve done a radio show through WOSU and have been regularly interviewed by print journalists (most recently Newsweek). I’m also an affiliated scholar with Public Religion Research Institute (PRRI), for whom I also sit on the Board of Directors.
Boyer, Ernest L. 1997. Scholarship reconsidered: Priorities of the professoriate. San Francisco, CA: Jossey-Bass.
Christie, Luke, Paul A. Djupe, Sean P. O’Rourke, and Elizabeth Smith. Forthcoming. “Whose Job Is It Anyway? The Place of Public Engagement in the Liberal Arts.” Journal of Higher Education Outreach and Engagement