Skip to content

Research Methods

The primary task of the Citation Project is to compile an aggregate portrait of the way college writers use sources in their source-based papers. We are studying these papers using what is variously known as “citation content analysis” or simply “citation analysis.” Harold White provides a useful history and overview of citation analysis in his 1994 “Citation Analysis and Discourse Analysis Revisited.” Of the various uses of citation analysis mentioned by White, the Citation Project focuses on one: classifying the “abstract features of the relationship between citing and cited work” (99).

Citation Project researchers began with these methods developed in information studies and applied linguistics, and have adapted them for inquiry in the field of composition and rhetoric. Useful in our adaptations is the work of Diane Pecorari. We first encountered Pecorari’s 2003 “Good and Original: Plagiarism and Patchwriting in Academic Second-Language Writing,” but she has since published several other insightful works that use citation content analysis to understand how student writers use their sources.

Working from Pecorari’s model, the pilot stage of our research examined 18 student papers, asking whether each citation employed quotation, summary, paraphrase, or patchwriting in its use of the cited source. Like Pecorari, we work from Howard’s definition of patchwriting (233), which we have revised: Patchwriting invloves restating a phrase, clause, or one or more sentences while staying close to the language or syntax of the source.

As we add student work from more institutions to our data pool, we are adding additional questions to our analysis. We are drawing on the work of information studies as we explore what kinds of sources are cited in addition to tracking how they are used and incorporated. As we understand the nature of the sources students are working with, we can better understand what causes them to choose quotation, summary, paraphrase, or patchwriting as their way of representing what is in the source. And then we will be in a position to make useful recommendations about the instruction in source-based writing that students need if they are to be able to handle sources in rhetorically effective and ethically responsible ways.

Methods and Documents

The specific methods we use to code papers and sources and train coders are described in the documents below. We have made these available to help people understand our data, not as an invitation to people to use our methods without communicating with us. The reliability and validity of Citation Project data comes from a methodology developed over half a decade and from careful training and calibration of coders. We believe that citation analysis can be a valuable pedagogical tool, a very effective part of faculty development, and a useful component in course and program assessment. We do not, though, invite people to use our methods and identify them as part of the Citation Project without our knowledge.

We have created a series of PowerPoints we use to train coders at individual schools and in workshops, and we offer training sessions for people who want to collect and code papers from their institution and include their data (or a portion of it) in our database. We also provide each participating school with data analysis, placed in the context of the overall data, and we help participating schools interpret their findings. For this reason we have developed some careful controls: We do not include data from schools that have coded their own papers; instead, we assign at least one coder from a different institution and only include double-coded data. This ensures that exactly the same methods are used across institutions, which increases the reliability of the results. More important, it increases the transparency of the process and the veracity of the individual campus reports.  As more institutions request to join the Citation Project, we pair them with three or more other schools and train participants to code each other’s papers. The same method is used for follow-up research.

To learn more about our methods or to request to be a participating institution, please contact us.

Links to PDFs

These documents are provided for information only, if you would like to code papers using our methods or adapt them to different uses please contact us for permission and additional documents and training materials.

Paper Coding FAQ’s (PC/Mac) – (Tablet)

Paper Coding Glossary (PC/Mac) – (Tablet)

Paper Coding Procedures (PC/Mac) – (Tablet)

Paper Coding Sheet (PC/Mac) – (Tablet)

Source Coding Procedures (PC/Mac) – (Tablet)

References

Howard, Rebecca Moore. “A Plagiarism Pentimento.” Journal of Teaching Writing 11.3 (Summer 1993): 233-46.
Pecorari, Diane. Academic Writing and Plagiarism: A Linguistic Analysis. New York: Continuum, 2008.
Pecorari, Diane. “Good and Original: Plagiarism and Patchwriting in Academic Second-Language Writing.” Journal of Second Language Writing 12 (2003): 317-345.
Pecorari, Diane. “Visible and Occluded Citation Features in Postgraduate Second-Language Writing.” English for Specific Purposes 25 (2006): 4-29.
White, Howard D. “Citation Analysis and Discourse Analysis Revisited.” Applied Linguistics 25.1 (2004): 89-116.