How ChatGPT Can Help Teach Students to Write Better Research Papers

A first foray into the AI classroom

A couple of weeks ago, I decided to teach a second-year digital humanities course next fall exploring how to use generative AI in research and writing. So last week I gave my current third year history students the option to write their major paper using ChatGPT. In a class of twenty-six, thirteen opted to take-up the challenge. It’s exciting and it forces me to articulate answers to two important questions: what does a history paper written with ChatGPT look like? What are we trying to teach and learn in the process?

I know my answers will change by next fall—things are evolving at breakneck speed and who knows what capabilities GPT-4 may bring—but right now I think AI is most effectively deployed as a co-pilot. Papers do not (yet) write themselves but AI can speed up the research process and improve writing. For students, I think its most important role is as a writing coach and tutor—one that is available 24/7 and does not get tired or annoyed.

Here is my hypothesis: the quality of average papers written with ChatGPT will improve significantly, but the effects will be less pronounced at both the bottom and top ends of the scale. I’ll let you know in a few weeks how it turns out, but for now here is how I am deploying AI in the classroom.

The Same Old Generic Research Paper

First some context. The assignment is a traditional 4,000 word research essay based on eight secondary sources and two to three primary sources, in this case on some aspect of the North American fur trade. Students submitted a research proposal and preliminary annotated bibliography weeks ago and received feedback on a research question and a list of sources. It’s a bread and butter history paper.

Spoiler: I suspect that AI won’t change the basic mechanics of assignments like this as much as some people fear. It is simply inefficient (at present) to use AI to generate long papers from scratch. To get good results, you need to understand how to use AI tools effectively and this itself is a learned skill. You also need a good general knowledge of the subject matter to know what to ask it to do and to spot hallucinations. This is especially important with sources: ChatGPT cannot be trusted on citations. This is why book reviews, short answer questions, and discussion questions are most vulnerable to disruption by plagiarism. Research papers, less so.

In thinking about how I wanted students to use ChatGPT, I drew on my own recent experience using it as I return to coding. In my younger days, I built websites and was fairly comfortable with Visual Basic so I have some grounding in the area. I have been learning Python from ChatGPT and it’s amazing. You can ask it to give you a series of lessons and then when you start writing code it will debug it, provide suggestions, and allow me to ask endless questions. I am shocked at how quickly I’ve progressed and I think it’s because the learning process is so streamlined. No more flipping through books or websites for 30 minutes looking for an example that fits your specific use-case. Now I just type: “I want to do x, show me how” and it does.

The Importance of Prompting

I think this is how chatbots will be most relevant to my history students too: they can use them to have an efficient conversation with a skilled guide as they navigate the process of researching and writing a paper. On their assignment sheet, my students’ first task is to talk to ChatGPT about their project, specifically their research question. The goal here is to get them familiar with using the tool, including how prompting can change its behaviour.

Prompting (especially the initial one) is essential. Although ChatGPT engages in conversations, it is still a computer program and prompting is the user’s way of programming the chatbot to get the desired type of response. An effective prompt should provide context for an AI to understand the problem, clear directions about what you want it to do and why, and information about how it should behave.

With ChatGPT, more context is almost always better and can include citations, parts of documents, and any other material you think it might need, up to around 3,000 words. Telling ChatGPT to adopt a certain persona (like that of a history professor) can also be helpful as it tells the AI what the conversation should “look like”. Think of this like a drama improve exercise: more information makes for a better scene.

AI Tutors and Co-Pilots

At the end of this initial conversation, students will hopefully come away with a more refined research question, but also more familiarity with how ChatGPT actually works. The next task is for students to actually conduct their own research, much in the same way they normally would. Again, ChatGPT will be their co-pilot.

One of the biggest problems students always seem to face is that they have difficulty effectively navigating the keyword search functions in databases like America: History and Life and JSTOR. Sure, for some people this might be laziness: it is always tempting to choose the first sources, rather than spending time finding the most relevant ones. But for most, the biggest thing is a lack of experience. It takes time to figure out how to sift through dozens of results efficiently.

For this assignment, I told my students to explain their research question to ChatGPT and the to use cut and paste to “show” it their keyword search terms as well as the results they get from the databases. They can also use the abstracts provided to ask it whether a specific source is likely to be relevant or not. In my experiments, I found that ChatGPT does a good job of suggesting better search phrases and identifying relevant results, even from a basic list of titles.

The goal is to get the students using the best sources quickly. Then they still need to read them and make notes—I know: so old fashioned. Yes, I also know that some students will use AI to speed up the process, asking it to summarize the papers etc for them. Nevertheless, I am confident that those who put in the work of actually reading will get the best results.

Making an Outline and Writing the Paper

The next stage is for students to use ChatGPT to develop and refine their thesis and a skeleton essay outline, with the AI acting as a personal tutor. In the assignment guide, I asked students to “show” ChatGPT their thesis and point form supporting arguments in order to get it to generate a fuller outline. From this, they will start to work-up their paper. I told them that they can do this in “chunks” with ChatGPT, using the sentences and even the paragraphs it generates in their paper. But I also explained that, although it might sound counterintuitive to those looking for a major shortcut, the reality is that ChatGPT is a better editor than it is a writer.

This is also why in finalizing the paper, students must pay close attention to sourcing and factuality. While it’s always important, with Generative AI it is an absolute necessity. Papers with unsourced information or factual errors cannot pass. They never should, but now they can’t. This is also why they must cite their use of ChatGPT in the paper’s first footnote, explaining how they used it in the research and writing process.

To my mind, this reflects how AI writing is and will be deployed in the “real world”: it’s main virtue is that it is a major timesaver, which only holds true if it is also accurate. In the very near future, effective writers will be adept at using AI to speed up the research and composition process, but this will also increase our expectations about the results they achieve. And that is why I am optimistic about how generative AI will impact the humanities: it simply puts a new spin on the skills we already claim to teach.

The Final Product

To return to my initial hypothesis, I suspect that “C” or “B” students will benefit the most from this process. “A” students too will improve their writing, to be sure, but the gains will be comparatively limited. At the other end of the spectrum, students that do not put in the effort—assuming ChatGPT will do all the work for them—stand to lose the most with unsourced and potentially error-ridden papers. I am anxious to see what the students think, because I have asked them to write up a brief reflection on the process.

In terms of the skills I hope students learn, my working assumption is that AI writing needs to be managed and that we need to teach our students to be its minders. This assignment is designed to teach students to maximize its (current) benefits while teaching them its limitations. I’ll keep you posted on how it goes!