1. Overview

Relevant OLI Principles & Tenets

Principle 3: Instructors and tutors should commit to regular, iterative processes of course and instructional material design, development, assessment, and revision to ensure that online literacy instruction and student support reflect current effective practices.

Tenet 5: Instructors and tutors should research, develop, theorize, and apply appropriate reading, alphabetic writing, and multimodal composition theories to their OLI environment(s).

Principle 4: Educators and researchers should initiate, support, and sustain online literacy instruction-related conversations and research efforts within and across institutions and disciplinary boundaries.

Tenet 3: Administrators, instructors, and tutors should be committed to ongoing research about, and exploration into, OLI. 

2. Introduction & Supporting Literature

[1] AI language tools became a more controversial issue with the release of OpenAI’s ChatGPT, but AI language tools, including Grammarly and paraphrasing tools, have been used by students before ChatGPT became widely available. Since then, the ecosystem of generative-AI (GenAI) writing tools has grown quickly. As Alharbi (2023) noted, since these tools are widely available and accessible, students will use them for communication and learning. If we are practicing OLI Principle 3 and committing to “regular, iterative process of course and instructional material design, development, assessment, and revision,” we cannot ignore the emergence and proliferation of GenAI writing tools. Ou, Stöhr, & Malmström (2024) explain, “As technology continues to advance, it is incumbent upon educators and researchers to discover effective and appropriate methods to enable students to make optimal use of these tools.”

[2] In order to prepare students for the future of writing tools and GenAI, we have to use OLI Principle 4 and “initiate, support, and sustain online literacy instruction-related conversations and research efforts.” And I add to this that because GenAI tools are rapidly developing and advancing, we must bring students into these conversations to equip them with the critical skills and to evaluate these emerging technologies and think critically about how and when to use them. As Ranade & Eyman (2024) explain in their introduction to a special issue of Computers and Composition dedicated to AI in Composition, instructors need to teach students literacy for AI tools and should be discussing these tools with each other. We may feel like we don’t know enough to teach students about these tools because we are not in computer science and because they are changing so rapidly and there are so many tools; but by engaging and experimenting with AI writing tools, we include students in our discussions and prepare them for the rapid advance of technology.

[3] Many researchers have been surveying students about AI tools to determine how they are using and thinking about these tools in local contexts (Abbas, et. al., 2024; Duong & Chen, 2025; Ma, et. al., 2024; Wang & Li, 2024). Zhao, H., & Min, Q. (2024), but the rapidly evolving tools require that we engage students in literacy practices for writing with AI tools and for reading AI outputs. Teaching students ways to engage and think critically about various tools is an important aspect for Online Literacy Instruction (OLI).

[4] In Fall 2023, when Generative-AI was still pretty new, I began addressing it in my composition classes by openly experimenting with ChatGPT with my students. I developed the following exercise for a personal essay assignment, which was our first full essay in this entry-level freshman writing class. I could see using a similar structure in advanced writing courses or even in courses on writing pedagogy. Having students evaluate AI-generated writing could be used in a variety of courses to check for writing skills, conventions, and accuracy. OLI Principle 3 stresses the importance of “regular, iterative process” of the materials we employ to teach students, and tenet 3.5 links this to “multimodal composition theories and OLI environment(s).” Through employing strategies to have students assess GenAI capabilities with us, we not only continue our own research and curiosity, but we highlight the importance of critical thinking surrounding our use of technology writing tools. Part of the OLI is the rapidly changing ecosystem of GenAI, and engaging our students with this conversation allows us to expand these conversations and research to include students as well as “across institutions and disciplinary boundaries” (OLI Principle 4). OLI Tenet 4.3 asks us to be “committed to ongoing research about, and exploration into, OLI,” which indicated that we cannot ignore the fact that AI tools are a part of the online learning environment avaliable to our students. This activity also encourages AI literacy among students by helping them think critically about the GanAI writing tools and their advantages and disadvantages. Since AI tools are evolving rapidly, it is important that we equip students with the critical agency to evaluate the work and ethics of AI use.

[5] I used ChatGPT to write sample essays that my students then read when practicing their peer review skills, but this could be done with a variety of AI writing tools, and it might even be interesting to compare how different LLMs write differently for the same prompt. I built this into our discussion and practice before students peer reviewed each other’s drafts. Before we started the peer review of ChatGPT’s personal essays, we had already brainstormed possible topics as a class, read sample student essays from professional publications as well as student essays from previous semesters, and discussed together the writers’ choices and what appealed to us or did not as readers. In my assignment sheet for this essay (see Figure 1), I link to outside samples and aids beyond my own lectures. This kind of open discussion and experimentation aligns with several OLI Principles, including regularly developing and revising our materials as well as teaching literacy of technology tools (OLI 2 and 3).

3. Outcomes

[6] I developed this practice to have students think through and practice peer review strategies and to learn to evaluate GenAI writing. At this time, most of my students said that they had not used ChatGPT or other AI tools for writing, so I thought we should just explore it together. I did have a few students who had tried using ChatGPT and two who used it in their jobs, one of whom used it to write real estate descriptions. Jane Rosenzweig, director of the Harvard College Writing Center, said in an interview, “Writing is hard because the process of getting something onto the page helps us figure out what we think—about a topic, a problem, or an idea. If we turn to AI to do the writing, we're not going to be doing the thinking either” (Eissner & Scott, 2024). What I wanted students to consider in using GenAI for writing is how it might not be able to demonstrate and connect with human thoughts and experiences. But I also wanted to help them develop their ability to critically read essays and offer responses and advice to each other before we got into peer review for their first essay. By mixing in ChatGPT-generated essays with student-written ones, students were able to practice being critical readers who could offer advice for revision.

4. Implementation

[7] When I start a peer review, we start by exploring what peer review is, reflecting on past experiences my students may have had, and collecting what ideas my students have for what they want in a peer review session. Then we do some peer review together. When I teach my students about peer review, I use this hyperDoc. This streamlines what students engage with to lead up to our first peer review. Suggested Revision: turn the hyperDOC into a figure on the page?

[8] First, students watch videos, then we all read “How to Write Meaningful Peer Response Praise” by Ron DePeter in Writing Spaces. Then we add to a shared Google Doc to discuss and create expectations for peer review together. We also talk about how we write feedback for authors and what kinds of feedback the students would like from peer review. We additionally talk about plagiarism and citation. In this class, we talked about ethical uses of AI for writing and classwork. Here is a link to a folder with some of the resources I used to facilitate a discussion of ethics and GenAI tools.

Suggested Revision: See Appendix A for a list of resources I used to faciliate a discussion of ethics and GenAI tools. 

[9] We also played around with ChatGPT for writing assistance. We used it to help brainstorm topics and ideas for the essay, we asked it to build an outline for some of the topics, and we asked it to revise our writing. In particular, I asked it to revise some tips we found online about personal essay writing, and we compared it to the original version. See Figure 2 for that comparison.

[10] Students have also used ChatGPT to help generate prompts and share results, including love poems and letters of resignation. We played around with having it revise the work it wrote for the prompts with different directions and ideas for revision. Through this, we learned that ChatGPT was vague, lacked sources, and that it tended to favor certain phrases and sentences, no matter the topic or genre of writing. I would say that while AI is advancing, many of these issues remain the same on the free versions of the platforms. We can watch for this to shift and change as ChatGPT has recently advanced with its “deep research” tool for paid subscribers, but right now this issue still exists in most of the GenAI writing models without using advanced prompting techniques to have AI revise its own writing.

[11] Before we did our peer review of their own essay drafts, I brought in sample past essays from students, and I mixed in essays written by ChatGPT. Students were assigned a group and essays that they read. They wrote feedback together in their groups on Google Docs; I encouraged them to use the essays on the Google Docs to comment on the essays and provide feedback together (Microsoft Office and OneDrive have similar functionality). In doing this, students could reflect together and practice providing feedback on essays. I did not tell them that some of the essays were written by ChatGPT and not by other students. I had four sample essays for peer review practice that were written by ChatGPT: 1. Learning to Read, 2. Literacy Narrative for Track and Field, 3. Growing up in Poverty, 4. Growing up Black and Hearing a Racial Slur for the First Time.

[12] After collectively annotating the essays, students had a whole-class discussion about the essays they reviewed and what they thought was valuable and what could be improved in the essay samples that they had read. During this discussion, students identified issues with the Learning to Read essay, including that it is “vague,” uses needlessly complex language, “does not focus on telling a story,” had “no characters” and “no plot,” and that it “needs action.” For the second sample, the Literacy Narrative about Track and Field, students noted that the author seemed to be confused about what a literacy narrative is and that it did not dive into the language of the sport. They also pointed out confusing sentences; for example, “But beyond the individual events, my track and field literacy narrative was enriched by the sense of community and camaraderie.” They also asked which track events this person competed in, which demonstrated a lack of clarity due to the essay being so general. For the third and fourth samples, about Growing Up in Poverty and Growing Up Black and Hearing a Racial Slur for the First Time, we moved the discussion beyond writing concerns and into the bias found in ChatGPT. For the essay on poverty, students were critical of the idea that there is a silver lining to growing up impoverished, pointing to the sentences that said, “Yet, amid the challenges, there were moments of profound richness. The support and love within my family became the narrative's anchor. Despite the financial constraints, laughter echoed in our home, and the bonds forged in times of scarcity became unbreakable threads in the fabric of our relationships.” Students also commented that this didn’t sound like their experiences of growing up in poverty—this person sounded wealthy and privileged to the students who grew up in poverty. For the last essay, students thought this one didn’t sound like someone who had experienced being called a racial slur. At an HBCU, this essay did not echo my students’ experiences. Several students related that the first time they were called a racial slur, they had to ask someone what it meant. But most importantly, they questioned the fact that this essay didn’t talk about learning or the emotions that came with realizing they were Black and how it felt to learn people saw them as different and to be called names because of the color of their skin. If someone wanted to replicate this, you could use the same essays I have here, or you could put in your own essay assignment and examples and ask GenAI to write an essay. I would also say you could have students perform this task as well and see what their results are. I would suggest teaching a little bit about prompt engineering as well if you have students prompt the GenAI.

Suggested Revision: Reproducing one of the linked ChatGPT essays as a Figure and then outlining student feedback or annotations on that particular essay as a paragraph below rather than including feedback about each of them?

[13] Later, I revealed to students which essays were composed by ChatGPT, and we furthered our discussion of these essays and what students liked or did not like about the essays they were assigned to read. We also had a discussion at this point about what AI might be able to help them with, and what writing skills GenAI lacks, as well as what biases are present in GenAI. We then turned our discussion to what we learned about how ChatGPT writes. Students pointed out that ChatGPT tends to use the same similes across different essays; for instance, many things were “like a vast tapestry.” ChatGPT also tends to repeat certain phrases, including “chapters of life.” They also noted that ChatGPT favors complex language, its writing is vague, it lacks human emotion, and lacks sensory details that are relatable. Most importantly, they said, “ChatGPT does not get being poor or being Black.”

[14] Through this critical insight, we delved into a conversation where we talked about what texts may have been used to train Large Language Models (LLMs) and how this might perpetuate biases in GenAI. Having students write about this and reflect on ChatGPT’s writing not only helped them think critically about using GenAI tools, but it also helped them practice critical literacy skills in reading and responding to writing. For many of the students, in their final reflective essays for the course, they noted that finding out these were written not by a person helped them feel like they had agency to critique the writing more openly than they would with one another. Additionally, several students also said that this helped them think through how they would like to write their own personal essays.

5. Efficacy and Relevance

[15] In looking back on this lesson, I think I would spend some time talking about practices for writing more effective prompts for GenAI using Mollick’s (2024) model. The fact that I concealed these essays were written by GenAI extended the lesson of our peer review by a few days, so I don’t think I would hide that these essays were composed by GenAI in the future. Additionally, students with lower reading skills really struggled with this assignment, and some of them said in a later reflection that they thought the writing was good because it was complex, and they couldn’t understand it very well. This demonstrates a view of GenAI that might be important to discuss with students who might be considered “underprepared” for college reading and writing.

[16] In reflecting back on this activity, I think I would use some ideas from Anders and Dux Speltz (2024) and their "Human in the Loop" activity for thinking through how technology tools, like AI, might help us with our writing workflows. However, looking at these personal essays together and looking at them compared to student-written essays was a really valuable exercise. When we tried something similar for a rhetorical analysis assignment, ChatGPT mostly wrote outlines (see a sample here). I think it would be valuable to revise and redevelop this activity (OLI Principle 3), given the rapidly evolving ecosystem of GenAI. I did not try anything similar for multimodal compositions in the course, but I think that might also prove to be interesting as GenAI has evolved.

Suggested Revision: Including the ChatGPT outline as a figure in-text

[17] We also had some valuable discussions, including how to cite ChatGPT or other GenAI, which reinforced the practice of citing other people’s thoughts. We had great discussions about the limitations of ChatGPT, and we all learned a lot by trying to use ChatGPT together and talking about our experiences. I would revamp this activity of peer review to incorporate the “DEER praxis” that Cummings et. al. (2024) outline of defining the stages of the project, evaluating a specific GenAI tool for those stages, encouraging students to explore a specific tool, and providing students with a place for reflection. While the reflection was encouraged with the entire class, and individually later in written assignments, I think starting with the stages of how this fit into a larger project and what GenAI tools might be useful for different workflows would be beneficial for students to have a foundation and structure for examining these essays, while incorporating more critical engagement with using GenAI tools for writing workflows. Overall, this was a really great critical thinking exercise that I would like to repeat with some revisions to my approach. I believe it is vital to continue to explore GenAI possibilities alongside our students, constantly testing, researching, and thinking critically about the possibilities and problems of GenAI tools. This is an important part of researching OLI and extending our conversations, and I find it valuable to think of students as our partners in our research and conversations (OLI Principal 4).

References

Abbas, M., Jam, F. A., & Khan, T. I. (2024). Is it harmful or helpful? Examining the causes and consequences of generative AI usage among university students. International Journal of Educational Technology in Higher Education, 21(1), 10–22. https://doi.org/10.1186/s41239-024-00444-7

Anders, A. and Dux Speltz, E. (2024). A human-in-the-loop approach: Designing an AI-assisted workflow" (2024). Teaching Repository of AI-Infused Learning, 8. https://stars.library.ucf.edu/traiil/8

Alharbi, W. (2023). AI in the foreign language classroom: A pedagogical overview of automated writing assistance tools. Education Research International, 2023(1), 4253331.

Cummings, R. E., Monroe, S. M., & Watkins, M. (2024). Generative AI in first-year writing: An early analysis of affordances, limitations, and a framework for the future. Computers and Composition, 71, 102827. https://doi.org/10.1016/j.compcom.2024.102827

Duong, T., & Chen, H. (2025). An AI chatbot for EFL writing: Students’ usage tendencies, writing performance, and perceptions. Journal of Educational Computing Research, 63(2), 406–430. https://doi.org/10.1177/07356331241312363

GSOLE. (2024). Online Literacy Instruction Principles and Tenets. Global Society of Online Literacy Educators. https://gsole.org/oliresources/oliprinciples

Ma, K., Zhang, Y., & Hui, B. (2024). How does AI affect college? The impact of AI usage in college teaching on students' innovative behavior and well-being. Behavioral Sciences, 14(12), 1223. https://doi.org/10.3390/bs14121223

Mollick, E. (2024 April 22). Innovation through prompting. One Useful Thing. https://www.oneusefulthing.org/p/innovation-through-prompting

Ou, A. W., Stöhr, C., & Malmström, H. (2024). Academic communication with AI-powered language tools in higher education: From a post-humanist perspective. System, 121, 103225. https://doi.org/10.1016/j.system.2024.103225

Ranade, N., & Eyman, D. (2024). Introduction: Composing with generative AI. Computers and Composition, 71, 102834. https://doi.org/10.1016/j.compcom.2024.102834

Wang, L., & Li, W. (2024). The impact of AI usage on university students' willingness for autonomous learning. Behavioral Sciences, 14(10), 956. https://doi.org/10.3390/bs14100956

Zhao, H., & Min, Q. (2024). Exploring continued usage of an AI teaching assistant among university students: A temporal distance perspective. Information & Management, 61(6), 104012. https://doi.org/10.1016/j.im.2024.104012