OLOR Series: | Research in Online Literacy Education |
Author(s): | Rebecca Hallman Martini |
Original Publication Date: | 15 September 2019 |
Permalink: |
<gsole.org/olor/role/vol2.iss2.e> |
This article presents findings from a study conducted at a large research institution that analyzed ten asynchronous online writing consultations in two different formats (in-text and end comment) and survey/interview data from tutors as well as students who received both kinds of feedback. Findings indicate that different formats impact the kinds of feedback tutors provide: compared with end comments, in-text comments were more frequent; focused on the sentence-level; and included more rewrites, questions, observations, and rule-based feedback.
Keywords: asynchronous, online writing center, tutoring, feedback, revision
[23] 2.3.2. Referent Types.Each comment was coded by the type of reference: full-text, paragraph, or sentence. One of this project’s aims was to determine how changes in commenting format changes how tutors address global (full-text) and local (paragraph and sentence-level) issues in student writing. |
Table 2: Codes for Degree of Directiveness
|
[25] 2.3.4. Writing Topics.After tutor responses were divided into comments and coded according to those comments, they were then coded according to the writing topics discussed. Four writing topics emerged from the data: audience, genre, rule/convention/strategy, and description. Comments were coded according to whether or not they addressed the topics below (Table 3): |
Table 3: Codes for Writing Topics Discussed
|
2.4. Analysis of tutor responses[26] A total of seventy virtual writing center responses with comments were collected. A sample of ten responses (two from each tutor, one in each format) were coded. Responses were examined for both differences and similarities between the in-text and end comment formats. An analysis of common trends in tutor responses collectively were considered, as well as how each individual tutor changed their response when the commenting format changed. 2.5. Tutor Interviews[27] Interviews were audio recorded and analyzed for both commonalities among tutors and differences in terms of approaches to using each commenting format, perceptions of how students understood their comments, reasons why tutors commented the way that they did, and preferred commenting format. These interviews were conducted with the goal of determining the tutors’ processes and perceptions of their commenting styles when using two different formats. 2.6. Student Surveys[28] Each student was given class time to complete a 20-question survey. Surveys began with a descriptive paragraph, explaining to students that they were being asked to compare the types of comments they received on the first two major assignments. These types were defined as “in-text (comments which appeared in bubbles in the margins)” and “end comments (comments which appear on a longer note, separate from the paper itself).” [29] The survey began by asking students to indicate which kind of comments they received on Assignment One and on Assignment Two. Survey questions asked students if they read comments for either Assignment One or Assignment Two right away. Then, a large portion of the survey questions asked students to compare the commenting formats and determine how they read comments, which format focused on which particular writing topics, and how the formats influenced their perceptions of revision. Students were also asked to explain why they preferred the commenting format that they did and to include any additional advantages or disadvantages involved with either format. 2.7. Student Interviews[30] Six students were selected for individual 30-minute interviews based on willingness to participate. These interviews aimed at determining how students understood both commenting formats and why they preferred one to the other. Students were asked about their first impressions of each set of comments, to pinpoint areas that were either clear or confusing, what they would/did change in revision, and why they preferred one format over the other. Students were provided with hard copies of both essays with comments during the interview. |
3. Results[31] This study found that: (1) commenting format had a significant impact on the kind of feedback tutors provided; (2) students and tutors both preferred in-text comments over end comments; and (3) different formats influenced students’ approaches to revision. 3.1. Impact of Format on Comment Type[32] 3.1.1. Tutors wrote more in-text comments but longer end comments. Table 4 indicates the average number of comments in the in-text responses contained over two and a half times as many comments as the end comment. Yet, the average number of words in the end comment was nearly three times that of the in-text comment. Thus, the end comment responses tended to be fewer but longer and in-text comments tended to be more frequent but shorter.
|
[33] Although the majority of end comments appeared in a format similar to or exactly as the one presented in Figure 1, two of the tutors used an end comment template developed by the Writing Center called the Standardized Asynchronous Response (SAR). This template encouraged tutors to consider particular higher order concerns with flexibility, but they were not required to use it (See Figure 4: Sample Online Writing Center End Comment SAR). [34] As shown in Figure 4, the SAR followed a letter format with bold, capitalized red headings which, in this particular example, read “This document,” “grammar,” “organization” and “relevance/answering all the questions,” and “future work.” In this example, Lindsey cut and pasted examples from the student’s text into the end comment document, which was a common strategy used by all tutors and also added to the overall length of the end comment. [35] 3.1.2. More in-text comments were sentence-level and more end comments were full-text. Table 5 shows that the end comment responses included nearly twice as many full-text comments overall compared to the in-text formats. In addition, these full-text comments made up 66% of all end comments. In contrast, the in-text format had nearly eight times as many sentence-level comments compared to the end comment format, culminating in 69% of all in-text comments. The difference in terms of paragraph-level comments between the two formats was minimal. |
[36] Table 6 indicates that over half the explanatory end comments were full-text referents while nearly two thirds of the in-text explanations were sentence-level referents. |
[37] 3.1.3. Praise was infrequent with the exception of one tutor.In this study, praise was surprisingly infrequent, making up only 8.3% (n=17) of all comments (n=204). In addition, 41% of the total praises (n=7) and all of the praises with explanations (n=3) were given by Lindsey One example of explanatory praise from Lindsey’s end comment response was:
[Name of student], you've given me great specific and concrete detail in this section—it says a lot about you as a person, too. Using the story of this son really does a lot to draw your reader in and make them understand just what kind of person you are. In this praise comment, Lindsey explained to the student not only that their detail was “concrete and specific” but she also suggested that it was important because it “draw[s] the reader in and make[s] them understand.” Thus, the comment suggested that the detail was effective because it drew in the audience and also provided information about this student’s personal character. [38] 3.1.4. More in-text comments were tutor re-writes and questions. Table 7 indicates that tutors provided over four times as many question comments in the in-text format compared to the end comment. Oftentimes, these question comments offered students multiple choices for revision, asked several questions at once, and/or asked clarifying questions.
|
[39] Of these rewritten comments, 56% of the end comments (n=5) and 92% of the in-text comments were at the sentence level. Both the end comment and in-text formats had the same number of paragraph level (n=2) and full-text (n=2) rewrites.
[40] For example, in an in-text response, Scott asked: Is this your thesis? Or is the first sentence your thesis? Does it represent an opinion, or some sort of narrowing down of the overall topic of the Crisis? What is the particular angle on the crisis that you’re going to talk about? In this question comment, Scott suggested that the thesis statement needed clarity. These questions encourage the student to figure out which statement is their thesis, what kind of statement (opinion or narrowing down) that thesis is, and what kind of angle they want to discuss.
[41] Another example of an in-text question comment comes from Alice, who asked a question clarification comment and also provided the students with options for revision: Do you mean the social worker? Social work would be the label for the field as a whole, while the social worker would designate the individual doing the work in the field. If you mean ‘worker,’ you should double check that it’s correct throughout your paper. In this sentence-level referent, Alice explained to the student the different implications involved in using “social worker” and “social work” and also encouraged them to be consistent throughout the entire essay.
[42] The end comment question comments also encouraged elaboration but oftentimes included an observation first. For example, Erica wrote: You mention that you would like to work with children, but can you say more about that? Do speech pathologists specialize in different areas? If so, can you say more about those? Are there specific groups of children or specific speech disorders that you would be more interested in working with? In a different end comment question comment, Alice used minimal marking and font color to indicate places where one student seemed to leave out important information for comprehension. Alice’s end comment read “In your paper, I’ve marked places where you seem to have left out some information with question marks in brackets, like this: [??] and changed the text to be blue.” Her in-text markings looked like this: |
[43] Although these question comments go beyond minimal marking, Alice’s decision to explain the overall writing issue trend in the end comment and then to mark the places in-text suggest a possible strategy that can be used to overcome the limitations of the end comment format. [44] Table 7 also indicates that in this study, tutor rewriting of student text occurred over five times as often in the in-text commenting compared to the end comment. The majority of rewriting in both response formats occurred as the sentence level. [45] For example, a typical rewritten comment by Scott was “‘marked by’ might be stronger.” In this instance, Scott did not literally rewrite the student text but he did provide a phrase that could immediately replace the student’s text without the student having to come up with original language or make a choice about which particular word to use. Instead, the student could delete the word of her text highlighted by the tutor and replace it with the tutor’s phrase, which the student may have decided was a better fit. [46] This same kind of rewriting also occurred in end comment responses, even though such comments were significantly less frequent. For example, Erica wrote: Be sure to tell your readers specific things about you that they did not already know. So you might begin by saying something like this: “I am applying for a master’s degree in speech-language pathology because…” Although only a partial sentence, this comment still provides the student with words constructed by the tutor that the student could cut and paste directly into her draft, which is a kind of rewriting. This particular practice has been noted by Hewett (2015a) as potentially dangerous for its fixing quality, yet the directness may also be necessary for teaching students who would otherwise be unsuccessfully guessing at what kinds of changes would be appropriate. One of the major contributions of Hewett’s work is her acknowledgement that the tendency to avoid more directive tutoring is primarily a result of rhetoric and composition’s valuing of social-constructivist approaches to teaching writing. Yet especially in asynchronous tutoring, clarity and explicitness sometimes requires more direct speech that can teach students how to revise their writing more effectively. [47] 3.1.5. More in-text comments were observations and rules. Table 8 indicates that the in-text format included more observation comments and writing rules. |
[48] However, observation comments in the end comment were on average 69 words per comment compared to an average word length of 18 in the in-text format. These observatory comments in the end comment were usually summative paragraphs in which the tutor explained how they understood the student’s major claims in the text. For example, Molly wrote the following end-comment observatory note: In your introduction, I see you setting up a structure that I’m having a hard time locating in your actual paper, and I don’t know if this is because I’m misunderstanding your thesis, or because you have two different theses operating right now. On the one hand, I see you setting up your paper to follow the three “paradigms” you introduce through C.S. Lewis – the pagan myth, Christian meta-narrative, and post-Christian ambiguity. My concern with this is that by the way you state them in the intro, I was expecting to follow this organization throughout. In this comment, Molly admitted her confusion with how the writer structured the piece, yet she prefaced those concerns with “I see you setting up…” which offered her perception of the moves the writer seemed to be attempting to make, which could then be read by the writer and used to evaluate whether or not those perceived moves were intentional. [49] While the end comment observations tended to be significantly more summative, the in-text observations were shorter and more connected to reader expectation. An in-text observation comment written by Erica was: “From this paragraph, I assume that you are going to discuss all three pieces that were part of the recital, but this is not the case.” In this instance, the tutor described how she thought the student’s essay was going to play out, but then admitted “this is not the case.” This kind of observation suggested there was an issue in how the introductory paragraph connected with the content of the essay, yet in this comment the tutor attempted to bring this observation to the student’s attention, rather than offering a way for them to rewrite or revise the sentence. [50] Table 8 also shows there were over twice as many writing rules included in the in-text comments compared to the end comment. In response to one student’s claim that “Due to the importance of eye contact in the western society this conduct must led [sic] to false impression when socializing with new people,” Alice highlighted the word “must” and wrote “Must suggests that it always will lead to misunderstanding. Generally in academic writing, the writer will offer a more tentative phrase, saying “’might lead.’” In this comment, Alice offered advice about “academic writing,” which, although related to the student text, was also framed outside the context of the particular piece of student writing. Instead, the focus was on the concept of “academic writing.” 3.2. Tutor Perceptions of Different Formats[51] By the time tutors were interviewed, they had been participating in this study for five months and many of them had completed or had nearly completed seven responses in each of the two formats. Thus, these tutors could draw on these experiences to articulate how their processes changed depending on which format they used. Perhaps because of their experience with each format and the absence of a collective tutor training in each format, these processes were nuanced and individualized, which made identifying trends among practices challenging. Yet, there were some commonalities in terms of understanding what the role of tutor should be, even though processes and preferences varied. [52] 3.2.1. Most tutors prefer writing in-text responses.Before participating in this study, three of the five tutors (Mary, Alice, and Erica) preferred in-text commenting. For Mary, this process was “more natural” and allowed her to provide an “in-depth, line-by-line reading of the student paper which they deserve… and it allows you to really dig deep and look at these, which I think is very important, stylistic issues and it allows you to be really precise.” In addition, Mary preferred the in-text format because she was able to focus more on closely reading the student text, rather than on constructing the end comment.[53] Alice preferred in-text commenting because it allowed her to respond as she read the paper and because the comments were “visually right there so students don’t have to go digging and guessing about where the thing you’re commenting on is or how many times they do it.” Similarly, Erica preferred in-text commenting because it allowed her to make comments when she saw issues them rather than having to rely on her memory. [54] In contrast, Lindsey had only ever done the end comment response and had not thought much about changing her style; Scott preferred the end comment response because of his own commenting style (see Figure 6: Example of Scott’s In-text Commenting), explaining that he had: a tendency to be heavy handed and I care very much about style and language as a poet…I have to recognize my weakness which his I want to fix sentences and that misunderstands the purpose of working in the writing center. After their participation in this study, two of these tutors changed their preferences, but three out of five still preferred the in-text format. Alice changed her preference from in-text to end comment over the course of the interview, and explained: I think 30 minutes ago I would have just said [I preferred] in-text because its faster, easier, more natural because I can just be a responder…but I think that, I guess after answering your questions and thinking about it more I think maybe that the comprehensive note is better. In contrast, Lindsey changed her preference from end comment to in-text because “being a reader is a way for me to ameliorate some of that stress so I can just be a reader instead of an instructor but I can’t deny that it’s faster and I’ve got lots to do.” [55] While these tutors had preferences before and after participating in this study, they also expressed a need to include a combination of both formats. Most commonly, tutors wrote in-text comments with a short summative end comment. Most tutors stated they wrote this way to “explain [to students] how the [in-text] comments work…and to wrap it up” (Lindsey) or as a way to summarize which “allow[s] the student to see that they shouldn’t despair about all the comments made” (Scott). Less often, tutors preferred to write a longer end comment with a few in-text comments to serve as examples they would refer to within the end comment. [56] 3.2.2. Physical and spatial constraints affect commenting style. Some tutors mentioned that the spatial conditions of either format significantly affected how their responses were constructed. Most notably, the text boxes constrained the in-text format and the end comments’ lack of structure created limitless space. Mary explained her confusion over how much the text boxes should determine the length of her comments, as well as her struggle to keep her end comments within 2-3 paragraphs, rather than going onto a second page response. She admitted the end comment seemed to lend itself better to explanatory comments, because:there isn’t the physical like spatial constraint [in the end comment] … you feel like you have to cut your text short in the in-text [format] because you realize your taking up most of the margin in the page and that’s ridiculous. Trying to focus on two to three major issues in the end comment and really expanding, then in the in-text there are lots of useful suggestions but not going in depth into why it’s important to change this. Scott expressed his concern for end comments that are too long, explaining that students would most likely not know what to do with so much feedback and only those highly invested would be willing to “figure out” how to use these comments. Lindsey, whose end comment responses tended to span over several pages, claimed that she appreciated how the in-text comments encouraged her to “push myself to be brief [and] get right to it.” The challenge end comments create, both in terms of how students navigate the feedback and whether or not they will read it all carefully, have also been acknowledged by Hewett (2015b) in Reading to Learn and Writing to Teach where she argues that online writing instruction requires students to read, interpret, and act upon written feedback, often in complex ways. [57] 3.2.3. The end comment is more future oriented and should be placed on the student text. Despite the overall preference for in-text commenting, most tutors identified strengths of the end comment that were better suited to that format than to the in-text format. For example, Scott explained that end comments encourage a kind of “moving forward,” because:We do feel like, especially because we’re instructors too, a need to give some kind of summative remarks… I would not feel right…without giving some directive comments at the end saying this is where you should go from here. Tutors also had much to say about how the end comment should appear in relation to student text. Mary said the end comment should “go directly on the draft not a separate sheet of paper,” and her reasoning was for convenience and practicality, “there’s a lot of unnecessary clicking and opening of things, you could save effort by having in-text and then summative end comments at the end.” [58] Alice explained that the end comment, which she called a “comprehensive note,” should appear on the student text. However, she purposefully constructed her note at the beginning of the document, before the student text, rather than at the end. Alice did this because I see that [comprehensive note] as saying, one I want to preview what you should expect to see [in the marginal comments] because I want to frame the way you’re going to read my comments, sometimes I over comment so I know it can be overwhelming…some people just think they did a bad job because there are lots of comments in the margin. Alice’s placement of the comprehensive comment at the beginning of the document sought to both frame her in-text commentary and reduce student anxiety about receiving comments on their text. Yet, of the five tutors who participated in this study, only Alice constructed the end note at the beginning of the student’s document. [59] 3.2.4. All tutors associated in-text with a “readerly” response and end comment with a “teacherly” response.One of the most striking findings was that all five of the tutors associated the in-text commentary with a “readerly” response and the end comment with a more “teacherly” response. In addition, all of the tutors made comments about teaching and how they responded to their own students’ texts as they answered questions about their roles as tutors. For example, Alice explained:
I like the in-text better because the commenting style is how I’m reading your paper as a reader not necessarily an evaluator, I mean it is an evaluation, but like, but as I read I’m thinking: this is where I’m confused, this is what I’m noticing, and with in-text I can put that right in... More like I’m saying these are the things I’m noticing, this is my response to your piece. Later during the interview, Alice referred to this practice as “reader response.” Prior to this project, Alice had responded to student texts in online writing conferences with primarily in-text comments, and she seemed to want to resist the role of the “evaluator” when writing from the position of the tutor. [60] In contrast, Lindsey was responding to her clients in the virtual writing center with an end comment response only. However, after participating in this project, she admitted her preference for in-text commenting. Like Alice, Lindsey also explained her role as that of a “reader” as she commented in the margins of student text instead of at the end. Lindsey explained: When I do in-text, I feel like I can insert myself as a reader into that text...it’s more like a reading mind...I think it’s a different kind of interacting actually, trying to put myself as a reader and give them my actual meta-commentary, but doing the end comment I feel a lot more teacherly. Lindsey also said that one of the weaknesses of the end comment response for her was that she “felt more like a teacher, [and that’s] not an actual rhetorical situation; I’m not acting as a reader, I’m acting as a teacher and giving them tools.” [61] Another reason Lindsey preferred the “readerly role” to the “teacherly role” was because she believed the power dynamics were more equalized when she responded as a reader. Lindsey explained that she found it essential for the tutor to provide the student writer with directions for how to read her marginal comments: you put yourself in as a reader which ameliorates some issues that come up with power—when you talk to student writers as a reader of text it helps level the playing field a little...having them see the instructor as a reader and having them think of you as an audience member gets them to think about audience more. In this explanation, Lindsey suggested “leveling the playing field” was important for tutors to consider while working with student writers and also in-text comments encouraged students to think about audience. Often, online writing conference tutors have even less context for the student’s original writing task; Lindsey’s desire for the student to maintain autonomy seemed especially important. When Lindsey restated her preference for the in-text format near the end of our interview, she connected this preference directly to her resistance to an authoritative role: I found that I prefer the in-text. I don’t know if that’s because I’m anxious about my identity as the assessor, so being the reader is a way for me to ameliorate some of that stress so I can just be a reader instead of an instructor. In contrast to Lindsey, Scott preferred the end comment format to the in-text format, yet his reasoning for this was because he considered himself a better teacher when he responded with the end comment. Scott preferred the end comment exclusively. This conflicted with the majority of the other tutors’ preference for in-text commenting, even though he, too, understood the end comment as a more “teacherly” kind of response. Scott explained, “The end comment is better because if not, they [the students] end up focusing only on the sentences I comment on so the more I keep my hands off the better of a teacher I am, I think.” Scott seemed unable to perceive the in-text commenting as a format with other kinds of potential beyond fixing, even though he did acknowledge some of its advantages, including the way it: lends itself to modeling and immediate intervention, [and it’s] very direct, very connected feedback can be really useful. [There is a] chance they will learn the lesson better that way because they actually can’t miss it. Of the tutors interviewed, Scott seemed most aware of his own tendencies as a responder to student texts. He admitted to his “struggle” with in-text commenting because of his tendency to “start talking grammar or a sentence fix, which I didn’t want to be wasting my time on but I just felt like I needed to say something and once I comment on a sentence I get really antsy and I can’t not fix that so it was a struggle to keep myself from doing that and I usually failed.” As a tutor-teacher, Scott explained he should be teaching students not only how to “fix” their writing, but also to identify that which needs to be fixed: I feel like being able to identify is just as important as being able to fix...but the problem is they couldn’t even identify until you pointed it out, so we have to figure out ways to help them identify, because you can’t fix without identifying. They[students] should begin being able to do their own teaching, fixing, learning, which I don’t think we do well when we only teach the fixing. Here, Scott suggested tutors should teach students to both identify and fix issues in their own writing. [62] He also said that in most sessions, he wanted to focus on higher order concerns, and the he was unable to see those higher order concerns when he commented using the in-text format because he was distracted by the sentence-level errors. Scott’s explanation of the tutor’s role as one who teaches students how to look at their own writing and the end comment’s ability to encourage that kind of work seemed similar to Lindsey’s articulation of the end comment’s strengths. Lindsey acknowledged that the end comment format: gets the writer to do the work [and I use] mini examples, and then tell them I noticed this here, here, and other places... I can see that helping student agency too, you’re giving them different tools...it encourages them to go use those tools on their writing. Lindsey and Scott suggested the end comment had potential beyond the “fixing” of a particular paper. 3.3. Student Responses to Different Formats[63] 3.3.1. Students in this study preferred in-text comments to end comments.Overall, this data found that students expressed a preference for in-text comments to end comments. The average preference for in-text comment over end comment on a four-point Likert scale, in which four was definitely end comment and one was definitely in-text, was 1.56 with a standard deviation of 0.89. Some students indicated on the survey that they preferred in-text comments because they made it “easier to locate and fix mistakes” or “revise the problems.” Others appreciated having the comments “right at the site of the error,” which, for this particular student, made it “easier to change it right then and there.” One student suggested the in-text comments helped her with revision because she “was able to…understand my mistakes better and [the in-text comments] allowed me to create a better paper.” Furthermore, two students noticed that in-text comments included both praise and criticism, because they pointed out “good and bad points and mistakes.”[64] Some of these students did not like the end comments because they found them to be too general, or “too vague and not specific enough.” Perhaps for this same reason, other students disliked the end comment because it was not helpful in terms of revision. For example, one student said the end comment “doesn’t help as much when I actually go to revise my paper,” and another confessed that the “end comment makes me feel hopeless to change a problem that seems to have been strong throughout the entire paper.” A third student explained the end comment as “more detailed and made it easier to understand my grade, but the in-text comments were easier when it came to revision.” This particular student seemed to associate in-text comments with revision and end comments with justification for a grade. Furthermore, end comments were criticized for “leav[ing] out information because the teacher goes through and has to remember what she wants to say at the very end,” which, according to one student, is an indication that “less effort was put into reading your paper.” [65] Students who preferred end comments appreciated them because “they were more broadly dealing with the entire paper,” provided “detailed explanations of what’s good information and what could use some work or needs to be taken out,” and they were “more personal.” While these students did not seem to criticize the in-text comments, one student did refer to the in-text comments as “corrections,” although this particular student also indicated a preference for them. [66] Interestingly, some students mentioned they would prefer a hybrid commenting format. For example, one student who circled “definitely in-text” as the preferred format further explained, “I actually think it would be most advantageous to use both in-text and end comments. That way nothing will be left out and that is how my papers have always been graded in the past.” Another student said: I actually would like to see both on my papers. They both help me and they work well together. Grammar and such could be in-text comments and opinions or organization would be good in the end comment. This student seemed aware of how different commenting formats might be better for different kinds of writing issues. [67] 3.3.2. Students perceived end comments to be more confusing and less helpful for revision.Table 9 shows that on average, students believed end comments were confusing, wishy-washy, and overwhelming. |
Table 9: Students’ Average Perception of Format Preference and Confusion (4-point Likert scale: 4- definitely end comment, 1- definitely in-text)
|
[68] These ideas were reflected in their short answer responses as indicated above. One student explained the overwhelming aspect of the end comment as “pil[ing] every error on top of you at one time.” Although most end comments in the sample collected did include praise as well as suggestions for improvement, most of these praises were general and either at the beginning or at the end of the end comment. This particular student seemed to appreciate how the in-text comments interspersed praise with suggestions. [69] Table 10 suggests students identified in-text comments as more helpful in terms of revision, confidence, and clarity. |
Table 10: Students’ Average Perception of Formats and Revision (4-point Likert scale: 4- definitely end comment, 1- definitely in-text)
|
[70] However, students indicated that in-text comments were slightly less useful in helping them understand why they needed to make changes. This was reflected in the students’ written responses about end comments mentioned above, which indicated that students appreciated the specific and detailed explanations in the end comment. [71] In addition, students indicated end comments were slightly more helpful in terms of identifying the most important thing to change. This was surprising, considering that one of the commonly understood strengths of the end comment is that instructors can prioritize (Bean 2011), whereas in-text comments often make prioritizing more difficult. Thus, students might not be reading end comments as intentionally ordered in terms of importance. For instance, during the interviews some students said they perceived end comments being in chronological order, when in actuality, they were not. [72] 3.3.3. Students perceived end comments to more fully address content and audience matters and in-text comments to address organization and grammar.Table 11 indicates that students perceived in-text comments to be slightly more helpful in terms of organization and grammar, and end comments to be slightly more helpful in terms of content and audience. |
Table 11: Students’ Average Perception of comment Formats and Writing Topics (4-point Likert scale: 4- definitely end comment, 1- definitely in-text)
|
[73] Students’ perceptions of in-text comments as more helpful with grammar and end comments as more helpful with content and audience is not surprising. However, the tendency for students to identify organizational comments with the in-text format was significant because organization is often considered to be a strength of the end comment, and a weakness of in-text comments. 3.4. Students Talk About Their Perceptions of Different Commenting Formats[74] 3.4.1. Students strongly preferred in-text comments. Of the students interviewed, all six of them preferred in-text commenting. The most common reason students provided for this preference was the location of the comment, directly next to their own text, making it easier for them to see what needed to be changed. Lisa explained, “I’m a very visual learner, so this is definitely more satisfying for me because I can see what I’m doing wrong at the exact position.” She also said, as a writer new to college-level writing, in-text comments were better:
I can see how the end comment would work really well with upper-level students, but I think for lower English classes a lot of people are still figuring out how to write academically, and I just feel like in-text is more nurturing that way. Overall, students interviewed in this study preferred in-text commenting because this kind of feedback was easier to understand and apply. For instance, Lisa acknowledged some students might prefer the in-text comments so they can “check off” the comments as they move through revision. In another interview, Penny explained she preferred in-text comments because “the arrows point to what needs to be changed and if it doesn’t need to be changed, there’s nothing there.” Both of these observations suggest students perceived revision to be necessary only in places identified by individual in-text comments. In addition, Molly explained, “When you’re revising, you want to do it quick, you don’t want to reread your paper over and over again.” [75] Another reason why many students preferred the in-text commenting format was because they indicated appreciation for praise along with criticism and suggestions. All but one student interviewed mentioned praise in relation to the in-text comments, and not the end comment. Julie explained, “I liked how she [the instructor] gave negative and positive feedback [in-text], it made me feel a lot more confident about my paper, rather than general stuff.” This particular student identified the following comment of praise to be especially helpful: |
[76] The small text box reads “You do a nice job of previewing the contents of the report: the types of writing that teachers do most often and the audiences that they typically write to.” This student, along with several others, appreciated the way that her own text was highlighted in connection to this comment, and she was able to understand where and why she did something well. Yet, this same student was frustrated by how praise appeared in her end comment. She explained: She [the instructor] does add good positive feedback [in the end comment] but at the same time it’s still too general for me to really see what my paper is doing well, especially since the positive feedback isn’t as long as the negative feedback and in the negative she’s asking all these questions but in the positive feedback it seems like I did that. The end comment this student was responding to appeared like this (Figure 8). The comments that seem most clearly to be praises are highlighted: |
[77] While the comments of praise are contained within a greater observational paragraph, this student still found the praise “too general.” The instructor did not explain why the student’s piece was a “very interesting narrative,” and perhaps in the second highlighted praise the student was unfamiliar with the idea of a “rite of passage” narrative. In addition, the student read the complimentary paragraph and the second paragraph as contradictory. [78] 3.4.2. End comments led to more drastic revisions of student texts. Despite Julie’s frustration with the end comment, she admitted she drastically revised the paper with an end comment. Julie was articulate about the way her instructor’s suggestion about the conclusion in the end comment caused her to revise. She explained this was the most important thing she needed to revise “because I did start my introduction with this [how her brother brought her into reading], but then I didn’t elaborate on it. So, I needed to go back and add more about it in my other paragraphs.” Yet, she also admitted, “I don’t know if I was adding it to the right spot…the one with the end comment would be more drastically changed, but not necessarily for the better, it could be for the worse.” This suggests students may experience more anxiety with their ability to interpret and apply the suggestions made in the end comment, which aligns with Hewett (2015a).
[79] Beth claimed the end comment led to profound change in her paper. She voiced extreme frustration with the end comment and admitted “I was confused [by the end comment] …I felt it [the paper] flowed and things were in good order, but what I had to do, it really didn’t give me a lot of detail as to what was wrong in the paper. I had to do another meeting with her [the instructor] to go through the entire paper.” The comment Beth received appears below: |
[80] While Beth expressed comfort with revising “the stance part” in the comment (labeled 2 by Casey), she said “the first comment was rocky” because “rewording the interview…I didn’t understand what she [Casey] meant.” Since the instructor did not write “reword the interview” in the end note above, Casey’s interpretation of this comment was either based on her own understanding or was something she realized after meeting one-on-one with her instructor. [81] Beth also admitted she “literally gutted this paper” and that “I think the revision was a lot better than the original. [But] it did take more time I had to figure out how I was going to say the things we talked about without saying ‘I said, she said.’” When asked about which commenting style she preferred, Beth said she strongly favored the in-text but then mentioned, “of course, these comments at the very end make me work a little harder, but the in-text are better for me because I can go through the paper and see step by step what it is that needs to be changed.” [82] 3.4.3. End comments made students feel less motivated to revise.Although Beth drastically revised her paper with end comments, other students voiced their frustration with the end comment and expressed less motivation for revision. For example, Lisa admitted her frustration with the end comment that appears below (highlighted text is the text the instructor cut and pasted from the student’s own text): |
[83] Lisa said she spent more time revising this literacy narrative because she “spent a lot of time trying to figure out how to relate the comments to the piece and trying to underline where the issues were, in that regard.” Even though a significant portion of this end comment was cut and pasted from Lisa’s text, she went on to say: Yeah, I definitely spent a longer time trying to figure out exactly what those comments meant in the end comment. Where…I guess when you’re considering the actual revision of my writing I spent more time on the piece that had the in-text notes because by the time I figured out what all the end comments meant, I was just kind of done with it (laughs)…But I was like this is already work in itself, so when I got around to the in-text ones it was just easier because I didn’t feel so mentally exhausted. Here, Lisa seemed to find the process of going back to her own paper and matching up the instructor’s comments with her own “underlin[ing]” time consuming and mentally exhausting. Molly seemed to share this concern, admitting that the first time reading her end comment was “overwhelming” because it had “big bulky paragraphs…when you see something like this, you’re like—I really don’t want to read all this.” [84] 3.4.4. Students use a variety of strategies to determine their plans for revision. Near the end of our interview, Lisa revealed the two ways she believed students decide what/how to revise. She first asked what instructors seemed to prefer. I admitted they seemed to prefer the in-text commenting, but they still expressed pressure to write end comments. I also told her most teachers said in-text commenting lent itself to lower order concerns, while the end comment focused more on organization and global issues. Yet Lisa explained that when students chose to focus on grammar instead of more global issues, it was because they cared less about getting a better grade (and were thus less invested in revising); “I think anyone can pick up on the fact that a punctuation error is far less important than a structural error or any error with clarity, I guess it depends on how much you care about getting a better grade.” This seemed to hold somewhat true in the case of Penny, who had already revised her paper when I interviewed her. She explained that she focused her revisions on the comment that “talk[ed] about the actual argument itself, which is what the paper’s about.”
[85] Lisa also claimed students who do want to revise determine what to revise based on the comments the instructor repeats in her response. She said: As far as the revision of the piece as a whole goes, I think as long as they’re more thorough comments about like the structure within the in-text revisions, then I think it works, at least for me, because even when she [the instructor] saw a problem with E12, that was a continuous seam, whenever she saw a problem with that, she would mark it again, so I think it just depends on, like the repetition. Lisa was able to prioritize which comment was most important by the number of times the instructor mentioned the issue throughout the in-text commentary. Lisa identified the following as the most important comment: |
[86] The comment box reads: “What exactly is included in the exhibit explanation and arts statements, and who reads them? Is the curator involved in writing the artist’s statement? What are the “documents” that include all the information that you list here. Say more about these different types of writing.” Lisa explained this comment had also appeared earlier (in E4, as “Say more about these. What are each of these statements exactly? What is included in each? Does the curator write the artist’s statement? If not, what role does the curator play in getting that writing to the audience at the gallery”) and then appeared later (in E18, as “What exactly do you mean?”). For Lisa, the repetition contributed to her awareness that clarity was the most important writing issue for her to focus on while revising. [87] In addition to these revision strategies, Molly said she determined the most important writing issue to focus on during her revision by determining which of the end comments was the longest. Her end comment appeared as follows: |
[88] Molly identified the fourth paragraph as the place where she would focus the most on revision, and the second paragraph as the comment she was most comfortable applying to her own revision. |
4. Discussion[89] This study found that formatting significantly impacted the content and style of tutor commenting. In-text comments were shorter, more frequent, consisted of primarily sentence-level referents, and included more questions, rewriting, and rules. End comments were longer, fewer, consisted primarily of full-text referents, and included slightly more praise comments. Additionally, both tutors and students preferred in-text commenting over the end comment because they found the comments were clearer and made revision easier. Yet, and a couple of students admitted during interviews that their writing was more significantly changed when they revised using the end comment response. While more revision may be assumed to result in a positive outcome, some studies have found that greater change in student writing does not always mean the writing has improved (Martin 2015). [90] One of the most surprising findings of this study was how students used praise to guide their revision and writing development as well as how rarely praise appeared in the online writing conferences. Furthermore, students acknowledged and claimed to use the praise they received through in-text comments as they revised their writing and seemed to overlook praise in the end comment because it was too general. The findings from this study extend Carol Dweck’s (2007) concept of “process praise,” which she says motivates students by acknowledging their persistence, effort, engagement, and improvement. While the findings from this study support Dweck’s claim for the importance of process praises and the need to encourage motivation, they also suggest that process praise must be clearly articulated with specific references to student text and explanations of how and why student writing is working effectively in order for praise to be most productive. Further, Maria Teglia’s (2009) claim that praise comments are often generic and do not lead to revision, since students are unlikely to make changes to text that instructors have already approved, must be reconsidered in light of this study’s findings. [91] Given that praise was used by students in this study to revise, it is crucial for tutors to become more aware of the praise, or the absence of praise, in their online writing conferences so they can intentionally use it to foster student revision. Rather than using praise for motivation and confidence building, the results from student interviews in this study suggest they used praise to help them understand what they are doing well, not only in the particular places where instructors praise the writing, but throughout the entirety of their own texts when they write in similar ways. In addition, students seemed to identify helpful praise more frequently in the in-text comments, which means that constructing explanatory praise in the end comment is essential to facilitating student revision. [92] In addition to what this study reveals about the lack of praise in online writing conferences, this study also shows tutors expressed a strong need to include a kind of hybrid response to student writing, even though they did prefer only in-text comments over only end comments. As mentioned in the methods section of this study, tutors were resistant to using one commenting style. Interviews with these tutors indicated they preferred to include short summative notes for students to read before the in-text responses. These notes were often included in addition to the end note comment, and found either attached to the student document before the student’s text began or in the body of an email sent to the student with the written response attached as a word document. [93] On a separate note, student and tutor preferences for in-text commenting, because of its perceived effectiveness in terms of revision, must be reexamined in light of the findings from student interviews, which suggested students drastically revised when they received end comment responses to their essays. While students indicated they were less comfortable making changes to their writing using end comment responses from their instructors because instructor expectations were not always clear, they also seemed to spend more time on revising and rewriting texts that received end comments. Since the end comment does make the process of “finding and fixing mistakes” more difficult for students, some indicated they spent more time trying to interpret these comments than they did revising their own essays. [94] Finally, these findings suggest the stage of writing and the sophistication of the text may help determine which commenting format is most effective. For example, when students are early on in the writing process, an end comment might serve them better to help them focus on overall perceptions of their writing. In a piece further along, in-text comments might be more useful because they are often more specific and anchored to the text. If the text will likely change, perhaps in-text comments are less necessary. Furthermore, students who are new to academic writing might benefit from the explicitness and directiveness of in-text comments because there may be less confusion. More advanced writers might be more capable of applying feedback from an end comment to their writing. |
5. Suggestions for Tutor Education[95] The findings from this study do not suggest that either in-text or end comment format should be practiced exclusively or consistently by all tutors with all student writers. This study also does not suggest that either format results in a higher quality of student writing or revision. Instead, this study shows that commenting format does result in different kinds of comments, tutor personas, and student perceptions. [96] Thus, this study implies there is great value in exploring different commenting formats as part of professional development for online writing tutors. Rather than imposing one particular structure for online tutoring, writing center administrators should explore a range of formats and have tutors practice using them. For example, tutors might take a paragraph or short sample of student writing and practice responding to it first with in-text commenting only and then with an end comment. Sharing the range of tutor responses as a group would be helpful, as would doing some basic analysis of comment number, length, and type. Or, tutors might be asked to respond to a piece of student writing in any way they chose as if they were participating in an actual online writing conference. Encouraging tutors to practice approaches to response that they may not automatically choose could be beneficial for them as well as for student writers. Finally, tutors might analyze sample online tutoring sessions to get a sense of different formatting styles [insert hyperlink for “Example Asynchronous Tutoring Responses” that could be used in professional development with online tutoring staff attached as four documents: Lindsey’s End Comment, Lindsey’s In-text, Scott’s End Comment, Scott’s In-text]. [97] Encouraging tutors to try out different approaches and then reflect on how those approaches impact their tutoring is an important part of helping them develop their tutor personas and strategies for responding to student writing. As mentioned earlier in the case of Lindsey, some tutors may be writing responses without an awareness of their options, and may eventually find out they have a preference for a different approach. Multiple tutors changed their minds about their preferred approaches to online tutoring by either trying out a new approach or discussing their process through interviews. Thus, the time for reflection and conversation, along with practice, is necessary as online writing tutors develop. |
6. Limitations and Future Studies[98] There are limitations to this study that open up questions for future research, the most significant of which is the small size of the study and single university context in which it took place. Additionally, coding and analyzing additional responses written by each tutor would provide a stronger sample for studying trends across tutors and individually. This study also would have benefited from interviewing students after multiple online writing conferences in different commenting formats with the same tutor over time. [99] One obvious drawback of this study is the artificiality of asking tutors to comment using either in-text or end comment format. Nearly all of the tutors interviewed preferred some kind of hybrid format, though they still usually favored and/or constructed the majority of their comments using one format or the other. However, the purpose of this study was to shed light on how these two different formats lend themselves to different kinds of comments. This change in content was especially clear in terms of directiveness (rewrites, questions, suggestions), explanations, and praise. If we understand how format affects the content of our comments, then perhaps we can make more conscious decisions about the ways we construct our feedback to student writers. [100] More research is needed on the ways students read and understand online writing conferences and on how graduate students and multilingual writers use and understand their responses in each of the formats. In addition, research into how different genres of writing may be better suited for different commenting formats is worth exploring. |
7. ReferencesBean, John. (2011). Engaging ideas: the professor's guide to integrating writing, critical thinking, and active learning in the classroom. San Francisco: Jossey-Bass. Calhoon-Dillahunt, Carolyn & Forrest, Dodie. (2013). Conversing in marginal spaces: Developmental writers’ responses to teacher comments. TETYC, 230-247. Conors, Robert Lunsford, Andrea. (1993). Teachers’ rhetorical comments on student papers. College Composition and Communication, 44, 200-223. Cox, Stephanie, Black, Jennifer, Heney, Jill, & Keith, Melissa. (2015). Promoting teacher presence: Strategies for effective and efficient feedback to student writing online. Teaching English in the Two-Year College, 42(4), 376-391. Denny, Harry. (2014). Writing centers research project survey. Retrieved from https://owl.english.purdue.edu/research/survey. Denton, Kathryn. (2017). Beyond the lore: A case for asynchronous online tutoring research. The Writing Center Journal, 36(2), 175-203. Dixon, Zachary and Moxley, Joe. (2013). Everything is illuminated: What big data can tell us about teacher commentary. Assessing Writing, 18, 241-256. Dweck, Carol. (2007) The perils and promises of praise. Educational Leadership, 65(2), 34-49. Ene, Estela. & Upton, Thomas (2014). Learner uptake of teacher electronic feedback in ESL composition. System, 46, 80-95. Gladstein, Jill & Fralix, Brandon. (2014). National census of writing. Retrieved from https://writingcensus.swarthmore.edu/survey/4?question_name=s4wc8#results. Hallman Martini, Rebecca & Hewett, Beth. (2018). Teaching tutors not to tutor themselves: Personality in online writing sessions. ROLE, 1(1). Heller, Dana. (1989). Silencing the soundtrack: An alternative to marginal comments. College Composition and Communication, 40(2), 210-215. Hewett, Beth. (2004-2005). Asynchronous online instructional commentary: A study of student revision." Readerly/Writerly Texts: Essays in Literary, Composition, and Pedagogical Theory. (Double Issue), 11 & 12(1 & 2), 47-67. Hewett, Beth. (2015). Reading to learn and writing to teach: Literacy Strategies for online writing instruction. Boston: Bedford/St. Martins Hewett, Beth. (2015). The online writing conference: A guide for teachers and tutors. Boston, MA: Macmillan. Retrieved from https://community.macmillan.com/docs/DOC-1474 Lunsford, Andrea & Lunsford, Karen. (2008). ‘Mistakes are a fact of life’: A national comparative study. College Composition and Communication, 59(4), 781-806. Martin, Robin. (2015). Rhetoric of teacher comments on student writing. Young Scholars in Writing, 8, 16-29. Martinez, Diane & Olsen, Leslie. (2015). Online writing labs. In B. Hewett & K. DePew (Eds.), Foundational practices of online writing instruction. Fort Collins: The WAC Clearinghouse and Parlor Press. Retrieved from https://wac.colostate.edu/books/owi/. MacAndrew, Donald & Reigstad, Thomas. (2001). Tutoring writing: A practical guide for conferences. Portsmouth: Boyton/Cook. Neaderhiser, Stephen & Wolfe, Joanna. 2009. Between technological endorsement and resistance: the state of online writing centers. The Writing Center Journal, 29(1), 49-77. Severino, Carol & Prim, Shih-Ni. (2015). Word choice errors in Chinese students’ English writing and how online writing center tutors respond to them. The Writing Center Journal, 34(2), 115-143. Simpson, Shelah. 2017. Student perceptions of online writing center designs for fully online programs. Dissertation. Smith, Summer. (1997). The genre of the end comment: Conventions in teacher responses to student writing. College Composition and Communication, 48(2), 249-268. Treglia, Maria. (2009). Teacher-written commentary in college writing composition: How does it impact student revision? Composition Studies, 37(1), 67-86. Wojahn, Patricia, Neuwirth, Christine & Bullock, Barbara. (1998). Effects of interfaces for annotation on communication in a collaborative task. CHI, 98, 18-23. |