Data Analysis
Data Collections Method
​
The following methods were selected to measure fifteen students’ progress and growth throughout the writing process. Though more students were in the class, I selected to only measure those students who completed the pre-test before strategies had begun being implemented to improve writing.
T-Test
Writing Process Checkpoints
The writing process requires specifically implemented steps to be done over a lengthy time frame. Student data was collected at each point in the writing process to check for completion. These checkpoints are displayed in the visual to the right. Each checkpoint required students to turn in the assigned task. These tasks ranged from a completed graphic organizer to worksheets to written products. The turned in assignments were then evaluated for student understanding and completion. These checkpoints allowed me to monitor student progress and make decisions about how to structure instruction moving forward. As an example, when students began turning in their RACES worksheets, it became apparent they were struggling with the "Explain" portion of the process. I was able to begin the following day's lesson with a review of what it means to explain cited evidence.
​
The method of using checkpoints was selected to ensure students were truly navigating the process and not simply skipping to the end. The strategies implemented can only be measured for effectiveness if students are completing each individual part of the process. The checkpoints measured both completion and if students met the deadline for each portion of the writing process. These checkpoints allowed for further analysis of specific student growth.
​
In order to measure improvement, the necessity of a benchmark was needed. The purpose of the research was to see if teaching the writing process improved student writing not only across the specific project, but also if it transferred to other writing assessments such as timed writes. The pre-test given to students was an objective timed write question asking students to write about their favorite foods. I wanted to assess an objective concept that did not act as a content obstacle and rather simply showcased a student’s writing ability. I chose a similar objective concept for students’ post-test in asking that they write about a favorite place. The two similar tests allowed for an accurate measure of progress due to the identical format of having students write on a simple subject over the course of one class period. The t-test also indicated areas where students needed to grow throughout the process and where they still needed to grow in future instruction.
Students were evaluated using “The ACT Writing Test Scoring Rubric.” The rubric broke down evaluation into four categories: Ideas and Analysis, Development and Support, Organization, and Language Use. Each category is allocated six points to be given based on the criteria provided. However, each final score was given out of twelve points. I used this format in evaluating my students’ pre and post tests. I converted the rubric as seen in the rationale section into a checklist, which can be seen on the left. Students used this checklist for both their "Peer Review Bingo" and self-reflection.
Interest Brainstorm
Questions
Outline
RACES
Rough
Draft
Peer
Review
Self
Reflection
Final
Draft
Observations
​
Throughout the course of the project, I made observations that were measured through my own perceptions and less based on numerical values. Though, the use of attendance numbers were used to gain an understanding of the effect of being in class versus being absent. Additionally, the data collected for checkpoints allowed me to see if my students were more likely to complete a checkpoint on time if class work time was more structured rather than an independent work day. I observed the way in which students used class time during less structured work days. I also observed length in paragraphs from the pre-test to the post-test. Finally, I made observations of students’ physical time spent in class and how that relates to the completion of tasks. These observations were used to track student progress such as whether they were understanding the tasks asked of them. The observations also helped me to make decisions about whether to adjust the time frame schedule to better meet the needs of my students.
Analysis
T-Test
In reviewing the final data for both tests, only one student’s score went down on the second test when compared to the results of the first test. Otherwise, every other student saw at least some growth. The greatest growth came from those who scored low on the initial test, leading me to believe that the strategies implemented throughout research had a larger impact on those students whose necessity for writing instruction was much greater. That is to say that students who performed poorly on the pre-test were able to close the gap through the application of the strategies provided.
For the student whose score went down, they had already performed higher than most of their peers on the pre-test. In looking at the general theme of less growth among those students who had initially scored higher on the pre-test, the decline could be explained more as a day-to-day fluctuation. Though the prompt was designed to remove subjective biases, from playing too large of a role in the writing evaluation, this student may have had a greater connection to the first topic than the second one. Additionally, the student may not have been as focused on this day compared to the day they completed the pre-test. Most importantly to note, the student suffered a family tragedy between the time when the first test was given and when the second test was given. It is impossible to know the impact that this had on the performance, but it is a factor that cannot be ignored.
Though the scores were not wholly in a range that would be considered proficient according to the rubric, that was never the intention of the research. In evaluating the data, it is safe to state that the implemented writing strategies had a positive impact on student growth from the pre-test to the post-test. It can be concluded that the skills developed as a result of writing process strategies have transferable traits that can be applied to other forms of writing.
Deadlines Met
20%
of Students
100% Met
75% Met
60%
of Students
25% Met
67%
of Students
5.23
Post-Test Average
6.87
Pre-Test Average
Writing Process Checkpoints
In order to measure if a student was making gains as a result of the writing process, it was necessary to know if they were engaging with each strategy implemented as part of the writing process. Students were evaluated after each strategy was implemented on two criteria: completion of the strategy and whether they met the deadline for implementation. The writing process relies on each step being done and within a specific time frame in order to be productive. By collecting data at these checkpoints, I was able to measure both. As the project began, I saw close to every student meet the deadline for the first strategy. Thirteen of the fifteen students being included in the research had completed the brainstorming strategy by the deadline. I saw this number drop to ten on the second strategy implemented, and then it continued to decrease the further into the process we had gone. By the time that the self-reflection strategy was implemented, only four of the fifteen had met the deadline.
At first, I was disappointed in these results. I felt that students had not met my expectations. Although, when I looked a little deeper, it was apparent that the decrease from one strategy to the next showed that they were taking the time to actually complete each strategy. It would have been easy for students to completely skip portions of the process in order to get caught up, but they were not doing this aside from a few areas where a student may have skipped one strategy or only partially completed the task. Students began to stall a little bit when it came to completing their RACES sheets. The task was more involved than the previous strategies and it required students to work independently as a result of applying their specific research. However, students were able to flourish on their rough draft as a result of completing their RACES sheets. Through using the RACES strategy, students had already successfully built their body paragraphs, making their rough drafts easy to produce.
Just because students appeared to be staying true to the process does not completely forgive the missing of deadlines. Timelines are part of the process and they are implemented to give an appropriate amount of time to reflect and make revisions. Some students needed more structure in order to meet deadlines. As a teacher, I should have taken this more into account. The data reveals that my students who met deadlines were more successful than those that did not. As referenced in the chart on the left, those students who met the deadlines received higher scores on their final drafts. These students had more time for revising and they also were able to complete more peer reviews. However, the data also indicates that just because deadlines were not met, students were not destined for low scores. Two students missed every deadline, but still scored higher than their peers. They still did not score as high as those students who met every deadline. The evidence shows that students who truly engaged with the process as outlined by the action plan, developed the highest quality essays.
Observations
One of the first aspects of the project that I noticed was that students were much more likely to meet deadlines earlier on in the process due to more structured work time. On strategies done during the pre-writing stage such as brainstorming and outlining, we worked on these strategies more as an entire class. I introduced the strategy, gave an example, and then gave students a few minutes to apply the strategy to their own interests or topic. I then brought everyone’s attention back to the whole class example. I built on the example before releasing them to apply the new portion of the strategy to their own work. This process repeated itself until the strategy had been completely implemented. Most students managed to complete their own work before the deadline as a result of this method.
As the process continued, there were fewer opportunities for this sort of instruction due to a number of reasons. First, the nature of the work was going to look slightly different from student to student. Many of the applicable aspects of the strategies had already been implemented and now students just needed to work. Second, more and more students were falling behind due to either absences or neglect. This was the portion of the project where certain students were expected to work on their own, while my co-teacher and myself helped those students who were either behind or struggled with specific concepts. I quickly observed that this free work time translated to less productivity from a handful of students.
As stated above, absences created an issue for working through the process. It quickly became clear that I could not move along in the process with everyone when I had a fourth of the class behind due to missed class time. Some of the students were simply not in class, while others were being removed for behavioral, medical, or other reasons. This was more impactful to my ability to implement strategies than I had originally anticipated. At times, it forced me to adjust my schedule slightly.
The final area of observation data was my visual survey of growth between the pre-test and the post-test. I noticed specifically two things. First, students’ paragraph length was longer as a whole on the post-test. Second, some students on the post-test had planned out what they were going to write prior to writing their essay. This was not observed on the pre-test. These two areas indicated growth beyond the evaluated criteria of the rubric. Though these areas of growth are not quantitatively evaluated, they showed an understanding of the strategies implemented, which lead to quantitative growth.
Triangulation
T-Test
Checkpoints
Observations
The success of growth on the T-Test was directly correlated to whether students could apply some of the writing process strategies to on-demand timed writing. Through simple observations such as the use of an outline and longer writing length, it was easy to see the impact that the writing process had on student growth from the pre-test to the post-test. Just simply noticing that most students showed quantitative growth between the tests showed that at the very least, the emphasis placed on writing over the course of six weeks helped students to improve their ability to write.
The data collected as part of the checkpoint process indicated that students were taking the time to engage with each strategy rather than skipping to the final product. These strategies allowed students to transfer skills to other forms of writing. As indicated above, the use of the outline and RACES paragraph format were implemented strategies that students were able to use on the post-test. Had they skipped ahead to the final draft of their essay and not familiarized themselves with these strategies, they would not have been able to apply them on the post-test.
Additionally, the checkpoints allowed me to further support my observation that students were more productive when they had structured strategies. The number of completed assignments that met the deadline were not only earlier in the process, but they also were done as part of a more structured whole class instruction. The later assignments with fewer on-time completions were subject to less structure.
Observational data collected on absenteeism ultimately affected my checkpoint data collection. I needed to adjust a few deadlines due to students missing. It also affected the way in which students implemented strategies. In order to help catch some students up, later checkpoints were relegated to more independent work and the latter part of gradual release. It was also apparent that those students who were missing many of the deadlines were also those missing the most class time.
In the case of student growth between the two T-Tests, absenteeism may have played a role in the one case where the student’s score went down on the second test. I would not have been able to make this deduction if I had not observed his absence and the reasoning behind that absence.
Impact
The data clearly showed a positive correlation between writing process strategies and application specifically to on-demand timed writing. Students were positively impacted as a result of the research project. Whether the impact will remain intact after some time has passed remains to be seen. It is likely that continued practice would be necessary to continue seeing growth.
The data also revealed that time plays a pivotal role in the ability to successfully implement all of the strategies of the writing process. When students were not in class, they were unable to meet the strategy checkpoint deadlines. Additionally, the way in which time was being spent with students also seemingly had an impact on student ability to successfully complete the strategies within the time frame. Meaning that when I worked through each strategy incrementally with the class as a whole, more students completed the strategy before the deadline. When I asked them to work on strategies independently, fewer completed the strategy before the deadline.
One of the major questions that arose as a result of examining the data is how time can be used more efficiently to better meet the needs of students. I felt as though I could have structured class time to better support those students who chose not to work when given class time. Given the circumstances of a tight time frame made the solution to this question much more difficult. It is certainly a question that should be answered in order to improve student impact. Additionally, a question of whether more efficient use of time would have lead to larger growth between the pre and post tests arose as a result of reviewing the data. I think it is reasonable to think it may have had an impact.