Background
In February 2020 representatives from the Digital Transformation division of The College Board approached the founders of essaypop to discuss conducting an efficacy study on the essaypop writing platform. The study was designed and administered by research scientists at The College Board's Learning Engineering Center. Broadly, the study aimed to answer the following question:
Is there an improvement in outcomes using essaypop’s writing frame system above and beyond the current pedological practice embedded within The College Board’s Pre-AP paragraph scaffolding tool called the Single Paragraph Outline (SPO)?
The SPO is a tool The College Board has been using for some time in conjunction with their middle school and Pre-AP writing curricula; the researchers wanted to see if a wireframe approach to writing like that offered by essaypop could measurably improve student results.
The research team determined that the study would measure three writing/learning skills: rhetorical awareness, original paragraph writing, and paragraph revision. These tasks represent real, business-as-usual activities that students encounter regularly in the classroom and on high-stakes exams.
Note: the rhetorical awareness component measures how well students are able to identify paragraph elements like a thesis statement, evidence, and analysis in previously written academic paragraphs.
At pretest and at posttest, students completed standards-based, response-to-literature paragraph writing prompts that were counterbalanced and randomly assigned. At posttest, students also revised their pre-test writing response.
Expert raters scored each response and were blind to condition and type. In other words, they did not know which students were in the treatment (essaypop) condition and which were in the control condition. They were also unaware of which writing samples were original responses and which responses were revisions.
The holistic scoring rubric used by these raters consisted of identifying proficiency on the following paragraph elements: Thesis sentence, research/evidence, interpretation/elaboration, closing sentence, organization, and use of conventions.
The participating students in both groups had recently completed 8th grade and were moving into 9th grade. At pretest, there were no significant scoring differences between the groups across all comparisons.
The Results
The rhetorical awareness posttest results reflected a small but meaningful difference in favor of the treatment (essaypop) group. Students were better able to identify the rhetorical elements within previously written academic paragraphs after using the essaypop platform several times.
The original writing posttest results showed a significant difference in favor of the treatment (essaypop) group. Students who had used essaypop several times during the study wrote more proficient, original prompt-based paragraphs on all components measured, across the board.
The paragraph revision posttest results were the most impressive of all, with the treatment (essaypop) group composing vastly more proficient academic paragraphs when they were able to revise their work using the essaypop writing platform.
Student Reactions to the Experience
The students in the study were also asked to respond to open-ended questions regarding their experiences using the essaypop writing platform. Here are some of their responses:
“For me, the most helpful and valuable part of the study was using the essaypop writing tool. It helped me better organize my writing and let me see the function of each part of the paragraph in a visual way. I think this made my writing more cohesive and logical, giving it better flow in general.”
“The most helpful activities were when I got to write in the boxes to help me organize my writing. Now, whenever I write a response to something I will always have those boxes ingrained in my mind.”
“My parents saw a big difference in my writing. Because of those boxes, I am now writing down to the point and I don't drift from subject to subject anymore.”
In a post-study survey, participating students indicated the following:
86.5 % either somewhat or strongly agreed that writing in frames helps them better organize their thoughts.
74.3% of students either somewhat or strongly agreed the color-coded writing frames help them stay better organized
81.1% of students indicated that they referred to the scaffolded riding models and the essaypop sidebar. Of those, 96.7% found these models to be helpful.
Summary
The results of The College Board study of the essaypop writing platform are clearly encouraging and reflect what more than 100,000 users across the nation have been expressing anecdotally since it launched in 2020 - essaypop works. The frame writing method resonates with students and allows them to internalize the elements of academic writing. Moreover, students transfer this knowledge when they write without the assistance of essaypop.
The College Board and essaypop plan to partner on additional studies this year. Specifically, the teams are eager to study the effect of peer-to-peer feedback on student writing performance. Essaypop’s collaborative and interactive Hive environment is uniquely set up to accommodate these observations, and both teams anticipate a fruitful study. In addition to taking a deep dive into essaypop’s efficacy, the teams also plan to study essaypop’s ability to drive student engagement. They are interested in essaypop’s potential as a social/emotional tool that can drive an overall love of writing. Needless to say, The College Board and essaypop still have a lot of research to accomplish.