Over the course of the last four week in the “Using Data to Make Data-Driven Instructional Decisions” series, we have outlined how to collect data, clean/organize data, and conduct univariate and multivariate statistical analysis on the data to transform it into newfound knowledge that can be used to make a decision. While this sounds like an extensive process, in practice, it is not extensive as it seems as seen in Part 1-3 of this blog series. It’s completely doable process that every K-12 educator and administrator can do in their capacity as an educator. As we progress through this process, it is now time to take that transformed data to use to make a decision.
Decision-Making Framework
Now, we are focusing on what do we do with the newfound knowledge we have been able to capture after we have transformed our data using statistics. Ultimately, there are many different avenues we can use the knowledge to make instructional decisions to help our students. However, to use this data effectively, there is a six step decision-making framework you can use to use the data to drive instructional decisions within a classroom, school site, or district. As a result, this six-step decision-making framework includes the following steps:
- Identifying the Problem by Analyzing Collected and Cleaned Student Data
- Involvement of Stakeholders, if applicable/needed
- Transforming the Data Using Statistics
- Summarize the Statistical Findings, Prioritize Specific Findings, and then Take Knowledge to Solve Problem
- Develop a Strategic Action Plan and Include the New Knowledge
- Monitor the Action Plan
By following the six-step decision-making framework, we can use data we collect and transform to solve many of the instructional challenges teachers and administrators face within classrooms and school sites because this new knowledge derived from the data can be used to connect our problems we encounter with solutions. When we first think about Action Plans, they seem to be very detailed. However, when it comes down to it, an Action Plan using this six-step decision-making framework does not have to be a challenge nor take long to create.
The goal of Part 4 of this blog series is to show you how you can use this six-step decision-making process to create instructional Action Plans to use the data you have collected and transformed as knowledge to solve instructional challenges in your classroom, school, or district.
Action Plans
Within an Action Plan, it has five components: 1) A problem with a baseline, 2) the Plan, 3) the goal(s) that can be monitored, 4) monitoring period(s) & data collection, and 5) conclusion. These five components do not need to be incredibly detailed. We want to ensure Action Plans are direct and easy to follow. Below is an example of an Action Plan a teacher can focus on for their students reading comprehension for the entire school year.
| Problem: The incoming fourth grade class of 200 students has 120 students below the Lexile reading level of 500L (4.0 GE). Of those students, 80 of the students are scoring under 50% on questions asking them to find key details. |
| The Plan: Improve the Lexile level of the 200 forth grade class by 175L (1.25 GE) and improve students answering reading comprehension questions that require students to find the key details by 25%. We fill focus on improving Lexile levels of all students by focusing our instructional strategies focused on helping students annotate, paraphrase, and share key details of the text collaborative and during independent practice. |
| Goal(s): 1) Increase Lexile level of 200 fourth graders by 175L 2) Improve student reading comprehension questions that require students to find the key details by 25%. |
| Monitoring Period & Data Collection How is Data Collected?: Data on goals is collected through MobyMax that is then exported to the an Excel or Sheets spreadsheet for data analysis. Monitoring: Data will be collected twice throughout the school year at the end of the first semester and before spring break. 1. Semester 1 Data Collection Summary: December 2020 We found an improvement in average scores by 95L. When we saw how students were doing on reading comprehension questions when students used annotations on the passage, a regression was ran and it predicted that when students annotated the passage before answering key detailed questions, they would get key detailed questions correct by 25% more than students who did not annotate. 2. Semester 2 Data Collection Summary: April 2021 |
| Conclusion: During the first collection of data, we found that annotating the passage students were working on demonstrated a higher predicted level of accuracy of students answer reading comprehension questions measuring their ability to find key details in the passage. |
This can be an Action Plan that will need to be completed throughout the school year. It does not take an extensive amount of time to set it up and to begin collecting, cleaning, and analyzing data during the monitoring period. Ultimately, Action Plans can also be used for the short term. Let’s see what that could look like with formative assessment data.
| Problem: On a recent Algebra formative assessment, students scored 75% average as a class on problems related to using the distributive property. However, students scored 50% average as a class on problems that require three or more steps to solve and require students to utilize PEMDAS to solve equations. |
| The Plan: Provide instruction based on using PEMDAS to solve equations. By the end of the week on a formative assessment, students will be scoring above 70% on solving equations that require three or more steps to solve and utilize PEMDAS. Review PEMDAS and provide multiple activities to help students see how PEMDAS works after the distributive property is utilized to simplify equations. |
| Goals: Improve the overall class average to 70% or higher in one week on solving equations that require three or more steps as well as require how to use PEMDAS to solve. |
| Monitoring Period: In one week data will be collected on students solving equations that require 3 or more steps and know-how of PEMDAS. |
| Conclusion: After a week of implementing the Action Plan, the class average increased 20% to 80% overall average on solving equations requiring 3 or more steps and PEMDAS. |
As we can see with the Action Plan above, it is very simple and be written in the matter of minutes. These types of Action Plans can be built using this template provided above for all age levels. Now, let’s focus on more ways to utilize the transformed data and the knowledge we gain from it.
Plugging in Knowledge into Instruction
When we transform student data using univariate and multivariate statistics, we can do many things in the classroom. First, for univariate statistics, assessment applications like Google Forms, GoFormative, and b.Socrativ provide teachers with a dashboard of information for them to view how well students performed. Data visualizations are already provided to teachers once the data has been collected by the students and automatically cleaned. What this does is provide a read out of the classes performance as well as the individual student performance as well as class and student performance for each question the assessment. Teachers can immediately use these data visualizations to see where gaps in student learning took place and then can quickly provide additional instruction and interventions to selected students. For both formative and summative assessments, this can take place.
Beyond data visualization, this analysis can take place on a spreadsheet. All types of edtech tools provide the opportunity to export data onto a spreadsheet as discussed in Part 1 of this blog series. As a result, teachers and school leaders can conduct the same univariate analysis as the automatic data visualizations. However, using many of the statistical formulas provided in Part 3 of this blog series, teachers can conduct a far more extensive analysis, which includes multivariate statistical analysis.
Within a multivariate statistical analysis, we can take grade equivalent Lexile reading levels of students and run a correlation with the their overall scores on a recent exam measuring their ability to synthesize key details from a story to create inferences and conclusions. What we can evaluate when the correlation is computed is to determine whether there is a statistically significant relationship between the students grade equivalent reading level and their overall performance on the assessment. The results of the correlation may tell us there may be a positive relationship between the students reading levels and the exam scores. However, the results could also state there is a negative relationship between reading levels and the exam score. Each scenario provides teachers with two important pieces of information. First, if there was a positive relationship between reading levels and exam scores, then there is a possibility the exam matched the students reading ability and their performance. This means the exam was aligned with their current reading skills. On the other hand, if a negative relationship existed, one possibility is that the exam did not match the students reading levels, which could mean the exam was not a good measure of their current reading level or their current reading level did not match the difficulty or types of skills measured on the exam. Using this information is helpful in determining whether the reading levels of students relates with the difficulty of the exam. To dig deeper, we could also run the same correlations for different segments of the class as well as the equation types presented on the exam. This information can tell us how students with higher or lower reading levels may relate to their higher or lower score on the exam. In addition, we can also see if the reading level of students had any relation with their performance on types of inferential and conclusion forming reading comprehension questions posed on the exam.
Note: When we discuss multivariate statistics, I will always say possibility and statistical significance because we can never prove causation. Even with a very large sample size, there will always be the possibility of intervening variables or the results of the calculated statistic are not statistically significant (which means the conversion rates between a given variation and the baseline is due to random chance; read more about the null hypothesis here). Ultimately, for each multivariate statistic calculation the p-value, which represents statistical significance must be less than or equal to .05 for the calculation to be statistically significant.
Other Considerations & Conclusion
What we discussed today is the tip of the iceberg in regards to what teachers and school leaders can do with the knowledge they have been able to gain from transforming the data they have collected and cleaned using statistics to make a decision. You have seen the power of Action Plans as well as how teachers can use the data they have transformed into powerful instructional knowledge to help them try and determine how their students are doing and what to do about it to bridge gaps in learning and instruction.
Beyond the classroom settings, what we have done throughout this blog series can be conducted at the grade level, school level, and district level. It can even get more complex by looking at specific demographics, question types, and standards. It can be increasingly complex. Currently in K-12 schools, there is an astronomical amount of data that can be utilized to make data-driven decisions to improve instructional and student outcomes. We, as educators, must collect that data do something with it. Too much of this data is wasted because it is not looked at further.
Hopefully from what you have seen in this blog series demonstrates there is a set of skills required to become data literate. At the same time, I hope you have seen how powerful transformed data can be to gain new insights on your students and instruction that you cannot see without collecting and analyzing the data. It is revolutionary. Think if all K-12 educators and school leaders had the data literacy skills to be able to use data on a daily basis to make efficient and effective decisions? What outcomes could there be for all students as well as the system of K-12 education?
Ultimately, there is much work to be done. This blog series was a preview of the curriculum required to for educators to become data literate, which is the ability to collect data, compile/clean data, conduct statistical analysis on the data, and to use the new knowledge gained from data to make effective decisions. We have seen major gains in becoming technologically literate. Now, we must become data literate to revolutionize how we teach and use our educational technology. It’s time all teacher and administrative preparation programs and districts to focus on curriculum to help build this capacity. This will be the new frontier as we progress through the next few years. It’s time for all educators to become data literate!
Thank you for reading this blog series on Using Data to Make Data-Driven Decisions. To review all of the previous posts, they are hyperlinked below. In addition, if you have any comments, please comment below or interact with me on Twitter @mattrhoads1990. I look forward to discussing with you these topics and concepts.
Using Data to Make Data-Driven Decisions Blog Series Parts
Part 1: Introduction & Collecting and Exporting Student Data from Edtech Tools
Navigating the Present and Future of Education 