Monday, June 11, 2012

More Than a Feeling: In Search of Hard Evidence

After my last post, I was challenged to think about convincing university faculty to apply Flipped Teaching. This is something I plan to do because I have seen both first-hand in my grad courses and also as a teacher that flipping a class has many positive effects. Unfortunately, these effects are just a "feeling" for me and I don't have any hard evidence. So I went on a journey to find some. 
I searched online and found which has a collection of data about flipped teaching. After parsing through the data, I found that there was no data that showed student learning outcomes.
I then found this post on the University of Wisconsin-Madison website called How 'Flipping' the Classroom Can Improve the Traditional LectureThe article originally comes from the Chronicle of Higher Education. Reading through this article I was unable to find any solid numbers - which is too bad because I am a bit of a mathematician - but I did find some valuable information.
"Research by Ms. Rhea and two colleagues suggests that Michigan's teaching methods have led to greater gains in conceptual understanding. The techniques have been lauded by the Association of American Universities, among others.
In 2008, Michigan gave concept inventories to students before they started calculus and after they finished, and calculated the difference relative to the maximum gain they could have made. Students in Michigan's flipped courses showed gains at about twice the rate of those in traditional lectures at other institutions who took the same inventories.
The students at Michigan who fared worst—a group of 12 who were at risk of failing the course—showed the same gain as those who demonstrated the largest increase in understanding from traditional lectures elsewhere."
I looked for Ms. Rhae's actual research, but was unable to find it. The results stated here are quite impressive. 
  • Gains in conceptual understanding at twice the rate 
  • The worst students showed as good of gains as the largest increase in traditional lectures
If you read the article, you will notice that there are some challenges with measuring student gains. Students are not likely to show much gains in basic knowledge, but it is when you start to look at their deeper understanding of concepts that you will start to see a difference. In order to truly measure the gains you have to use a high quality assessment tool.

For faculty there are downsides to using a flipped model. The same article states that:

"Harvard colleagues have tried flipping, [...], but few have stuck with it. It demands that faculty members be good at answering students' questions on the spot, even when their misconceptions are not yet clear because they are still processing the information.

It can also be very labor-intensive for faculty members who do not have teaching support [...] if it requires a professor to read questions that students submit before class (which is characteristic of just-in-time teaching)...

...But her chief critique is based on the intensity of students' responses. The average score on a student evaluation of a flipped course is about half what the same professor gets when using the traditional lecture."

The downsides for faculty are:
  • forces the professor to be much more flexible and adaptable 
  • requires extra time before classes   
  • student evaluation of faculty is often lower - likely due to the students being uncomfortable with something "different"
These are going to be definite roadblocks when convincing faculty to implement such a model. 
In response, one of the proponents of flipped teaching commented on the student evaluations by saying:
"Liking the class is ultimately beside the point, Mr. Mazur says. He says his results from using peer instruction show that [...] nonmajors who take his class outperform physics majors who learn in traditional lectures.
"You want students to like class, but that's not the goal of education," Mr. Mazur says. "I could give them foot massages and they'd like it."

I continued to search and found some more soft data to support flipped teaching. A press release posted May 15, 2012 on The Business Journal's website called Sophia Survey Finds Student Grades Improve When Teachers "Flip" Their Classroom. In this, I found that "More than 85 percent of teachers who have used the "flipped" classroom model said they saw improvement in student grades, according to a recent study of more than 400 teachers".
From the data I have found, I would say that there are indications that Flipped Teaching does improve student learning. Opponents may argue that the improvement may not actually be from the videos or out-of-class portion, but rather the improvements could be from the new use of class time. Michelle Naidu argued this in my video interview Flipped Pessimism: What are the opponents saying?
At the end of the day, convincing faculty to adopt this model will be a slow process. It will be a tremendous amount of work and a major shift in philosophyIt is imperative that proponents of the method begin collecting really strong quantitative data to show the gains. Without hard evidence, the positives of Flipped Teaching are just a feeling.


  1. Go get 'em, Ryan! I challenged you, and you rose to the challenge. What a wonderful moment-- I hope for both of us. I think you'll continue to find the same claim (great gains) without the kind of evidence you seek. And that doesn't mean the gains aren't there. We have a collision of epistemologies, IMHO.

    And you clearly point out why faculty might not try, or stay with, flipped approaches. Yes. Your explanations make sense to me. But you will also definitely find faculty who are willing to make the kinds of investments necessary to make this approach work (yes, I hope in ETAD).

    But in the end, it really does come down to a shift in culture: where teaching and learning really matters. Where the sacred bovines of research and grant $$ doesn't trump what happens in our classroom and for our students. I know it is a grand hope, but I think it's out there.

  2. I have been wanting to search for actual data and you gave me the impetus to do it. Thanks for the push!

    I hope that I do find faculty that are willing to make these investments in their teaching. When they look at the big picture it becomes clear that a better education and learning experience will lead to better researchers in the future. And at the end of the day, everybody wants to see great gains in his/her field.

    Thanks again for the comments.

  3. Hi Ryan,
    I just came across your blog and am excited to read some of your old posts. I put together to connect flipping teachers with each other, so it's not a "data" site in that form of the word. On my actual blog at, I do have some rough data posted seeing student improvement as compared to traditional ( If you look at the flipped class ning (, there are also some discussion threads of research/data.

    Hope that helps in your continued search!

    1. Thanks for stopping by, Crystal. I appreciate you pointing me to your data. I just quickly looked at it, and it looks like you have done a good analysis. I will definitely take a longer look at it later!

      Thank you and enjoy ISTE!