A new way to get students to read the text

Ask any instructor and she'll tell you that one of the biggest challenges is making sure that students read the textbook. I've tried several approaches to this problem, from begging and pleading (no effect) to requiring students to outline the text (seemingly effective, but very time-consuming). In my flipped class (described elsewhere in this blog) I rely on the in-class activities to force students to engage with the material in the text, and I think this approach works pretty well. However, the jigsaw activities we do in class really only require students to engage with the material their learning groups are responsible for (just a small portion of any chapter), and they get the rest of the material from their peers in the teaching groups. So when my Pearson book rep told me about a new offering for my textbook that would require more student engagement with the text, I was ready to listen.

She told me about a new service called REVEL. REVEL really is a number of related things. First, the REVEL service provides students with an electronic version of the entire text, from pictures, to bold key words, to chapter summaries, everything that appears in the paper version appears in the e-version, plus there are interactive elements like brief activities and videos. The etext is in the cloud and is accessible on most platforms, from PCs to tablets to smartphones. So students can access the etext anywhere they have internet access including over wifi and cell service. Students also get an unbound 3-hole-punch version of the text identical to the traditional bound version. So, in sum, they get the same text they used to get, but now they get it electronically and in paper form, with added electronic features (including narration of the text). This alone might have tempted me to adopt this version, just for the increased access and portability. But there's much more to REVEL.

The most useful aspect of REVEL are the assignments that Pearson has built into the etext. As an instructor I get an account on REVEL that allows me to select from a list of assignments for each chapter in the etext. I can set up a due date for each assignment and REVEL will track each student's performance and progress on the assignments. The assignments themselves range from short quizzes covering small sections of the chapters to shared reflections on the material. The short quizzes can be taken multiple times, although correct answers diminish in value with each attempt. I really like this mastery approach. There are also quizzes that cover the entire chapter. I can select whatever type of assignment to use, covering any part of a chapter or all of it. The semester hasn't started yet so I don't know how well the analytics work, but it looks very helpful from the instructional videos. 

Setting up the assignment on Pearson's REVEL website does take some time and effort, and at this point REVEL doesn't interface with our LMS (Blackboard) so I'll have to manually import the results into the LMS and my gradebook, but that looks relatively painless. But I really like the ability to control when the assignments are available so they track my course schedule.

So the idea is that students are forced to attempt the assignments I selected (I chose the section quizzes, but not the chapter-length quizzes). They can relatively easily jump to the quizzes without reading the associated material, but their performance on those quizzes will likely suffer for it. My hope is that the REVEL assignments will lead to more student engagement with the text (obviously).

Pearson suggests that you make the REVEL assignments worth 15-35% of their final grade, but I decided that they would be worth 40% of the final grade for my class (and 40% from the in-class quizzes, and 20% from a paper assignment).

So, again, if REVEL only provided a more accessible etext, and the online assignments, I would have been very likely to adopt it. But on top of that, the cost of the REVEL package is substantially lower than the traditional bound text. I'm talking $105 for the REVEL package compared to about $240 for the bound text. With the spiraling cost of textbook (and higher education in general), I felt nearly compelled to adopt the REVEL version.

I'm pretty excited about using REVEL in my class. I promise to blog about how well it works during the semester. Stay tuned.

Difficulty ratings and TripAdisor-ing your class

I just got back from attending The Teaching Professor Technology Conference for the first time. It was a very well-run conference of about 600 attendees. That many people passionate about teaching (and technology) energized me. Of course there were presentations about the latest tech tools, but several people, including the keynote speaker, talked about putting pedagogy before the flashy tools. Spot on.

I love it when I learn something that seems obvious in hindsight. Here's one of those for me: students (especially first-generation students like I have at EKU) don't necessarily know how much effort we expect them to put into each assignment. Something we might think should take 1-2 hours they might think they can shoot off in 20 minutes. I wager that faculty's time estimates are inflated, so perhaps time is not the best metric to use. But imagine if faculty gave students some indication of the effort they estimate and expect an assignment would take.

One of my classes is about information literacy and I've been thinking of indicating the difficulty (or expected effort) of each assignment using a information source analogy. For example, easy assignments could get a rating of text or Facebook post (quick and easy to read). More difficult assignments could get blog or news article, and the most difficult could get journal article. You could also use a more familiar star rating or some other symbol. I think this might help students schedule their studying and avoid surprises. I'm a bit concerned about the difficult ratings on the hardest assignments scaring or intimidating students. Perhaps including some language about growth mindset would be helpful ("We haven't done an assignment this tough in class -- yet").

Another cool idea is to have the students provide their own difficulty or effort ratings of the assignments (after the fact) and then report those ratings the next semester. Perhaps students will trust and take heed of other students' ratings more than professor ratings. This is sort of a TripAdvisor approach to the issue.

A similar idea is to have students write instructions for the course ("read the text first, then do the discussion boards" or "preparing for the tests took more time than I thought"). I've done this sort of thing in some of my classes by asking students at the end of the semester what they wish they had known at the start of the semester, and then giving these to students the next semester. I have gotten some advice that I did not pass along ("drop this class now"), but most of the advice is appropriate and helpful. Again, it has the cache that my advice does not.

Flipping with a Jigsaw, epilogue

I'm now in my third semester teaching my flipped social psychology course. Everything is going well -- I'm now to the pleasant stage of not needing to revise my materials much -- but I did make some changes. I led a Professional Learning Community last spring about metacognition (you can see some info about that here) and one idea I got from that literature is the importance of retrieval in learning. I admit that I used to think of tests as mostly assessments of learning. But research in psychology (where most of the ideas about metacognition come from) clearly demonstrates that learning is strengthened by retrieval. So, this means that tests can (should) be thought of as formative tools that enhance learning. This was the motivation for having so many smaller quizzes that are lower stakes in my flipped course. But if taking the quiz once is good, perhaps taking it a second time would be better. So this is why I changed my class structure.

I used to have the students engage in the jigsaw, take the quiz, and then we'd go over the quiz right away. Now we do the jigsaw and take the quiz, but we don't go over it right away. Instead, the next day they come in and take the exact same quiz a second time, using the same quiz itself with all their notes and comments on it. This means that between taking the quiz the first and second time they can go back to the text or recorded lectures to look up the answers to the questions on the quiz. Or they can talk to other students about them. Either activity would be beneficial in terms of learning the material, I believe. The idea is that thinking of the answers to the questions twice is twice as much retrieval.

Correct answers are worth one point on the first version of the quiz, but only half a point if they get it wrong first and right the second time. So it's in their interest to do as well as possible the first time.

This has lead to higher final grades on the quizzes. Scores on the first six quizzes are on average 10.6% higher than in the previous two semesters (combined). That's a letter grade higher -- not insignificant. So the question is does that reflect a commensurate increase in learning. I don't have data that speaks to that definitively. And I do think the students have pretty quickly figured out a strategy for the second quiz: if they were trying to deiced between two options, pick the best one the first time and the second best the second time. Fair enough, and clever. But they still get more points for the first choice if they get it right, and they still have to narrow it down to two options from four. I wish I had good data about their learning that was not contaminated by this strategy, but I just don't. I just have to trust the research on retrieval, I suppose.

This has introduced some logistical complexity. Each day starts with taking the previous day's quiz a second time, then we go over the quiz, then do the jigsaw on that day's material, and finally take the quiz on that material. So twice as many scantron forms used and twice as much time set aside for quizzes. And the quiz grading is more complex because I have to compare both versions to give credit for getting it right the second time. (A dependable graduate assistant is worth her weight in gold!) Overall I think the increase in learning as result of added retrieval is worth the added complexity.

My courses are always little laboratories. I'll see how this turns out.

Two new activities based (loosely) on metacognition

It's a new academic year and I'm busy getting ready for the new year, like every other teacher. I'm always on the hunt for new ideas to imporve my teaching, and a couple of seemingly serendipitous events have lead to two new activities.

The first activity came as a result of my Chair forwarding this piece about the effect of asking students to set goals for their education. The author of that piece refers to research about goal setting that certainly is encouraging. Two caveats to that research: 1) the intervention in the main study he refers to (described here) is quite labor- and time-intensive, and 2) the author tried a more modest intervention and referred to it as a "failure". Despite those issues I decided to attempt to create an activity that would get at the same effect. In the past I've asked students on the first day to report their reasons for taking my class, but that usually produces non-helpful (but honest) answers like "to fulfill requirements" or  "it fit my schedule". In this activity I start out asking the same question (reason for taking this class), and what grade they want to earn in the course. But I then move on to getting them to think about their goals for the course other than a particular grade. I then ask them to reflect on their habits that both get them closer to and prevent them from attaining their educational goals. Hopefully this elicits some honest reflection. Then, and I think here is where the action takes place, I ask them to imagine their life after they graduate if they were somehow able to increase and improve their helpful habits and minimize or eliminate their un-helpful habits, not just in this course but in their life as a whole. Describe what their life would be like if that happened. Conversely, I also ask them to consider what their life would be like if they decreased their helpful habits and increased their un-helpful habits. Finally, capitalizing on a little cognitive dissonance research about public commitments, I ask them to at least try to minimize the un-helpful and maximize the helpful habits for the next semester, and ask them to sign and date the document. 

I plan to hand out the activity either the first or second class of the term and get them to return it by the next class. I'll hold on to them until perhaps midterm (this is a flipped class that has quizzes in every class, otherwise I would hold them until just after the first exam) and them return the activities to them and ask them to consider their progress on their habits. Reflecting on their process is a metacognitive activity. My activity can be seen here, feel free to use it as you see fit.

The other activity came about because I got an email from the Director of the Honors program at my university talking about the students in Honors electives. I teach several such courses and was a bit startled to be reminded that the students were mostly first-semester sophomores. I had thought of them as more advanced students, so the reminder was important. My Honors electives are typically taught as seminars, much like a graduate seminar, and I often assign empirical articles that would be appropriate for graduate students, but perhaps a new type of reading for these students. Now, I don't shy away from assigning challenging readings (indeed there's research about the value of doing so), but I also don't want to overwhelm them early on. So, I created an activity I plan to have students complete the first day we discuss a reading -- before we get to the discussion. I first ask if they had done the assigned reading. Then, assuming they had, I ask them how they approached the reading activity -- did they read alone, was music playing, etc. Then I ask them how they did the reading -- did they read it all at once, did they write in the margin, highlight, take notes, etc. Then I ask them what their goal was while they were reading and I give them a list of options based on Anderson and Krathwohl's taxonomy (similar to Bloom's taxonomy), including understanding, analysis, synthesis, etc. They can choose as many as apply. I bet this will be the first time they realize that they can read with different goals in mind. Finally, and this mirrors the previous activity, I ask them about what they did that helped them understand the reading and what they did that did not help them. I plan to have them share their responses with a classmate in the hopes that they would get some helpful tips from others, or at least realize that they are not alone in struggling with the reading. Again, I plan to collect the completed activities and then return them a month or so later so they can compare their current practices with what they said was helpful and not helpful. The activity can be seen here. 

These are new activities for me so I can't report here on their effectiveness. Perhaps I will comment about that in a future post. Good luck with the new year!

The Dreaded Teaching Evaluations (good news!)

Where I work we use the Student Ratings of Instruction (SRI) from IDEA. They are useful instruments meant to be used to modify and improve instruction, but of course we mainly use them (inappropriately) to evaluate instruction. I must admit that I've never found them too useful in terms of improvement, and often found the results to be annoying (the adjusted scores drive me nuts). But that may change now that I've heard a presentation by the president of IDEA, Ken Ryalls.

Dr. Ryalls (he's a former faculty, in fact a social psychologist like myself) explained that IDEA has always been about improving instruction, but that they are now trying to emphasize and enhance that function of the IDEA reports that faculty get about their classes.

Faculty get to indicate how important 12 different objectives are for their courses (from no importance, minor importance, to essential). The objectives range from gaining factual knowledge, to learning to apply course material, to developing a clearer understanding of, and commitment to, personal values. You can select them all to be essential or none, it's up to you. You then get a single score that indicates how well you did on the objectives you said were most important, and this number is said to be the most important in the whole report. I understood all of this pretty well. What Dr. Ryalls told me that I did NOT know was that IDEA has a set of papers that target each of the objectives (find them here). These papers are written by faculty with expertise in thes relevant areas, and are research-based. So, if you say that getting students to apply course content is important to you but your results indicate that you did not do so well on that objective, you can go to the website and learn about ways to improve in this area. That is pretty handy.

But IDEA doesn't stop there. They also have no less than 57(!) papers about a variety of pedagogical topics, like the flipped classroom, deep learning, team teaching, getting students to read, the list goes on! They also have teamed up with the POD network (Professional Organizational Development) to write papers about the other items on the SRI questionnaire. So if your students say that you don't show a personal interest in their learning, there's a paper about how to do that better. Perhaps you want to get your students to think creatively -- there's a paper on that.

In short, there's a lot more to the IDEA organization than those pesky teaching evaluations. Used correctly (as in, as IDEA means for them to be used), the IDEA reports can really help faculty target their efforts to improve instruction on the objectives they say are the most important. I've been doing it wrong all this time!

Flipping with a Jigsaw, part 6

Part 6: Conclusions and suggestions

So that’s how I flipped my Social Psychology course. Overall it required a considerable amount of work, but no more than prepping a class for the first time. And, like most courses, now that I have completed the first time teaching it, the second time is much less work. In fact I am teaching it for the second time now, and I can report that it going just as well as the first time (and requires much less work).

Here are a few of what I think are the main points from this series of blogs:

• I challenge instructors to really consider the Dee Fink question: are you creating courses that will result in significant learning experiences that will endure well past graduation? How would you change your course if every student was as eager and able as your best student?

• We preach about the importance of active learning and many instructors insert episodes of active learning into their lectures. Flipping takes you all in; class sessions are (or can be) devoid of lecturing and all active engagement by the students.

• Flipping requires the willingness to give up lecturing at least to some degree, and perhaps totally. I admit that this was hard for me. I was used to receiving the students’ attention, with all the power and status that affords. Giving that up is a sacrifice. But I believe the benefit to the students is more important than the boost I get to my ego or my entertainment.

• There are many things you can do during the class time instead of lecturing. I chose the Jigsaw and can enthusiastically recommend it. The Jigsaw has a proud history and impressive empirical support. It’s also messy, loud, and chaotic. And I mainly stand around doing nothing while the students are working – but that’s the point: they are engaging with the material in active ways, not me. And I think that’s the way it should be.

Here are my suggestions:

• Realize that flipping a course will require a significant amount of prep work. I started designing my flipped class and the materials months before the class started. Build in that prep time.

• Read up on and consider the Jigsaw. It can work in nearly any discipline, from sciences to the humanities. It does require that you trust your students to not only learn material but also teach it to their peers. The success of the Jigsaw relies on the materials you design. And remember that the Jigsaw demands the engagement of every student in class, something that I’ve struggled to achieve in other formats.

• Use existing resources whatever you decide to do during the flipped class sessions. I used the essay questions I had crafted over the years and many active learning materials as the prompts for the Jigsaw. I also used many existing multiple choice questions for the quizzes. Take advantage of the work you have already done. I also suggest you look (again) at the resources that come with your textbook. In my opinion these have gotten better over the years and are worth a new look.

• And finally, don’t fear change. I had my Social Psychology course ‘in the can’. I could walk into almost any day in the semester and rattle off the lecture without any prep, if I had to. And I think I did a pretty good job of it, too (of course). But Dee Fink’s presentation made me question the effectiveness of my course in achieving a broader set of goals. I’ve enjoyed treading the line between being fearless and foolhardy. I’ve not always fallen on the right side of that line, but I also know that line will take me where I really want to be.