SXSWedu day 2

Day 2 has come and gone, and I have to say that I'm exhausted. Today was another very full day, with more

walking than day 1. Plus I visited the hotel gym this morning -- perhaps not a good idea. Oh well.

I'll follow the same format as Day 1's blog: general comments followed by notes from individual sessions.

General comments

I'm starting to see the excellent organization of this conference. There are SXSWedu volunteers all over, all eager to answer questions about locations of venues and anything else. The sessions start and finish on time, the tech is nearly invisible (a good thing), and the app is very functional. The wifi coverage is excellent across all venues.

Designing and Sustaining Transformative PD

 

I was really excited for this session. The description made it sound like it would align with the efforts of a committee I'm on at my institution to create a system for professional development on-demand. We certainly want the PD to be transformative. The room was beyond standing room only; I was nearly pushed out by the SXSWedu staff. I had to sit on the floor (not good for the post-gym aches). The description did not clearly state that the intended audience was K12 teachers, but in retrospect I should have surmised that by the offer of CPE credits. The presenters and most of the audience were indeed K12 teachers and administrators. Somewhat worse, and I have mixed feelings about saying this, the presenters set up the session as a 'workshop', meaning that they wanted the audience to interact and do much of the work. As I was not at a table that was difficult, even had I been inclined to participate. Which I wasn't. Plus, it was a 2-hour session. I left after 1 hour for a session I didn't think I would get to, but looked good... The presenters did talk about iTunes U, and I was able to find some interesting resources on that platform during the session. So, not a total loss.

How Universities are Crowdsourcing Innovations

This was a panel discussion between a reporter and three university Presidents (UKansas, Georgia State, Arizona State), all of which are members of the University Innovation Alliance, which I mentioned in my Day 1 blog. They do have some impressive outcomes in terms of raising the retention and graduation rates of poor and minority students. Given the student body and service region of my university (some of the poorest counties in the U.S.) this effort should be made known to the relevant administrators. I did tweet at my university president about UIA; he was not aware but wanted more info. I would love to follow up with him. If Georgia State and UAz can improve, we can too.

ImagineCon: The Future of Student Success

This was less a session than a fair for organizations that won a grant from the Bill & Melinda Gates Foundation last year.

Readers of this blog (shout out to my mother and a few others) will recall that I've been working with publishing companies to inform my Dept. colleagues about their online resources, especially their adaptive and personalized learning platforms. And I've been impressed by most of them. But I learned at this session that there are lots of non-profit organizations that are trying to do the same thing but with open-access content, reducing the cost to the student all the way down to $10 - $25, much less than the ~$100 cost from the publishing companies. Some of these

organizations include lumen, Open Learning Initiative, and more. Most of these companies have materials for Introduction to Psychology, or it's in their pipeline. I hope my Dept. will consider these resources when we next adopt a text for that class.

From South L.A. to the Ivy League

I hadn't planned on attending this session, but I had a few minutes between other sessions so I dropped in. As you might deduce from the title, this was a panel of two students and one of their teachers from L.A. The students are now attending Harvard and Yale, but came from an impoverished area of L.A., although they somehow attended a charter school. To be clear, these two young men were about as impressive as could be, still grounded and focused. Inspiring. Both mentioned individual teachers in the charter school as keys to their current Ivy League status. A nice boost for every teacher that aims to make a difference.

Evolve or Die: Why Education Has to Change

I was attracted to this session because I believe its title is true. The presenter, Jim Deters, started by making the (now) commonly understood points that the jobs of the future do not exist now, that future workers will change jobs (and careers) more than in the past, and that higher education has to change to respond to this new landscape. His focus is on coding and data skills; he made the point that many large companies that we do not associate with software (Goldman Sachs, G.E.) are now employing large numbers of coders. He's an entrepreneur, and the session quickly took on the feel of a sales pitch. Eventually one of the questions from the audience was about the cost of their training (~$21,000 for a 6-month course); he did try to make the point that some help with tuition was possible. His model may become more frequent but I'm not sure I buy that it will supplant traditional higher ed. What is the role of the liberal arts in his model? Not mentioned, even in passing.

That's a wrap for day 2. Another full day tomorrow. Whew.

SXSWedu day 1

Well, it's finally here. I've been thinking about coming to SXSWedu for more than a year, and with the support of several key people (shout outs to Dean Zeigler, TLC honchos Sweet and Blythe, and Noel boss Carpenter) I'm soaking up the energy of this conference and the host city of Austin, TX. I'll start with some general comments about the conference, and then some highlights of the sessions I attended today.

General Comments

This is a huge conference. Not as big as some of the major disciplinary conferences I've been to like APA and APS, but still sizable. The audience is more varied than I'm used to seeing at psychology conferences. There are lots of K12 teachers, passionate advocates all, and many quite young. But there aren't the horde of graduate students here I'm used to seeing. Doesn't make me feel quite so old. There are also the more familiar faculty types. The other less-familiar bunch are the people representing start-ups of all kinds. The panelists at one of the sessions I attended walked out into the crowd to 'interview' audience members, roving-reporter style. Many of the interviewees introduced themselves, and their affiliations took almost as much 'airtime' as their comments. This industry is new to me, or at least the extent of it. That's something I'll keep an eye on for the next three days.

Sessions

Temple Grandin. 

I've seen Grandin speak before, but it was still entertaining to see her today. She really seemed

to have a good time in front of a large capacity crowd -- unusual for someone with Autism. Someone even asked in the Q&A if she enjoyed giving talks like this, and she said it took some getting used to. There was a momentary technical glitch (embarrassing for such a technology-enfused event) involving her slides, and she was clearly disturbed by the prospect of presenting without them. Her main points were familiar to anyone who has heard her or watched her online: people think in different ways, we need to design our pedagogies to accommodate these various ways of thinking, and that there are many highly successful people 'on the spectrum'. She added 'keep Austin weird' several times, to the enjoyment of the crowd.

Wisdom of Higher Ed Pioneers in 7 Minutes or Less

They really packed a lot of info into this session. I'm not sure how these 8 presenters were affiliated, other than they were all interested in innovation, broadly speaking. Here are some of the important ideas: innovation and scalability are important, but diffusion of innovation is key; higher ed needs to credential smaller bits of education to prepare future students for the future job market; the 5th wave of evolution of higher ed will need to work on all scales (individual learners and large groups), and teachers need to get off the stage and work to create personalizable learning environments; we can't let the publishing companies keep the data about learning behind a proprietary wall, we have to open that data up to the light of science; and Dr. Pennebaker from UT-Austin talked about changing the way students think (I believe he was getting at metacognition), not just change their knowledge. Check out this organization.

Several speakers were talking around the scholarship of teaching and learning (SoTL) without actually using the term. That was an interesting omission during day 1: no one I saw said SoTL explicitly, but lots of presenters talked about collecting data and assessment. Is SoTL a dirty word here, or passe?

The Subtle Psychology of Motivation and Learning

This was a real research team doing SoTL (although they didn't call it that). They reported on their efforts to improve the outcomes of both junior-high and community-college students. Their approach was impressively broad, including individual (growth mindset, persistence) and social factors (peer ties, faculty support). They have published their research in leading journals (Journal of Educational Psychology) and worked with edtech start-ups like Analytics for Learning.

I was pleased to hear that retrieval practice and interleaving were part of their intervention. 

There was more, much more. But this is tl;dr already, and I plan to blog at the close of every day of the conference. So if you want more from SXSWedu, stay tuned.

A new way to get students to read the text

Ask any instructor and she'll tell you that one of the biggest challenges is making sure that students read the textbook. I've tried several approaches to this problem, from begging and pleading (no effect) to requiring students to outline the text (seemingly effective, but very time-consuming). In my flipped class (described elsewhere in this blog) I rely on the in-class activities to force students to engage with the material in the text, and I think this approach works pretty well. However, the jigsaw activities we do in class really only require students to engage with the material their learning groups are responsible for (just a small portion of any chapter), and they get the rest of the material from their peers in the teaching groups. So when my Pearson book rep told me about a new offering for my textbook that would require more student engagement with the text, I was ready to listen.

She told me about a new service called REVEL. REVEL really is a number of related things. First, the REVEL service provides students with an electronic version of the entire text, from pictures, to bold key words, to chapter summaries, everything that appears in the paper version appears in the e-version, plus there are interactive elements like brief activities and videos. The etext is in the cloud and is accessible on most platforms, from PCs to tablets to smartphones. So students can access the etext anywhere they have internet access including over wifi and cell service. Students also get an unbound 3-hole-punch version of the text identical to the traditional bound version. So, in sum, they get the same text they used to get, but now they get it electronically and in paper form, with added electronic features (including narration of the text). This alone might have tempted me to adopt this version, just for the increased access and portability. But there's much more to REVEL.

The most useful aspect of REVEL are the assignments that Pearson has built into the etext. As an instructor I get an account on REVEL that allows me to select from a list of assignments for each chapter in the etext. I can set up a due date for each assignment and REVEL will track each student's performance and progress on the assignments. The assignments themselves range from short quizzes covering small sections of the chapters to shared reflections on the material. The short quizzes can be taken multiple times, although correct answers diminish in value with each attempt. I really like this mastery approach. There are also quizzes that cover the entire chapter. I can select whatever type of assignment to use, covering any part of a chapter or all of it. The semester hasn't started yet so I don't know how well the analytics work, but it looks very helpful from the instructional videos. 

Setting up the assignment on Pearson's REVEL website does take some time and effort, and at this point REVEL doesn't interface with our LMS (Blackboard) so I'll have to manually import the results into the LMS and my gradebook, but that looks relatively painless. But I really like the ability to control when the assignments are available so they track my course schedule.

So the idea is that students are forced to attempt the assignments I selected (I chose the section quizzes, but not the chapter-length quizzes). They can relatively easily jump to the quizzes without reading the associated material, but their performance on those quizzes will likely suffer for it. My hope is that the REVEL assignments will lead to more student engagement with the text (obviously).

Pearson suggests that you make the REVEL assignments worth 15-35% of their final grade, but I decided that they would be worth 40% of the final grade for my class (and 40% from the in-class quizzes, and 20% from a paper assignment).

So, again, if REVEL only provided a more accessible etext, and the online assignments, I would have been very likely to adopt it. But on top of that, the cost of the REVEL package is substantially lower than the traditional bound text. I'm talking $105 for the REVEL package compared to about $240 for the bound text. With the spiraling cost of textbook (and higher education in general), I felt nearly compelled to adopt the REVEL version.

I'm pretty excited about using REVEL in my class. I promise to blog about how well it works during the semester. Stay tuned.

Difficulty ratings and TripAdisor-ing your class

I just got back from attending The Teaching Professor Technology Conference for the first time. It was a very well-run conference of about 600 attendees. That many people passionate about teaching (and technology) energized me. Of course there were presentations about the latest tech tools, but several people, including the keynote speaker, talked about putting pedagogy before the flashy tools. Spot on.

I love it when I learn something that seems obvious in hindsight. Here's one of those for me: students (especially first-generation students like I have at EKU) don't necessarily know how much effort we expect them to put into each assignment. Something we might think should take 1-2 hours they might think they can shoot off in 20 minutes. I wager that faculty's time estimates are inflated, so perhaps time is not the best metric to use. But imagine if faculty gave students some indication of the effort they estimate and expect an assignment would take.

One of my classes is about information literacy and I've been thinking of indicating the difficulty (or expected effort) of each assignment using a information source analogy. For example, easy assignments could get a rating of text or Facebook post (quick and easy to read). More difficult assignments could get blog or news article, and the most difficult could get journal article. You could also use a more familiar star rating or some other symbol. I think this might help students schedule their studying and avoid surprises. I'm a bit concerned about the difficult ratings on the hardest assignments scaring or intimidating students. Perhaps including some language about growth mindset would be helpful ("We haven't done an assignment this tough in class -- yet").

Another cool idea is to have the students provide their own difficulty or effort ratings of the assignments (after the fact) and then report those ratings the next semester. Perhaps students will trust and take heed of other students' ratings more than professor ratings. This is sort of a TripAdvisor approach to the issue.

A similar idea is to have students write instructions for the course ("read the text first, then do the discussion boards" or "preparing for the tests took more time than I thought"). I've done this sort of thing in some of my classes by asking students at the end of the semester what they wish they had known at the start of the semester, and then giving these to students the next semester. I have gotten some advice that I did not pass along ("drop this class now"), but most of the advice is appropriate and helpful. Again, it has the cache that my advice does not.

Flipping with a Jigsaw, epilogue

I'm now in my third semester teaching my flipped social psychology course. Everything is going well -- I'm now to the pleasant stage of not needing to revise my materials much -- but I did make some changes. I led a Professional Learning Community last spring about metacognition (you can see some info about that here) and one idea I got from that literature is the importance of retrieval in learning. I admit that I used to think of tests as mostly assessments of learning. But research in psychology (where most of the ideas about metacognition come from) clearly demonstrates that learning is strengthened by retrieval. So, this means that tests can (should) be thought of as formative tools that enhance learning. This was the motivation for having so many smaller quizzes that are lower stakes in my flipped course. But if taking the quiz once is good, perhaps taking it a second time would be better. So this is why I changed my class structure.

I used to have the students engage in the jigsaw, take the quiz, and then we'd go over the quiz right away. Now we do the jigsaw and take the quiz, but we don't go over it right away. Instead, the next day they come in and take the exact same quiz a second time, using the same quiz itself with all their notes and comments on it. This means that between taking the quiz the first and second time they can go back to the text or recorded lectures to look up the answers to the questions on the quiz. Or they can talk to other students about them. Either activity would be beneficial in terms of learning the material, I believe. The idea is that thinking of the answers to the questions twice is twice as much retrieval.

Correct answers are worth one point on the first version of the quiz, but only half a point if they get it wrong first and right the second time. So it's in their interest to do as well as possible the first time.

This has lead to higher final grades on the quizzes. Scores on the first six quizzes are on average 10.6% higher than in the previous two semesters (combined). That's a letter grade higher -- not insignificant. So the question is does that reflect a commensurate increase in learning. I don't have data that speaks to that definitively. And I do think the students have pretty quickly figured out a strategy for the second quiz: if they were trying to deiced between two options, pick the best one the first time and the second best the second time. Fair enough, and clever. But they still get more points for the first choice if they get it right, and they still have to narrow it down to two options from four. I wish I had good data about their learning that was not contaminated by this strategy, but I just don't. I just have to trust the research on retrieval, I suppose.

This has introduced some logistical complexity. Each day starts with taking the previous day's quiz a second time, then we go over the quiz, then do the jigsaw on that day's material, and finally take the quiz on that material. So twice as many scantron forms used and twice as much time set aside for quizzes. And the quiz grading is more complex because I have to compare both versions to give credit for getting it right the second time. (A dependable graduate assistant is worth her weight in gold!) Overall I think the increase in learning as result of added retrieval is worth the added complexity.

My courses are always little laboratories. I'll see how this turns out.

Two new activities based (loosely) on metacognition

It's a new academic year and I'm busy getting ready for the new year, like every other teacher. I'm always on the hunt for new ideas to imporve my teaching, and a couple of seemingly serendipitous events have lead to two new activities.

The first activity came as a result of my Chair forwarding this piece about the effect of asking students to set goals for their education. The author of that piece refers to research about goal setting that certainly is encouraging. Two caveats to that research: 1) the intervention in the main study he refers to (described here) is quite labor- and time-intensive, and 2) the author tried a more modest intervention and referred to it as a "failure". Despite those issues I decided to attempt to create an activity that would get at the same effect. In the past I've asked students on the first day to report their reasons for taking my class, but that usually produces non-helpful (but honest) answers like "to fulfill requirements" or  "it fit my schedule". In this activity I start out asking the same question (reason for taking this class), and what grade they want to earn in the course. But I then move on to getting them to think about their goals for the course other than a particular grade. I then ask them to reflect on their habits that both get them closer to and prevent them from attaining their educational goals. Hopefully this elicits some honest reflection. Then, and I think here is where the action takes place, I ask them to imagine their life after they graduate if they were somehow able to increase and improve their helpful habits and minimize or eliminate their un-helpful habits, not just in this course but in their life as a whole. Describe what their life would be like if that happened. Conversely, I also ask them to consider what their life would be like if they decreased their helpful habits and increased their un-helpful habits. Finally, capitalizing on a little cognitive dissonance research about public commitments, I ask them to at least try to minimize the un-helpful and maximize the helpful habits for the next semester, and ask them to sign and date the document. 

I plan to hand out the activity either the first or second class of the term and get them to return it by the next class. I'll hold on to them until perhaps midterm (this is a flipped class that has quizzes in every class, otherwise I would hold them until just after the first exam) and them return the activities to them and ask them to consider their progress on their habits. Reflecting on their process is a metacognitive activity. My activity can be seen here, feel free to use it as you see fit.

The other activity came about because I got an email from the Director of the Honors program at my university talking about the students in Honors electives. I teach several such courses and was a bit startled to be reminded that the students were mostly first-semester sophomores. I had thought of them as more advanced students, so the reminder was important. My Honors electives are typically taught as seminars, much like a graduate seminar, and I often assign empirical articles that would be appropriate for graduate students, but perhaps a new type of reading for these students. Now, I don't shy away from assigning challenging readings (indeed there's research about the value of doing so), but I also don't want to overwhelm them early on. So, I created an activity I plan to have students complete the first day we discuss a reading -- before we get to the discussion. I first ask if they had done the assigned reading. Then, assuming they had, I ask them how they approached the reading activity -- did they read alone, was music playing, etc. Then I ask them how they did the reading -- did they read it all at once, did they write in the margin, highlight, take notes, etc. Then I ask them what their goal was while they were reading and I give them a list of options based on Anderson and Krathwohl's taxonomy (similar to Bloom's taxonomy), including understanding, analysis, synthesis, etc. They can choose as many as apply. I bet this will be the first time they realize that they can read with different goals in mind. Finally, and this mirrors the previous activity, I ask them about what they did that helped them understand the reading and what they did that did not help them. I plan to have them share their responses with a classmate in the hopes that they would get some helpful tips from others, or at least realize that they are not alone in struggling with the reading. Again, I plan to collect the completed activities and then return them a month or so later so they can compare their current practices with what they said was helpful and not helpful. The activity can be seen here. 

These are new activities for me so I can't report here on their effectiveness. Perhaps I will comment about that in a future post. Good luck with the new year!

The Dreaded Teaching Evaluations (good news!)

Where I work we use the Student Ratings of Instruction (SRI) from IDEA. They are useful instruments meant to be used to modify and improve instruction, but of course we mainly use them (inappropriately) to evaluate instruction. I must admit that I've never found them too useful in terms of improvement, and often found the results to be annoying (the adjusted scores drive me nuts). But that may change now that I've heard a presentation by the president of IDEA, Ken Ryalls.

Dr. Ryalls (he's a former faculty, in fact a social psychologist like myself) explained that IDEA has always been about improving instruction, but that they are now trying to emphasize and enhance that function of the IDEA reports that faculty get about their classes.

Faculty get to indicate how important 12 different objectives are for their courses (from no importance, minor importance, to essential). The objectives range from gaining factual knowledge, to learning to apply course material, to developing a clearer understanding of, and commitment to, personal values. You can select them all to be essential or none, it's up to you. You then get a single score that indicates how well you did on the objectives you said were most important, and this number is said to be the most important in the whole report. I understood all of this pretty well. What Dr. Ryalls told me that I did NOT know was that IDEA has a set of papers that target each of the objectives (find them here). These papers are written by faculty with expertise in thes relevant areas, and are research-based. So, if you say that getting students to apply course content is important to you but your results indicate that you did not do so well on that objective, you can go to the website and learn about ways to improve in this area. That is pretty handy.

But IDEA doesn't stop there. They also have no less than 57(!) papers about a variety of pedagogical topics, like the flipped classroom, deep learning, team teaching, getting students to read, the list goes on! They also have teamed up with the POD network (Professional Organizational Development) to write papers about the other items on the SRI questionnaire. So if your students say that you don't show a personal interest in their learning, there's a paper about how to do that better. Perhaps you want to get your students to think creatively -- there's a paper on that.

In short, there's a lot more to the IDEA organization than those pesky teaching evaluations. Used correctly (as in, as IDEA means for them to be used), the IDEA reports can really help faculty target their efforts to improve instruction on the objectives they say are the most important. I've been doing it wrong all this time!