Assessment, Make It Stick Monday, Pedagogy, TLTCon

Killing our Darlings

The following is the third post on the book Make It Stick by Brown, Roediger, and McDaniel.  Henry L. Roediger, III will be the keynote speaker at this year’s TLTCon, May 16-17, 2019, on the campus of the College of Charleston.  Attendees will receive a free copy of Make It Stick at a registration event on March 14, 2019 to promote Roediger’s visit and the learning experience. Click the hyperlinks to read blog posts on “effortful retrieval” and “varied practice” as learning strategies.

“In writing, you must kill all your darlings.”  That’s advice from William Faulkner where he is encouraging writers not to shy from deleting favorite but useless passages from manuscripts.  It may be helpful advice for teachers, too.   Few theories are as dear as learning styles theory (LST), which urges teachers to use a variety of presentation methods to meet preferred modes of learning.  If you believe the grammar of chapter six’s title “Get beyond Learning Styles” expresses the writers’ true feelings about LST, you’re correct.  Move past it.  Get over it.  Research suggests the theory is overrated at best and dispiriting at worst.

We shouldn’t be surprised at their argument, however, given LST’s “perverse effect” (148) of subordinating hard work to style preference.  Brown, Roediger, and McDaniel have made it abundantly clear that effort is key to learning.  That’s the one note they’ve been blowing all along. The sooner we jettison “easy learning is the best learning” the better.

That said, we should consider whether a chapter debunking learning style theory is nothing short of a “Get out of jail free” card for us instructors—i.e., we don’t have to worry about how we teach since learning is really up to the learner.  Such is not the case.  In a previous blog on “effortful retrieval” (ET) I highlighted the limitations of “dipstick testing”—i.e., testing that measures a student’s short-term memory.  These tests can also be called “static testing” because they measure a student’s learning at a specific time in the same way that a dipstick tells us where the oil level is while we’re at the BP on Highway 17 at 3:15 pm on Monday, November 26, 2018.  The information is helpful for the moment but could quickly become irrelevant if an engine valve is going out or the oil plug is faulty.  In short, there are more precise measurements to take if we really want to know how the Honda Accord is running.  Enter dynamic testing, a term we could have predicted.  Dynamic testing aims at

determining the state of one’s expertise; refocusing learning on areas of low performance; follow-up testing to measure the improvement and to refocus learning so as to keep raising expertise.  Thus, a test may assess a weakness, but rather than assuming that the weakness indicates a fixed inability, you interpret it as a lack of skill or knowledge than can be remedied (151).

If this description rings with Carol Dweck’s concept of a growth mindset, you’re right, and there are at least two important takeaways for instructors.  The first is represented directly from the above excerpt: offer testing regularly for students to assess their weaknesses.  They can redouble efforts to improve weak areas and check for improvement with subsequent testing.  Certain course formats are more suited for this type of testing and follow-up testing, gaming being perhaps the best.

The second takeaway is more applicable to a wider variety of course designs: clearly identify what skill(s) we are testing.  As teachers, we can too easily be guilty of giving tests that simply cover content areas.  Ill-defined testing yields useless information.  Call it non-information or—better yet—Statistically Hopeless Ill-defined Testing.  Would any one of us be satisfied by going to the doctor and having to sit through a barrage of tests only to be told we’re “sick” at the end of the appointment?  “Sick with what?” we demand.  That diagnosis isn’t good enough for you or the doctor, and more testing will ensue.  The same holds true in cases where Professor Z announces, “There will be a quiz on chapter 10, pages 253-75.”  What’s being measured?  Students who earn a D on a quiz so poorly defined only know that they are below average in chapter 10, pages 253-75.  That’s not helpful, and it’s not education.

Providing our students with clearly articulated objectives prior to testing is essential to dynamic testing.  The clarity lets the students know exactly what skill they are being tested on.  As instructors, we should be able to finish this statement for every graded assignment: “This [test, quiz, writing assignment, etc.] measures the student’s ability to . . .”  (Nota bene: I recommend Bloom’s Revised Taxonomy Action Verbs to formulate and scale objectives.) The student should be able to verbalize the objective(s) in return.  If we can’t say precisely what we are testing, let’s save ourselves the irritation when grading and students the bewilderment of blindly reading over information in hopes of reaching unforeseen goals.  That’s a darling everyone can do without.

Make It Stick Monday, TLT, TLTCon

Cross-training and Learning

Nota bene: The following post considers “Mix Up Your Practice,” the third chapter of Make It Stick: The Science of Successful Learning by Brown, Roediger, and McDaniel.  Dr. Roediger will be the keynote speaker at the College of Charleston during TLTCon, which will be held May 16-17, 2019.

“Students just didn’t perform as well as I thought.” “I’ve spent my afternoon grading a lousy round of quizzes.” “I didn’t think my expectations were too high, but these tests proved they were.” Ever say one of these? Numerous times in my thirteen years of teaching I’ve felt the exasperation of wondering how my painstaking trek through lessons, skills, and concepts has left me with the same expression Stephen Colbert wore on the eve of November 8, 2016: “I’m not sure it’s a comedy show anymore.”

For our “Make It Stick Monday” blogpost, we are highlighting some surprising findings regarding skill mastery: “Practice that’s spaced out, interleaved with other learning, and varied produces better mastery, longer retention, and more versatility” (47).  Sounds messy, right?  Perhaps even chaotic, but Roediger and company offer some helpful insights.  Here’s a sample of their evidence:

A group of eight-year-olds practiced tossing beanbags into buckets in gym class.  Half of the kids tossed into a bucket three feet away.  The other half mixed it up by tossing into buckets two feet and four feet away.  After twelve weeks of this they were all tested on tossing into a three-foot bucket.  The kids who did the best by far were those who’d practiced on two-and four-foot buckets but never on three-foot buckets (46).

I’ll pass over the fact that kids tossed beanbags into buckets for twelve weeks and consider a study that looks a little more relevant to us at College of Charleston.  After being shown how to find the volumes of four geometric solids, one group of college students practiced solving problems grouped by type while another group practiced solving them with a mixed (i.e., interleaved) sequence.  Just to be clear, the first group had four problems on the wedge, four on the spheroid, four on the spherical cone, and four on the half-cone; the second did the same problems but in random order. Initially, group one outperformed group two, averaging 89% of the problems correct to group two’s 60%.  A week later, nevertheless, group one was out-performed by group two, averaging 20% correct to group two’s 63%.  The authors conclude that “The mixing of problem types, which boosted final test performance by a remarkable 215 percent, actually impeded performance during initial learning” (50).  Do they have our attention now?

Let’s assume for the time being that the findings are correct.  What do these studies advocate?  The initial takeaway is the focus on long-term results instead of short-term gains.  There are pros and cons to this perspective for both students and teachers.  Students are not going to feel the satisfaction of a mastering step 1 before moving on to step 2 and so on.  Keep in mind, however, that mastery is not a short-term issue.  To master something is to demonstrate it in the final analysis, and spaced out, interleaved, and varied practicing is going to feel a lot more like real work—an ethic completely compatible with critical thinking.  When is the last time you’ve had a student tell you, “I love hard work”?  Don’t hold your breath waiting for that response.

The other takeaway, though, may compel us to give the “mix it up” concept a chance.  These studies insist that we must be able to articulate very clearly what we are guiding the students toward in terms of functionality.  What are students ultimately being trained to do?  What are the “movements” they must be capable of performing successfully?  Most of my teaching has been in ancient languages, which means that I ultimately want students to read in the original language Plato’s “Allegory of the Cave,” Pericles’ funeral oration for the Athenian dead, Cicero’s orations, or Horace’s satires.  Their ability to move about in text thick with Genitive and Accusative absolutes, mixed conditional statements, subjunctive and optative moods, etc. hinges upon their ability to pull all of their knowledge together, not just parse φαντασία or mensa in isolation.  “I read Greek and Latin because I love grammar” is not something I’ve ever said or heard from a colleague.  Yet, that’s the impression our textbooks often give.

In thinking about this concept of spaced out, interleaved, and varied practice, I wondered if there are parallels in physical exercise that suggest something similar to this type of learning exercise.  Something about Make It Stick smacked of cross-training and—lo and behold—some thoughts on the popular CrossFit website caught my attention:

CrossFit is constantly varied functional movements performed at high intensity.  All CrossFit workouts are based on functional movement, and these movements reflect the best aspects of gymnastics, weightlifting, running, rowing and more.  These are the core movements of life.

That last statement gets me: “these are the core movements of life.”  Functionality in the physical sense means that I need to be able to walk, lift, bend, sit, stand, run, twist, turn, pull, etc. and do them in unlimited combinations.  An inability in any one of these movements leaves me struggling to get into my car for the ride to work.  Or think of functionality in terms of competitive sport.  How many times did my basketball coach have us stop running drills so that we could simply scrimmage?  Why? Because ultimately, he wanted us to be a great basketball team.  Who cares if we were able to run a 1-3-1 trap defense in practice on the right side of the court when an opposing guard during a scrimmage saw we were unable to cope or when he brought the ball on the left side or down the middle?  Scrimmaging made us functional as a team because it forced us to connect our various movements.

The importance Make It Stick places on spaced out, interleaved, and varied practice reinforces a certain skepticism I’ve long harbored toward venting sessions, where instructors place all of the blame on the shoulders of seemingly incapable or “lazy” students.  Granted, I’m still waiting on a student to tell me he loves figuring out Thucydides’ use of the Dative case, but then again I never confessed to loving Callimachus’ poetry during Professor Depew’s “Hellenistic Greek Poetry” class.  (Take my word for it: that stuff is hard, and we should all just leave it to Aaron Palmore over in Randolph Hall to have his fun.)  It stands to reason, though, that if a high percentage of the students can’t seem to “get it”—whatever that may mean in the various contexts of our courses—we may need to “mix up [our] practice.”  It can’t hurt to try.


Innovative Instruction, Pedagogy, TLTCon

The Path of Least Resistance Makes Both Men and Memories . . . Duller?

Effortful retrieval—bleh!  The terminology ages more like milk than wine.  Fortunately, it’s a concept with substance and one of the main learning strategies promoted in Make It Stick: The Science of Successful Learning. Most educators have fallen for gimmicks that claim to make learning easier.  I know I have. The authors of Make It Stick don’t buy it.  Two hundred and fifty pounds will always be difficult to press for most people, and for most people memorizing the essential two hundred and fifty verbs of any foreign language will always take a lot of effort.  The solution?  Effortful retrieval—”ET” from now on.  Recollection that takes a lot of umph.

We are quite familiar with ET’s relative, aka “dipstick tests.”  These are often prevalent in online courses.  Read, test, move on; read, test, move on.  It’s not the worst type of learning.  We could force ourselves to sit through fifty-minute lectures and then take a test once every six weeks. But the problem with dipstick testing is that we are compelling students to load and unload information.  Like the recycle bin. Knowledge retention is rarely a deliberately calculated objective.  We assume students will remember because—why?  The content is important to us, the professors?

The data, however, doesn’t support our assumption, which is an unfortunate predicament given the fact that knowledge retention is one of the basic steps in getting to the higher order thinking skills.  There’s a reason that Bloom’s Taxonomy keeps “Remembering” at level one, and it’s not because level one is least important. The level is foundational.  Foundational concepts are built upon brick by brick so that the edifice—the evidence of our ability to create with the knowledge we retain—changes our horizons.  Languages are especially vulnerable to the insufficiencies of dipstick tests; ET aims at retaining knowledge over an extended time period.

So, what is ET really?   ET inculcates obstacles for the sole purpose of challenging the student to put more effort into the remembering process with strategies like delayed testing, alternative scenarios, and different but appropriate terminology.

Think of a language course, though the leap to other disciplines is minimal.  Students often struggle to remember simple vocabulary words.  Add to this difficulty verb conjugations, singular/plural differences, gendered nouns, and the inflections to boot.  It’s a lot to remember.  If dipstick’s virtue is providing regular testing, ET enhances the regular testing strategies by forcing the student to relearn material multiple times.  A very simple strategy can be implemented with vocabulary quizzes in a secondary language class.  For Chapter 1, there’s an initial quiz on thirty vocabulary words.  When students get to Chapter 2, the instructor gives another vocabulary quiz but this time selects twenty words from Chapter 2 and ten from Chapter 1.  For the Chapter 3 quiz, fifteen words come from Chapter 3, eight from Chapter 2, and seven from Chapter 1.  You get the picture.  Of course, students are informed that they’ll be tested on previous vocabulary chapters, which is the point.  They’ll need to relearn previously studied chapters.

But that’s just one simple application of delayed testing.  Here’s a more creative ET strategy.  I remember when Asher, my older son, tried Boy Scouts for a year.  The camp leader told me that the students would learn all of the knots from memory.  Eventually, they would not only practice knot-tying indoors at camp meetings but also outside in the dark, which is the more likely scenario of when campers’ abilities would really be tested.  What sort of alterations would force the student to remember differently, to retrieve the necessary information within a different context, scenario, location?  That is the core set of questions ET attempts to answer.  “The greater the effort to retrieve learning, provided you succeed,” claims Roediger, “the more that learning is strengthened by retrieval.”  Make It Stick maintains that there’s no easy way to learn, but certainly a better way.

Maybe you can imagine a time before we Googled what we forgot.  Someone asks you to recall the name of an actor from a particular film—say, the lead role in The Shawshank Redemption.  You can remember his face, the way he climbs through the sewers of the prison, his triumphant emergence from the culvert and into freedom, and even Morgan Freeman’s great supporting role as Red.  But you can’t name the leading actor.  So, you go through the letters in the alphabet: A, no; B, huh-uh; C, nein; D, nyet—all the way to R.  Something about saying “R” sounds right, and so you attempt a couple of R-names until “Robbins” emerges from the rubble, and you’re good.  His first name is Tim.  Guess what.  You won’t so easily forget his name next time.


8 Engaging Ways to use Technology in the Classroom

I wanted to share the article 8 Engaging Ways to use Technology in the Classroom to Create Lessons That Aren’t Boring from EmergingEdTech that offers up some strategies and tools for the classroom at this time because a few ideas noted in the article will be covered by CofC Faculty at the upcoming TLT Conference which takes place March 8, 9, and 10th.  There are still a few spaces available so register now at

The article mentions Socrative and Plickers.  To learn more about these tools register for the Faculty Discovery Lab and Lunch on 3/9 from 11:50-1:15

Google Drive is another tool in the article.  CofC has adopted Google apps for Education and Google Drive is available to all faculty and students.  There will be a number a session on Google Drive: “Using Google docs for a final project in place of a final exam,” “Introducing Students to Collaboration Using Google Docs,” “Improve Collaboration and Efficiency with Google Docs” as well as “E-portfolios, Google Sites and Digital Projects,”  and Using Blogger for Class Notes.” Check the Conference Schedule for dates and times.

PollEverywhere is listed in the article and although there will not be a session on it at the upcoming conference I think it is important to note that CofC does have a educational license to PollEverywhere.  To learn more about it and view step by step tutorials visit:

Like PollEveyerywhere, both Twitter and PowToon make an appearance in the article and are not featured sessions at the Conference, but TLT has created step by step instructions for these tools and if you would like to learn more about them contact your Instructional Technologist.


#tltcon It's Going to Be Epic!
Events, Faculty Showcase, Faculty Technology Institute, TLT, TLTCon

#TLTCon Featured Session: Casting Your Web

Are you interested in learning about web building tools?
Do you want to learn how to create a custom website for your course?
If so, you’re going to LOVE this session!

Casting Your Web: Building a custom website for your course (for free!)

Paul Colling head shotPresenter:  Paul Collins

Session Date:  March 9th

Session Time:  1:30- 3:00 pm

Sitting down to build a website can seem an intimidating and daunting proposal. But, as technology advances make it easier and easier (not to mention cheaper) to have a web presence, it is worth exploring some of the tools that have become available to use in our classes. A number of pretty good tools have made it possible to build websites to help to support our teaching. Weebly and Wix are two of these. I have used both services to support both my classes and my professional practice. I will briefly present the websites that I have created, and discuss how each has been useful to my work both within the College and outside of it. I will then go ‘under the hood’ to show the customization tools at work.

In the second part of the workshop, participants will explore the interface of each of these services, and experiment with some of the tools that are available to the user. I will then give a tutorial in which participants will be able to build webpages that can be published on the web by the end of the session!

In order to ‘play along’ in the web-building part of the workshop, participants will need a laptop (as opposed to a mobile device), as the editing engines do not yet work on mobile platforms. The demonstration portion of the workshop will work equally well on laptop and mobile.



*Teaching, Learning, and Technology Conference brought to you by TLT and the FTI