Assessment, Google Apps, Presentation, TLT

#OneNewThing: Creating Infographics Using Google Slides

Infographics are graphic visual representations of information, data, or knowledge intended to present information quickly and clearly. (wikipedia)  For the layman, they can make difficult to understand data or concepts more accessible by delivering only the most important data in an understandable way.  When students are asked to create infographics, it requires them to understand the data and concepts deeply enough to be able to distill them into digestible chunks that a layman can understand.  Because of this, I highly recommend incorporating an infographic assignment into your teaching.


While there are some free applications (Canva.com) that can be used to create infographics, my recommendation is to use Google Slides.  Google Slides is completely free, easy to use, familiar, and the students already have an account!  Here’s how you can do it:

Page Size:

Don’t worry about being restricted to the standard presentation size you see in Google Slides.  You can change the page to be any size you want!

Adding Images:

  • screenshotYou can search the web for royalty free images directly from Slides or you can upload images from your computer (note: Pixabay.org gives you free images to use).
  • You can crop your images into shapes and add shadows and reflections.
  • You can also add a color overlay to some images to make them work for showing a percentage of people for instance.

Add Charts:

  • chart screenshotYou can add charts (pie, line, bar, and column charts) from directly within Slides.
  • You can also connect a chart from Google Sheets that will automatically update when you change the data in your spreadsheet!  Can’t do that in Canva.

 

Add Diagrams:

You can add anything from an org. chart to a timeline from within Slides as well.  They are professional looking and easy to update.

screenshot of diagrams

Add Shapes:

Shapes can be used to highlight or group text or data areas on an infographic.  Slides offers a wide range of shapes and call outs.

Add Text:

Slides offer a variety of font styles and sizes to meet your needs and the best part is that these will look the same on every computer regardless of the fonts installed on the viewer’s computer, which is critical on an infographic.

Shareable and Collaborative:

Because it’s a Google product it’s easy to share it for group or parter collaboration in creating and easy sharing to turn it in the instructor.

[button link=”https://docs.google.com/document/d/1uK22xphnEWkXn7nMDDn67NvU6cQMz9qcbkSeBVJDZOA/edit?usp=sharing” newwindow=”yes”] Access the tutorials for you and your students[/button]

Pedagogy, TLT

Pedagogy Pointer: How to write your own PBL problems

Problem-based Learning

“Problem-Based Learning (PBL) is a teaching method in which complex real-world problems are used as the vehicle to promote student learning of concepts and principles as opposed to direct presentation of facts and concepts.” (University of Illinois Urbana-Champaign)  If you’re not familiar with the term, the goal of PBL is to encourage interdisciplinary thinking, collaborative learning, ethical and quantitative reasoning, and critical thinking all while working on authentic, relevant and real issues.

Regardless of the type of PBL you want to use in your teaching, it all relies on the “problem” that will be resolved.  Therefore, learning to write a good problem is critical to the success of PBL.

Use existing problems

You can find existing problems on the internet or maybe even on your textbook site.  The University of Delaware Institute for Transforming University Education has a PBL problem database that is free to use as well.

Write your own

Writing your own problems ensures that the problems and questions align with your course learning outcomes.  Therefore, before setting out to write a problem the first step is to make sure you have written your learning outcomes.

STEP 1: Write your learning outcomes for the PBL assignment.

STEP 2: Write a “hook.”  This is a story or statement that draws the students into the problem and makes them want to find a solution.

STEP 3: Use the rubric below to help you craft a fully engaging problem.

STEP 4: Review the problem, to ensure it has enough complexity to support group work.

STEP 5:  Make sure it encourage solutions that may: require a decision or recommendation; be open-ended or depend on assumptions.

Rubric
rubric provided by the University of Delaware.


REMEMBER…

The PBL activity is based on a problem.

Write the problem and the corresponding questions that will lead the students to demonstrate that they understand and can achieve the learning outcomes.

Assessment

Akindi has great new features for online classes

In response to everyone going online, Akindi (our bubble sheet scanning application) has created new features that allow you do use the program in your online class.  This option is great for quizzes, tests, and exams that are time consuming to recreate online such as those with music notes, math formulas, etc.

NOTE: Akindi does not offer all of the anti-cheating features available in OAKS.

Check out the A Beginner’s Guide to Akindi Online for all the details and tutorials.

If you are curious as to how it works for the students check out Student Experience of Akindi Online Assessment

If you still have questions you can get answers at https://help.akindi.com/en/collections/2215712-online-bubble-sheet-assessments

 

Minimum Final: F, Maximum Final: A, Current Grade: A
Assessment

Cool OAKS Tip to Find At Risk Students

OAKS contains data that will help you know which students may be in trouble.  NOTE: this will only work if you have released the Final Calculated Grade.

  1. In OAKS, go to Communication > Classlist
  2. From the dropdown arrow next to your first student choose View Progress
  3. Above the Grade area for that user you will see three grades, Current, Maximum, Minimum.

The Maximum grade will give you a guide as to how that student will do in your class based on acing all of the remaining assignments in the grade book.

Grades: Minimum F, Current A, Maximum A

Important things to note before this will work properly:

  • Your Final Calculated/Adjusted Grade must be released for the student to view.
  • All of your gradeable items must be in the grade book (Grades > Grades).

Again, it’s a guide that you can use to find the at risk students and to help them make the best decisions.

Team-based Learning: a quick guide to understanding
Assessment, Best Practices, Collaboration, Innovative Instruction, Pedagogy

Team-Based Learning Quick Guide

[et_pb_section bb_built=”1″][et_pb_row][et_pb_column type=”4_4″][et_pb_text _builder_version=”3.0.101″ background_layout=”light”]

What is Team-Based Learning?

“Team-Based Learning is an evidence-based collaborative learning teaching strategy designed around units of instruction, known as “modules,” that are taught in a three-step cycle: preparation, in-class readiness assurance testing, and application-focused exercise. A class typically includes one module.” 1

[/et_pb_text][/et_pb_column][/et_pb_row][et_pb_row][et_pb_column type=”1_2″][et_pb_text _builder_version=”3.0.101″ background_layout=”light” border_radii=”on|7px|7px|7px|7px” border_width_all=”2px” border_color_all=”#b72d21″ box_shadow_style=”preset3″ custom_padding=”15px|15px|15px|15px”]

Why incorporate Team-based Learning?

TBL covers all types of learning:

  • rote and concept learning tested by the individual assurance testing (iRAT)
  • collaborative learning when discussing and coming to consensus on the team readiness assurance test (gRAT/tRAT)
  • application and creative learning during the team case portion

In addition, it also encourages additional skills necessary to succeed in work/life today, such as:

  • problem-solving
  • teamwork
  • consensus
  • cooperation
  • leadership
  • listening skills
  • collaboration

 

[/et_pb_text][/et_pb_column][et_pb_column type=”1_2″][et_pb_text _builder_version=”3.0.101″ background_layout=”light” custom_padding=”15px|15px|15px|15px” border_radii=”on|7px|7px|7px|7px” border_width_all=”2px” border_color_all=”#b72d21″ box_shadow_style=”preset3″]

When should you incorporate Team-based Learning?

TBL is most successful when used on a consistent basis throughout the semester.  This is because the critical component to TBL is the ongoing, consistent team!  CIEL at Vancouver University states, “Groups are collections of individuals. Teams are groups who have developed a shared purpose and sense of collective responsibility. Groups evolve into teams when an instructor creates the proper conditions for effective collaboration.” 2  In order for these teams to gel and be successful they need to meet and work together on a regular basis otherwise, it’s just in class group work.

TBL can be used in any discipline so don’t shy away from the idea because you don’t immediately see how this will work for you.   A little web research will show you many case studies and problems that you can use to teach your concepts.  When choosing a case or problem remember, the teamwork is most effective “when used with assignments where students are asked to converge their diverse thinking in making a single, collective decision, much like a deliberative body.”2

[/et_pb_text][/et_pb_column][/et_pb_row][et_pb_row _builder_version=”3.0.101″][et_pb_column type=”1_3″][et_pb_image _builder_version=”3.0.101″ src=”https://blogs.cofc.edu/tlt/files/2018/10/workplace-1245776_640-24ojw3x.jpg” show_in_lightbox=”off” url_new_window=”off” use_overlay=”off” always_center_on_mobile=”on” force_fullwidth=”off” show_bottom_space=”on” /][/et_pb_column][et_pb_column type=”2_3″][et_pb_text _builder_version=”3.0.101″ background_layout=”light”]

Creating the Teams

The teams are the most important part of TBL.  Here are a few rules to follow when making the groups:

  1. never use student-selected teams
  2. create diverse teams (balanced intellectual and personality resources)
  3. make the selection process transparent
  4. 5-7 students per team
  5. decide what criteria are important to the groups in your class, as well as detrimental.  Ex. had previous courses in the program.
  6. prioritize your criteria (good and bad)
  7. call out the first criteria and allow the students to self-determine if they meet the criteria or not

Learn more about creating your teams at Team Formation for TBL.

[/et_pb_text][/et_pb_column][/et_pb_row][et_pb_row _builder_version=”3.0.101″ border_radii=”on|7px|7px|7px|7px” border_width_all=”2px” border_color_all=”#b72d21″ box_shadow_style=”preset3″ custom_padding=”15px|15px|15px|15px” background_color_gradient_start=”rgba(183,45,33,0.79)” background_color_gradient_end=”rgba(96,7,31,0.3)”][et_pb_column type=”2_3″][et_pb_text _builder_version=”3.0.101″ background_layout=”light”]

The Process

Taught in modules (usually one per class) in three-step cycles: preparation, in-class readiness assurance testing, and application focused exercise.  

  • Student Preparation:
    • must be done before the class – watching, reading, completing a worksheet, etc.
    • some give a reading/watching guide of things to look for and vocab to know.
  • In-class Readiness Assessment Test (RAT):

Step 1:  Students complete an individual RAT (5-20 questions) and submit it (this is not on the if-at) a.k.a. iRAT
These questions are based on the reading(s) and shouldn’t be an easy yes/no answer.  They are multiple choice but should require some thought and application.

Step 2: Students get into their teams and take the same RAT together (uses if-at) a.k.a. tRAT or gRAT
All answers must be agreed upon by the entire team so if there is a discrepancy, the students have to try to convince the other students on the team until they come to a consensus.  This is the same test they took earlier as an individual.  

Team reads the question and discusses it.
They then scratch off the answer they agree upon on the If-At scratch-off.
If it is correct they see a star and get full points.
If it is incorrect they have to discuss again and give it another go.
They continue to scratch answers until they receive the correct one.  Their points decrease every time they incorrectly scratch.

Step 3: Teams are given the opportunity to appeal answers they got incorrect.  This is a formal process in writing where they state their Argument then provide Evidence with page numbers from the readings that back their argument.

Step 4: Professor conducts a clarifying lecture of what the students didn’t grasp, based on the RAT scores.

  • Application Exercise:
    • students are given a problem or challenge and they must come to a team consensus to choose the “best” solution.  These problems do not have one right answer.
    • the teams discuss their findings and solution with the class.

The application-based exercises are very case-based and should include the following:

  • Significant: demonstrates a concepts usefulness.
  • Specific choice: based on course concepts.  Ex which procedure is BEST to use and why.
  • Same problem: all teams receive the same problem.
  • Simultaneous report to the class in a discussion.

Scaffolding

  • Instructors can give a worksheet to the teams that teach them to think through a problem by walking them through the process, how to dissect a statement and make an argument.

[/et_pb_text][/et_pb_column][et_pb_column type=”1_3″][et_pb_image _builder_version=”3.0.101″ src=”https://blogs.cofc.edu/tlt/files/2018/10/6145799166_c7a48edd6d_b-r9208i.jpg” show_in_lightbox=”off” url_new_window=”off” use_overlay=”off” always_center_on_mobile=”on” force_fullwidth=”off” show_bottom_space=”on” /][et_pb_image _builder_version=”3.0.101″ src=”https://blogs.cofc.edu/tlt/files/2018/10/images-1-1e5f8in.jpeg” show_in_lightbox=”off” url_new_window=”off” use_overlay=”off” always_center_on_mobile=”on” force_fullwidth=”off” show_bottom_space=”on” /][et_pb_image _builder_version=”3.0.101″ src=”https://blogs.cofc.edu/tlt/files/2018/10/IF-AT-card-2bmh677.jpg” show_in_lightbox=”off” url_new_window=”off” use_overlay=”off” always_center_on_mobile=”on” force_fullwidth=”off” show_bottom_space=”on” /][/et_pb_column][/et_pb_row][et_pb_row][et_pb_column type=”4_4″][et_pb_text _builder_version=”3.0.101″ background_layout=”light”]

Student to Student feedback at midterm and final

This feedback is critical to the success of a long-term team so these evaluations are an important part of the process.  The feedback should be positive and constructive.  Here are some ideas for questions:

  • One thing they appreciate about this team member
  • One thing they request of this team member
  • Distribute points among the members
    • Look at Preparation, Contribution, Gatekeeping, Flexibility
  • Also, include what they appreciate/request about the instructor

[/et_pb_text][et_pb_divider _builder_version=”3.0.101″ show_divider=”on” color=”#b72d21″ divider_weight=”2″ /][et_pb_text _builder_version=”3.17.2″]

Sample Case Repositories

Public Health

Exercise Science

PEHD

 

COFC ONLY – Does this seem at all interesting?  If so, contact me and I’ll give you the IF-AT scratch-off cards to use in your class.  They include instructions and a test-maker!  This offer is first come, first serve so don’t wait!  Email benignim@cofc.edu using your CofC email to let me know you want them.

[/et_pb_text][et_pb_divider _builder_version=”3.0.101″ show_divider=”on” color=”#b72d21″ divider_weight=”2″ /][et_pb_text _builder_version=”3.17.2″ background_layout=”light”]

Resources:

Team-Based Learning Collaborative

Team-Based Learning Video

Yale Center for Teaching and Learning: Team-based Learning

What is Team-Based Learning? from the Center for Innovation and Excellence in Learning

 

 

 

[/et_pb_text][/et_pb_column][/et_pb_row][/et_pb_section]

Small teaching tip number 8: incorporate informal early feedback rather than rely solely on end-of-semester course evaluations
Small Teaching Tip, Teaching Advice

Small Teaching Tip #8: The Problem with Student Course Evaluations

We are rapidly approaching the end of the semester.  Soon, faculty will receive the results of their course and teaching evaluations. . . Well, perhaps it’s more accurate to say some will receive evaluations of their teaching.  Many more will receive evaluations of their personality, wardrobe, voice, sense of humor, and physical attractiveness. . .

When I first began teaching, I agonized over my students’ evaluations.  I can still quote some of their comments five years later.  Some evaluations made me feel like I could soar while others crushed me.  I’ve since learned to take student course evaluations with a grain of salt.  There are simply too many flaws that make these evaluations an unreliable measurement, including that they are administered at the very end of the semester.

This is problematic for numerous reasons:  First, human memory is notoriously unreliable so student recollections may not be accurate.  Second, the end of the semester is when student stress peaks, which could result in venting negative feelings about their professors.  Finally, students’ opinions can only be used to change future courses rather than being used to improve the course during the semester.

Despite these weaknesses, student perceptions matter and it’s important to provide a platform for their voices to be heard.  What can we do as individual instructors to better assess student learning and satisfaction?  I believe the simplest and most effective solution is to administer student evaluations throughout the semester.  This is sometimes called “Informal Early Feedback.”

How to Incorporate Informal Early Feedback

Gathering students’ opinions multiple times during the semester solves many of the problems associated with end-of-term evaluations.  Also, responding to students’ comments by discussing them in class and making changes as appropriate can have a powerful and positive impact on the classroom culture.  Here are a few ideas to incorporate into your classes:

Exit Tickets:  These are quick formative assessments that allow instructors to check students’ understanding and identify areas of struggle.  They’re called exit tickets because they are typically administered at the end of each class period.  They can take any form and ask any question.  For example, some instructors simply ask students to write responses on scrap paper.  Others incorporate instructional technologies, such as Poll Everywhere, Socrative, Plickers, or Google Forms.  These are two of my favorite exit ticket prompts:

  • 3-2-1:  Ask students to list three concepts they learned, two ways they contributed to today’s class, and one question they still have about the material. This allows the instructor to compare the learning outcomes he/she set for that class with what students are actually retaining.  It also provides insight into how students perceive their participation as well as identifies concepts that students may need further help understanding.
  • Muddiest point: Ask students to identify the most challenging concept discussed in class or in the readings.  This provides a safe way for students to communicate what they’re struggling with so you can determine if additional class time is warranted or if individual interventions are needed.

Keep, Stop, Start:  Ask students to write on a Post-It note one thing they wish would remain the same, one thing they wish would stop, and one thing they wish would start happening.  For example, a student may comment that they like the flipped classroom structure, but they wish the weekly quizzes would be eliminated, and instead be replaced with journaling.  I ask students to not write their names on the Post-It and to stick them to the wall on their way out.  This helps to ensure anonymity and, therefore, more honest feedback.

Post-it notes with students' feedback about the class stuck to the wall outside the classroom door.

Describe Our Class:  Around midterm time, I ask students to compose a letter to a friend who is interested in taking the course.  I ask them to describe the class, including how each class period is typically structured, how I interact with students, what types of readings are assigned, what types of assignments are completed, what he/she is learning, and whether or not he/she is enjoying the experience.  This exercise gives me fantastic insight into how students’ perceptions compare to my own.

It’s easy to allow student course evaluations to distress us.  When so much of our identities is connected to teaching, it’s painful to be criticized or even attacked.  If you receive negative evaluations, seek out the counsel of your Department Chair or ask a colleague to observe your teaching.  And instead of relying only on this one snapshot to assess your teaching, consider implementing informal early feedback throughout the semester.  I’ve found that these exercises have actually improved the quality of my end-of-semester evaluations.


This post is part of a series which presents low risk, high reward teaching ideas, inspired by James Lang’s book Small Teaching: Everyday Lessons from the Science of Learning.

Easier Paper Grading with Google Classroom
Assessment, Collaboration, Google, Google Apps

Easier Paper Grading with Google Classroom

Hurricane Matthew forced TLT to cancel our session on “Easier Paper Grading with Google Classroom.”  We had several people ask if we could reschedule, so to meet the needs of more faculty we decided to do a recorded version of the class.  Check out the playlist to view the entire session, or click on the three lines in the upper right corner to view specific videos in the series.

 

Dear TLT
Assessment, Dear TLT, TLT

Can I grade one OAKS Dropbox Assignment using two rubrics?

DEAR TLTDear TLT,

I have an assignment that has two components but produces one grade.  I’d like to use a different OAKS rubric to evaluate each component.  Is it possible to do this within one OAKS Dropbox so that there is only one grade in the grade book?  I really want to keep them as one assignment.

Sincerely,

Karen HB
Health and Human Performance


 

Dear Karen,

The answer is yes and no.  The OAKS Dropbox allows the instructor to attach multiple OAKS rubrics to one assignment and use both of them to interactively grade the work.  However, it will only automatically load the calculated score from the first rubric.  As you can see from the screenshot below, only Rubric 1’s score has been entered into the Score area for the assignment.

Two graded rubrics with the score from the first one transferred
Two rubrics added and graded. Notice only the first rubric score transferred automatically to the assignment score.

 

 

You will just need to manually enter the appropriate grade into the Score area, based on the outcome of the two rubrics.

The other option to consider would be to create only one rubric in OAKS that has two Criteria Groups.  Group 1 is for the first component of the assignment and Group 2 is for the second part.  The benefit of using the groups is the you can use different scoring levels per section.  Note: this may not produce the same outcome as the two rubrics so be sure to test this before applying it to a live assignment.

Multipgroup rubric

Sincerely,

TLT

Teacher and students engaged in discussion
Assessment, Best Practices, Pedagogy, Teaching Advice

The Essential Role of Memory Retrieval in Student Learning

Too often, at professional development workshops or on education blogs, there’s an emphasis on designing courses that encourage students to reach the summit of Bloom’s pyramid.  There’s absolutely nothing inadvisable about helping students analyze, evaluate, and explore.  But in our race to the top, we often overlook the importance of remembering, understanding, and even applying (especially in our upper-level courses).  According to cognitive psychologists, this is a mistake that can have damaging effects on student learning.  Without foundational knowledge, it is difficult, if not impossible, for students to demonstrate higher order levels of thinking.  According to cognitive psychologist Daniel Willingham:

“Thinking well requires knowing facts, and that’s true not simply because you need something to think about. The very processes that teachers care most about–critical thinking processes such as reasoning and problem-solving–are intimately intertwined with factual knowledge that is stored in long-term memory (not just found in the environment).” (quoted in Lang, 2016, p. 16)

Without a solid understanding of basic concepts, theories, and processes, a student cannot think creatively or critically about a discipline’s body of knowledge.  This academic groundwork allows students to integrate new knowledge in deeper ways and make more sophisticated connections.

Unfortunately, students often make poor choices when they attempt to learn new information.  Have you ever asked your students (maybe after the class did terribly on an exam) how they studied?  Often, students will say things like “I re-read my notes” or “I made flash cards and read them over and over again.”  Research has demonstrated that these are some of the least effective strategies for committing information to long-term memory.  Thus, if we care about our students’ learning, then we must design our courses in ways that actually help students learn, not simply cram and forget.  

Exams are considered by many to be the gold standard of measuring student learning.  However, most instructors are not familiar with the cognitive science literature and, therefore, do not design exams that actually result in student learning.  Better understanding the retrieval effect (sometimes called the testing effect) will help us to create more effective assessments.

How many times have you claimed your “brain is full” or “you can only remember so much”?  Our long-term memories are actually capable of holding quite a lot of information.  Cognitive psychologist Michelle Miller argues “the limiting factor is not storage capacity, but rather the ability to find what you need when you need it.  Long-term memory is rather like having a vast amount of closet space–it is easy to store many items, but it is difficult to retrieve the needed item in a timely fashion” (quoted in Lang, 2016, p. 28).  She explains that each time we recall a piece of information, we strengthen the neural pathways that move the information from our long-term memories to our working memories.  This is key.  The more times we retrieve the information, the better.

Book Small Teaching by James Lang

To encourage your students to practice retrieval, try these strategies from James Lang’s book Small Teaching:

The Retrieval Syllabus.  Most of us distribute our syllabi on the first day of class and never bring it up again, until a student violates a policy or makes a complaint.  Instead of thinking of your syllabus as a contract, envision it as a resource that is continuously referred to throughout the semester.  Fill out the course schedule with details that will help students see how the course will progress, how topics connect to one another, and how knowledge is organized in your discipline.  Then, during class, ask students to look at the document to orient themselves as well as remind them of what has been discussed thus far.

Warm-up Review.  In the first few minutes of class, ask students to write down on a scrap sheet of paper the topics that were covered the class period before or the main themes from the reading.  Ask students to share their “take aways”: What do they think was the most important point?  What struck them?  What piqued their interest?

I’ve done something similar with my students, but I simply asked the class to provide a review orally.  Typically, the same few students are the only ones who reply.  Thus, not everyone is encouraged to practice retrieval, so this method is less effective than asking all students to write down their recap.  This simple exercise has the added benefit of an intellectual “warm-up” — prepping students for learning and participating during class.

Exit Tickets.  Similarly, at the end of class, have students to complete an exit ticket.  For example, you could ask students to write down two things they learned and one question they still have.  This requires retrieval as well as provides valuable information about what students identify as important and what they are struggling with.  This can serve as a great jumping off point for the next class period.

What is absolutely essential for both warm-ups and exit tickets is that students are told not to consult their notes or textbook when responding.  If students look up the answers, they are not practicing retrieval.  It’s also important to explain to students the purpose of these exercises.  You’re not trying to test them or give them busy-work; you’re trying to help them learn more effectively.

Frequent Quizzing.  Frequent, low-stakes quizzes are one of the best ways for students to strengthen their retrieval muscles.  Remember that the more we recall information, the stronger the neural pathways between long-term and working memory.  When creating quizzes, it’s essential that they are not weighted heavily.  The point is to encourage retrieval, not stress students out.  It’s also important to include question types that will be similar to what students can expect on exams.  This allows students to familiarize themselves with those formats so the exam is a test of knowledge instead of exam-taking ability.

If you don’t have enough class time to devote to frequent quizzes, consider using online quizzes, such as through your Learning Management System (LMS).  Most textbook publishers provide gigantic test banks that provide more than enough questions to create multiple quizzes throughout the semester. These banks are designed to be quickly imported into your LMS and quizzes can be automatically-graded, making quiz creation and administration simple.  To ensure students are practicing retrieval, restrict the time limit so they don’t have the leeway to look up every answer in their notes or book (30-60 seconds per multiple choice question is advisable).

Space Out Due Dates.  Students should complete multiple smaller assessments throughout the semester (as opposed to only one midterm and one final exam).  Intersperse lower stakes assessments (e.g. weekly quizzes, practice problems, minute papers) with higher stakes assessments (e.g. exams, research papers, lab reports).  According to James Lang, “the more frequently that your students have to check in and offer some demonstration of their learning, the more often you are giving them retrieval practice” (2016, p. 36).

Providing frequent opportunities for retrieval will not only help your students remember important information, it will also open the door to higher levels of cognition.  I’ve shared simple but powerful ways to help your students learn that do not require extra preparation, overwhelming amounts of grading, or even that much class time.  Want more ideas?  Check out James Lang’s fantastic book Small Teaching and then ask yourself, “what small changes can I make to help my students learn?”

Evolution of a Group Research Project
1-1-1, Assessment, Best Practices, Research, TLT

Faculty Guest Post: Evolution of a Group Research Project

Today’s Faculty Guest Post is from Chris Mothorpe, Assistant Professor of Economics.  Chris attended TLT’s 2015 Faculty Technology Institute.  In this post, he reflects on the process of revising and improving a group research project in two of his courses: Urban Economics and Economics of Geography and Transportation.  This is an excerpt from Chris’ own blog.  To read the entire post, please visit: https://sites.google.com/site/chrismothorpe/home/group-research-projects


I am writing this blog post based on my experience conducting research projects in my upper level economic classes over the past three semesters. This post will not discuss the research project in its entirety; instead, it will provide a general overview of the project and then focus on specific challenges I have faced each semester and different strategies I have employed (or I am planning on employing to overcome them).  There are two main challenges I will discuss: 1) group formation; and 2) peer evaluations.

Project Overview

I decided to require a group research project after reading several magazine and newspaper articles discussing what companies are looking for in college graduates.  Atop many of the surveys were not the hard-technical skills taught in the classrooms, but many soft-skills developed in the non-academic, extracurricular setting.  These soft-skills include: 1) leadership; 2) ability to work in a team; 3) written communication skills; 4) problem solving skills; 5) work ethic; 6) verbal communication skills; 7) initiative; 8) interpersonal skills; 9) creativity; and 10) organizational ability.  Conducting a group-based research project provides students the opportunity to practice many of these skills — practice they would otherwise not receive if the class is taught in a more traditional manner.   A second motivating factor is to allow the students the opportunity to apply economic models to real world problems.

I decided to require a group research project after reading several magazine and newspaper articles discussing what companies are looking for in college graduates.  Atop many of the surveys were not the hard-technical skills taught in the classrooms, but many soft-skills developed in the non-academic, extracurricular setting.  These soft-skills include: 1) leadership; 2) ability to work in a team; 3) written communication skills; 4) problem solving skills; 5) work ethic; 6) verbal communication skills; 7) initiative; 8) interpersonal skills; 9) creativity; and 10) organizational ability.  Conducting a group-based research project provides students the opportunity to practice many of these skills — practice they would otherwise not receive if the class is taught in a more traditional manner.   A second motivating factor is to allow the students the opportunity to apply economic models to real world problems.

The stated objectives for the research project are:
  1. Analyze a contemporary economic issue or social issue using economic theory and models
  2. Demonstrate versatile and competent written, oral and digital communication skills
  3. Evaluate communication situations and audiences to make choices about the most effective ways to deliver messages
  4. Appraise written communication skills through self and peer evaluations
  5. Manage diverse teams successfully

The project is set up as a paper submission to the (fictional) Charleston Journal of Economics, which I reside over as Editor.  At the beginning of the semester, I pass out the Fall/Spring 20XX Charleston Journal of Economics (CJE) Request for Papers (RFP), which contains the objectives of the journal, the strategic areas, scoring criteria, formatting requirements, and examples of correctly formatted submissions. Throughout the semester, groups are required to submit portions of their project to the Editor and receive feedback (in the form of a letter from the editor). I have required the research project in the Spring of 2015, the Fall of 2015 and the Spring of 2016.  These three iterations have proven valuable as I continually update the project to improve on its effectiveness and efficiency in delivery.

Group Formation

In the first iteration (Spring 2015) of the research project, I allowed each student to write his/her own paper and choose any topic as long as it was related somehow Urban Economics.  While allowing each student the opportunity to write their own research paper provides the best learning opportunity for the student (since he/she receives individualized feedback), it is much harder (time consuming) on me. I realized that there were three main consequences to allowing students to complete their own project:
  1. Grading fatigue
  2. Increase time until work is returned to students
  3. Grading research projects detracts from other activities such as research

In the second iteration (Fall 2015), I switched from individual research projects to group based projects.  I allowed the groups to form endogenously — students selected their own groups.  Each research group was required to have 3-4 individuals.  The main problem that arose from students selecting their own groups is that the groups were not interdisciplinary in nature.  For example, Group A consisted of three Transportation and Logistics Majors.  One of the comments Group A received on one of their drafts was that their paper lacks a sufficient economic model.  The feedback I received from Group A was that there is not a economic major (or minor) in the group, and as a result no one is familiar with economic models.

In the second iteration, I also began restricting the topic selection by requiring each group’s research question to at least fall within one of the strategic areas of the Charleston Journal of Economics.  The strategic areas are:
  1. Transportation Infrastructure
  2. The Port of Charleston Expansion
  3. Coastal Community Resilience and the Impacts of Sea Level Rise/Climate Change
  4. The Long Savannah Development

In the third iteration (Spring 2016), I attempted to correct for the lack of interdisciplinary majors within a research group by assigning research groups.  To aid in the assignment of research groups, each student completed an Oaks quiz that asked the following questions:

  1. List the strategic areas in order of greater interest to least interest
  2. For your top ranked strategic area, list keywords of interest
  3. For your second ranked strategic area, list keywords of interest
  4. List your major(s)
  5. List your minor(s)
  6. List individuals you would like to work with

Students submitted their responses via an Oaks quiz and then I used their responses to assign groups.  Matches were made based on strategic areas and keywords; however, not all students receive their top ranked strategic area (most did) as I also sought to ensure that each group contained at least one each major or minor.  This mechanism worked well in solving the interdisciplinary problem previously encountered; however, the new problem that arose was that group members wanted a greater say about who was in their group as the “Free-Riding” problem arose in several groups.  The Free-Riding problem occurs when not all members contribute equally to the project, yet all group members receive the same grade.  Of the 8 research groups in the Spring of 2016, at least 4 registered complaints about one of their group members not contributing.

The Free-Rider Problem

I am planning on implementing two strategies to attempt to mitigate the Free-Riding Problem.  First, I plan on introducing a mechanism that will allow students to reveal information about themselves (e.g. work ethic) to other members in the class.  This mechanism is a series of group-based homework problem sets in the first few weeks of class and before the assignment of groups.  Groups will be randomly assigned.  The random assignment of groups will ensure that students are meeting and learning about other members of the class.  After the problem sets, students will again be asked to complete an Oaks quiz, but on their quiz there will be additional questions aimed at revealing their preferences for who they do and do not want to work with.

 

The second strategy is to have students submit peer evaluations of their group members when assignments are due.  A portion of the peer evaluation is a Grade Multiplier.  Each member of the group assigns every other member of the group a multiplier, which gives each group member control over every other group member’s grade.  The purpose of the multiplier is to provide incentive to group members to work hard towards the completion of the project.  In the Spring of 2016, I required the students to submit Peer Evaluations at the end of the semester; however, this did not provide strong incentives to students since at the time of submissions final class grades were almost known.  It was recommended to me, by a student, to conduct the peer evaluations more frequently.

 

Peer Evaluations are a useful tool that provide students with information on their performance over the course of the research project.  Since the goal of the project is to aid students in developing soft skills, the peer evaluations are particularly effective, since they address each student individually.  Herein lies the main problem since each time I require a peer evaluation I cannot write 20-40 individual letters commenting on their performance.  The remainder of this blog post discusses the tools I have developed to create individualized letters based on peer reviews in an (semi) automatic fashion.  Creating letters in this manner allows me to provide individualized feedback to students while at the same time not spending hours drafting letters.

 

The letter-creation process requires the following programs/files:
  1. The Form Letter – Microsoft Word Template
  2. Oaks Quiz and Excel File of Modified Data
  3. Microsoft Word Template File
  4. Microsoft Excel Template File
  5. Microsoft Excel Addin ExcelToWord

The procedure behind the automated process is to have students complete their peer evaluations through an Oaks quiz, text-mine their responses, and populate a form letter with student responses.  Note that this process relies on student responses on the peer evaluation but does leave open the possibility of directly editing the individualized letters.

[TLT Note: On his own blog, Chris provides instructions for using OAKS, Microsoft Word, and Microsoft Excel to facilitate the peer evaluations described above.  He also provides templates and examples. To access this information, please visit  https://sites.google.com/site/chrismothorpe/home/group-research-projects]

In this blog, I have discussed the research project that I conduct in my upper level economics classes, two of the challenges that have arisen, and various strategies I have or will employ to overcome the challenges.  To overcome group formation problems, I am employing an Oaks quiz and group based homework assigned in order to allow students the opportunity to reveal information about themselves to other students in the class as well as myself.  To overcome the “Free-Riding” problem, I am planning on employing a series of peer evaluations, which gives all members in the group some control over the grades of the other group members.

One key to conducting peer evaluations is returning individualized feedback to the student based on their performance.  I have also discussed a set of tools which will enable me to create individualized letters in a timely manner.  Providing timely and individualized feedback also enhances the learning outcomes of the research project since the project is geared towards student practice of their “soft” skills.  Receiving individualized feedback allows students to learn from their experience and develop a stronger set of skills that they can employ in the future.