An Instructional Challenge is a difficulty, obstacle, or bottleneck that hinders students as they attempt to achieve a specific learning outcome or cluster of learning outcomes. An Instructional Challenge might be cognitive or motivational. For example, a student may have difficulty with a particularly abstruse theory, or with memorizing an abundance of new terms, or with visualizing the relationships among members in a complex system, all of which might be considered Instructional Challenges that are cognitive in nature. On the other hand, a student might have difficulty staying on task because she finds the material boring, or because she cannot see its relevance to her prior learning, or even because the content itself conjures up negative associations, all of which might be considered Instructional Challenges that are motivational in nature. Obviously, it's not possible for instructors to anticipate every Instructional Challenge that their students will experience, since many of them will be unique, if not idiosyncratic, to certain students. However, often an entire class, or at least a large number of students in a class, will experience the same obstacle in their learning, a pattern that might recur every time that course is taught. A list of about 70 common instructional challenges is available here.

Clickers are small, handheld units that are intended to increase student participation and engagement in class by allowing students to easily (and anonymously) respond to an instructor's multiple-choice questions; these responses are instantly tabulated so that the instructor, and potentially the students, can see the results. More information is available here.

pkates@uwaterloo.ca	519-888-4567 x 37047	MC 3044
As a CTE Faculty Liaison, Paul Kates helps instructors in the Faculty of Mathematics integrate technology into their teaching through innovative learning activities. He also serves as guide for instructors to access other CTE resources. Prior to joining the Centre for Teaching Excellence Paul lectured in the School of Computer Science at the University of Waterloo. He received his BMath from the University of Toronto.

mmorton@uwaterloo.ca	519-888-4567 x 37765	MC 4050
As Senior Instructional Developer (Emerging Technologies), Mark Morton helps to manage projects and disseminate knowledge pertaining to new learning technologies. Prior to joining the Centre for Teaching Excellence, Mark taught for twelve years in the English Department at the University of Winnipeg. He received his PhD in 1992 from the University of Toronto, and is the author of four books: Cupboard Love; The End; The Lover's Tongue; and Cooking with Shakespeare. Mark has broadcast dozens of columns pertaining to language for CBC radio, some of which are available as MP3s here. His latest obsession is the Arabic language, which he has been studying at Renison College since 2008. Mark has three dogs, two children, and one wife.
Selected Professional Highlights Presenter at the 2006 McGraw-Hill Ryerson Conference in Ottawa Keynote panelist at the 2007 Canadian Art Gallery Educators Conference Maintains a learning technology blog with Educause. Facilitated a "training the trainers" session for the Centre for Professional Development at the University of California at Long Beach. Recipient of the 2005 Alexander Isbister Award for Nonfiction. Nominated for the 2005 Stephen Leacock Award Recipient, with Svitlana Taraban-Gordon, of a WatCACE Grant for a research project pertaining to the unique challenges facing international students in UW's Co-op Program.

Faculty Testimonials:

• Ryerson University
• The University of California
• Ohio University
• The University of Iowa
• The University of Minnesota

Broadly spread histogram:

Once the students have responded to a clicker question, the results for the class are tabulated and displayed in a histogram, that is, a bar graph that visually depicts the frequency of the various responses. A broadly spread histogram – that is, one in which the bars of the graph are roughly of equal height – is probably an indication of a good question, because the diversity of response is likely to pique the interest of students, and has great potential for fuelling small-group discussions. On the other hand, a histogram that has just a single high bar probably indicates an ineffective clicker-question, because it means that all the students are in agreement. For example, if the clicker-question were a simple poll – such as, “Which of the following five movies is your favourite?” – and if everyone chose, say, Gladiator, then their general agreement would not foster much debate. Likewise, if the clicker question is one in which there was a definite right answer, there will be limited room for debate if everyone gets it correct (or even if all get it wrong). A nice “spread” of responses is more liable to provoke animated discussion.

Although many students will be motivated to respond to clicker questions simply because they find those questions inherently interesting, not all students will be so keen. Without an extrinsic motivation, some students will simply not pick up their clicker or will choose a response at random. Accordingly, it’s probably best to assign a grade to the clicker questions. The grade should not be too high, however, as that might encourage academic dishonesty (such as students gawking to see which clicker button their neighbour has pushed). Attaching a “low stakes” grade to the clicker questions should be sufficient motivation to get almost all the students to genuinely – and honestly -- respond to those questions. Some instructors have found it effective to award full credit for a clicker question if the student gets it right and partial credit if he or she simply makes an attempt.

Conceptual:

Most courses demand that students learn both facts and concepts. Facts might include dates, terminology, atomic weights, and so on, while concepts encompass ideas, perspectives, relationships, and more. Clickers can be used to efficiently assess how well students have learned the "facts" of a given unit; after all, without a basic command of, say, the terminology of a discipline, students will flounder, so it is indeed important to ascertain this basic aspect of student learning. The best clicker questions, however, are those that exercise the students' conceptual understanding of the material: questions that require the students to apply their knowledge, to make comparisons, to analyze and synthesize, to critique -- in other words, questions that require students to use their faculty of reason rather than their ability to merely recognize and recall. To put this another way, the clicker questions that will best aid a student in achieving deep and integrated learning are those that dwell in the "upper" levels of Benjamin Bloom's taxonomy of the cognitive domain.

Counter-intuitive:

Clicker-questions can be an effective means to help students to recognize misconceptions that they have about some aspect of course content. Sometimes those misconceptons exist because the "facts" seem counter-intuitive. For example, how many US states have land that is north of the southern-most tip of Canada? (Answer: 27) Misconceptions can also exist simply because students have previously been taught incorrectly. For example, is it grammatically correct to begin a sentence with the word "and"? (Answer: yes) And of course the media can also distort realities into misconceptions. For example, which is bigger, Texas or British Columbia? (Answer: British Columbia).

Posing such questions to students, and having them actually commit to a response via a clicker, helps students "feel" the incorrectness of their response. It's one thing to be simply told a fact (in, say, a lecture) that seems counter-intuitive; it's another thing to commit yourself to an answer and then discover, to your surprise, that you are wrong.

Frequency:

Most instructors have found that 2 to 5 clicker questions for every hour or so of class time is optimal. Anything less and the clickers will seem superfluous, causing students to forget to bring them to class or not to procure them in the first place. Anything more, and the instructor ends up limiting the discussion time that can be allotted to each question.

Ambiguity:

"Ambiguity is a good quality of a CCS question. It sensitizes students to the ambiguous point's implications, trains them to pay attention to subtleties in a situation, and motivates a discussion about what aspects of a question statement are important and how they matter. All this may not help students reach a correct answer to the question at hand, but the answer isn't the goal: it helps students learn to reason and think defensively and to answer future questions, especially the vague, fuzzy kind often encountered outside the classroom." -- Beatty, Ian et al. (2004). Transforming student learning with classroom communication systems. Educause Center for Applied Research. 2004 (1), 1-13.

Clicker solution #3 (for students being bored in class) is the same as Solution #2 except that the instructor takes it further. Rather than interspersing his or her lecture with clicker questions (answered individually, then discussed as a group, and then answered individually again), the instructor replaces the entire lecture with a series of well-designed clicker questions.

"Many of the courses that use clickers have abandoned lecture altogether or at least reduced it to a smaller component of class time (Draper et al., 2002; Cutts et al., 2004; Knight and Wood, 2005). These 'interactive engagement' or peer instruction' methods are quite powerful, but still fairly new to most instructors." -- -- Caldwell, J.E. (2007). Clickers in the large classroom: current research and best practice tips. CBE, Life Sciences Education. Vol. 6, Spring, 8-20

"A wealth of journal articles explore the uses, outcomes, and benefits of clicker use, and some good reviews exist (McDermott and Redish, 1999; Roschelle et al., 2004a; Duncan, 2005; Simpson and Oliver, 2006). Most reviews agree that 'ample converging evidence' suggests that clickers generally cause improved student outcomes such as improved exam scores or passing rates, student comprehension, and learning and that students like clickers." -- Caldwell, J.E. (2007). Clickers in the large classroom: current research and best practice tips. CBE, Life Sciences Education. Vol. 6, Spring, 8-20.

"The statistical analyses of grade data collected for the 11 parallel courses between fall 2004 and fall 2005 showed a statistically significant impact of clicker use on student performance.... There was an increase of 2.23% in the number of students obtaining a grade of C or better in the courses that used clickers. This increase is consistent with the qualitative data obtained from faculty and students, which show that they believe the use of clickers had a positive impact on learning." -- Kaleta, Robert. "Student Response Systems: A University of Wisconsin Study of Clickers." 2007.

"There is great agreement that CRSs [clickers] promote learning when coupled with appropriate pedagogical methodologies.... The literature also indicates that CRS-supported environment lead to greater learning gains than traditional learning environments." -- Fies, Carmen et al. (2006). Classroom response systems: a review of the literature." Journal of Science Education and Technology, Vol. 15, No. 1, 101-109.

"Clickers appear to reduce student attrition compared with lecture without clickers. Table 1 compares the attendance at the beginning and end of the semester in two courses conducted with and without clickers. With clickers, roughly 4% of students stopped attending by the final exam. This attrition rate was noticeably higher without the clickers, ranging from 8% to nearly 12%. A possible explanation is related to the regular attendance encouraged by daily clicker questions and attendance checks. Students were either better prepared for the exam and chose to attend or were more invested in the course after having spent so much time attending—regardless of preparation. In any case, it is interesting to note that attrition was dramatically reduced during fall semester, when freshmen are typically adjusting to college life." -- -- Caldwell, J.E. (2007). "Clickers in the large classroom: current research and best practice tips. CBE, Life Sciences Education. Vol. 6, Spring, 8-20.

Enhanced learning outcomes:

"Faculty agreed or strongly agreed that there was greater student engagement (94%), participation (87%), and interaction (68%) in class as a result of clicker use. In their narratives, faculty reported that one of their main uses of clickers was to stimulate discussion, and most of them (82%) felt that clickers allowed them to do this. Faculty reported that clickers equalized the classroom, allowed everyone to have a voice, and led to more stimulating discussions. One faculty member commented, 'The clickers were very effective in stimulating discussion...I think that seeing that range of opinions made the students a little more willing to talk about their opinions.' The majority of students also agreed or strongly agreed that the use of clickers made them feel more engaged (69%) in class, increased participation (70%), and helped them pay attention (67%). Students reported that clickers allowed them to participate in class discussion by answering questions without risking embarrassment for incorrect or naive answers. A student included on the survey that 'I liked how the clickers started discussions, especially if the question was especially difficult. The clickers also made me more active in class.'" -- Kaleta, Robert et al. (2007). Student Response Systems: a University of Wisconsin system study of clickers." Educause Center for Applied Research. 2007 (10), 1-12.

Clicker solution #2 for students being bored by lectures:

A multiple choice question is projected onto the screen in the classroom, students spend a minute or two thinking about it individually, and then individually use their clickers to select their answer. After the answers are displayed on the screen, and students note the diversity of responses, they engage in small group discussion, trying to persuade one another of the answer. They then use their clickers again to answer individually, and the new cluster of answers are tabulated and displayed. The key here is not that the question breaks the "monotony" of the lecture, but that having to choose and defend an answer prompts the students to become emotionally invested in the question -- i.e. to care about it.

"Thalheimer (2003) states that unless a student cognitively processes a question and participates in answering it (even if mentally), that learning does not take place. Frase et al. (1970) discovered that less-motivated students’ academic performance approaches that of students who are more highly motivated when asked questions 'frequently'....Kulhavy (1977) posits that students who received feedback before they have formulated their own answer do not learn as much as students who formulate an answer before receiving feedback. This suggests that student who do not attempt to answer questions in class learn very little from listening to the interactions of other students with the instructor.... Kulik and Kulik (1988) learned that immediate feedback resulted in more learning than delayed feedback. This suggests that students learn more from feedback relating to their answers to questions in the classroom than from feedback from graded projects and exams." -- Guthrie, Rand. (2004). Waking the dead: using interactive technology to engage passive listeners in the classroom. Proceedings of the Tenth Americas Conference on Information Systems, New York, New York, August 2004. 1-8.

Moreover, because of shared attitudes and common experience, students are often "better at clearing up each other's confusions and misconceptions" than their instructor -- Wood, W. B. (2004). Clickers: a teaching gimmick that works. Dev. Cell. 7(6), 796–798.

Clicker solution #1 for students being bored by lectures:

A multiple choice question is projected onto the screen in the classroom, students spend a minute or two thinking about it individually, and then they individually use their clickers to select their answer." The key here is not that the question breaks the "monotony" of the lecture, but that having to choose an answer prompts the student to become emotionally invested in the question -- i.e. to care about it.

"Thalheimer (2003) states that unless a student cognitively processes a question and participates in answering it (even if mentally), that learning does not take place. Frase et al. (1970) discovered that less-motivated students’ academic performance approaches that of students who are more highly motivated when asked questions 'frequently'....Kulhavy (1977) posits that students who received feedback before they have formulated their own answer do not learn as much as students who formulate an answer before receiving feedback. This suggests that student who do not attempt to answer questions in class learn very little from listening to the interactions of other students with the instructor.... Kulik and Kulik (1988) learned that immediate feedback resulted in more learning than delayed feedback. This suggests that students learn more from feedback relating to their answers to questions in the classroom than from feedback from graded projects and exams." -- Guthrie, Rand. (2004). Waking the dead: using interactive technology to engage passive listeners in the classroom. Proceedings of the Tenth Americas Conference on Information Systems, New York, New York, August 2004. 1-8.

I have no evidence, other than my own teaching experience, to support my claim that many students are motivated to work harder when they realize that their performance is lacking, in comparison with that of their classmates.

A "clicker" solution for students not knowing whether they are "getting it":

At the conclusion of a content unit, the instructor gives the students a clicker-quiz on that content. After the students have completed the quiz, the instructor presents the tabulated data to them so that each student can see how he or she did in relation to the rest of the class. Students who achieve a low rank, in relation to their classmates, may be motivated to intensify their efforts. If the intention of these quizzes is only to allow students to self-assess (that is, if the instructor has no interest in discerning whether his or her students are "getting it"), then there is probably no need to assign any grade value to these quizzes.

"By providing feedback to an instructor about students' background knowledge and preconceptions, CCS-based [clicker] pedagogy can help the instructor design learning experiences appropriate to students' state of knowledge and explicitly confront and resolve misconceptions. By providing frequent feedback about students' ongoing learning and confusions, it can help an instructor dynamically adjust her teaching to students' real, immediate, changing needs. In the absence of such feedback, lesson plans tend to be 'ballistic': they are designed and launched, and, after the fact, the instructor learns whether the target was hit. When first using a CCS, many instructors are quite shocked by how incorrect their expectations are of students' comprehension." -- Beatty, Ian et al. (2004). Transforming student learning with classroom communication systems. Educause Center for Applied Research. 2004 (1), 1-13.

A "clicker" solution for the instructor not knowing whether students are "getting it":

At the conclusion of a content unit, the instructor gives the students a clicker-quiz on that content. If most of the students do well on that quiz, the instructor can move on to the next content unit. If most of the students do poorly on that quiz, the instructor should probably spend more time on that unit's content, before moving on to the next unit. A grade-value, albeit minimal, should probably be allotted to each of these quizzes to discourage students from intentionally "bombing" the quiz in order to slow the pace of instruction.

"Clickers offer an efficient way to hold all students accountable for preclass preparation. Students who were regularly quizzed on readings prepared more for class, but didn't seem to mind so long as they earned something toward their final grades. This offers an instructor a way out of two common dilemmas: the need to 'cover' the material in lecture leaves little time for more interactive teaching, and many students in a standard lecture course disregard reading assignments because they believe the important material will be covered in class. If clickers are used for brief quizzes on assigned readings or homework to encourage preparation, then class time can then be spent in more productive ways than 'coverage'" (e.g., Knight and Wood, 2005). -- Caldwell, J.E. (2007). Clickers in the large classroom: current research and best practice tips. CBE, Life Sciences Education. Vol. 6, Spring, 8-20.

A "clicker" solution for students coming unprepared to class:

At the beginning of a class, the instructor gives students a quiz based on previously assigned readings. The questions for that quiz are projected at the front of the class, and students use their clickers to respond to the questions. A grade-value, albeit minimal, should be assigned to each quiz.

"When linked to grades, and particularly if it becomes a daily feature of the class, an Audience Response System increases attendance (Cue, 1998; Jackson and Trees, 2003). Physics instructors report that when clicker scores accounted for 15% or more of the course grade, attendance rose to 80-90%....Other instructors, however, report that when clickers contribute 5% or less to the course grade, their effect on attendance remains negligible." -- Caldwell, J.E. (2007). Clickers in the large classroom: current research and best practice tips. CBE, Life Sciences Education. Vol. 6, Spring, 8-20.

A "clicker" solution for students not attending class:

Coercing students to attend class can be achieved by alloting a portion of their grade to their mere attendance, and clickers can facilitate the taking of attendance. Essentially, when students arrive at class, they simply "log in" to the clicker system, which immediately records them as "present"; the record of attendance for the entire class can then be imported into the gradebook for the course. Such an approach is not without its problems, though. A student might, for example, bring the clicker of his or her absent friends and sign in on their behalf. Moreover, is a room full of students, half of whom don't really want to be there, better than a room half full of students all of whom do want to be there?