Welcome to Assessment – all about managing tests, projects, and the grade center within BlackBoard.
Today we will use Standards 3, 4, and 5 of QM to guide us in approaching the assessment of student learning and interactions. This includes various types of learning assessments, including tests/quizzes, assignments, projects, presentations, and discussions. The idea we should have is one where both formative and summative assessment methods are used to find out whether students are learning, and where they might yet be struggling. In this way, we can take continual “snapshots” of how they are doing over time in the class, and adjust accordingly when we find there are gaps in student knowledge. If we can diagnose issues while students are still with us for the term, there is a better chance that we will be able to help them be their best when it comes time to complete final projects and tests at the end of the semester.
Standard 2 is the first of many standards that look at the idea of alignment – or ensuring that your course objectives, assessments, and activities all match up in terms of level and expectations. Consider what you want your students to come away from your course knowing or being able to do. Now, put that in terms of some action verbs – what would you be observing or seeing them do if they are successful in your class? Under what condition do you know they are successful? To what degree should they be able to perform these observed behaviors? These are things to keep in mind as you go. Ideally, avoid words like “understand”, “learn,” “appreciate,” or anything else that isn’t a directly observable behavior. We can’t see what’s going on in students’ heads, but we can see them explaining what they know or feel, creating a presentation, or writing a paper.
Remember Bloom’s Taxonomy of the three different domains? This will most definitely come in handy as you try to determine what kinds of assessments will go along with each of your objectives and related activities. Go back to your objectives and use them as a guide when developing your assessments – you will always be right on target if you use those objectives as your blueprint. See the following documents for additional help and explanation of each area: http://assessment.uconn.edu/docs/LearningTaxonomy_Affective.pdf http://assessment.uconn.edu/docs/LearningTaxonomy_Cognitive.pdf http://assessment.uconn.edu/docs/LearningTaxonomy_Psychomotor.pdf
Here is a wonderful and very helpful wheel for different types of verbs that can be used for different kinds of objectives. Another good basic list is at http://www.teach-nology.com/worksheets/time_savers/bloom/. Searching for “Bloom’s Taxonomy verbs” helps you find good verbs that you can match up with your activities, as well as determine what level you asking students to do things at. For example, if you find you have all low-level objectives (explain, identify, discuss) in a 400-level class, you might consider changing things up and adding some application, evaluation, and synthesis objectives to the mix. Student products they might actually do as a result that you can assess are correlated as well, so you can easily build objectives for your course just by using this handy wheel. What kinds of objectives can you build?
The SOLO taxonomy is an alternative to Bloom’s that looks at the attainment of objectives in a more developmental way. The idea of assigning action verbs to behaviors we want students to accomplish is the same, but the SOLO Taxonomy includes the idea that the same objective can be achieved at different levels depending on the student’s development at that time. For example, in understanding photosynthesis, a beginning student may only be able to describe the basic concept, but a more advanced student will be able to design experiments to demonstrate photosynthesis in action.
Some more examples of good objectives are here, along with ways to measure them. For instance, in discussion and identify objectives, a discussion board or a multiple choice test might be a good option. On the other hand, if you want students to be able to analyze or create something, they should be asked to engage in some sort of project, or perhaps design their own tests. Consider your own objectives and what you want students to achieve, and this should lead you to the best assessments for the objective. And of course, in some cases you might be going through a little trial and error – this is normal! Sometimes what you think will work well with an objective does not bring forward the picture of student learning that you would like. Don’t be afraid to tweak and experiment to get the best results.
Assessments in your course should match up with what students need to know and be able to do. It sounds like common sense, but it’s easy sometimes to design assessments that contain information that might be important to you, but not to the course. It’s also easy to take assessments to higher or lower levels than where your students are at.
Richard Mayer is somewhat famous in the instructional design world as a major contributor to the body of research in multimedia. He has studied how people learn through computers and multimedia (the mix of text, graphics, and video on a screen) for many years, and has come up with a few important theories. Here are just a few of these research-tested theories. Of them, the modality principle and multimedia effect are probably the most helpful to teachers and instructional designers, as they help us understand how words, images, and audio can be most optimally used to enhance learning.
When you’re looking for something beyond PowerPoint, consider options like Prezi.com, a very interactive and “fun” tool that produces very attractive presentations. There is also VoiceThread.com, where you can create something that’s more like a cross between a video and a presentation. It also allows you to collaborate with others to turn your work into a truly interactive presentation experience. Of course, there are many other tools out there as well, such as Animoto, PreZentit, and several others. The Cool Tools for Schools wiki has a lot of excellent options that are worth exploring in more detail. Try something and see what you get! You never know – you may be pleasantly surprised by how powerful a free piece of software can be.
Many students appreciate video because it can illustrate important points even better than just text or graphics alone. Good videos, of course, implement those multimedia principles that Mayer researched, and will include voice, onscreen text, moving images, and sometimes still images to fully explain concepts across both channels of the brain (audio and visual). And luckily, the Internet – especially YouTube – is absolutely stuffed with amazing videos on just about every subject available. It is really worth checking out what is available through YouTube, Vimeo, and even your textbook publisher, to find helpful supplements to your lessons.
“Mashups” in BlackBoard refer to bringing in content from other online sources such as YouTube, Slideshare.com, Flickr, and other places. These little gems within BlackBoard can turn an otherwise boring course into one that is rich with different sorts of media, all without you having to do a lot of extra work! With each of these tools, you can search the available libraries right within BlackBoard, saving you time. Note that Kaltura is the only exception – here is your repository of your own videos, such as things you record from WebEx. When you create a WebEx recording (and more about that in a moment), the recordings go to your Kaltura library and can be pulled into BlackBoard from there. Find all Mashup options under the Build Content menu in BlackBoard.
There are several different lecture capture options available to you, including live, in-classroom recording and software where you can record your presentation at your desk. Echo360 is our live lecture capture option on campus, and it allows for recording everything that happens in your classroom, including the screen of your computer and the audio in the room. After about 12-24 hours, your link is available for viewing. One link is all that is needed to share with others, as every video that goes along with your course will become part of your “EchoCenter”. Camtasia is the university’s pre-recorded capture software of choice, and it can capture everything on your screen as well as your voice and images from your webcam. You can choose what feeds you wish to have on (such as turning off the camera if you prefer not to be on it!) at any time. Camtasia is fairly easy to use, and can take video of any portion of the screen that you wish. You can upload your work directly to YouTube or the company’s own Screencast.com site, or you can save as MP4 to upload into BlackBoard via the Kaltura Mashup option.
Check our website for information about Echo360, including where it is available on campus and tutorials on how it can be used. You can contact IS through firstname.lastname@example.org to set up your account and get your sessions scheduled for automatic recording if you so desire. You can also set them up manually or start a recording at any time once your account is set up. Each Echo session will have its own unique link, but everything gets assembled at your EchoCenter for each class, allowing you to provide one link to students that is good all semester long.
Pre-recorded videos are great for online classes, or flipped classrooms (students watch your lecture outside of class, then come to class ready for activities) Record everything that’s happening onscreen, then edit and publish to Blackboard or YouTube Camtasia makes this process easy – record from your desk You can make high-def videos in the Video Express Room (LSF-253) and edit in Camtasia later
Camtasia is available free to Purdue staff on and off-campus, on both Mac and Windows platforms. It is able to publish to many formats and has features that make it worth learning and using. I personally use Camtasia for the many, many videos that I record for my students each semester, and receive good feedback on them. To request Camtasia, log in at itap.purdue.edu/learning/tools/camtasia and follow the prompts.
The Camtasia interface is relatively simple and provides a place for tools and options, a place to preview your video as it is being edited, and a timeline to show where your tracks are, and where your audio is. You can even see the waveforms in your audio to see whether you have any particularly loud or soft spots, and these can be edited with volume tools, or deleted if you don’t need them. You can use the Red and Green slider tool at the top of the timeline to select portions of the video to cut, copy, move, or delete.
These are some resources to help with understanding best practices in using Mayer’s principles
There are many tools available for assessment at all kinds of levels, including both within BlackBoard and outside of BlackBoard. We can use technology in all kinds of interesting ways to encourage students to be more creative and provide us with many different ways to figure out if they know what we need them to. A few examples include discussing on forums, producing multimedia, presenting with slides and other media, and creating books, blogs, wikis, journals, and videos.
Tests in BlackBoard can be designed as either “regular” assessments or as mobile-compatible tests, meaning they can be accessed through a mobile device or the BlackBoard app on iPad/Android. This is a great option if you have a lot of mobile users in your class, although be warned that there are far fewer options with mobile-compatible tests, and fewer test choices. You can import tests through Respondus if you wish to set up tests in advance in a Word document and upload them. This is a great option for those converting their tests into an online format, or for those who download test banks from publishers that are not yet BlackBoard-ready. Respondus can be downloaded by contacting Purdue ITAP – see the link in the slide for directions. Note that tests have many options that can help limit cheating and also ensure that all students have equal access. This includes the test availability exceptions, which allow for various exceptions for individuals or groups of students. This is especially useful for those with extra time accommodations, or for those who are making up late work. Lockdown Browser and Respondus Monitor are other tools that can be used to limit cheating. LockDown Browser works best in lab environments, as it limits student access to other applications while taking a test. However, if you are giving quizzes online, you can combine Lockdown Browser with Monitor, a new tool that will be available by the fall to our campus, which sets up video and audio surveillance of students taking their tests. The system prompts the student to go through various actions to prove that they are alone and that they are who they say they are, and then the video of each test-taker is recorded and stored for later viewing by the instructor. Should you have concerns about a student’s test, you can go back to the video at any point to review whether there was any wrong-doing going on.
Some tips for giving successful tests in BlackBoard include checking your Test Options very carefully. Above all, avoid turning on Force Completion. This idea sounds great – it limits the student to taking the test in only one sitting – but the problem is that if the student’s Internet connection fails for any reason while taking the test, the test is forced to conclude. This means that students who legitimately lost their connection cannot continue with the test, and they will contact you frantically asking for a reset. They cannot return to their test in progress, so they would have to start all over again if you chose to allow them to go back to the test! This can be very unpleasant for all involved, so the rule of thumb is to avoid Force Completion all together. Test Availability Exceptions, as mentioned before, can be wonderful for make-ups and accommodations – just set up your conditions for any student who needs it. Auto-Submit is a good option that you may wish to turn on, as it forces the quiz to submit even if the student runs out of time on a timed exam. Otherwise, when the timer runs out, the student may still continue if she is not done with the test. Finally, take note of the options for receiving feedback and ensure that you are giving students every chance at feedback that you wish to give them at any point in time. By default, no feedback other than the score is provided upon submission, but you can choose to give students an overview of the correct answers or the entire test, either right away or after all other students have finished the exam. Note that if you do choose to only release feedback after everyone has submitted, this will include any student who did not take the exam. You will need to manually assign them a grade of 0 for a missing assessment if you wish for BlackBoard to release the feedback under that condition.
In order to grade things like papers, projects, and presentations, you’ll probably want a rubric. A rubric is a comprehensive way to grade, because it provides you with objectives and observations to look for in a student’s work and compare to a standard view of what is “good.” This makes the act of grading much less subjective, as can often happen with things like presentations and multimedia. When students know what you’re looking for, they are much more likely to perform where you want them to, and it takes the mystery out of learning. No student likes to be held to standards that they don’t understand. Providing rubrics and good objectives will help with this.
Some great example rubrics and information about them can be found here. BlacKboard even has a nice tool for creating rubrics and applying them to assignments, discussions, and other items students can turn in to you.
BlackBoard rubrics take some of the pain out of scoring projects, because you can just click the little bubbles by each performance a student makes and it totals up their score for you. You can even apply the same rubric to multiple assignments or discussions, so if you score everyone the same way on every discussion, for example, you do not have to create new rubrics every time. Rubrics can also be imported and exported between courses, so they can be shared with your colleagues who might teach the same or similar activities.
You can access work that is ready to be graded through the Needs Grading option under the Grade Center tool in your left-hand toolbar. Optionally, you can go into the Full Grade Center and look for work with an exclamation mark by it – this work will be submitted but ungraded. Click on the arrow next to the exclamation mark to Grade User attempt or activity as needed.
The Full Grade Center allows you to see all grades at a glance in a spreadsheet-like view. You can actually enter items into the gradebook just like a spreadsheet, in fact – simply click in a box and enter a score, then hit Enter to save the grade. This is just one way of grading work, and necessary for any manually entered columns you may have. You can create new columns for all kinds of purposes, such as work that was submitted in class rather than in BlackBoard, or for special calculated columns like special percentage columns, letter grades, and more. BlackBoard will do a fair amount of calculations to your gradebook data, including weighted percentages, dropping highest or lowest grades, and other useful tricks. See the resources at the end of this presentation for specific tutorials. You can use the Manage button to organize all of your gradebook columns as well, and the “arrow menus” by each column name and within each cell of your gradebook include options for editing, sorting, and getting statistics as needed.
Some cool tools that may be useful to you and to your students include BlackBoard’s various tools, WebEx for live collaboration, Google Docs, Prezi, new templates from Microsoft that add more spice to your presentations or papers, Screencast-o-matic or Jing/Camtasia, Wordpress, or SimpleBooklet. These are all tools that students can use freely to promote their own learning, and they often require minimal knowledge or effort from you. These tools can all be figured out by students pretty easily, so giving them some options and letting them “run with it” is not a bad way to go about things. Try it and see what happens!
Here are some more useful resources on assessments and rubrics, including Rubistar which will help fill in the blanks for you in creating rubrics and take even more pain out of the process. Yes, creating rubrics and authentic assessments does take a little more time than the traditional testing route. However, it is well worth it when you consider how much more your students will be able to do as a result of your course in the long run!
Please contact us and visit http://centers.pnw.edu/teaching for all workshop notes, links, and training needs. Thank you!
QM Standards 2, 3, & 4: Objectives and Alignment
Quality Matters Boot Camp 2016
Use the Quality Matters rubric and Standards
2, 3, & 4 to help guide your course
Create measurable course-level and unit-level
Discuss appropriate assessment strategies in
your course course and discipline
Use a variety of formative and summative
assessment methods to maximize your
picture of how students are doing
Alignment and orienting your students to the learning
tasks isVERY important
Use this formula:
Audience: who are the learners?
Behavior: what do you want to be able to observe them
Condition: under what conditions will they do this?
Degree: to what degree must they perform to be
For example: After the course, students will be able to
discuss at least three major outcomes from the
American CivilWar, including social and political
This stuff helps you find and create the most appropriate assessments!
Type of Objective How to Measure
Discuss/Recall/Identify Discussion board, summary paper,
Apply/Use Essay or fill-in test, labs, report paper
Analyze Problem-solving, analysis paper, case
Create/Design Research paper, creative essays, art,
prototypes, plans, student-created tests,
Evaluate/Judge Journals, case studies, debates, peer
Make sure that the information that you’re
testing students on it directly related to what
students are supposed to know and be able
Nice-to-know information is great, but
shouldn’t be included if it’s not directly tied to
Make sure your assessment matches the level
of the objective and is not above or below the
students’ skill level
Multimedia effect: words and pictures are more
powerful than words alone
Continuity: related words and pictures should be
near each other onscreen
Personalization: students learn better from more
informal, conversational styles
Coherence: Extraneous or “nice to know”
information does not help student learning
Modality: Students learn better when their visual
channel is not overloaded (words as speech rather
than onscreen text)
Prezi – for the cool factor
VoiceThread – for the interactive and
TheVideo Express Room (LSF 253): put on a
live presentation in hi-def!
Camtasia: record and edit screencasts from
anywhere (including fromVideo Express)
esentation+Tools for many more choices
Sometimes, a video is
needed to show crucial
Luckily, the Internet is
full of wonderful
videos for all sorts of
Available for streaming on-
demand about 24 hours after
Can be scheduled for your
class time – no button
Share one link with students
for the whole semester
Example of Echo360:
CAMTASIA /VIDEO EXPRESS
Captures everything on the
screen, plus voice and camera
Excellent for presentations,
or showing students how to
do a task on the computer
Can take video of any portion
of the screen that you wish
Can be uploaded directly to
YouTube or saved for
Find out if Echo is in your classroom:
Get your account set up: email
email@example.com or fill out a ticket request
Echo can be automatically scheduled to come on
when you are teaching, and shut off when you’re
Each session has a unique link but are all
assembled at your EchoCenter, which has one
Great for online classes, or flipped classrooms
(students watch your lecture outside of class,
then come to class ready for activities)
Record everything that’s happening onscreen,
then edit and publish to Blackboard orYouTube
Camtasia makes this process easy – record from
You can make high-def videos in theVideo
Express Room (LSF-253) and edit in Camtasia
Purdue has a university license for you to have
Camtasia in your office and on your home
machine (Mac and Windows)
asia/ to download the license request form and
wait approximately 24-48 hours for response
You will be able to download from a secure
Filelocker the Camtasia version of your choice,
along with SnagIt – a great tool for capturing
and editing still, single-frame screen captures
Mayer’s multimedia theory: http://www.learning-
Common but questionable principles of multimedia
10Tools to FlipYour Class (tip: most are screen-capture
Flipped class best practices:
Quizzes and tests
Rubrics for the creation of:
Multimedia (text + images, video, etc)
Digital video and audio
E-books, wikis, blogs
Can be mobile-capable or browser-only
Can be supported through Respondus
LockDown Browser and Respondus Monitor
You can create tests from scratch or import from
RespondusTest Generator (or other tools)
Test exceptions settings allow different options
for different students (like extra time)
tutorials/ for all videos on tests
for info on downloading Respondus
Edit theTest Options -
check your settings
Do NOT turn on Force
Exceptions for make-ups
Auto-Submit is ok –
forces the student to
stop when timer ends
Decide on options for
how and whether
students can receive
Need help with the
text for your rubric?
Try rubric-makers that
make your job easier!
You can grade using rubrics right within BlackBoard
Rubrics make your job a little easier when grading
more extensive assessments
Video tutorials available:
Use the left-hand
sidebar and choose
Needs Grading to view
when new work has
been turned in
Alternately, in the Full
submitted will have a
Allows you to see all
grades at a glance –
works like a spreadsheet
You can create columns
for various purposes,
Use the Manage button
to change column
Use the “arrow-menus”
to change settings, view
and edit grades