When we give standard written tests in bachelor-level mathematics, we roughly want to assess three skills:

– executing computations,

– understanding of definitions and statements,

– writing of a mathematical argument.

Based on my experience in the last several years, I am convinced that executing computations and understanding of definitions and statements can be very successfully tested in a digital test with automatic grading. I have done it many times on three different testing platforms (ANS, Grasple, Sowiso). Gradually, this has led to the exam system that I will describe in this post. The system can be implemented in many variations, I will explain how we did it with Lotte Weedage, see more details in this article.

Our exam consisted of two parts.

Part 1 of the exam. Digital test. The digital test may have about 8-10 questions and take, say, 1.5 hour. The goal is to test executing computations and understanding of definitions and statements. Lotte and I required a score of at least 50% for passing the course..

– Executing computations is easy to test with questions that required only a numerical answer. Whenever possible, I use questions with randomized numbers, so that each student has a different answer. Btw, a numerical answer doesn’t need to be just a number; for example, Grasple allows answers even in a form of a vector and a matrix, and understands parametrized solutions of linear system.

– To test understanding of the definitions and statements, I never use vanilla multiple choice with one correct answer. Modern testing platforms allow so much more! I often use multiple choice questions with multiple correct answers, e.g.: `Which of the following 4 statements are correct?’ Below is an example of such question made for a basic probability course by Fabian Burghart, a postdoc at the TU/e. But there are even more creative ways to test understanding. For instance, you can ask the students to fill out the gaps in a definition, or mark errors in a proof. Usually students find such theoretical questions quite hard, and they cannot get any points by just memorizing the definitions!

Example of a question that tests understanding of Binomial distribution

 

Part 2 of the exam. Written test. Here we test writing of a mathematical argument. In this particular implementation, this part lasted 1,5 hours and had two problems: an easier one and a more difficult one. I believe that two problems are sufficient to evaluate student’s ability to write mathematics. Some teachers say, they would give three problems, but no one said they needed more than three. So, the idea is: give 2-3 representative problems and require a very high quality of writing. A correct solution is not enough! Lotte and I asked the students to write their solution in Polya’s 4 steps (analyse the problem, make a plan, execute the plan, reflect on the solution), and correct execution alone was worth 4 points out of 10. Students had to score at least 50% in this part to pass the course. When we required a good write-up only in two proof-type problems, we could be very strict about it. This way, I believe, we managed to communicate to the students that writing is indeed important in mathematics.

Investment in students’ practice. Importantly, for both parts of the exam, the students need to practice. For Part 1, the students need practice questions, ideally a large practice question bank. For Part 2, the students must understand the requirements, and they need a dedicated time during the course for practice and feedback. There are many ways to do this. Lotte and I had a weekly mandatory tutorial, called `proof of the week’, with homework, group discussion in class, and feedback of the teacher to each group; see details here. All these require time investments. But it is worth it! Creating questions, organizing practice and giving feedback is much more rewarding than grading huge piles of substandard work. And, once developed, materials will serve you for many years.