Online Assessment: Smarter Balance

Online assessments are becoming the norm. There are online assessments for most tests including the Graduate Record Examinations (GRE) and the Michigan Test for Teacher Certification (MTTC). Personally, I've found most of the computer based assessments I've encountered to be be reasonable. Unfortunately, the Smarter Balanced online assessment that Michigan has adopted to administer K-12 standardized assessments is far from that.

I had the opportunity to browse through the some of the tests offered by Smarter Balanced, which covered both the Math and English topics and spanned across K-12 grades. Although I am an avid technology user, I personally found the online test to be cumbersome and difficult to use. It made me imagine how much more difficult using this assessment might be for students who may not be as tech savvy.

The overall structure of the online test is not bad, but several things are formatted in ways that makes the user interface unfriendly. For example, the question information, and the question itself were placed in 2 panels side to side. The left panel would include the necessary background information to answer the question, and the right panel would contain the actual questions. The problem arises when the background information is a long piece of text or includes some tables and graphs, and the student has to scroll up and down, and even worse, scroll sideways just to see the information. The student can expand the width of the panel to see the information better, however it then covers up the question. This type of structure makes is difficult to navigate because a student has to keep switching back and forth between panels and has to keep scrolling to see various parts of the information.

I also found it frustrating that the test wouldn't let me move around between questions. Even though I could flag a question I wasn't sure about, I couldn't skip it and come back to it later. I had to provide some sort of answer to be able to move on. This is unlike the GRE; I clearly remember being able to move back and forth between questions in each section, making it easier for me to answer the questions I felt ready for. Also, every time I would do something not allowed by the system (such as skip a question), I got a scary looking error message, with extraneous information! I don't think that the student needs to know that this error is "[Message Code 12784]". Why is that even there? I found it to be really distracting, and a bit distressing to see this message every time.

On the math tests, the calculator is pretty bulky, and a bit hard to use. I kept trying to press "Ans" to get the answer but it wouldn't work, until I realized there is a small button with the = sign that I'm supposed to be pressing. Some questions asked the students show their work, but the answer space looked like it was to provide a paragraph of text. It would be really difficult showing work that would ideally need a diagram, involves division and other operations by just typing it in like regular text. It would make it very difficult to read, and I'm sure would lead to higher rate of mistakes. Also, specifically in the G7 Math Performance Task, it says that the test has questions 1-5 at the top, but then there are only 4 questions. I had to spend some times trying to figure out if I had missed any of the questions.

On the English Language Art tests, some of the words were underlined lightly, and wasn't clear why that was until I realized that if I clicked on those words, a small box appeared with its definition. This is actually useful, but I wish was presented in a different way as I found this hard distracting. Also, some of questions were in multiple choice format, and asked the student to pick one correct answer. Although the students should only pick one answer, sometimes the system would only allow students to pick one answer, whereas other times it allows them to pick more than one. I think this just increases confusion and difficulty because of the inconsistency presented in multiple choice answers.

Overall, I think online assessment tools can be a useful thing. It saves paper, and could allow for immediate results. However, with an online assessment tool such as the Smarter Balance, the test starts to test how well the student can overcome some of the user interface challenges, rather than the material itself. This is especially true if we consider that students can easily get nervous about tests and the technological difficulties might hinder their performance. Another scenario is that the students realize this test doesn't affect their life directly, and so when they see that the test is difficult to use and navigate, they give up easily. So what exactly is this tool measuring? Is it really measuring student performance, or is the student performance becoming overshadowed by the tool itself? How fair is it to base teacher evaluation on "student performance" when it is gathered by online assessment tools such as this one?