Internet Learning Volume 1, Number 1, Fall 2012 | Page 26

Assessing Virtual Students’ Quiz Performance in Web Mediated Synchronous Instruction 25
Data
Within the four semesters, the middle portion of the course served as the sample, omitting the
initial three and last three classes in a 14-week semester. The rationale for leaving out quizzes
administered during the initial classes was that explaining the option for web-based attendance,
covering the logistics of attending remotely, and assembling a reasonable number (three or more)
of virtual attendees generally occupied the first three classes. Delaying measurement also
allowed for the learning curves of both attending virtually and taking quizzes. The rationale for
omitting classes late in the semester is an artifact of the grading policy, which mandated
attendance of 10 classes. After the required 10 attendances, completed quizzes were assigned
grades of “A” automatically. Another atypical facet of end-of-semester classes was the
scheduling of guest lectures, when no quizzes were administered.
A total of 88 students participated in the four sections of the public budgeting course,
accounting for 567 total attendances. One important difference with previous research is that the
groups of students attending face-to-face and attending virtually are not discrete. Approximately
half the students attended in both modes, with most of those choosing a single mode opting for
face-to-face attendance (36 percent of all students), versus 14 percent selecting entirely virtual
attendance. For the 50 percent of the class attending both virtually and face-to-face, the median
number of virtual attendances was three (3) and the mean was 3.8.
The implication of these patterns of attendance is that the populations of virtual and faceto-face attendees were intermingled substantially. Dual-mode attendees accounted for 42 percent
of the quizzes administered in person and 64 percent of the virtual quizzes. The mixing of these
populations tended to dampen effects reflecting the characteristics of early-adopting students,
since deliberate and even cautious adopters were also included among the virtual attendees. The
in-person attendances, with the majority (58 percent) being attributable to solely in-person
attendees, were potentially more representative of late-adopting attributes. But other reasons
could contribute to exclusively in-person attendance, such as the scheduling of other on-campus
activities in proximity to the budgeting class (including consecutive classes, quite common since
MPA classes were offered primarily in the evenings).
Dependent Variable
Quiz grades serve as the dependent variable, normalized for relative performance using ordinal
values. Students earning the highest grade on a particular quiz, always an “A,” were assigned a
value of four (4). The next-highest grade, which varied from “B” to “A-,” resulted in an
assigned value of three (3), and so forth to the lowest grades, which resulted in assigned values
ranging from two (2) to zero (0): with none of the quizzes producing more than five grade levels.
Normalizing the grading distribution preserved the order of student performance, which more
closely tracked the phenomenon of interest. Selection of a categorical dependent variable
necessitated use of ordinal logistic regression, as opposed to multiple regression, which assumes
a normal distribution for the dependent variable.
Quizzes usually included two questions, composed of true-false or multiple choice types,
sometimes requiring a brief explanation to support the student’s selection. Quizzes were
intended to be confirmatory, reinforcing points emphasized during lectures, rather than
challenging students to integrate concepts. Accordingly, the distribution of quiz grades was
skewed toward the highest grade, with nearly 70 percent of quizzes graded “A.”