BestChoice, a Model for Interactive Web-Based Teaching
Sheila Woodgate and David Titheridge
The University of Auckland, Auckland, New Zealand
BestChoice is an open-access interactive
web site (www.bestchoice.che.auckland.ac.nz) that was developed initially to
support learning in large first year Chemistry classes at The University of
Auckland. More recently BestChoice has been expanded to provide
learning opportunities for a wider ranger of users. The model underpinning
BestChoice learning activities is simulation of the interchange of a
student with an experienced teacher. Thus student responses on
BestChoice question pages generate instant assessment and feedback.
BestChoice is innovative in its emphasis on teaching both concepts and
problem-solving strategies by guiding students in ways that promote their
understanding.
View a selection of
BestChoice Question Pages, by clicking on
Live (during 2006).
View screen shots of
BestChoice Question Pages by clicking on
For Archive (after 2006).
The discussion below considers the
Design considerations
Facilitating both teaching and learning using web-based activities
Teaching Chemistry to large classes at first year university is
largely transmission of information to students who adopt a passive role.
However, for quality learning to occur, the student must assume an active role
and engage with the content through problem-solving. Often the subject expert is
not readily accessible to offer guidance to the student during the learning
process. This may cause
- uncertain learners to become discouraged and to conclude that chemistry is
"too hard".
- confident learners to solve problems algorithmically without developing
full understanding.
BestChoice activities have been designed
to support the learning process by acting as a bridge between "hearing about it
in lectures" and "doing it yourself". The authors' aim was
- to construct activities that students like to use and which help them to
learn
- to develop methods for tracking how students use the activities in order
to detect how better to support learning
The process by which we have
achieved these objectives is described below.
Establishing a connection between web-based learning activities and other
study resources
Traditionally, students study using print materials. They
solve problems using pen and paper. Many students prefer to have model answers
readily accessible.
If BestChoice activities are to be valued as
learning tools, students should be able to relate these activities to other
study tools. The authors have established this connection by
- constructing BestChoice pages that are visually appealing and use
conventional symbolism in display of mathematical and chemical concepts
- developing activities that help students learn how to solve problems that
appear on written assessments
- giving users ready access to correct answers and to background material
required.
Overcoming some of the limitations of teaching over the
web
Reading from the screen is more difficult than reading
from a printed page.
BestChoice has been designed to present information
in a way that prevents cognitive overload. The emphasis is on requiring users to
answer questions. Content is then exposed in small quantities in the feedback
generated by the user's response.
Waiting for downloads
or for the computer to respond is frustrating.
BestChoice pages are text-based. Graphics
and animations are used only where necessary. This allows BestChoice to
be used on a dial-up connection.
Common web browser
answer-input devices (text-boxes, dropdown lists) do not support subscripts and
superscripts.
This limitation has been overcome by developing answer input
devices having support for both images and formatted text.
Constructing BestChoice modules to simulate a one-on-one tutorial
An
experienced teacher would, in a one-on-one tutorial, mix discussion of concepts
with questions to probe the student's level of understanding. Concepts are
presented as Review Pages in BestChoice. These are typically brief and
highlight only major principles.
Some examples of Review Pages can be viewed by clicking
Live (during 2006).
Screen shots of
Review Pages are available on clicking
For Archive (after 2006).
Most
modules begin with a Review Page. Concepts are then developed systematically
through the
- questions on Question Pages that follow each Review Page.
- feedback displayed when users answer questions on Question Pages.The
feedback either relates to the previous Review page by reiterating principles
or presages the next Review Page by applying and extending the principles.
Thus, during completion of a BestChoice module, learning is
scaffolded by the constant feedback and guidance provided by the system. Some
students struggle to understand what the question requires or may not be able to
enter the correct answer. These BestChoice users have the option for
the system to show them the correct answer so that they can overcome this
barrier and move on to complete the problem.
Connecting the BestChoice activities with the student's course of study
Teachers choose course resources to help students study. In order for
BestChoice learning activities to be accepted as a course resource, we
constructed groupings of modules for target groups both in first year Chemistry
courses at The University of Auckland and in Chemistry classes at New Zealand
high schools. We also constructed a General course (aimed at General Chemistry
courses) and an Organic Chemistry course for users outside the target groups.
Implementation of BestChoice
BestChoice for the web 2002
The BestChoice web site
went live in 2002, with a total of 1900 pages in 60 modules available. The
content was taken from a DOS version of BestChoice that had been
developed by one of us. 90% of the pages were Question Pages; thus the emphasis
was, as intended, on users entering responses that generate feedback.
The use of BestChoice was recommended to students in two
first-year courses at The University of Auckland in Semester 2, 2002. An
end-of-semester survey in one of these courses endorsed the approach taken in
BestChoice modules. The survey asked "What feature of BestChoice
did you like best?". The two most common responses to the question were
"combination of questions and review pages" and "feedback".
While we had
put in place the structure described above and had some validation of the
BestChoice approach to supporting learning, BestChoice 2002
was limited to one answer per question page; thus multistep problems extended
over several pages. Furthermore, the lack of control over the positioning of the
answer fields and their associated feedback was a severe hindrance in authoring.
BestChoice for the web 2003-2006
Both the BestChoice content
and the BestChoice system have been upgraded and expanded on a
continual basis during the last three years. BestChoice currently has
2500 different pages in more than 100 modules with 6000 possibilities for
interaction that result in instructive feedback. A variety of new features have
been developed so that the author can
- construct Question Pages having any number of answer fields.
- control the positioning of the answer fields and their associated feedback
on Question Pages.
- use a variety of types of answer fields on the same Question Page.
- use correct answers to one part of a Question Page to trigger the
appearance of a second part.
The BestChoice Question Pages
accessed using the links below show that we have made significant progress
toward mimicking stepwise problem-solving for a variety of problem
types.
To see actual
BestChoice Question pages, click on
Live (during 2006)
To access screen shots
of
BestChoice Question pages, click on
For Archive (after 2006).
Evaluation of BestChoice using an on-line survey
Establishing mechanisms for on-line evaluation of BestChoice by students
We wanted our student users to indicate what they liked
about BestChoice and how BestChoice could be improved.
Therefore we incorporated on-line evaluation mechanisms into the
BestChoice system, in order to get a continual feedback from our users.
In BestChoice 2002 students could enter feedback using a text
form accessed by clicking a Comments button that appears at the top of
every page. Very few comments were received. As a consequence, for 2003 we added
the survey below on the last page of each module. This allows the user to enter
comments and to rate the modules on a six-point scale. Each user could enter one
rating and one comment per module.
The comments entered in the survey indicate the student
perception of BestChoice
This on-line survey has been a very rich
source of student comment, most of which is pertinent to teaching and learning.
2100 complimentary comments were entered during 2003-2005. A few unedited
examples are given below.
User 1: Wonderful! Sooooo
helpful! Best feature is that you can read the theory, then do the quiz! (3 Nov
03)
User 2:This was fun and helped me understand
much easier. I was not placed under pressure at all, because I was allowed to
make mistakes. I really enjoyed this. (22 Mar 04)
User 3:
This has made so much that I didnt understand much clearer. The
little amounts of information followed by heaps of questions makes it really
easy to absorb!(04 Aug 03)
User 4: Man, you have
no idea how helpful Bestchoice is....the questions on them are relavent to what
we are learning and to make it even more helpful....the hint and give up
function help alot....thank you so much! (15 Jun 03)
User 5: This site is AWESOME!!! im really glad theres something like
this up and running to help students who are willing to study at home. it gives
a break from just reading and doing examples from books and really helps!!! im
sure my marks are going to improve!!!! THANKS!!! (30 Apr 03)
User 6: i learnt alot, & it went smoothly,
gradually getting harder & making me learn much more efficiently. it is very
helpful thank you (9 May 03)
The comments entered during 2003-2005 have
been assigned to categories. Three of these categories (compliment,
suggestion and criticism) are relevant to teaching and learning. 78% of comments
have been assigned to one of these. 22% of comments either do not pertain to
BestChoice or highlight typographical errors and bugs that have been
fixed.
Many of the comments relevant to teaching and learning have
multiple threads. Any comment that includes a compliment is assigned to
compliment. Critical comments that include a suggestion are assigned
to suggestion. The table shows the distribution of comments in these
three categories. The trend is toward a lower percentage in the
criticism category and higher percentages in the compliment
and suggestion categories as a result of enhancement both of the system
and the content.
| Comments entered in on-line survey |
| Year |
Total |
Compliment |
 |
Suggestion |
|
Criticism |
 |
| 2005 |
1173 |
68% |
15% |
17% |
| 2004 |
1134 |
67% |
14% |
19% |
| 2003 |
401 |
64% |
12% |
24% |
These data show that the learners perceive
that BestChoice is beneficial to them and helps them to learn.
Furthermore users suggest ways in which BestChoice can be improved
(even through critical comments). One advantage of a web-based system is that
problems identified can be rectified and the revised version made available
within a short time frame.
The module ratings (out of 6)
indicate that most students find BestChoice modules helpful
Students
indicate how helpful they have found a module by choosing one of the six radio
buttons on the end-of-module survey form. These range from not at all
(1) to fantastic (6). The average response rate to this part of the
survey over all modules and all cohorts of users is 30%. As shown below, the
radio buttons corresponding to 4,5 and 6 ratings are much more commonly chosen
than those corresponding to 1,2 and 3 ratings. The most frequent choice in each
year is fantastic (6).
| Module ratings using a 6 point scale |
| Year |
Total responses |
Average |
1 |
2 |
3 |
4 |
5 |
6 |
| 2005 |
6866 |
4.64 |
253 |
275 |
717 |
1383 |
2034 |
2204 |
| 2004 |
6442 |
4.71 |
186 |
266 |
570 |
1309 |
2019 |
2092 |
| 2003 |
2010 |
4.53 |
83 |
106 |
223 |
437 |
578 |
583 |
Using BestChoice as part of course materials
BestChoice for Chemistry students at The University of Auckland
In 2003, use of BestChoice was compulsory in two first
year Chemistry courses at The University of Auckland. In 2006 use of
BestChoice is compulsory in five first year Chemistry courses. In total
this involves approximately 2000 users, some of whom are enrolled in more than
one course.
Although one of the authors has had extensive experience
teaching first year Chemistry, neither of us is currently lecturing in any of
the first year Chemistry courses. Therefore, support of course coordinators and
teachers is crucial. These academic staff
- introduce their class to BestChoice, providing details of the
registration process.
- promote BestChoice activities as learning experiences.
- encourage students to use BestChoice to support their learning
during the entire course. One lecturer has recently referenced relevant
BestChoice modules in the course notes.
What form does the compulsory usage
take?
BestChoice modules are currently used as
assignments in four courses.
Cohort: Students in one of these courses are
science majors. Students in the other three courses are studying Chemistry at
pre-university level.
Contribution to overall assessment: All
courses have a maximum of 100 marks. Ten modules at 0.5 marks each are
compulsory in the course for science majors and in one of the pre-university
courses. Five modules at 1 mark each are compulsory for the other two
pre-university courses.
One of these courses was the first to use BestChoice
as an assessment component. Based on that experience, course coordinators
specify the modules to be completed and set completion deadlines. This ensures
that students begin to use BestChoice early in the
course.
BestChoice modules are used in one course as both
pre-lab activities and assignments
Cohort: In semester 1 these students are
intending Health Science students and science majors (including Chemistry). In
Semester 2 they are science majors.
Assessment of prelabs:
Evidence for completion of the module is submission of a printout of the
last page of the module with the lab script. For some experiments, the last page
is part of the background information/data required for the experiment.
Contribution of assignments to overall assessment: The course
maximum is 100 marks. Four modules at 0.5 marks each are compulsory.
The BestChoice pre-labs replace written pre-labs
that were marked by supervisors at the beginning of the lab session. Because
BestChoice pages are marked as they are completed, the supervisors now
have more time to help students with their practical work at the beginning of
the lab. The course coordinator also believes that there is less "blind copying"
and more opportunity for learning when the students complete pre-lab exercises
on line
This course coordinator has decided, for both the pre-lab and
assignment modules, to disable the Give Up and Show Next
options that allow BestChoice users to view the correct answer. Despite
this, these modules have high student ratings, and students do not comment on
the absence of these features in the on-line
survey.
BestChoice sessions in a computer lab were also a
feature of our four day catch-up course in 2006.
Each day 80 students used BestChoice modules that
reinforced lecture material.
BestChoice for New Zealand high school students
New Zealand high school
students use BestChoice on a voluntary basis. High schools in New
Zealand have computer labs, and students are often introduced to
BestChoice through their teacher booking a period in the school
computer lab. Teachers are made aware of BestChoice nationally through
an electronic newsletter circulated by a high school teacher.
Evaluating BestChoice through usage data
Usage of BestChoice has grown steadily
BestChoice
collects a variety of data for each user. This includes demographic details
(user-entered), registration date, number of logins, and a count of pages on
which answers have been entered as well as how many attempts were required to
get the correct answer and the time interval over which the question was
answered.
The table below shows that there has been a steady increase in
the number of registrations per year over 2003-2005. The increase in 2005 is
largely due to New Zealand high school students.
| Year |
Number of registrations |
Number of Active Users |
Logins |
| 2005 |
7 560 |
4 000 |
50 190 |
| 2004 |
5 983 |
3 200 |
37 916 |
| 2003 |
3 380 |
1 400 |
15 581 |
There has also been
growth in the number of active users. These users have answered
questions on more than 20 pages. In 2003 an active user entered at least 20
answers. In 2004-2005 an active user will have have entered at least 100
answers. There is a correlation between the percentage increase in either logins
or registrations over an entire year and the percentage increase in active
usage.
Active usage data highlights the importance of the connection to the user's
course of study
Users are asked on registration to identify themselves with
one of the institutions shown in the table below. The table also shows the
number of active users associated with each institution for 2005.
| Institution |
Active users in 2005 |
| The University of Auckland |
2230 |
| NZ high schools |
1670 (from 210 schools) |
| Other NZ Universities |
88 |
| Outside NZ |
12 |
The largest groups of active
users are those for whom specific courses have been created. The difference
between the two large cohorts is that while use of BestChoice is
compulsory for most university users, New Zealand high school students use
BestChoice on a voluntary basis.
The timing of logins gives evidence for how these two cohorts use BestChoice
The graph below shows the number of logins by students at The University of
Auckland for each month during 2005. This indicates that usage of BestChoice
during the semester (March-June and then July - November) is fairly
consistent and that usage drops during the inter-semester period. Comparison
with data from 2003 and 2004 shows that, due to the support of the course
coordinators, students in each subsequent year use BestChoice both
earlier in the semester and more consistently throughout the semester.
The graph above for New Zealand high school students logins
shows that their highest usage is at the end of the year. Thus this cohort
regards BestChoice primarily as an exam preparation tool. This
perception may, however, be changing because the percentage of logins to
BestChoice during the period February through September is higher for
2005 (53%) than for 2004 (41%).
In conclusion
What have we found?
The BestChoice project has shown that
web-based activities can facilitate learning by requiring the user to take an
active role, with the system then responding to the user's input. The guidance
offered by BestChoice makes successful completion of the problem more
accessible for more learners who, over time, should gain the skills to answer
the questions without guidance. Consider the unedited comments.
User 7: The main thing I liked
is that it gave me the opportunity to practice and learn from my mistakes. (31
Mar 06)
User 8: liked this topic. Being able to
see and work everything out myself really helped me understand this. Thanks (31
Mar 06)
User 9: It helped very much. It was much
easier to understand than my notes, textbook and lecturer! The reviews are so
concise and great. Thanks! (31 Mar 06)
BestChoice activities also
provide capable learners with an enhanced learning experience as insights are
revealed during problem-solving and appear in the form of feedback just as the
user has entered the answer that they thought was correct.
User 10: Very good, now I understand alot more
little things I wouldve never thought of asking. (19 Mar 06)
User 11: it lead us to think critically and
logically, and not just know facts that have already been provided for us in the
course notes. (19 Mar 04)
The BestChoice project has also
shown that users can inform the design of systems to support their learning. The
extent and quality of feedback entered by users has been overwhelming.
User 12: BC helps me understand
chemistry in a way that is both faster and more convenient than tedious text
book exercises - almost like having a personal tutor watching over my shoulder.
Thank you and keep up the good work! (2 Apr 06)
User
13: Bestchoice is simply the best learning technique i ever came
across. It walks me through the important steps to follow inorder to solve a
question and makes the theory more simpler. (26 Mar 06)
How does BestChoice compare to other learning tools?
A paper survey
asking "What feature of the course most helps you to learn?" has been circulated
in some courses where use of BestChoice is compulsory.
(Science majors) 100 surveys, 62 responses to this
question, BestChoice 41; Labs 39
(Science majors) 170 surveys, 111
responses, BestChoice 37; Handouts 44; Labs 25; Lecturers 20
(Summer pre-university) 80 surveys, 63 responses, BestChoice 25;
Lectures/lecturers 28; Handouts 13
The student responses indicate that
they view BestChoice as an effective learning tool that complements
traditional learning tools. BestChoice is not, and was never intended
to be, a replacement for these.
It is a challenge for lecturers to
accommodate the variety of learning styles and academic backgrounds in the large
classes at first year university level. Web-based activities like those in
BestChoice can be an addition to the suite of tools available to help
meet this challenge. Unlike the most other learning tools, use of
BestChoice by students generates data that can provide information on
how to support learning more effectively and efficiently.
Where to now?
The most frequent suggestion entered in our survey is
"more questions". New Question Pages are currently being added on a continual
basis. An important aspect of the writing of these pages is that new ways being
explored to present content interactively, including working toward enabling the
user to choose the extent of guidance offered on a BestChoice
page.
The discussion above is a broad analysis of data pertinent to
student perception and usage. It is now appropriate to extend these
investigations by considering student response data relevant to individual
modules and questions to determine whether areas of difficulty for students can
be identified. Once barriers to learning have been identified, the focus would
be on improving support for learning in these areas.
In conclusion, it
is evident that a large number of users believe that BestChoice is very
effective in helping them learn Chemistry. Furthermore, once a system like
BestChoice is in place, it thrives on student usage because usage
creates data that highlights where the system needs improving.
BestChoice (www.che.auckland.ac.nz/bestchoice)
is open access. Use Demo mode (no registration, no marks stored) or register. In
either case, we would appreciate your feedback.
Acknowledgement: We acknowledge Drs David Salter and
Judy Brittain, and Ian Torrie. David and Judy coordinate Chemistry courses that
use BestChoice. Ian advertises BestChoice to high school
teachers. We thank the Chemistry Department and the Teaching Improvement Grants
Committee of The University of Auckland for financial support.
