Berkeley Open Learning Technology (Bolt)

T(DesignDocument)?

Bolt is a software infrastructure for creating web-based training courses, and is designed to meet the needs of:

• Skill aggregation projects, where volunteers must be trained to perform various tasks;
• Volunteer computing projects, where educating participants can increase their enthusiasm and commitment.

These areas have properties that are much different from those of formal education:

• Churn: constant turnover, thousands of new students per day;
• Wide geographical distribution;
• Wide age distribution;
• Motivation: most volunteers have a pre-existing interest in the topic, and are motivated by acknowledgement (e.g. being marked as an "expert" on the project web site).

What Bolt does

Using Bolt, you can

• Create exercises of various types: multiple-choice, fill-in-the-blank, graphical, etc.
• Specify a course as a sequence of lessons and exercises.

Given such a course, Bolt does the following:

• It guides students sequentially through the course;
• If the student fails a quiz, they repeat one or more lessons and retry the quiz (Bolt courses are designed to be "fail-proof");
• Each student's progress is recorded in a database, and when they return to the course later they resume at that point.
• Bolt maintain an estimate of each student's mastery of the course material.

In addition, Bolt lets you create better courses; specifically, you can

• find out exactly how effective each lesson is;
• make statistically valid comparisons of alternative lessons;
• make "adaptive" courses in which different lessons are used for different groups of students

This is done as follows:

• Bolt records the timing and results of each student interaction (viewing a lesson or completing an exercise) in a database.
• Demographics (age, sex, education level, nationality) are stored for each student.
• Course documents can have various types of "control structures". For example, they can specify that a lesson should be chosen randomly from a given set, or should be chosen based on student demographics.
• Bolt offers macro-analytic tools that let you see the overall flow of students through your course (in the style of Charles Minard's map of Napoleon's march to Moscow in the war of 1812), revealing the points where they are getting bored or discouraged.
• Bolt offers micro-analytic tools that let you compare a set of alternative lessons, identifying those which are uniformly better, or are better for a particular demographic subgroup.

Creating exercises

An exercise is a PHP script. Here's an example containing a multiple-choice question (the '2' indicates that the correct answer is the last one):

<?php
require_once('bolt.php');
echo 'Conifers are so named because:';
bolt_exclusive_choice(
   array(
      'They are cone-shaped.',
      'They originated during the Coniceous era.',
      'They carry their seeds in cones.'
   ),
   2
);
?>

Here's an example that shows an image; a correct answer is a click in the indicated subrectangle.

<?php
require_once('bolt.php');
echo "Click on the dog's nose:";
bolt_image_rect(
   'dog.jpg',
   array(100, 60, 110, 70)
);
?>

Bolt supplies functions for other types of questions, such as inclusive multiple-choice and fill-in-the-blank. An exercise can include multiple questions.

At the implementation level, an exercise has three functions:

• When invoked with $mode_show set, it shows the exercise.
• When invoked with $mode_score set, it computes a score based on the responses stored in $_GET, and assigns the score to $score. Its text output, if any, is ignored.
• When invoked with $mode_answer, it shows and "answer sheet" based on the responses stored in $_GET. If the response is correct and no answer sheet is to be shown, it sets $no_answer_sheet.

Bolt's exercise primitives perform all these functions for you; however, you're free to implement your own exercises.

Course documents

The structure of a Bolt course is defined by a ​JSON document. Here's an example of a course with two lessons followed by an exercise:

{
   "name": "Identifying Sierra Conifers",
   "description: "Learn to identify the major conifers of California's Sierra Nevada",
   "items": [
      {
          "type": "lesson",
          "name": "Introduction",
          "file": "intro.html"
      },
      {
          "type": "lesson",
          "name": "The Linnaean hierarchy",
          "file": "linnaean.html"
      },
      {
          "type": "exercise",
          "file": "linnaean.php"
      }
   ]
}

Course items can be grouped into sets; for example:

{
   "type": "set",
   "show_n": 1,
   "order": "random",
   "items": {
      {
      ...
      }
   }
}

The attributes of a set include:

• show_n: the number of items in the set to show
• order: whether to show the items sequentially or randomly

Items (lessons, exercises, and sets) can include properties, e.g.:

{
   "type": "lesson",
   "name": "The Linnaean hierarchy",
   "file": "linnaean.html"
   "properties": {
      "verbal_level": 12,
      "detail_level": 0.8
   }
},

When Bolt has a choice of items (e.g. when it encounters a set from which a single item is to be shown) it calls, for each item, a course-supplied matchmaking function, passing to it the student object (which includes demographics such as age) and the item's properties (represented as a PHP object). The matchmaking function returns a number representing the estimated effectiveness of that item for that student, and Bolt chooses the item with the highest value.

Memory refresh

Bolt offers a memory refresh system that periodically repeats exercises and, if necessary, lessons. Memory research suggests that this is necessary for students to shift learning to long-term memory. This mechanism works as follows:

• A sequence of inter-refresh intervals is defined. For example, (7, 28) means that an exercise should be repeated 7 days after it is first taken, and then every 28 days thereafter.
• Bolt provides a function that returns the set of items, for a given student, for which refresh is due. Your course can use this function to implement a "Review now" button on web pages.
• Bolt provides a "review mode" in which the student is presented with exercises due for review.

The bigger picture

Bolt's primary goal is to serve the needs of volunteer computing and skill aggregation projects. However, we believe that it can also become a tool for research in education and cognitive science, for a variety of reasons:

• Experiments can be deployed with large sample sizes.
• Experiments can be deployed immediately, with no dependence on the academic calendar. Significant results are available in days rather than months or years.
• The student population varies widely in age, language, and education level.
• The student population is self-selecting for interest in the topic area, and has diverse learning goals.
• Experiments are not limited by standards or syllabi.
• Experiments can be conducted without dependence on educational institutions or teachers.