A Comparison of Online MOOC Versus On Campus Course Delivery

Posted on November 20, 2013 by

20


I report on the demographics and experiences of two cohorts of  students who completed an 8 week course. The cohorts include: 29,000 students who took the course via MOOC, and 100 students who took the course on campus at Georgia Tech. Of those students, the data is based on 879 respondents who completed the course online and 57 who completed the course on campus.  Both groups were provided the same curricula and were assessed according to the same criteria.

Focus of this article

In this article I  present  data without commentary. However, I would like to provide this important note: Please keep in mind that this is a rather advanced course at the intersection of Computer Science and Finance. Statistics are accordingly different than one might expect for introductory level courses.

Executive summary

Some of the more interesting results of this study include:

  • The completion rate for MOOC students who invested $40.00 at the beginning of the course for a validated certificate was 99.0%.
  • The completion rate for on campus students was 74.0%.
  • The completion rate for MOOC students overall was 4.5%.
  • MOOC students who completed the course reported a higher satisfaction with the course than on campus students according to all quantitative measures.
  • A plurality of MOOC students are US citizens: 32%
  • Most on campus students are from China and India: 39%, 35% respectively, 14% US.
  • MOOC students are 60% White.
  • On campus students are 76% Asian.
  • Proportionally more African Americans completed the MOOC than the on campus course: 1.2% vs 0.0%
  • Proportionally more women completed on campus than in the MOOC: 30% vs 6.3%

Background and overview

In order to assess the quality and impact of Massive Open Online Course (MOOC) delivery of curricular material we  recently concluded the following experiment: We taught the same curriculum to students via a Coursera MOOC and also via traditional in-person delivery to students on the Georgia Tech campus.

The material is based on the first 8 weeks of Georgia Tech’s graduate level course CS 7646: Machine Learning for Trading. At Georgia Tech, this course is taught primarily to graduate students  with majors in Computer Science and Quantitative and Computational Finance.

The students who enrolled for online delivery outside Georgia Tech followed a conventional MOOC process of viewing recorded lectures and responding to multiple choice response quizzes. In the case of programming assignments, the students responded to questions that can only be answered using the output of the program they created.

The students on campus at Georgia Tech were provided in-class delivery of the same material as well as access to the MOOC videos.  They submitted their work via the MOOC protocols, and their work was assessed in the same way as those who took the online course.

Resource differences between online and on campus

The course experience for the two cohorts differed primarily according to the resources available to them. I summarize these below.

Resources available to students outside Georgia Tech, enrolled in the MOOC:

  • Video lectures
  • Online discussion forum (via piazza.com)

Resources available to students on campus at Georgia Tech:

  • Video lectures
  • Online discussion forum (via piazza.com)
  • In class lectures by the instructor
  • In class Q&A with the instructor
  • Access to in-person interaction with a Teaching Assistant
  • Self-organized on campus in person study groups

Students who completed the course online received a digital certificate that validates their completion.  For the on-campus students, their scores are integrated into the overall course grade.

Enrollment and completion rates

29,000 students enrolled in the online MOOC.  Of those enrolled in the MOOC, 378 followed Coursera’s “signature track.” This is a fee-based offering ($40) by Coursera in which students’ identities are validated.  Completion rates for each group were as follows:

  • MOOC Signature Track: 99.0%
  • On campus: 74.0%
  • Overall MOOC: 4.5%

Demographics

Results for each data element are reported as MOOC / On Campus

Age of students who completed the course (MOOC / On Campus):

  • Average age: 35.0 / 24.1
  • Max age: 73.0 / 38.0
  • Min age: 16.0 / 21.0

Gender of students who completed the course (MOOC / On Campus):

  • Male: 93.7% / 70.0%
  • Female 6.3% / 30%

Race of students who completed the course (MOOC / On Campus):

  • White: 60.0% / 10.0%
  • African American: 1.2% / 0.0%
  • Asian: 31.4% / 76.7%
  • Native American or Pacific Islander: 0.0% / 0.0%
  • Other: 4.3% / 6.7%

Country of origin of students who completed the course (MOOC / On Campus):

  • US: 32.2% / 14.3%
  • India: 7.9% / 35.7%
  • China: 3.4% / 39.3%
  • UK: 4.8% / 0.0%
  • Russia: 4.2% / 0.0%
  • Canada: 4.1% / 0.0%
  • Germany: 3.4% / 0.0%
  • Korea: 0.0% / 3.6%
  • Spain: 3.4% / 0.0%
  • Brazil: 2.6% /0.0%
  • Australia: 2.5% / 0.0%
  • Other: 31.3% / 7.1%

Highest education level attained (MOOC / On Campus):

  • No formal school: 0.2% / 0.0%
  • High School: 4.8% / 0.0%
  • Some College: 4.2% / 0.0%
  • Graduated College: 26.7% / 56.7%
  • Some Grad School: 8.5% / 13.3%
  • MS Degree: 43.7% / 30%
  • Ph.D.:  11.9% / 0.0%

Quality of experience / satisfaction metrics

Students were provided a number of statements and asked to assess them on a 5 choice scale from “Strongly Disagree” to “Strongly Agree.” We report below the fraction of students who selected “Agree” or “Strongly Agree” (MOOC / On Campus):

  • Considering everything, the instructor was an effective teacher: 87.5% / 85.7%
  • For the amount of time I invested in this course, I’m happy with what I learned: 91.3% / 75.0%
  • The course materials were presented in an engaging manner: 82.4% / 75.0%
  • I would like to take a more advanced course on this topic: 94.0% / 67.8%
  • I found this course personally fulfilling: 86.1% / 51.8%
  •  I learned what I was hoping to learn in this course: 78.0% 67.9%

Acknowledgements

Sourabh Bajaj and Weiyi Chen assisted significantly in the delivery of this course.

About these ads