I report on the demographics and experiences of two cohorts of students who completed an 8 week course. The cohorts include: 29,000 students who took the course via MOOC, and 100 students who took the course on campus at Georgia Tech. Of those students, the data is based on 879 respondents who completed the course online and 57 who completed the course on campus. Both groups were provided the same curricula and were assessed according to the same criteria.
Focus of this article
In this article I present data without commentary. However, I would like to provide this important note: Please keep in mind that this is a rather advanced course at the intersection of Computer Science and Finance. Statistics are accordingly different than one might expect for introductory level courses.
Executive summary
Some of the more interesting results of this study include:
- The completion rate for MOOC students who invested $40.00 at the beginning of the course for a validated certificate was 99.0%.
- The completion rate for on campus students was 74.0%.
- The completion rate for MOOC students overall was 4.5%.
- MOOC students who completed the course reported a higher satisfaction with the course than on campus students according to all quantitative measures.
- A plurality of MOOC students are US citizens: 32%
- Most on campus students are from China and India: 39%, 35% respectively, 14% US.
- MOOC students are 60% White.
- On campus students are 76% Asian.
- Proportionally more African Americans completed the MOOC than the on campus course: 1.2% vs 0.0%
- Proportionally more women completed on campus than in the MOOC: 30% vs 6.3%
Background and overview
In order to assess the quality and impact of Massive Open Online Course (MOOC) delivery of curricular material we recently concluded the following experiment: We taught the same curriculum to students via a Coursera MOOC and also via traditional in-person delivery to students on the Georgia Tech campus.
The material is based on the first 8 weeks of Georgia Tech’s graduate level course CS 7646: Machine Learning for Trading. At Georgia Tech, this course is taught primarily to graduate students with majors in Computer Science and Quantitative and Computational Finance.
The students who enrolled for online delivery outside Georgia Tech followed a conventional MOOC process of viewing recorded lectures and responding to multiple choice response quizzes. In the case of programming assignments, the students responded to questions that can only be answered using the output of the program they created.
The students on campus at Georgia Tech were provided in-class delivery of the same material as well as access to the MOOC videos. They submitted their work via the MOOC protocols, and their work was assessed in the same way as those who took the online course.
Resource differences between online and on campus
The course experience for the two cohorts differed primarily according to the resources available to them. I summarize these below.
Resources available to students outside Georgia Tech, enrolled in the MOOC:
- Video lectures
- Online discussion forum (via piazza.com)
Resources available to students on campus at Georgia Tech:
- Video lectures
- Online discussion forum (via piazza.com)
- In class lectures by the instructor
- In class Q&A with the instructor
- Access to in-person interaction with a Teaching Assistant
- Self-organized on campus in person study groups
Students who completed the course online received a digital certificate that validates their completion. For the on-campus students, their scores are integrated into the overall course grade.
Enrollment and completion rates
29,000 students enrolled in the online MOOC. Of those enrolled in the MOOC, 378 followed Coursera’s “signature track.” This is a fee-based offering ($40) by Coursera in which students’ identities are validated. Completion rates for each group were as follows:
- MOOC Signature Track: 99.0%
- On campus: 74.0%
- Overall MOOC: 4.5%
Demographics
Results for each data element are reported as MOOC / On Campus
Age of students who completed the course (MOOC / On Campus):
- Average age: 35.0 / 24.1
- Max age: 73.0 / 38.0
- Min age: 16.0 / 21.0
Gender of students who completed the course (MOOC / On Campus):
- Male: 93.7% / 70.0%
- Female 6.3% / 30%
Race of students who completed the course (MOOC / On Campus):
- White: 60.0% / 10.0%
- African American: 1.2% / 0.0%
- Asian: 31.4% / 76.7%
- Native American or Pacific Islander: 0.0% / 0.0%
- Other: 4.3% / 6.7%
Country of origin of students who completed the course (MOOC / On Campus):
- US: 32.2% / 14.3%
- India: 7.9% / 35.7%
- China: 3.4% / 39.3%
- UK: 4.8% / 0.0%
- Russia: 4.2% / 0.0%
- Canada: 4.1% / 0.0%
- Germany: 3.4% / 0.0%
- Korea: 0.0% / 3.6%
- Spain: 3.4% / 0.0%
- Brazil: 2.6% /0.0%
- Australia: 2.5% / 0.0%
- Other: 31.3% / 7.1%
Highest education level attained (MOOC / On Campus):
- No formal school: 0.2% / 0.0%
- High School: 4.8% / 0.0%
- Some College: 4.2% / 0.0%
- Graduated College: 26.7% / 56.7%
- Some Grad School: 8.5% / 13.3%
- MS Degree: 43.7% / 30%
- Ph.D.: 11.9% / 0.0%
Quality of experience / satisfaction metrics
Students were provided a number of statements and asked to assess them on a 5 choice scale from “Strongly Disagree” to “Strongly Agree.” We report below the fraction of students who selected “Agree” or “Strongly Agree” (MOOC / On Campus):
- Considering everything, the instructor was an effective teacher: 87.5% / 85.7%
- For the amount of time I invested in this course, I’m happy with what I learned: 91.3% / 75.0%
- The course materials were presented in an engaging manner: 82.4% / 75.0%
- I would like to take a more advanced course on this topic: 94.0% / 67.8%
- I found this course personally fulfilling: 86.1% / 51.8%
- I learned what I was hoping to learn in this course: 78.0% 67.9%
Acknowledgements
Sourabh Bajaj and Weiyi Chen assisted significantly in the delivery of this course.
the augmented trader 
Jim Foley
November 20, 2013
Tucker, thank you for doing this summary, it is important to have data like this. I am wondering if you have any assessment data on the students – they all did the multiple choice quizzes and programming assignments. Did you see any difference in their performance?
Of course to fully evaluate performance data, we need to have a pre-test of knowledge prior to taking the course, and also some data on the aptitude of MOOC students and the on-campus students. When I did my “flipping the classroom” experiment, I was able to use student GPA as a proxy for aptitude.
Tucker Balch
November 21, 2013
Jim, we defined “completion” as a score of 70% or better. Because MOOC students drop out along the way at arbitrary times, and therefore do not complete the later assignments it is difficult to assess “performance” other than as completion.
I fully agree with the pre-test. I was hoping to do that for this study, but was not able to get it in place in time.
Gary Patterson
November 21, 2013
This was one of my favorite MOOCs so far. I fell behind but I still did the projects even though there was no credit. The QSTK software is so very cool to play with. Tucker, please schedule the course again soon so I can keep up and get a certificate!
I attended Ga Tech years ago (1977) when we didn’t see many Asians in class. I wonder why the demographic has changed so much? Is the in-state tuition really high? What’s wrong with all those Georgia boys? Seriously, what’s the explanation for the current demographic?
Tucker Balch
November 21, 2013
Gary, Undergraduate demographics are much different from graduate demographics. Many more US folks in the undergraduate population.
Brian Mullen
November 21, 2013
Hi Tucker, I joined the course in the final weeks so I was not able to complete it in time but I did find it extremely interesting. I will read the suggested supplemental reading and will take the course with the intent of earning the certificate the next time it is offered. I ‘got out’ from Tech in ’99 with a BCMPE degree and now work as a full-time software engineer. I passively trade stocks and sell covered option calls.
Tucker Balch
November 21, 2013
Glad you enjoyed the course. It may be offered again in the Spring, and then again next Fall.
Sudhir K
November 22, 2013
Hi Dr. Tucker Balcch, Though, I had signed up for many courses in the past, this course happened to be the first one to complete and get a certificate. I have found it of great value. Even though I work in Wall street as a programmer for many years, the information I found in this course was really useful for me. I am glad that I stayed on track till the end. I am really looking forward to the II part of the course! Thank you!.
Tucker Balch
November 22, 2013
It is great to learn that the course is useful for folks like you.
pedrots Trullenque
November 22, 2013
Hi Tucker,
I enrolled in your class but I didn’t finish it on time, mostly because I work full time. I’ve been watching the videos and reading the material on my weekends and It has been really fun.
As this course was named Computatational Investing Part I, is there any chance that we could see part II in the near future?
Tucker Balch
November 23, 2013
Yes. Part II will come next Fall.
gerardorosiles
November 22, 2013
Dr. Balch,
I thoroughly enjoyed this course. My objective was to gain more experience with Python and was really curious as to how to apply computational methods (and machine learning on the next course) to financial data. Now when I visit yahoo or msn money and use the available indicators, my experience is totally different.
BTW, I wanted to share anecdote that I did not relaize it was relevant to you until I visited your faculty webpage. Back in the late 90’s I was walking from Van Leer to the College of Computing. At the entrance of the building there were many students and administrative staff looking up to the sky. I asked one of the admins and she told me one of the COO graduate students who was also a pilot was about to do a fly by. I don’t think too many GT alumni fit that profile, so most probably it was you (as you reference in your webpage). So we waited for a few minutes and if I am not mistaked we saw two fighter planes making a hard turn.
Anyways, thanks for your time and dedications.
G. Rosiles
Tucker Balch
November 23, 2013
Ha! Yes, that was me.
yflorida
November 23, 2013
Thanks Dr. Balch, great course, I really enjoyed it. I’m Neither in computer science nor finance but I like both, and wish to be a trader.
I feel the course content and the student data should be profiled for WSJ/NYT for two reasons:
1) It is an excellent course, it deserves a larger audience
2) MOOC has a future, it offers people who can’t afford the time and money to take the LIVE course a chance to take the course online and appreciate the course.
michalsourek
November 27, 2013
Appreciate your work — as always, Tucker, good job! If I may add a small note, the Computational Investment MOOC, created and performed by you & your TA-s team, created an incredibly high level of personal, almost one-to-one approach to subject focused discussions during the whole course ( which must have been a tough work within that many thousands of participants ) Once comparing your approach with the Coursera’s father — Stanford prof. Andrew NG’s, Machine Learning, you, Tucker, have set such a yardstick that others will hardly get close to. The more I thank you very much.
Tucker Balch
November 27, 2013
Wow, thanks Michal for the kind words! Much of the success is owed to our great TAs.
Mohammed Sule Allscience
December 4, 2013
Can we the Africans get a scholarships at Coursera to study on campus for period of the week?
Mohammed Sule Allscience
December 4, 2013
Hello Dr. Tucker how can we help each other on educating everyone on the basis of computer science in this Modern world.Here in Ghana everything over here is hard and weeping is the greatest of all economic hardship children.So please help us succeed in education.Thanks.
Anonymous
December 6, 2013
Hey Tucker, thanks a lot for the course and for this statistical summary. Just a small note: I appear to be the 0.02% “No formal school” statistic. However, “Some high school” is probably a more accurate title for me and I think you should consider including that on the next survey. In Australia, leaving high school around 10th grade is common for people pursuing apprenticeships and so forth.
Henry
January 21, 2014
I just found this course and am interested in watching the video and work through the assignments. However, for some reason I can’t get access to the lecture slides and quizzes. I can only watch pre-views. Are you planning on offering the course again or is there a way to get those materials?
Tucker Balch
January 23, 2014
We’ll be offering it again in late Spring.
herrington61
August 10, 2014
Hi. Late to the party, but I have an observation. There may be many like me (early 50’s) who don’t really have any intention to ‘trade’ on the information, so I feel there are issues in evaluating MOOC attendee completion in the way you are doing it.
Essentially what I mean by ‘trade’ is not stock market trading; it is that a large percentage of people will not seek to use any qualification gained (whether through signature track or not) or the skills acquired, for any unified, identifiable purpose (whether grade accumulation, employment pursuit and so on). It enters the realm of a hobby.
Since there is no need to have the skill formally recognised then there is comparatively little need to aim for the pass / completion. In other words, older, qualified, more professionally settled people are simply less likely to proceed to formal completion, once they have the information they are seeking. I feel that demographic filtering in the course survey ought to be able to show this. It may require that the survey be tweaked; many of Coursera’s surveys do implicitly expect that people to be using courses they sign up to as a career choice.
Tucker Balch
August 11, 2014
Thanks for that feedback. I think you are correct in your assessment.