Core Methods in Educational Data Mining: Syllabus

• Course: [HUDK 4050, Teachers College, Columbia] (http://www.columbia.edu/~rsb2162/EDM2015/index.html)
• Instructor: Charles Lang, [email protected], @learng00d, #TCLA16
• Day/Time: Tuesdays/Thursdays, 11:00am - 12:40pm
• Location: GDH 535
• Instructor Office Hours: Thursdays, 1:00pm - 3:00pm or by appointment in GDH 540
• Prerequisite: HUDM 5122 or HUDM 5126 or approved statistics/computer science data mining course.
• Credits: 3
• Required Technology: Laptop with RStudio installed, Phone with the Sensor Kinetics Pro app installed

Course Description

The Internet and mobile computing are changing our relationship to data. Data can be collected from more people, across longer periods of time, and a greater number of variables, at a lower cost and with less effort than ever before. This has brought opportunities and challenges to many domains, but the full impact on education is only beginning to be felt. Core Methods in Educational Data Mining provides an overview of the use of new data sources in education with the aim of developing students’ ability to perform analyses and critically evaluate their application in this emerging field. It covers methods and technologies associated with Data Science, Educational Data Mining and Learning Analytics, as well as discusses the opportunities for education that these methods present and the problems that they may create.

Course Goals

The overarching goal of this course is for students to acquire the knowledge and skills to be intelligent producers and consumers of data mining in education. By the end of the course students should:

• Systematically develop a line of inquiry utilizing data to make an argument about learning
• Be able to evaluate the implications of data science for educational research, policy, and practice

This necessarily means that students become comfortable with the educational applications of three domain areas: computer science, statistics and the context surrounding data use. There is no expectation for students to become experts in any one of these areas but rather the course will aim to: enhance student competency in identifying issues at the level of data acquisition, data analysis and application of analyses to the educational enterprise.

Assessment

In EDCT-GE 2550 students will be attempting several data science projects, however, unlike most courses, students will not be asssessed based on how successful they are in completing these projects. Rather students will be assessed on two key components that will contribute to their future sucess in the field: contribution and organization. Contribution reflects the extent to which students participate in the course, whether or not students complete assignments and quizzes, attend class, etc. Organization reflects how well students document their process and maintain data and software resources. For example, maintaining a well organized bibliographic library with notes, maintaining a well organized Github account and maintaining organized data sets that are labelled appropriately. To do well in HUDM 4050 requires that students finish the course with the resources to sucessfully use data science in education in the future. Do the work and stay organized and all will be well!

Tasks that need to be completed during the semester:

Weekly:

• Attend class
• Weekly readings
• Comment on readings on Twitter
• Maintain documentation of work (Github, R Markdown, Zotero)

One time only:

• Ask one question on Stack Overflow
• In person meeting with instructor
• 8 short assignments (including one group assignment)
• Group presentation of group assignment, 3-5 students each
• Submit Zotero file with semester's notes
• Record a tutorial session

Week-by-week

Unit 1: Introduction

Unit 2: Data Sources & Their Manipulation

Unit 3: Structure Discovery

Unit 4: Prediction

Unit 1: Introduction

Class 1 - Introduction (9/7/16)

Learning Objectives

• Be familiar with course philosophy, logic & structure
• Install and be familiar with the software to be used in the course
• Appreciate the importance of tightly defining educational goals

Class 2 - LA, EDM and the Learning Sciences (9/13/16)

Learning Objectives

• Be familiar with the kinds of work done in the fields of LA and EDM

Tasks to be completed:

Read/watch and comment:

• Siemens, G. and Baker, R.S.J. d. 2012. Learning Analytics and Educational Data Mining: Towards Communication and Collaboration. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (New York, NY, USA, 2012), 252–254.
• Educause 2015. Why Is Measuring Learning So Difficult?

Due: Assignment 1 - Set up

Class 3 - Data Sources (9/15/16)

• Be familiar with a range of data sources, formats and extraction processes
• Be familiar with R & Github & markdown

Tasks to be completed:

1. Read/watch and comment:

• The R Markdown Cheat sheet: 2014.

Unit 2: Data Sources & their Manipulation

Class 4 - Data Wrangling (9/20/16)

Learning Objectives:

• Understand the importance of workflow and recording workflow

Tasks to be completed:

Read/comment:

• Leong, B. and Polonetsky, J. 2016. Passing the Privacy Test as Student Data Laws Take Effect. EdSurge.
• Greller, W., & Drachsler, H. (2012). Translating Learning into Numbers: A Generic Framework for Learning Analytics. Journal of Educational Technology & Society, 15(3), 42–57.

Class 5 - Data Wrangling (9/22/16)

Learning Objectives:

• Be able to perform a data tidying workflow

Tasks to be completed:

Read:

• Saturday Morning Breakfast Cereal: 2016.

• Data Wrangling Cheatsheet: 2015.

Class 6 - Data Wrangling (9/27/16)

Read/Comment:

• Clow, D. 2014. Data wranglers: human interpreters to help close the feedback loop. Proceedings of the Fourth International Conference on Learning Analytics And Knowledge (2014), 49–53.
• Young, J.R. 2014. Why Students Should Own Their Educational Data. The Chronicle of Higher Education Blogs: Wired Campus.

Class 7 - Data Wrangling (9/29/16)

Learning Objectives:

• Be able to perform a data tidying workflow
• Be familiar with a range of data manipulation commands

Tasks to be completed:

Read:

• Data Wrangling Cheatsheet: 2015.

Unit 3: Structure Discovery

Class 8 - Visualization (10/4/16)

Learning Objectives:

• Understand the place of data visualization in the data analysis cycle
• Be familiar with a range of data simulation commands

Tasks to be completed:

Read:

• Kay, J., Maisonneuve, N., Yacef, K., & Reimann, P. (2006). The Big Five and Visualisations of Team Work Activity. In M. Ikeda, K. D. Ashley, & T.-W. Chan (Eds.), Intelligent Tutoring Systems (pp. 197–206). Springer Berlin Heidelberg.

Class 9 - Visualization (10/6/16)

Learning Objectives:

• Be able to generate basic visualizations during on-the-fly analysis

Watch:

• Chapter 6 in Baker, R. (2014). Big Data in Education: video 1, video 3, video 4

Class 10 - Networks (10/11/16)

Learning Objectives:

• Understand the basic premise of graph theory applied to social networks

Tasks to be completed:

Read/Comment:

• Grunspan, D. Z., Wiggins, B. L., & Goodreau, S. M. (2014). Understanding Classrooms through Social Network Analysis: A Primer for Social Network Analysis in Education Research. CBE-Life Sciences Education, 13(2), 167–178.

Class 11 - Networks (10/13/16)

Learning Objectives:

• Conceptualize a data structure suitable for network analysis, generate a network and produce basic summary metrics

Tasks to be completed:

Read/Comment:

• Hanneman, R.A. and Riddle, M. Chapter 1: Social Network Data. Introduction to Social Network Methods.

Due: Assignment 2 - Social Network

Class 12 - Clustering (10/18/16)

Learning Objectives:

• Understand the basic principle and algorithm behind cluster analysis

Tasks to be completed:

Read/Comment:

• Bowers, A.J. (2010) Analyzing the Longitudinal K-12 Grading Histories of Entire Cohorts of Students: Grades, Data Driven Decision Making, Dropping Out and Hierarchical Cluster Analysis. Practical Assessment, Research & Evaluation (PARE), 15(7), 1-18.

Class 13 - Clustering (10/20/16)

Learning Objectives:

• Create a suitable data structure and perform clustering on a sample

Tasks to be completed:

Watch:

• Chapter 7 in Baker, R. (2014). Big Data in Education: video 1, video 2

Due: Assignment 3 - Clustering

Class 14 - Principal Component Analysis (10/20/16)

Learning Objectives:

• Be familiar with the basic ideas behind dimension reduction and the reasons for needing it
• Understand the basic principles behind Principal Component Analysis

Tasks to be completed:

Read/Comment:

• Konstan, J. A., Walker, J. D., Brooks, D. C., Brown, K., & Ekstrand, M. D. (2015). Teaching Recommender Systems at Large Scale: Evaluation and Lessons Learned from a Hybrid MOOC. ACM Trans. Comput.-Hum. Interact., 22(2), 10:1–10:23.

Class 15 - Principal Component Analysis (10/20/16)

Learning Objectives:

• Perform principal component analysis

Tasks to be completed:

Watch:

• Georgia Tech 2015. Feature Selection. Youtube.

Due: Assignment 4 - Principal Component Analysis

Class 16 - Domain Structure Discovery (11/8/16)

Learning Objectives:

• Be familiar with the range of strategies for mapping domains and skills

Tasks to be completed:

Read/Comment:

• Matsuda, N., Furukawa, T., Bier, N., & Faloutsos, C. (2015). Machine Beats Experts: Automatic Discovery of Skill Models for Data-Driven Online Course Refinement. International Educational Data Mining Society.

Class 17 - Domain Structure Discovery (11/10/16)

Learning Objectives:

• Be familiar with the Q-matrix method

Tasks to be completed:

Watch:

• Chapter 7 in Baker, R. (2014). Big Data in Education:video 6

Unit 4: Prediction

Class 18 - Prediction (11/8/16)

Learning Objectives:

• Understand why prediction is desireable goal, the various meanings of the word and general strategies employed across statistics, machine learning and experimental psychology

Tasks to be completed:

Read/Comment:

• Kucirkova, N. and FitzGerald, E. 2015. Zuckerberg is Ploughing Billions into “Personalised Learning” – Why? The Conversation.
• San Pedro, M.O.Z., Baker, R.S.J.d., Bowers, A.J., Heffernan, N.T. (2013) Predicting College Enrollment from Student Interaction with a Intelligent Tutoring System in Middle School. Proceedings of the 6th International Conference on Educational Data Mining, 177-184.

Class 19 - Prediction (11/10/16)

Learning Objectives:

• Employ a linear prediction model

Tasks to be completed:

Watch:

• Chapter 1 in Baker, R. (2014). Big Data in Education: video 1

Class 20 - Classification (11/15/16)

Learning Objectives:

• Understand the concept of classification and its relationship to modeling

Tasks to be completed:

Read/Comment:

• Cortez, P., & Silva, A. M. G. (2008). Using data mining to predict secondary school student performance. In Proceedings of 5th Annual Future Business Technology Conference. Porto, Spain: EUROSIS.
• Baker, R.S.J.d., Corbett, A.T., Roll, I., Koedinger, K.R. (2008) Developing a Generalizable Detector of When Students Game the System. User Modeling and User-Adapted Interaction, 18, 3, 287-314.

Class 21 - Classification (11/17/16)

Learning Objectives:

• Implement a CART model

Tasks to be completed:

Watch:

• Chapter 1 in Baker, R. (2014). Big Data in Education: video 3 & video 4

Class 22 - Data Jam (11/22/16)

Learning Objectives:

• Practice of methods learned so far using real world data

Tasks to be completed:

• Analysis of data ad generation of model

Class 23 - Diagnostic Metrics (11/29/16)

Learning Objectives:

• Understand and apply the following diagnostic metrics to models: Kappa, A', correlation, RMSE, ROC

Tasks to be completed:

Read:

• Zheng, A. 2015. Evaluating Machine Learning Models. O’Reily Media. Chapter 2: Evaluation Metrics p.7-18

Watch:

• Chapter 2 in Baker, R. (2014). Big Data in Education: video 2, video 3 and video 4

Class 24 - Diagnostic Metrics (12/1/16)

Learning Objectives:

• Understand over-fitting and institute cross-validation

Tasks to be completed:

Watch:

• Chapter 2 in Baker, R. (2014). Big Data in Education: video 5
• Georgia Tech 2015. Cross Validation. Youtube.

Class 25 - Knowledge Tracing (12/6/16)

Learning Objectives:

• Understand the concepts behind Bayesian Knowledge Tracing

Tasks to be completed:

Read: Corbett, A.T., Anderson, J.R. (1995) Knowledge Tracing: Modeling the Acquisition of Procedural Knowledge. User Modeling and User-Adapted Interaction, 4, 253-278.

Class 26 - Knowledge Tracing (12/8/16)

Learning Objectives:

• Understand Bayesian Knowledge Tracing

Tasks to be completed:

Watch:

• Chapter 4 in Baker, R. (2014). Big Data in Education: video 1

Class 27 - Class Presentations (12/13/16)

Class 28 - Class Presentations (12/15/16)

Class 28 - Data Jam (12/20/16)

Class 28 - Data Jam (12/22/16)

Fine Print

1. All examinations, papers, and other graded work and assignments are to be completed in conformance with the [Academic Integrity Policy](http://www.tc.columbia.edu/administration/diversity/index.asp? Id=Civility+Resources+and+Policies&Info=Civility+Resources+and+Policies&Area=Studen t+Miscon duct+Policy). Students who intentionally submit work either not their own or without clear attribution to the original source, fabricate data or other information, engage in cheating, or misrepresentation of academic records may be subject to charges. Sanctions may include dismissal from the college for violation of the TC principles of academic and professional integrity fundamental to the purpose of the College.

2. The College will make reasonable accommodations for persons with documented disabilities. Students are encouraged to contact the Office of Access and Services for Individuals with Disabilities for information about registration (166 Thorndike Hall). Services are available only to students who are registered and submit appropriate documentation. As your instructor, I am happy to discuss specific needs with you as well.

3. The grade of Incomplete will be assigned only when the course attendance requirement has been met but, for reasons satisfactory to the instructor, the granting of a final grade has been postponed because certain course assignments are outstanding. If the outstanding assignments are completed within one calendar year from the date of the close of term in which the grade of Incomplete was received and a final grade submitted, the final grade will be recorded on the permanent transcript, replacing the grade of Incomplete, with a transcript notation indicating the date that the grade of Incomplete was replaced by a final grade. If the outstanding work is not completed within one calendar year from the date of the close of term in which the grade of Incomplete was received, the grade will remain as a permanent Incomplete on the transcript. In such instances, if the course is a required course or part of an approved program of study, students will be required to re-enroll in the course including repayment of all tuition and fee charges for the new registration and satisfactorily complete all course requirements. If the required course is not offered in subsequent terms, the student should speak with the faculty advisor or Program Coordinator about their options for fulfilling the degree requirement. Doctoral students with six or more credits with grades of Incomplete included on their program of study will not be allowed to sit for the certification exam.

4. Teachers College students have the responsibility for activating the Columbia University Network ID (UNI) and a free TC Gmail account. As official communications from the College – e.g., information on graduation, announcements of closing due to severe storm, flu epidemic, transportation disruption, etc. -- will be sent to the student’s TC Gmail account, students are responsible for either reading email there, or, for utilizing the mail forwarding option to forward mail from their account to an email address which they will monitor.

5. It is the policy of Teachers College to respect its members’ observance of their major religious holidays. Students should notify instructors at the beginning of the semester about their wishes to observe holidays on days when class sessions are scheduled. Where academic scheduling conflicts prove unavoidable, no student will be penalized for absence due to religious reasons, and alternative means will be sought for satisfying the academic requirements involved. If a suitable arrangement cannot be worked out between the student and the instructor, students and instructors should consult the appropriate department chair or director. If an additional appeal is needed, it may be taken to the Provost.

6. All students must bring delicious food on the last day of class (DN)