Home

TaskTracker is a toolkit for collecting and processing data of student activity during problem solving. TaskTracker allows to

• create and customize programming tasks
• distribute the tasks and collect results through plugins installed in user IDEs
• store, process, and analyze the results.

Structure

TaskTracker has a client-server architecture:

1. TaskTracker plugin for IntelliJ-based IDEs (PyСharm, IntelliJ IDEA, CLion and others) captures snapshots of code and interactions with the IDE, thus providing data to analyze the process of writing code in Python, Java, Kotlin, and C++. Support for other programming languages integrated with students’ IDEs can be easily added using the TaskTracker server.
2. TaskTracker server allows to configure the database for the plugins remotely—for example, by updating the list of tasks—and to collect data in a centralized manner. Remote or local configuration is possible.
3. Data post-processing and visualization modules allow to perform basic analysis, also for different subsets of collected data:
   • merging of activity logs and code snapshot files per task
   • task scoring based on your test files
   • filtering out of intermediate diffs by various criteria
   • creating charts: distributions of participants and tasks, action sequences, score dynamics, etc.

Benefits

• With just one monitoring tool, you can track your students’ problem solving in a considerable number of supported professional IDEs and programming languages.
• The tool is customizable: you can add tasks and plugin interface texts in any language as shown in the wiki.
• TaskTracker plugins for IDEs are easy to install and use. The server part is easy to deploy and administer.
• User privacy is preserved: all recorded data is anonymized (no user names, paths or data unrelated to tasks are sent) and stored only on your server. When preparing your datasets, you can also anonymize names of classes, methods, etc. and delete comments from code using our Dataset Anonymizer.

TaskTracker creates a dedicated file for each task selected by the user and tracks user activity only in this file. All consecutive code snapshots and user actions are collected grouped by task and stored in .csv format.

The basic version provides English and Russian interfaces, six tasks described in English and Russian with the opportunity to submit solutions in Python, Java, Kotlin, and C++. You can set up more human and programming languages and write your tasks in any language you added. If the programming language you add is not supported by any IntelliJ IDE, you can still solve tasks in this language in any IntelliJ IDE. The only problem will be the absence of syntax highlighting.

Goals

The primary goal of this project is data collection. First, tracking and analysis of students’ coding behavior can be a valuable source of insight into the learning process: for example, help the teacher to improve their course and understand which topics and assignments may be more difficult for students. Second, such data may be used in computing education research. If you choose to share your problem-solving data with us, it will help our studies in generating personalized hints during problem solving.

For more information about TaskTracker, read the research paper or presentation, or watch the demo.