C0D3 DOJO Design doc

[anso3]

Intro

Current UI design link: click me

Goals

One of the main benefits of learning is for users to engage creatively with the exercise sets. Their learning would be improved if they could solve creative uses of the concepts they are learning, and also come up with their own creative exercises.

This feature is designed to engage users by giving them more practice exercises and helping them engage other students with their own creative exercises.

Problem

Currently, the only engagement that users have in their learning journey is submitting challenge exercises for each lesson module. 8-12 challenges per module. Besides this, it is all reading and self paced, self guided exercises. Most students do not do all of the exercises, and new exercises are difficult to add, so content is a bit old. New exercises are added at a rate of a few exercises a year!

Solution

There are two types of student groups in our solution. First is the learning group, second is the mentor group.

Learners Group

When a student starts a module, they could jump right into exercises and get quick feedback by submitting answers or writing code, and getting quick feedback (your answer or implementation is right / wrong). Each exercise should also support a discussion where students could ask questions. This approach has several benefits:

• Quick feedback - When users submit a solution, right away they know whether it is correct or wrong. This is a huge improvement over our existing "reading" format where the users could have made a mistake and would never know.
• Direct discussion and higher student - student engagement - If a student has a question, they could directly ask on the page, on the specific exercise. Currently, when a student has a question on the exercise section, it is difficult to ask for help. The student would have to share a link to the exercise and ask on discord. And not every student has done the exercise before so a student currently don't get quick support when they ask.
• Fast navigation and review - System could keep track of what the users are weak at. Students are able to pickup where they left off.

Sample UI - Note: A exercise could also ask users to submit a function to do X. The check for correctness would then be a test suite instead of a hard-coded string.

When user clicks on View Discussion, the would be taken to a discussion page below. The discussion page mock should have included the original question for students to main context

Notes:

• Infinite number of exercises for each section. Some students feel comfortable after solving 3 exercises, some after 100. Up to them!
• What happens when users got the question wrong? Shake and red border.
  • We want to give users an ability to try again but also don't want to stress the student so after they get an answer wrong, there should be a "view solution" button where we give them the answer.
    • There will be no explanation of the answer because we don't want to add friction when creating exercise problems and we want users to engage with each other. Also, alot of questions will have similar explanations so it will get very repetitive.

Mentors Group

Mentors are users who has completed a module. They could then submit exercise to challenge the future generations of students (which would show up in the prior example UI). Submitting exercises could be fun (and therefore more engaging for the students) and educational in the following ways:

• Seeing all the wrong submissions from students: This will help the mentor gain a deeper understanding of where common traps students may fall for, which would help them pay attention to their own code in the future!
• Seeing how other students approach their exercises: Gives mentors new perspectives on their exercises.
• Observe how impactful their exercises were: Mentors should be able to see how many students decided to "save" their example exercises.
• Extra practice writing tests / coming up with impactful exercises
• Observe and interact through discussions - By allowing discussions on the exercises, mentors are able to engage more deeply with future students and identify common learning confusion points.

A tab will only show up when users have completed the lesson

Notes

• We believe that discussions are important, so to facilitate new students being able to explain concepts to newer students, each question and discussions around that should have a expiration time.
  • However, some core questions should be immune from expiration time.
• Flagged problems: ie a question was posted in JS1 but the question requires knowledge of Arrays (covered in JS2).
  • Flagged problems are taken out of circulation and put into admin page (maintained by eng team).
  • Admin should be able to: unflag a problem mark as hidden.
• Problem state transitions:
  • normal → flagged: (when user flags it, the provide a reason).
  • flagged → removed: Admin could hide the questions from view because of irrelevance or incorrect information
  • removed → flagged: When user updates the problem (by correcting it)
  • flagged → normal: With admin approval, problem goes to normal state.

Admin Group

• /admin/exercises would display a list of all flagged problems with 3 action buttons
  • Unflag - puts the exercise back to normal mode
  • Remove - puts the exercise to removed mode

Metrics for measurement

To measure for improvement over time:

• Number of users who complete module / Number of users who submits exercises.
  • Increase means students found the engagement rewarding / fun. Decrease means there were no incentive to submit exercises, and we have to look into why that is.
• Number of questions users answers before completing module.
  • Increase means users are finding the questions rewarding and worth repeating over and over again.
  • Decrease means lower quality questions.

Implementation

Database Schema

Considerations

• Each exercise needs to be associated with:
  • associated with a lesson module AND submodule. I.e. Functions section of JS0.
    • Exercise: We don't have a list of defined submodules / labels in the current lessons table.
  • associated with a user (creator)
• Each exercise has the following resources:
  • Discussion table
    • How do we store the question being asked, and its corresponding answers?
    • Do we want upvotes to highlight the most relevant / educational questions?
  • Answer (which belongs to a student)
  • How to we query like... "top 5 answers" for a specific exercise?

Proposed Schema

id, createdAt, and updatedAt will always be present, so omitted from the schema definitions.

module - stores a list of submodules for each lesson

• lessonId - foreign key to reference the lesson
• name - name of submodule
• content - text/blob, will hold markdown string
• Example:
  • lessonId - 3, name - function

exercises - stores exercise exercises

• submoduleId - foreign key to submodule
• description - markdown string
• answer - answer string
• testable - boolean to represent whether this is a testable exercise or not
  • true - when evaluating answer, run the answer against the testStr column
  • false - string matching against answer column
• testStr - test string to be run.

ExerciseInteraction - stores the interactions students have with each exercise

• Fields:
  • exerciseId - foreign key
  • userId - foreign key
  • note - note str
    • If a note is empty for a user, it is by default skipped
  • flag - string: reason why its flagged. Maybe answer is wrong?
  • answer - first answer str that user created
  • isCorrect - boolean
• Can user have > 1 interaction with an exercise? No. It does not make sense for users to have multiple notes / flags per exercise
• Sample queries:
  • Select * from ExerciseInteraction where note is not empty and exerciesId = 5;
    • This will display the # of favourited types

ExerciseQuestion - stores questions that the user asked

• Fields:
  • ExerciseId, userId → foreign keys
  • title → string
  • content → markdown string (blob / text?)

ExerciseAnswer - Stores answers that users submitted

• Fields:
  • exerciseQuestionId, userId → foreign keys
  • content → markdown string

ExerciseDiscussionLikes - stores user likes

• The reason why we want to have this table separately instead of putting a column in ExerciseAnswer table is because we eventually want to have a feature where students could see the answers they have "liked" so they could use it as a bookmark for great answers.
• Fields:
  • exerciseQuestionId, exerciseAnswerId, userId

GraphQL Queries

Student Page

On page load, initial request should query for

• All modules for lesson
• All user question interactions
• All questions for each lesson

User Interactions

• Queries
  • Next question → query for next question
  • Discussion → Pulls discussion and answers for the exercise problem
• Mutations
  • Add question, add answer, upvote answer, flag, add note

Mentor Page

On page load, initial request should query for

• all questions for the module that the user has created
  • For each question, include subsequent stats
• mutations
  • Add an exercise

Admin Page

On page load, initial request should query for all lessons

User Interactions

• When admin selects a lesson
  • Query for all modules / flagged questions for each module
• Mutations
  • Create module (after lesson selection)
  • For flagged questions, state transition to hide or unflag

Launch Dependencies

Rather than launching everything at once, we should incrementally release features so work could be done in parallel.

Databases tables we would end up creating: module, exercises, exerciseInteraction, exerciseQuestions, exerciseAnswer,exerciseDiscussionLikes

Phase 1 - Modules

It takes alot of time to curate exercise content, so phase1 would be optimized for the ability to create module content

• Admin page: https://www.c0d3.com/admin/lessons/js0
  • GraphQL
    • There should be an admin path to CRUD lesson modules
      • Read: graphql endpoint should send back a list of all lesson modules.
      • Create: Send graphQL mutation to create a module
        • takes in lessonId, name, content
      • Update: GraphQL Mutation to update a module
        • takes in moduleId, name, content
      • Delete: GraphQL Mutation to delete a module
        • takes in a moduleId
  • UI - Similar to https://www.c0d3.com/admin/lessons
    • Users should be able to directly create a module
    • Click on a list to select existing module to update it (or delete it)

Phase 2 - Exercises

Mentor Group

• Mentor should see a nav bar next to lessons challenges review Edit Lessons
  • Clicking on Edit Lessons should take users to https://www.c0d3.com/editLessons/js0
  • GraphQL - CRUD exercises
    • Create: mutation to create an exercise problem
      • Takes in moduleId, description, answer, test
  • UI
    • Should see a list of lesson modules on a left nav bar (like https://www.c0d3.com/admin/lessons )
    • On selection, should have create button option to create an exercise
    • Also should see a list of exercise problems that user has created previously

Student Group

• Based on mocks - ExerciseInteraction
• User interactions

Phase 3 - Discussion & Stats

ExerciseDiscussionQuestion, ExerciseDiscussionAnswer, ExerciseDiscussionAnswerLikes

Stats to the mentor group page

[evo777]

This is good.

[songz]

@all-contributors please add @mino323 for documentation

[allcontributors]

@songz

I've updated the pull request to add @mino323! 🎉