Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Programming exercises: Add server support for feedback suggestions #7136

Merged

Conversation

pal03377
Copy link
Contributor

@pal03377 pal03377 commented Sep 3, 2023

Warning

This PR includes a migration! Do not deploy to a normal test server. Instead, use ma-schwind.ase.cit.tum.de.

Feature proposal: #7094

Checklist

General

Server

  • Important: I implemented the changes with a very good performance and prevented too many (unnecessary) database calls.
  • I followed the coding and design guidelines.
  • I added multiple integration tests (Spring) related to the features (with a high test coverage).
  • I added pre-authorization annotations according to the guidelines and checked the course groups for all new REST Calls (security).
  • I documented the Java code using JavaDoc style.

Client

  • Important: I implemented the changes with a very good performance, prevented too many (unnecessary) REST calls and made sure the UI is responsive, even with large data.
  • I followed the coding and design guidelines.
  • Following the theming guidelines, I specified colors only in the theming variable files and checked that the changes look consistent in both the light and the dark theme.
  • I added multiple integration tests (Jest) related to the features (with a high test coverage), while following the test guidelines.
  • I documented the TypeScript code using JSDoc style.
  • I added multiple screenshots/screencasts of my UI changes.
  • I translated all newly inserted strings into English and German.

Motivation and Context

We want to actually show feedback suggestions from Athena in the programming exercise tutor assessment editor.

Description

  • I generalized the server code to work for more types of specific exercises, namely text and programming exercises.
  • I added support for sending structured grading instructions to Athena and receiving them from Athena.
  • I adapted the client code as needed to receive the feedback suggestions from the server.

Steps for Testing

Prerequisites:

  • 1 Programming/Text exercise, each with at least one submission. The programming exercise has to be set up for manual assessment (has a due date and is switched from "automatic" to "manual" assessment on the settings page)
  • 1 Instructor
  • 1 Tutor
  • Deployment of Athena#125 with GitHub label like in Artemis

Pre-configured example exercises: https://ma-schwind.ase.cit.tum.de/courses/7/exercises

  1. As an instructor, go to the settings page of each exercise and ensure that you can turn on the option for "Athena feedback suggestions". It should be disabled for programming exercises with automatic assessment: You can enable it by choosing manual assessment for the exercise.
  2. As a tutor (or still instructor), go to the exercise with the existing submissions
  3. Go to the assessment dashboard and start assessing a submission.
  4. You should see feedback suggestions.
  5. Test that the suggestions behave like the following:
  • For text exercises, the suggestion is automatically applied. You can edit or delete it.
  • For programming exercises, the suggestions are visible with lightbulbs in the sidebar of the editor. You can accept them or reject them. After accepting, you can modify them.
  • If you have a programming exercise feedback suggestion that is neither accepted nor rejected: When you submit, there is a confirmation dialog to confirm that you dismiss all pending suggestions.

(New) The first referenced test feedback will be associated with a grading instruction.

After testing, ask Paul to look into the Artemis+Athena server logs to confirm that there were no new problems with submission sending, feedback sending, and submission selection either.

Review Progress

Performance Review

  • I (as a reviewer) confirm that the client changes (in particular related to REST calls and UI responsiveness) are implemented with a very good performance
  • I (as a reviewer) confirm that the server changes (in particular related to database calls) are implemented with a very good performance

Code Review

  • Code Review 1
  • Code Review 2

Manual Tests

  • Test 1
  • Test 2

Test Coverage

Client

Class/File Line Coverage Confirmation (assert/expect)
athena.service.ts 91.83%
unreferenced-feedback-detail.component.ts 100%
code-editor-tutor-assessment-container.component.ts 87.03%
improved from 86.81% on develop
code-editor-ace.component.ts 84.78%
improved from 83.39% on develop
feedback-suggestion-badge.component.ts 100%
feedback-suggestions-pending-confirmation-dialog.component.ts 50% The component contains no logic that can be tested
unreferenced-feedback.component.ts 96.61%
text-submission-assessment.component.ts 90.82%
text-exercise-update.component.ts 86.27%
increased from 84.61% on develop

Server

Class/File Line Coverage Confirmation (assert/expect)
TextSubmissionService.java 88% ✅ same as on develop
AthenaDTOConverter.java 81% ✅ only exceptions that should never happen are untested
AthenaFeedbackSendingService.java 94%
AthenaFeedbackSuggestionsService.java 100%
AthenaModuleUrlHelper.java 76%
Only an exception that should never happen is untested
AthenaRepositoryExportService.java 92%
AthenaSubmissionSelectionService.java 100%
AthenaSubmissionSendingService.java 92%
AthenaScheduleService.java 44% kept as before
AthenaResource.java 77% ✅ ❌
ProgrammingAssessmentResource.java 89% ✅ Up from 88% on develop
TextAssessmentResource.java 90%

Screenshots

Suggestions in Text Exercise (looks a bit weird because I hardcoded the suggestion index ranges to better test):
screenshot-2023-10-24_002038

Suggestions in Programming Exercise:
screenshot-2023-10-24_002039

Confirmation if there are pending feedback suggestions (programming exercises only):
screenshot-2023-10-24_002040

pal03377 and others added 30 commits August 25, 2023 23:29
egekurt123
egekurt123 previously approved these changes Nov 20, 2023
Copy link
Contributor

@egekurt123 egekurt123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested at a testing session, works as expected

vinceclifford
vinceclifford previously approved these changes Nov 20, 2023
Copy link

@vinceclifford vinceclifford left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as expected. Tested during testing session

basak-akan
basak-akan previously approved these changes Nov 20, 2023
Copy link
Contributor

@basak-akan basak-akan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested during the testing session, worked well.

Copy link
Contributor

@florian-glombik florian-glombik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on https://ma-schwind.ase.cit.tum.de/course-management/7/text-exercises/35 and worked fine!

  • suggestions are displayed for programming and text exercises
  • when disabling suggestions there are no errors displayed for the text exercises
  • suggestions can be applied and rejected

Only a small comment (no need to change it now): I think that Delete and the trash bin is a quite strong wording - so I would either directly apply the suggestions to programming exercises (similar to the text exercises) or make the wording less strong

laurenzfb
laurenzfb previously approved these changes Nov 20, 2023
Copy link
Contributor

@laurenzfb laurenzfb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested in testing session. Found everything working as described!

terlan98
terlan98 previously approved these changes Nov 20, 2023
@b-fein b-fein added this to the 6.7.0 milestone Nov 20, 2023
Copy link
Member

@maximiliansoelch maximiliansoelch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good after the latest commit.
Just found some minor copy & paste mistakes in the comments that can be corrected in a follow up

Comment on lines +18 to +20
/**
* Create a new TextExerciseDTO from a TextExercise
*/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/**
* Create a new TextExerciseDTO from a TextExercise
*/
/**
* Create a new ProgrammingExerciseDTO from a ProgrammingExercise
*/

Comment on lines +14 to +19
* Creates a TextFeedbackDTO from a Feedback object
*
* @param exerciseId the id of the exercise the feedback is given for
* @param submissionId the id of the submission the feedback is given for
* @param feedback the feedback object
* @return the TextFeedbackDTO
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy & paste mistake:

Suggested change
* Creates a TextFeedbackDTO from a Feedback object
*
* @param exerciseId the id of the exercise the feedback is given for
* @param submissionId the id of the submission the feedback is given for
* @param feedback the feedback object
* @return the TextFeedbackDTO
* Creates a ProgrammingFeedbackDTO from a Feedback object
*
* @param exerciseId the id of the exercise the feedback is given for
* @param submissionId the id of the submission the feedback is given for
* @param feedback the feedback object
* @return the ProgrammingFeedbackDTO

@maximiliansoelch maximiliansoelch added the maintainer-approved The feature maintainer has approved the PR label Nov 24, 2023
…ggestions-server

# Conflicts:
#	src/main/java/de/tum/in/www1/artemis/repository/ProgrammingExerciseRepository.java
@krusche krusche merged commit d1ee9ca into develop Nov 25, 2023
21 of 24 checks passed
@krusche krusche deleted the feature/programming-exercises/feedback-suggestions-server branch November 25, 2023 00:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
client Pull requests that update TypeScript code. (Added Automatically!) config-change Pull requests that change the config in a way that they require a deployment via Ansible. cypress Pull requests that update cypress tests. (Added Automatically!) database Pull requests that update the database. (Added Automatically!). Require a CRITICAL deployment. documentation maintainer-approved The feature maintainer has approved the PR ready to merge server Pull requests that update Java code. (Added Automatically!) tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.