Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Iris: Lecture Ingestion Web hook #8512

Closed
wants to merge 260 commits into from
Closed

Iris: Lecture Ingestion Web hook #8512

wants to merge 260 commits into from

Conversation

yassinsws
Copy link
Contributor

@yassinsws yassinsws commented Apr 30, 2024 •
edited
Loading

Checklist

General

Server

  • Important: I implemented the changes with a very good performance and prevented too many (unnecessary) database calls.
  • I strictly followed the server coding and design guidelines.
  • I added multiple integration tests (Spring) related to the features (with a high test coverage).
  • I added pre-authorization annotations according to the guidelines and checked the course groups for all new REST Calls (security).
  • I documented the Java code using JavaDoc style.

Motivation and Context

We want Iris to be able to respond to the students questions based on the lecture content. For that we need to send the lectures to Pyris, everytime new lecture slides or attachments get updated or deleted.

Description

Every lecture that gets updated or created is sent to the Pyris through a rest call. If a course, lecture or a lectureUnit gets erased the webhook is also executed and the files are erased from the Pyris side.

Steps for Testing

Prerequisites:

  • 1 Instructor
  • 1 course
  • Pyris and Artemis running locally, with the PR Ingestion Pipeline
  1. Log in to Artemis
  2. Go to the course page
  3. Upload a lecture unit attachment -> Check if the file is sent correctly by checking that the log has the right course and lecture unit name.
  4. Delete a lecture unit, delete a lecture, and delete a course. -> check that the files are sent correctly to the Pyris side. The name of the lecture units sent will be logged after they get sent successfully.

Testserver States

Note

These badges show the state of the test servers.
Green = Currently available, Red = Currently locked



Review Progress

Performance Review

  • I (as a reviewer) confirm that the client changes (in particular related to REST calls and UI responsiveness) are implemented with a very good performance
  • I (as a reviewer) confirm that the server changes (in particular related to database calls) are implemented with a very good performance

Code Review

  • Code Review 1
  • Code Review 2

Manual Tests

  • Test 1
  • Test 2

Test Coverage

Screenshots

julian-christl and others added 30 commits October 30, 2023 13:20
@julian-christl
fix dialect
ebaa97b
@julian-christl
fixed jpql queries
2254b16
@julian-christl
fix JPQL queries
3471327
@julian-christl
Merge branch 'develop' into chore/spring-boot-3.x
559d358
@julian-christl
fix SecurityConfiguration
eb0eb06
@julian-christl
fix configurations
25126e1
@julian-christl
Merge branch 'develop' into chore/spring-boot-3.x
e346c2e
@julian-christl
fix configurations
3000faa
@julian-christl
upgrade hibernate
1228bdc
@julian-christl
generalise authentication
3ade8b0
@julian-christl
fix client paths
a831089
@julian-christl
Merge branch 'develop' into chore/spring-boot-3.x
d57ecd4
@julian-christl
remove unnecessary annotations
6b2bcf8 
- join column defined in collection table
- element collection always cascading
@julian-christl
change some versions, fix some issues
f684f2c
@julian-christl
fix lti javax issues
da9743d
@julian-christl
bump run configuration
650ab4e
@julian-christl
fix nulling annotations
d8ef0a5
@julian-christl
Merge branch 'develop' into chore/spring-boot-3.x
934226b
@julian-christl
replace competency query with specification
e1f2d00
@julian-christl
fix filter
ae06982
@julian-christl
fix some test rest paths
acc84d0
@julian-christl
fix specification casting
11ba29c
@julian-christl
fix OnlineCourseConfigurationServiceTest
050aa99
@julian-christl
Merge branch 'develop' into chore/spring-boot-3.x
4dcd676
@julian-christl
Merge branch 'develop' into chore/spring-boot-3.x
f77df0a
@julian-christl
Merge branch 'develop' into chore/spring-boot-3.x
ae0dcce
@julian-christl
Merge branch 'develop' into chore/spring-boot-3.x
f93a8c3
@b-fein
temporarily disable parallel tests
16a49bf 
Might reduce the used heap space enough so that at least the tests in
the CI pipeline run through.
Cause for the increase in space usage needs to be checked and when found
the parallel testing should be re-enabled before merge.
@julian-christl
Merge branch 'develop' into chore/spring-boot-3.x
173fb27
@krusche
Merge branch 'develop' into chore/spring-boot-3.x
e9f8f91
@yassinsws yassinsws requested a review from Hialus May 3, 2024 11:43
@github-actions github-actions bot added the client Pull requests that update TypeScript code. (Added Automatically!) label May 3, 2024
@yassinsws yassinsws changed the title Ingestion Pipeline Iris: Lecture Ingestion Web hook May 3, 2024
@yassinsws yassinsws added component:Iris ready for review and removed client Pull requests that update TypeScript code. (Added Automatically!) labels May 3, 2024
@yassinsws yassinsws requested a review from kaancayli May 3, 2024 14:05
@github-actions github-actions bot added client Pull requests that update TypeScript code. (Added Automatically!) docker labels May 3, 2024
@yassinsws yassinsws force-pushed the feature/iris/IngestionPipeline branch from 18e1144 to f9fab7f Compare May 4, 2024 08:54
@yassinsws yassinsws removed the docker label May 4, 2024
@yassinsws yassinsws dismissed Hialus’s stale review May 4, 2024 16:43

implemented requests

Hialus and others added 7 commits May 6, 2024 13:46
@Hialus
Implement Stephans feedback
57f1025
@Hialus
Fix missing JavaDoc
df532cd
@Hialus
Fix missing JavaDoc
c35e0a7
@Hialus
Merge branch 'develop' into feature/iris/pyrisV2
af36966
@yassinsws
merge
7a49d5b
@yassinsws
Merge remote-tracking branch 'origin/feature/iris/pyrisV2' into featu…
aab74ef 
…re/iris/IngestionPipeline

# Conflicts:
#	docker/playwright-E2E-tests-mysql-localci.yml
#	docker/playwright-E2E-tests-mysql.yml
#	docker/playwright-E2E-tests-postgres.yml
#	docker/playwright.yml
#	src/main/java/de/tum/in/www1/artemis/domain/AssessmentNote.java
#	src/main/java/de/tum/in/www1/artemis/repository/ProgrammingExerciseRepository.java
#	src/main/java/de/tum/in/www1/artemis/repository/ProgrammingExerciseStudentParticipationRepository.java
#	src/main/java/de/tum/in/www1/artemis/repository/ResultRepository.java
#	src/main/java/de/tum/in/www1/artemis/repository/SubmissionRepository.java
#	src/main/java/de/tum/in/www1/artemis/service/connectors/pyris/PyrisConnectorService.java
#	src/main/java/de/tum/in/www1/artemis/service/dto/FinishedBuildJobDTO.java
#	src/main/java/de/tum/in/www1/artemis/service/notifications/push_notifications/PushNotificationService.java
#	src/main/java/de/tum/in/www1/artemis/web/rest/open/PublicPyrisStatusUpdateResource.java
#	src/main/java/de/tum/in/www1/artemis/web/rest/programming/ProgrammingExerciseExportImportResource.java
#	src/main/resources/config/liquibase/changelog/20230628215302_changelog.xml
#	src/main/resources/config/liquibase/changelog/20240404040404_changelog.xml
#	src/main/webapp/app/admin/standardized-competencies/standardized-competency-management.component.html
#	src/main/webapp/app/admin/standardized-competencies/standardized-competency-management.component.ts
#	src/main/webapp/app/assessment/assessment-layout/assessment-layout.component.html
#	src/main/webapp/app/assessment/assessment-note/assessment-note.component.html
#	src/main/webapp/app/course/competencies/competency-form/competency-form.component.html
#	src/main/webapp/app/course/competencies/competency-management/competency-management.component.html
#	src/main/webapp/app/course/competencies/import-standardized-competencies/course-import-standardized-competencies.component.html
#	src/main/webapp/app/course/competencies/import-standardized-competencies/course-import-standardized-competencies.component.ts
#	src/main/webapp/app/localci/build-queue/build-queue.component.html
#	src/main/webapp/app/localci/build-queue/build-queue.component.ts
#	src/main/webapp/app/localci/build-queue/build-queue.service.ts
#	src/main/webapp/app/overview/course-conversations/course-conversations.component.html
#	src/main/webapp/app/overview/course-conversations/course-wide-search/course-wide-search.component.html
#	src/main/webapp/app/overview/course-overview.component.html
#	src/main/webapp/app/overview/course-overview.component.scss
#	src/main/webapp/app/shared/standardized-competencies/standardized-competency-detail.component.html
#	src/main/webapp/app/shared/standardized-competencies/standardized-competency-filter-page.component.ts
#	src/main/webapp/app/shared/standardized-competencies/standardized-competency-filter.component.html
#	src/test/java/de/tum/in/www1/artemis/exercise/programmingexercise/AuxiliaryRepositoryServiceTest.java
#	src/test/java/de/tum/in/www1/artemis/metis/linkpreview/LinkPreviewIntegrationTest.java
#	src/test/java/de/tum/in/www1/artemis/service/BuildLogEntryServiceTest.java
#	src/test/java/de/tum/in/www1/artemis/util/RequestUtilService.java
#	src/test/javascript/spec/component/iris/ui/chatbot-widget.component.spec.ts
#	src/test/javascript/spec/component/localci/build-queue/build-queue.component.spec.ts
#	src/test/javascript/spec/component/overview/course-conversations/course-wide-search.component.spec.ts
#	src/test/playwright/e2e/exercise/programming/ProgrammingExerciseParticipation.spec.ts
#	src/test/playwright/support/constants.ts
#	src/test/playwright/support/pageobjects/exercises/programming/ProgrammingExerciseOverviewPage.ts
@yassinsws
add execute webhook to the pyris connector service
85198db
Base automatically changed from feature/iris/pyrisV2 to develop May 10, 2024 18:19
@krusche krusche dismissed coderabbitai[bot]’s stale review May 10, 2024 18:19

The base branch was changed.

@yassinsws yassinsws closed this May 20, 2024
@yassinsws yassinsws deleted the feature/iris/IngestionPipeline branch June 14, 2024 23:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@coderabbitai coderabbitai[bot] coderabbitai[bot] left review comments

@Hialus Hialus Awaiting requested review from Hialus

@kaancayli kaancayli Awaiting requested review from kaancayli

Assignees

@yassinsws yassinsws

Labels
client Pull requests that update TypeScript code. (Added Automatically!) component:Iris ready for review server Pull requests that update Java code. (Added Automatically!) tests
Projects
Artemis Development
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
9 participants
@yassinsws @Hialus @julian-christl @b-fein @krusche @bassner @Strohgelaender @pzdr7 @coolchock