Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrated code lifecycle: Add filter and search to finished build jobs #8702

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
9a7e1ec
add filter to buildjob queue view
BBesrour May 22, 2024
4a910cd
Add en text. TODO: german
BBesrour May 22, 2024
00d76aa
add new filters
BBesrour May 24, 2024
06f57b1
Merge branch 'refs/heads/develop' into feature/integrated-code-lifecy…
BBesrour May 29, 2024
d5b9f8d
wip
BBesrour May 30, 2024
1313f7f
- fix localstorage issues
BBesrour May 30, 2024
3da5ece
improve doc
BBesrour May 30, 2024
4c5bfc4
- move logic to service
BBesrour May 30, 2024
4821662
fix wrong url
BBesrour May 30, 2024
246b07e
fix server style
BBesrour May 30, 2024
b02ed5b
fix server style
BBesrour May 30, 2024
4a344b9
fix missing fetches
BBesrour May 31, 2024
57e37a9
fix client tests and remove useless methods
BBesrour May 31, 2024
ec98a7a
adjust tests
BBesrour May 31, 2024
7729c95
adjust tests
BBesrour May 31, 2024
d40d005
adjust tests
BBesrour May 31, 2024
9280888
fix missing condition in query
BBesrour May 31, 2024
603d888
fix wrong translations and reduce debounce time
BBesrour May 31, 2024
e01f683
Merge branch 'develop' into feature/integrated-code-lifecycle/add-fil…
BBesrour Jun 2, 2024
8a2b767
Add tooltip for search field
BBesrour Jun 2, 2024
f53dd20
convert dto to record
BBesrour Jun 2, 2024
dcf3300
adjust tests
BBesrour Jun 2, 2024
67e35a1
adjust tests
BBesrour Jun 2, 2024
ad1106f
adjust tests
BBesrour Jun 2, 2024
72dcb5f
add annotation
BBesrour Jun 2, 2024
716e33c
adjust tests
BBesrour Jun 2, 2024
98ed15c
Merge branch 'develop' into feature/integrated-code-lifecycle/add-fil…
BBesrour Jun 2, 2024
bb8ad74
change requests
BBesrour Jun 3, 2024
96729c3
change requests
BBesrour Jun 5, 2024
741957a
change requests
BBesrour Jun 5, 2024
643a7ac
Merge branch 'develop' into feature/integrated-code-lifecycle/add-fil…
BBesrour Jun 5, 2024
b285bcb
fix issue with postgresql
BBesrour Jun 5, 2024
51a39f5
fix issue with postgresql
BBesrour Jun 5, 2024
20fabf3
Merge branch 'develop' into feature/integrated-code-lifecycle/add-fil…
BBesrour Jun 5, 2024
c096a94
Test more functions
BBesrour Jun 5, 2024
384e7dd
use specificationAPI
BBesrour Jun 6, 2024
595c750
Revert "use specificationAPI"
BBesrour Jun 6, 2024
1895b09
Revert specificationAPI and improve query performance
BBesrour Jun 6, 2024
8bde3e0
Add duration to query
BBesrour Jun 6, 2024
9c7eb88
use specification API
BBesrour Jun 6, 2024
9134570
fix wrong attribute name
BBesrour Jun 7, 2024
4c8cac7
Revert "fix wrong attribute name"
BBesrour Jun 7, 2024
cb66d93
Revert "use specification API"
BBesrour Jun 7, 2024
3affe89
Merge branch 'develop' into feature/integrated-code-lifecycle/add-fil…
BBesrour Jun 12, 2024
ba7a9e1
Merge branch 'develop' into feature/integrated-code-lifecycle/add-fil…
BBesrour Jun 14, 2024
f2e81ae
Merge branch 'develop' into feature/integrated-code-lifecycle/add-fil…
BBesrour Jun 20, 2024
6f8cfbc
Merge branch 'develop' into feature/integrated-code-lifecycle/add-fil…
BBesrour Jun 21, 2024
efe7a0e
Merge branch 'refs/heads/develop' into feature/integrated-code-lifecy…
BBesrour Jun 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

import static de.tum.in.www1.artemis.config.Constants.PROFILE_CORE;

import java.time.Duration;
import java.time.ZonedDateTime;
import java.util.List;
import java.util.Optional;
import java.util.Set;
Expand All @@ -17,6 +19,7 @@

import de.tum.in.www1.artemis.domain.BuildJob;
import de.tum.in.www1.artemis.domain.Result;
import de.tum.in.www1.artemis.domain.enumeration.BuildStatus;
import de.tum.in.www1.artemis.repository.base.ArtemisJpaRepository;
import de.tum.in.www1.artemis.service.connectors.localci.dto.DockerImageBuild;
import de.tum.in.www1.artemis.service.connectors.localci.dto.ResultBuildJob;
Expand All @@ -33,6 +36,36 @@ public interface BuildJobRepository extends ArtemisJpaRepository<BuildJob, Long>
@EntityGraph(attributePaths = { "result", "result.participation", "result.participation.exercise", "result.submission" })
Page<BuildJob> findAll(Pageable pageable);

// Cast to string is necessary. Otherwise, the query will fail on PostgreSQL.
@Query("""
SELECT b.id
FROM BuildJob b
LEFT JOIN Course c ON b.courseId = c.id
WHERE (:buildStatus IS NULL OR b.buildStatus = :buildStatus)
AND (:buildAgentAddress IS NULL OR b.buildAgentAddress = :buildAgentAddress)
AND (CAST(:startDate AS string) IS NULL OR b.buildStartDate >= :startDate)
AND (CAST(:endDate AS string) IS NULL OR b.buildStartDate <= :endDate)
AND (:searchTerm IS NULL OR (b.repositoryName LIKE %:searchTerm% OR c.title LIKE %:searchTerm%))
AND (:courseId IS NULL OR b.courseId = :courseId)
AND (:durationLower IS NULL OR (b.buildCompletionDate - b.buildStartDate) >= :durationLower)
AND (:durationUpper IS NULL OR (b.buildCompletionDate - b.buildStartDate) <= :durationUpper)

""")
Page<Long> findAllByFilterCriteria(@Param("buildStatus") BuildStatus buildStatus, @Param("buildAgentAddress") String buildAgentAddress,
@Param("startDate") ZonedDateTime startDate, @Param("endDate") ZonedDateTime endDate, @Param("searchTerm") String searchTerm, @Param("courseId") Long courseId,
@Param("durationLower") Duration durationLower, @Param("durationUpper") Duration durationUpper, Pageable pageable);
BBesrour marked this conversation as resolved.
Show resolved Hide resolved

@Query("""
SELECT b
FROM BuildJob b
LEFT JOIN FETCH b.result r
LEFT JOIN FETCH r.participation p
LEFT JOIN FETCH p.exercise
LEFT JOIN FETCH r.submission
WHERE b.id IN :buildJobIds
""")
List<BuildJob> findAllByIdWithResults(@Param("buildJobIds") List<Long> buildJobIds);
BBesrour marked this conversation as resolved.
Show resolved Hide resolved

@Query("""
SELECT new de.tum.in.www1.artemis.service.connectors.localci.dto.DockerImageBuild(
b.dockerImage,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import static de.tum.in.www1.artemis.config.Constants.PROFILE_LOCALCI;

import java.time.Duration;
import java.time.ZonedDateTime;
import java.util.ArrayList;
import java.util.List;
Expand All @@ -14,6 +15,8 @@
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Profile;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.PageImpl;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;

Expand All @@ -23,11 +26,14 @@
import com.hazelcast.map.IMap;
import com.hazelcast.topic.ITopic;

import de.tum.in.www1.artemis.domain.BuildJob;
import de.tum.in.www1.artemis.repository.BuildJobRepository;
import de.tum.in.www1.artemis.service.ProfileService;
import de.tum.in.www1.artemis.service.connectors.localci.dto.BuildAgentInformation;
import de.tum.in.www1.artemis.service.connectors.localci.dto.BuildJobQueueItem;
import de.tum.in.www1.artemis.service.connectors.localci.dto.DockerImageBuild;
import de.tum.in.www1.artemis.web.rest.dto.pageablesearch.FinishedBuildJobPageableSearchDTO;
import de.tum.in.www1.artemis.web.rest.util.PageUtil;

/**
* Includes methods for managing and retrieving the shared build job queue and build agent information. Also contains methods for cancelling build jobs.
Expand Down Expand Up @@ -279,4 +285,24 @@ public void cancelAllJobsForParticipation(long participationId) {
}
}

/**
* Get all finished build jobs that match the search criteria.
*
* @param search the search criteria
* @param courseId the id of the course
* @return the page of build jobs
*/
public Page<BuildJob> getFilteredFinishedBuildJobs(FinishedBuildJobPageableSearchDTO search, Long courseId) {
Duration buildDurationLower = search.buildDurationLower() == null ? null : Duration.ofSeconds(search.buildDurationLower());
Duration buildDurationUpper = search.buildDurationUpper() == null ? null : Duration.ofSeconds(search.buildDurationUpper());

Page<Long> buildJobIdsPage = buildJobRepository.findAllByFilterCriteria(search.buildStatus(), search.buildAgentAddress(), search.startDate(), search.endDate(),
search.pageable().getSearchTerm(), courseId, buildDurationLower, buildDurationUpper,
PageUtil.createDefaultPageRequest(search.pageable(), PageUtil.ColumnMapping.BUILD_JOB));

List<BuildJob> buildJobs = buildJobRepository.findAllByIdWithResults(buildJobIdsPage.toList());

return new PageImpl<>(buildJobs, buildJobIdsPage.getPageable(), buildJobIdsPage.getTotalElements());
}
BBesrour marked this conversation as resolved.
Show resolved Hide resolved

}
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,12 @@
import org.springframework.web.servlet.support.ServletUriComponentsBuilder;

import de.tum.in.www1.artemis.domain.BuildJob;
import de.tum.in.www1.artemis.repository.BuildJobRepository;
import de.tum.in.www1.artemis.security.annotations.EnforceAdmin;
import de.tum.in.www1.artemis.service.connectors.localci.SharedQueueManagementService;
import de.tum.in.www1.artemis.service.connectors.localci.dto.BuildAgentInformation;
import de.tum.in.www1.artemis.service.connectors.localci.dto.BuildJobQueueItem;
import de.tum.in.www1.artemis.service.dto.FinishedBuildJobDTO;
import de.tum.in.www1.artemis.web.rest.dto.pageablesearch.PageableSearchDTO;
import de.tum.in.www1.artemis.web.rest.util.PageUtil;
import de.tum.in.www1.artemis.web.rest.dto.pageablesearch.FinishedBuildJobPageableSearchDTO;
import tech.jhipster.web.util.PaginationUtil;

@Profile(PROFILE_LOCALCI)
Expand All @@ -37,13 +35,10 @@ public class AdminBuildJobQueueResource {

private final SharedQueueManagementService localCIBuildJobQueueService;

private final BuildJobRepository buildJobRepository;

private static final Logger log = LoggerFactory.getLogger(AdminBuildJobQueueResource.class);

public AdminBuildJobQueueResource(SharedQueueManagementService localCIBuildJobQueueService, BuildJobRepository buildJobRepository) {
public AdminBuildJobQueueResource(SharedQueueManagementService localCIBuildJobQueueService) {
this.localCIBuildJobQueueService = localCIBuildJobQueueService;
this.buildJobRepository = buildJobRepository;
}

/**
Expand Down Expand Up @@ -170,11 +165,14 @@ public ResponseEntity<Void> cancelAllRunningBuildJobsForAgent(@RequestParam Stri
*/
@GetMapping("finished-jobs")
@EnforceAdmin
public ResponseEntity<List<FinishedBuildJobDTO>> getFinishedBuildJobs(PageableSearchDTO<String> search) {
log.debug("REST request to get a page of finished build jobs");
final Page<BuildJob> page = buildJobRepository.findAll(PageUtil.createDefaultPageRequest(search, PageUtil.ColumnMapping.BUILD_JOB));
Page<FinishedBuildJobDTO> finishedBuildJobDTOs = FinishedBuildJobDTO.fromBuildJobsPage(page);
HttpHeaders headers = PaginationUtil.generatePaginationHttpHeaders(ServletUriComponentsBuilder.fromCurrentRequest(), page);
public ResponseEntity<List<FinishedBuildJobDTO>> getFinishedBuildJobs(FinishedBuildJobPageableSearchDTO search) {
log.debug("REST request to get a page of finished build jobs with build status {}, build agent address {}, start date {} and end date {}", search.buildStatus(),
search.buildAgentAddress(), search.startDate(), search.endDate());

Page<BuildJob> buildJobPage = localCIBuildJobQueueService.getFilteredFinishedBuildJobs(search, null);

Page<FinishedBuildJobDTO> finishedBuildJobDTOs = FinishedBuildJobDTO.fromBuildJobsPage(buildJobPage);
HttpHeaders headers = PaginationUtil.generatePaginationHttpHeaders(ServletUriComponentsBuilder.fromCurrentRequest(), buildJobPage);
return new ResponseEntity<>(finishedBuildJobDTOs.getContent(), headers, HttpStatus.OK);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
package de.tum.in.www1.artemis.web.rest.dto.pageablesearch;

import java.time.ZonedDateTime;

import com.fasterxml.jackson.annotation.JsonInclude;

import de.tum.in.www1.artemis.domain.enumeration.BuildStatus;

@JsonInclude(JsonInclude.Include.NON_EMPTY)
public record FinishedBuildJobPageableSearchDTO(BuildStatus buildStatus, String buildAgentAddress, ZonedDateTime startDate, ZonedDateTime endDate, Integer buildDurationLower,
Integer buildDurationUpper, SearchTermPageableSearchDTO<String> pageable) {
}
Original file line number Diff line number Diff line change
Expand Up @@ -20,17 +20,15 @@

import de.tum.in.www1.artemis.domain.BuildJob;
import de.tum.in.www1.artemis.domain.Course;
import de.tum.in.www1.artemis.repository.BuildJobRepository;
import de.tum.in.www1.artemis.repository.CourseRepository;
import de.tum.in.www1.artemis.security.annotations.EnforceAtLeastInstructor;
import de.tum.in.www1.artemis.security.annotations.enforceRoleInCourse.EnforceAtLeastInstructorInCourse;
import de.tum.in.www1.artemis.service.AuthorizationCheckService;
import de.tum.in.www1.artemis.service.connectors.localci.SharedQueueManagementService;
import de.tum.in.www1.artemis.service.connectors.localci.dto.BuildJobQueueItem;
import de.tum.in.www1.artemis.service.dto.FinishedBuildJobDTO;
import de.tum.in.www1.artemis.web.rest.dto.pageablesearch.PageableSearchDTO;
import de.tum.in.www1.artemis.web.rest.dto.pageablesearch.FinishedBuildJobPageableSearchDTO;
import de.tum.in.www1.artemis.web.rest.errors.AccessForbiddenException;
import de.tum.in.www1.artemis.web.rest.util.PageUtil;
import tech.jhipster.web.util.PaginationUtil;

@Profile(PROFILE_LOCALCI)
Expand All @@ -46,14 +44,10 @@ public class BuildJobQueueResource {

private final CourseRepository courseRepository;

private final BuildJobRepository buildJobRepository;

public BuildJobQueueResource(SharedQueueManagementService localCIBuildJobQueueService, AuthorizationCheckService authorizationCheckService, CourseRepository courseRepository,
BuildJobRepository buildJobRepository) {
public BuildJobQueueResource(SharedQueueManagementService localCIBuildJobQueueService, AuthorizationCheckService authorizationCheckService, CourseRepository courseRepository) {
this.localCIBuildJobQueueService = localCIBuildJobQueueService;
this.authorizationCheckService = authorizationCheckService;
this.courseRepository = courseRepository;
this.buildJobRepository = buildJobRepository;
}

/**
Expand Down Expand Up @@ -163,11 +157,11 @@ public ResponseEntity<Void> cancelAllRunningBuildJobs(@PathVariable long courseI
*/
@GetMapping("courses/{courseId}/finished-jobs")
@EnforceAtLeastInstructorInCourse
public ResponseEntity<List<FinishedBuildJobDTO>> getFinishedBuildJobsForCourse(@PathVariable long courseId, PageableSearchDTO<String> search) {
public ResponseEntity<List<FinishedBuildJobDTO>> getFinishedBuildJobsForCourse(@PathVariable long courseId, FinishedBuildJobPageableSearchDTO search) {
log.debug("REST request to get the finished build jobs for course {}", courseId);
final Page<BuildJob> page = buildJobRepository.findAllByCourseId(courseId, PageUtil.createDefaultPageRequest(search, PageUtil.ColumnMapping.BUILD_JOB));
Page<FinishedBuildJobDTO> finishedBuildJobDTOs = FinishedBuildJobDTO.fromBuildJobsPage(page);
HttpHeaders headers = PaginationUtil.generatePaginationHttpHeaders(ServletUriComponentsBuilder.fromCurrentRequest(), page);
Page<BuildJob> buildJobPage = localCIBuildJobQueueService.getFilteredFinishedBuildJobs(search, courseId);
Page<FinishedBuildJobDTO> finishedBuildJobDTOs = FinishedBuildJobDTO.fromBuildJobsPage(buildJobPage);
HttpHeaders headers = PaginationUtil.generatePaginationHttpHeaders(ServletUriComponentsBuilder.fromCurrentRequest(), buildJobPage);
BBesrour marked this conversation as resolved.
Show resolved Hide resolved
return new ResponseEntity<>(finishedBuildJobDTOs.getContent(), headers, HttpStatus.OK);
}

Expand Down
Loading
Loading