Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pipelines): ✨ Config allowed pipeline branches #500

Open
wants to merge 6 commits into
base: next
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -301,6 +301,7 @@ metadata:
gitlab.com/project-slug: 'project-slug' # group_name/project_name
# or
gitlab.com/instance: gitlab.internal.abcd # abcd, represents local instance used
gitlab.com/pipeline-refs: 'main,develop,feature/*,refs/merge-requests/1234/merge' # Optional comma seperated lists of branches/refs to show in pipeline table, default is all, accepts wildcard "*".
spec:
type: service
# ...
Expand Down
2 changes: 2 additions & 0 deletions packages/gitlab/dev/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,8 @@ createDevApp()
'gitlab.com/project-id': `${projectId}`,
'gitlab.com/codeowners-path': `CODEOWNERS`,
'gitlab.com/readme-path': `README.md`,
'gitlab.com/pipeline-refs':
'master,refs/merge-requests/3678/*',
},
name: 'backstage',
},
Expand Down
3 changes: 2 additions & 1 deletion packages/gitlab/src/api/GitlabCIApi.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,8 @@ export type GitlabProjectCoverageResponse = {

export type GitlabCIApi = {
getPipelineSummary(
projectID: string | number
projectID: string | number,
refList?: string[]
): Promise<PipelineSchema[] | undefined>;
getContributorsSummary(
projectID: string | number
Expand Down
21 changes: 18 additions & 3 deletions packages/gitlab/src/api/GitlabCIClient.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,10 @@ import {
OAuthApi,
} from '@backstage/core-plugin-api';
import { PeopleCardEntityData } from '../components/types';
import { parseCodeOwners } from '../components/utils';
import {
convertWildcardFilterArrayToFilterFunction,
parseCodeOwners,
} from '../components/utils';
import {
ContributorsSummary,
GitlabCIApi,
Expand Down Expand Up @@ -146,7 +149,8 @@ export class GitlabCIClient implements GitlabCIApi {
}

async getPipelineSummary(
projectID?: string | number
projectID?: string | number,
refList?: string[]
): Promise<PipelineSchema[] | undefined> {
const [pipelineObjects, projectObj] = await Promise.all([
this.callApi<PipelineSchema[]>(
Expand All @@ -155,12 +159,23 @@ export class GitlabCIClient implements GitlabCIApi {
),
this.callApi<Record<string, string>>('projects/' + projectID, {}),
]);

if (pipelineObjects && projectObj) {
pipelineObjects.forEach((element) => {
element.project_name = projectObj.name;
});
}
return pipelineObjects || undefined;

const relevantPipelineObjects = refList
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doing this can reduce the number of pipelines you get:

Example

Having this annotation:

gitlab.com/pipeline-refs: 'main,develop'

if in your last 50 pipelines, 49 are in branches different from main and develop you will get only one pipeline in your card. Then you can make a request for each branch ex. /projects/2416/pipelines?ref=${branch} (with this approach you get more pipelines than before) or better you can use graphql

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @antoniomuso, good catch haven't forgotten about this - just swamped with other work. Hoping to pick this back up in early aug :)

? pipelineObjects?.filter((pipeline) =>
convertWildcardFilterArrayToFilterFunction(
pipeline.ref,
refList
)
)
: pipelineObjects;

return relevantPipelineObjects ?? undefined;
}

async getIssuesSummary(
Expand Down
13 changes: 13 additions & 0 deletions packages/gitlab/src/components/gitlabAppData.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ export const GITLAB_ANNOTATION_PROJECT_SLUG = 'gitlab.com/project-slug';
export const GITLAB_ANNOTATION_INSTANCE = 'gitlab.com/instance';
export const GITLAB_ANNOTATION_CODEOWNERS_PATH = 'gitlab.com/codeowners-path';
export const GITLAB_ANNOTATION_README_PATH = 'gitlab.com/readme-path';
export const GITLAB_ANNOTATION_PIPELINE_REFS = 'gitlab.com/pipeline-refs';
const defaultGitlabIntegration = {
hostname: 'gitlab.com',
baseUrl: 'https://gitlab.com/api/v4',
Expand Down Expand Up @@ -102,3 +103,15 @@ export const gitlabReadmePath = () => {

return readme_path;
};

export const gitlabPipelineRelevantRefs = () => {
const { entity } = useEntity();

const relevant_refs =
entity.metadata.annotations?.[GITLAB_ANNOTATION_PIPELINE_REFS];

// I'd prefer to return an array here, but annotations being strings is a requrement of the backstage entity model
return relevant_refs
? relevant_refs.split(',').map((ref) => ref.trim())
: undefined;
};
53 changes: 53 additions & 0 deletions packages/gitlab/src/components/util.test.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
import { convertWildcardFilterArrayToFilterFunction } from './utils';

describe('convertWildcardFilterArrayToFilterFunction', () => {
it('should return true if input matches any of the validInputs', () => {
const input = 'foobar';
const validInputs = ['foo*', 'bar', 'baz'];
const result = convertWildcardFilterArrayToFilterFunction(
input,
validInputs
);
expect(result).toBeTruthy();
});

it('should return false if input does not match any of the validInputs', () => {
const input = 'foobar';
const validInputs = ['baz', 'qux'];
const result = convertWildcardFilterArrayToFilterFunction(
input,
validInputs
);
expect(result).toBeFalsy();
});

it('should account for multiple wildcards in the validInputs', () => {
const input = 'foobar';
const validInputs = ['foo*', '*bar', 'baz'];
const result = convertWildcardFilterArrayToFilterFunction(
input,
validInputs
);
expect(result).toBeTruthy();
});

it('should always return true if any of the valid inputs is *', () => {
const input = 'foobar';
const validInputs = ['*'];
const result = convertWildcardFilterArrayToFilterFunction(
input,
validInputs
);
expect(result).toBeTruthy();
});

it('should account for ** in the validInputs', () => {
const input = 'foobar';
const validInputs = ['foo**', 'bar', 'baz'];
const result = convertWildcardFilterArrayToFilterFunction(
input,
validInputs
);
expect(result).toBeTruthy();
});
});
11 changes: 11 additions & 0 deletions packages/gitlab/src/components/utils.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -114,3 +114,14 @@ const parseCodeOwnerLine = (rule: string): FileOwnership => {
// ensures that only the following patterns are allowed @octocat @octocat/kitty [email protected]
const codeOwnerRegex =
/(^@[a-zA-Z0-9_\-/]*$)|(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*|"(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])*")@(?:(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\[(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?|[a-z0-9-]*[a-z0-9]:(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21-\x5a\x53-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])+)\])/;

export const convertWildcardFilterArrayToFilterFunction = (
input: string, // eg: 'foobar'
validInputs: string[] // eg: ['foo*', 'bar', 'baz']
) => {
const regex = new RegExp(
`^${validInputs.join('|').split('*').join('(.+)')}$`
);

return regex.test(input);
};
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
import React from 'react';
import React, { useMemo } from 'react';
import { Table, TableColumn, Progress } from '@backstage/core-components';
import Alert from '@material-ui/lab/Alert';
import { useAsync } from 'react-use';
import {
gitlabInstance,
gitlabPipelineRelevantRefs,
gitlabProjectId,
gitlabProjectSlug,
} from '../../gitlabAppData';
Expand Down Expand Up @@ -32,19 +33,23 @@ export const PipelineDenseTable = ({
];
const title = 'Gitlab Pipelines: ' + projectName;

const data = summary.map((pipelineObject) => {
return {
id: pipelineObject.id,
status: pipelineObject.status,
ref: pipelineObject.ref,
web_url: pipelineObject.web_url,
created_date: getElapsedTime(pipelineObject.created_at),
duration: getDuration(
pipelineObject.created_at,
pipelineObject.updated_at
),
};
});
const data = useMemo(
() =>
summary.map((pipelineObject) => {
return {
id: pipelineObject.id,
status: pipelineObject.status,
ref: pipelineObject.ref,
web_url: pipelineObject.web_url,
created_date: getElapsedTime(pipelineObject.created_at),
duration: getDuration(
pipelineObject.created_at,
pipelineObject.updated_at
),
};
}),
[summary]
);

return (
<Table
Expand All @@ -60,6 +65,7 @@ export const PipelinesTable = ({}) => {
const project_id = gitlabProjectId();
const project_slug = gitlabProjectSlug();
const gitlab_instance = gitlabInstance();
const gitlab_relevant_refs = gitlabPipelineRelevantRefs();

const GitlabCIAPI = useApi(GitlabCIApiRef).build(
gitlab_instance || 'gitlab.com'
Expand All @@ -72,9 +78,13 @@ export const PipelinesTable = ({}) => {
if (!projectDetails)
throw new Error('wrong project_slug or project_id');

const summary = await GitlabCIAPI.getPipelineSummary(projectDetails.id);
const summary = await GitlabCIAPI.getPipelineSummary(
projectDetails.id,
gitlab_relevant_refs
);

if (!summary) throw new Error('Merge request summary is undefined!');

return { summary, projectName: projectDetails.name };
}, []);

Expand Down
Loading