Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add test for combine tools script #3136

Merged
merged 82 commits into from
Dec 9, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
7bbc0a8
tools test added
vishvamsinh28 Jul 28, 2024
527cf9f
tools test updated for errors
vishvamsinh28 Jul 29, 2024
878023a
tools test upadted
vishvamsinh28 Jul 31, 2024
aedbb7c
Merge branch 'master' into combineToolsTest
vishvamsinh28 Aug 9, 2024
86b914e
Merge branch 'master' into combineToolsTest
vishvamsinh28 Aug 31, 2024
be80885
refactor build-tools
vishvamsinh28 Aug 31, 2024
b04532e
Merge branch 'master' into combineToolsTest
vishvamsinh28 Sep 11, 2024
3ea1852
Merge branch 'master' into combineToolsTest
vishvamsinh28 Sep 20, 2024
1dfd188
87% coverage
vishvamsinh28 Sep 21, 2024
103d643
fewgw
vishvamsinh28 Sep 21, 2024
b3a87dd
fewgw
vishvamsinh28 Sep 21, 2024
8357f56
fewgwfwf
vishvamsinh28 Sep 21, 2024
db96f45
wf
vishvamsinh28 Sep 21, 2024
2b392d1
fwefgwe
vishvamsinh28 Sep 29, 2024
877990e
test update
vishvamsinh28 Sep 29, 2024
88b1491
test update
vishvamsinh28 Sep 29, 2024
dd90f3c
test update
vishvamsinh28 Sep 29, 2024
c347dbf
test update again
vishvamsinh28 Sep 29, 2024
71e4f13
coverage 98%
vishvamsinh28 Sep 29, 2024
03e0664
fefefge
vishvamsinh28 Sep 29, 2024
18e9d98
100% coverage
vishvamsinh28 Sep 29, 2024
b3ae94b
test cases for error added
vishvamsinh28 Sep 30, 2024
014fdb6
fixures
vishvamsinh28 Sep 30, 2024
e74251e
fixture update
vishvamsinh28 Sep 30, 2024
07a7dc3
fixutures again
vishvamsinh28 Sep 30, 2024
511c00f
Merge branch 'master' into combineToolsTest
vishvamsinh28 Sep 30, 2024
91ad534
Merge branch 'combineToolsTest' of https://github.com/vishvamsinh28/w…
vishvamsinh28 Sep 30, 2024
a78dfa1
idjwqf
vishvamsinh28 Sep 30, 2024
8577070
Merge branch 'master' into combineToolsTest
vishvamsinh28 Oct 4, 2024
57f4dc3
fwqefqe
vishvamsinh28 Oct 4, 2024
0f9052e
tests updated
vishvamsinh28 Oct 4, 2024
a8ab07c
Merge branch 'master' into combineToolsTest
vishvamsinh28 Oct 7, 2024
a454c8d
Merge branch 'master' into combineToolsTest
vishvamsinh28 Oct 11, 2024
b7bf6c9
Merge branch 'master' into combineToolsTest
vishvamsinh28 Oct 13, 2024
f95ecce
buildtools update
vishvamsinh28 Oct 13, 2024
acb1ec3
add empty line at last
vishvamsinh28 Oct 13, 2024
61ac558
Merge branch 'master' into combineToolsTest
vishvamsinh28 Oct 14, 2024
906a873
Merge branch 'master' into combineToolsTest
anshgoyalevil Oct 15, 2024
5c4c192
update path
vishvamsinh28 Oct 16, 2024
6940c12
Merge branch 'master' into combineToolsTest
anshgoyalevil Oct 17, 2024
aeabab2
wfq
vishvamsinh28 Oct 18, 2024
db33c88
supress logs
vishvamsinh28 Oct 18, 2024
820de27
test and function update
vishvamsinh28 Oct 18, 2024
b16fca9
commit to re run the workflow
vishvamsinh28 Oct 18, 2024
995b9bd
commit to re run the workflow again
vishvamsinh28 Oct 18, 2024
4a4dcb7
Merge branch 'master' into combineToolsTest
vishvamsinh28 Oct 19, 2024
bd6424d
added retry mechanicasm for enoent errors
vishvamsinh28 Oct 19, 2024
233efdd
use fsextra
vishvamsinh28 Oct 19, 2024
4b13fc8
update test
vishvamsinh28 Oct 19, 2024
0489f2d
fqewfewqf
vishvamsinh28 Oct 19, 2024
774da3f
fqewfewqf
vishvamsinh28 Oct 19, 2024
7c24517
fqwefge
vishvamsinh28 Oct 19, 2024
7279d79
commit to re run the workflow
vishvamsinh28 Oct 19, 2024
2eccf52
Merge branch 'master' into combineToolsTest
vishvamsinh28 Oct 20, 2024
7601440
test upadted
vishvamsinh28 Oct 21, 2024
9afb39b
Merge branch 'combineToolsTest' of https://github.com/vishvamsinh28/w…
vishvamsinh28 Oct 21, 2024
464b1dc
Merge branch 'master' into combineToolsTest
vishvamsinh28 Oct 27, 2024
496de09
Merge branch 'master' into combineToolsTest
vishvamsinh28 Nov 1, 2024
2419f06
Merge branch 'master' into combineToolsTest
vishvamsinh28 Nov 4, 2024
11fcfe6
test updated
vishvamsinh28 Nov 5, 2024
7536a96
add empty line at the end
vishvamsinh28 Nov 5, 2024
5164452
fege
vishvamsinh28 Nov 5, 2024
ca72a7f
fqefe
vishvamsinh28 Nov 5, 2024
65fcfbd
Merge branch 'master' into combineToolsTest
vishvamsinh28 Nov 8, 2024
302b332
Merge branch 'master' into combineToolsTest
vishvamsinh28 Nov 13, 2024
1698ce2
fege
vishvamsinh28 Nov 13, 2024
31c66ec
Merge branch 'master' into combineToolsTest
vishvamsinh28 Nov 15, 2024
3ea3753
Merge branch 'master' into combineToolsTest
vishvamsinh28 Nov 22, 2024
fa11169
Merge branch 'master' into combineToolsTest
vishvamsinh28 Nov 23, 2024
7a55807
Merge branch 'master' into combineToolsTest
vishvamsinh28 Nov 28, 2024
0a7e3c2
Merge branch 'master' into combineToolsTest
vishvamsinh28 Nov 29, 2024
993310e
removed try/catch block from tests related to errors
vishvamsinh28 Nov 29, 2024
97905ba
Merge branch 'master' into combineToolsTest
vishvamsinh28 Nov 29, 2024
94f362c
Merge branch 'master' into combineToolsTest
vishvamsinh28 Dec 2, 2024
9d3aa7c
Merge branch 'master' into combineToolsTest
vishvamsinh28 Dec 4, 2024
05b14a2
Merge branch 'master' into combineToolsTest
vishvamsinh28 Dec 5, 2024
8587110
Merge branch 'master' into combineToolsTest
vishvamsinh28 Dec 8, 2024
1cc25cb
Merge branch 'master' into combineToolsTest
vishvamsinh28 Dec 9, 2024
40dcc54
Merge branch 'combineToolsTest' of https://github.com/vishvamsinh28/w…
vishvamsinh28 Dec 9, 2024
5d23088
remove new line
vishvamsinh28 Dec 9, 2024
fca4dcc
fix indentation errors
vishvamsinh28 Dec 9, 2024
98f3238
updated error logging
vishvamsinh28 Dec 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions scripts/build-tools.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,19 @@ const manualTools = require('../config/tools-manual.json')
const fs = require('fs');
const { resolve } = require('path');

let toolsPath = resolve(__dirname, '../../config', 'tools.json')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is using let a good option here?

let tagsPath = resolve(__dirname, '../../config', 'all-tags.json')
let automatedToolsPath = resolve(__dirname, '../config', 'tools-automated.json')

const buildTools = async () => {
try {
let githubExtractData = await getData();
let automatedTools = await convertTools(githubExtractData);
fs.writeFileSync(
resolve(__dirname, '../config', 'tools-automated.json'),
automatedToolsPath,
JSON.stringify(automatedTools, null, ' ')
);
await combineTools(automatedTools, manualTools);
await combineTools(automatedTools, manualTools, toolsPath, tagsPath);
} catch (err) {
console.log(err);
throw err
Expand Down
13 changes: 3 additions & 10 deletions scripts/tools/combine-tools.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ const fs = require('fs')
const schema = require("./tools-schema.json");
const Ajv = require("ajv")
const addFormats = require("ajv-formats")
const { resolve } = require('path');
const Fuse = require("fuse.js");
const ajv = new Ajv()
addFormats(ajv, ["uri"])
Expand Down Expand Up @@ -106,7 +105,7 @@ const getFinalTool = async (toolObject) => {

// Combine the automated tools and manual tools list into single JSON object file, and
// lists down all the language and technology tags in one JSON file.
const combineTools = async (automatedTools, manualTools) => {
const combineTools = async (automatedTools, manualTools, toolsPath, tagsPath) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does tagspath signify??

for (const key in automatedTools) {
let finalToolsList = [];
if (automatedTools[key].toolsList.length) {
Expand Down Expand Up @@ -136,14 +135,8 @@ const combineTools = async (automatedTools, manualTools) => {
finalToolsList.sort((tool, anotherTool) => tool.title.localeCompare(anotherTool.title));
finalTools[key].toolsList = finalToolsList
}
fs.writeFileSync(
resolve(__dirname, '../../config', 'tools.json'),
JSON.stringify(finalTools)
);
fs.writeFileSync(
resolve(__dirname, '../../config', 'all-tags.json'),
JSON.stringify({ languages: languageList, technologies: technologyList }),
)
fs.writeFileSync(toolsPath,JSON.stringify(finalTools));
fs.writeFileSync(tagsPath,JSON.stringify({ languages: languageList, technologies: technologyList }),)
}

module.exports = { combineTools }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix this linter error

17 changes: 17 additions & 0 deletions tests/fixtures/tools/automated-tools.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"category1": {
"description": "Sample Category",
"toolsList": [
{
"title": "Tool B",
"filters": {
"language": "Python",
"technology": ["Flask"]
akshatnema marked this conversation as resolved.
Show resolved Hide resolved
},
"links": {
"repoUrl": "https://github.com/asyncapi/tool-b"
}
}
]
}
}
12 changes: 12 additions & 0 deletions tests/fixtures/tools/manual-tools.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
[
{
"title": "Tool A",
"filters": {
"language": "JavaScript",
"technology": ["Node.js"]
},
"links": {
"repoUrl": "https://github.com/asyncapi/tool-a"
}
}
]
187 changes: 187 additions & 0 deletions tests/tools/combine-tools.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,187 @@
const fs = require('fs');
const path = require('path');
const { combineTools } = require('../../scripts/tools/combine-tools');
const { createToolObject } = require('../../scripts/tools/tools-object');

jest.mock('../../scripts/tools/tags-color', () => ({
languagesColor: [
{ name: 'JavaScript', color: 'bg-[#57f281]', borderColor: 'border-[#37f069]' },
{ name: 'Python', color: 'bg-[#3572A5]', borderColor: 'border-[#3572A5]' }
],
technologiesColor: [
{ name: 'Node.js', color: 'bg-[#61d0f2]', borderColor: 'border-[#40ccf7]' },
{ name: 'Flask', color: 'bg-[#000000]', borderColor: 'border-[#FFFFFF]' }
]
}));

jest.mock('../../scripts/tools/categorylist', () => ({
categoryList: [
{ name: 'category1', description: 'Sample Category 1' },
{ name: 'category2', description: 'Sample Category 2' }
]
}));

jest.mock('../../scripts/tools/tools-object', () => ({
createToolObject: jest.fn((tool, _, __, isAsyncAPIrepo) => {
return { ...tool, isAsyncAPIrepo };
})
}));

const readJSON = (filePath) => JSON.parse(fs.readFileSync(filePath, 'utf-8'));

describe('combineTools function', () => {
const toolsPath = path.join(__dirname, '../', 'fixtures', 'tools', 'tools.json');
const tagsPath = path.join(__dirname, '../', 'fixtures', 'tools', 'tags.json');
const manualToolsPath = path.join(__dirname, '../', 'fixtures', 'tools', 'manual-tools.json');
const automatedToolsPath = path.join(__dirname, '../', 'fixtures', 'tools', 'automated-tools.json');

let manualTools;
let automatedTools;

let consoleErrorMock;

beforeAll(() => {
manualTools = readJSON(manualToolsPath);
automatedTools = readJSON(automatedToolsPath);

consoleErrorMock = jest.spyOn(console, 'error').mockImplementation(() => {});
});
Copy link

@coderabbitai coderabbitai bot Nov 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Move console.error mock setup to beforeEach and reset mocks between tests

Currently, the console.error mock is set up in beforeAll and not reset between tests. This can lead to cross-test pollution and affect the reliability of your tests. Consider moving the mock setup to a beforeEach block and resetting it in an afterEach block to ensure test isolation.

Apply the following changes:

 beforeAll(() => {
   manualTools = readJSON(manualToolsPath);
   automatedTools = readJSON(automatedToolsPath);

-  consoleErrorMock = jest.spyOn(console, 'error').mockImplementation(() => {});
 });

+ beforeEach(() => {
+   consoleErrorMock = jest.spyOn(console, 'error').mockImplementation(() => {});
+ });
+ 
+ afterEach(() => {
+   jest.clearAllMocks();
+ });
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
consoleErrorMock = jest.spyOn(console, 'error').mockImplementation(() => {});
});
manualTools = readJSON(manualToolsPath);
automatedTools = readJSON(automatedToolsPath);
});
beforeEach(() => {
consoleErrorMock = jest.spyOn(console, 'error').mockImplementation(() => {});
});
afterEach(() => {
jest.clearAllMocks();
});

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vishvamsinh28 Apply this suggestion

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!


akshatnema marked this conversation as resolved.
Show resolved Hide resolved
afterAll(() => {
if (fs.existsSync(toolsPath)) fs.unlinkSync(toolsPath);
if (fs.existsSync(tagsPath)) fs.unlinkSync(tagsPath);
});

it('should combine tools and create correct JSON files', async () => {
await combineTools(automatedTools, manualTools, toolsPath, tagsPath);
akshatnema marked this conversation as resolved.
Show resolved Hide resolved

const combinedTools = readJSON(toolsPath);
expect(combinedTools).toHaveProperty('category1');

const tagsData = readJSON(tagsPath);
expect(tagsData).toHaveProperty('languages');
expect(tagsData).toHaveProperty('technologies');
expect(tagsData.languages).toContainEqual({
name: 'JavaScript',
color: 'bg-[#57f281]',
borderColor: 'border-[#37f069]'
});
expect(tagsData.languages).toContainEqual({
name: 'Python',
color: 'bg-[#3572A5]',
borderColor: 'border-[#3572A5]'
});
expect(tagsData.technologies).toContainEqual({
name: 'Node.js',
color: 'bg-[#61d0f2]',
borderColor: 'border-[#40ccf7]'
});
expect(tagsData.technologies).toContainEqual({
name: 'Flask',
color: 'bg-[#000000]',
borderColor: 'border-[#FFFFFF]'
});
});

it('should handle tools with missing language or technology', async () => {
const manualToolsWithMissingData = [
{
title: 'Tool C',
filters: {},
links: { repoUrl: 'https://github.com/asyncapi/tool-c' }
}
];

await combineTools({}, manualToolsWithMissingData, toolsPath, tagsPath);

const combinedTools = readJSON(toolsPath);
expect(combinedTools).toHaveProperty('category1');
});

it('should sort tools alphabetically by title', async () => {
const manualToolsToSort = {
category1: {
description: 'Sample Category',
toolsList: [
{
title: 'Tool Z',
filters: { language: 'JavaScript' },
links: { repoUrl: 'https://github.com/asyncapi/tool-z' }
},
{
title: 'Tool A',
filters: { language: 'Python' },
links: { repoUrl: 'https://github.com/asyncapi/tool-a' }
}
]
}
};

await combineTools(manualToolsToSort, {}, toolsPath, tagsPath);

const combinedTools = readJSON(toolsPath);
const toolTitles = combinedTools.category1.toolsList.map(tool => tool.title);
expect(toolTitles).toEqual(['Tool A', 'Tool Z']);
});


it('should log validation errors to console.error', async () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Errors shouldn't be validated via console messages. You should formally return a error using Promise.reject.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vishvamsinh28 Have you worked on this?

const invalidTool = { title: 'Invalid Tool' };
const automatedTools = {
'category1': {
description: 'Category 1 Description',
toolsList: []
}
};
const manualTools = {
'category1': {
toolsList: [invalidTool]
}
};

createToolObject.mockImplementation((tool) => Promise.resolve(tool));

await combineTools(automatedTools, manualTools, toolsPath, tagsPath);

const errorCalls = console.error.mock.calls;

expect(errorCalls[0][0]).toBe('Script is not failing, it is just dropping errors for further investigation');
expect(errorCalls[1][0]).toBe('Invalid Invalid Tool .asyncapi-tool file.');
expect(errorCalls[2][0]).toBe('Located in manual-tools.json file');
expect(errorCalls[3][0]).toEqual(expect.stringContaining('Validation errors:'));

expect(fs.existsSync(toolsPath)).toBe(true);
expect(fs.existsSync(tagsPath)).toBe(true);
});

it('should handle tools with multiple languages, including new ones', async () => {
const toolWithMultipleLanguages = {
title: 'Multi-Language Tool',
filters: {
language: ['JavaScript', 'Python', 'NewLanguage'],
technology: ['Node.js']
},
links: { repoUrl: 'https://github.com/example/multi-language-tool' }
};

const automatedTools = {
'category1': {
description: 'Category 1 Description',
toolsList: [toolWithMultipleLanguages]
}
};

await combineTools(automatedTools, {}, toolsPath, tagsPath);

const combinedTools = readJSON(toolsPath);
const tool = combinedTools.category1.toolsList[0];

expect(tool.filters.language).toHaveLength(3);
expect(tool.filters.language).toContainEqual(expect.objectContaining({ name: 'JavaScript' }));
expect(tool.filters.language).toContainEqual(expect.objectContaining({ name: 'Python' }));
expect(tool.filters.language).toContainEqual(expect.objectContaining({ name: 'NewLanguage' }));

const tagsData = readJSON(tagsPath);
expect(tagsData.languages).toContainEqual(expect.objectContaining({ name: 'NewLanguage' }));
});
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix this one also

Loading