Skip to content

Commit

Permalink
Adds easy Java setup. Adds testRunner.js and --clearLogs flag.
Browse files Browse the repository at this point in the history
  • Loading branch information
jeremy-then committed Jun 19, 2024
1 parent 6d0cfe0 commit ffad403
Show file tree
Hide file tree
Showing 8 changed files with 167 additions and 68 deletions.
1 change: 1 addition & 0 deletions .env-example
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ POWPEG_NODE_JAR_PATH=<path_to_powpeg_repo>/powpeg-node/build/libs/federate-node-
CONFIG_FILE=regtest-all-keyfiles
BITCOIND_BIN_PATH=/<path_to_bitcoind>/bitcoind/bin/bitcoind
LOG_HOME=/<path_to_logs/directory>/logs
JAVA_BIN_PATH=/<path_to_java_bin>/java
40 changes: 26 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ All private keys used in the library are for testing only and not used in any pr
<details open>
<summary>Java 8</summary>
- install java the x86 version to be used with rossetta in case of ARM arch (1.8, 11 and 17 can be used).

- Or simply download it and update the `JAVA_BIN_PATH` variable in the `.env` file to point to the path of the `java` binary. See `.env-example`. This is helpful when you don't want to fully install a specific version of Java in your system. This way, you can use any Java version downloaded without really installing it. If not specified, the `java` in the environment will be used.
</details>

<details open>
Expand Down Expand Up @@ -75,7 +77,7 @@ All private keys used in the library are for testing only and not used in any pr

TODO: add more config files with HSM4 and HSM5 nodes.

For the `BITCOIND_BIN_PATH` variable, set the path of the `bitcoind` binary. This is helpful when you don't want to fully install bitcoind in your system. This way, you can use any bitcoind version downloaded without really installing it. If not specified, the `bitcoind` in the environment will try to be used.
For the `BITCOIND_BIN_PATH` variable, set the path of the `bitcoind` binary. This is helpful when you don't want to fully install bitcoind in your system. This way, you can use any bitcoind version downloaded without really installing it. If not specified, the `bitcoind` in the environment will be used.


```bash
Expand Down Expand Up @@ -204,27 +206,31 @@ runners {
NODE_ENV=mynewenv npm test
```

## Clearing the logs automatically before running tests

To clear the logs before running new tests, run the following command:

```
node testRunner.js --clearLogs
```

## Running the tests multiple times

### Running all tests multiple times from scratch

To run all tests multiple times from scratch, right after all tests finished running, run:

> node testRunner.js
Or with npm:

> npm run run-tests-multiple-times
> node testRunner.js --clearLogs --runTestsMultipleTimes
You can specify how many times to run all tests from scratch by adding the times to run them in the `RUN_ALL_TESTS_THESE_TIMES` property in the `.env` file.

Remember to copy the `.env-example` file and rename it as `.env` file.

Or, you can pass te times you want to run the tests from scratch using the `--times` flag, like this:

> npm run run-tests-multiple-times --times 3
> node testRunner.js --clearLogs --runTestsMultipleTimes --times=3
The `npm run run-tests-multiple-times` command is executing the file `testRunner.js`, which uses `shelljs` to execute the script `npm run test-fail-fast` in a loop and counts the failures and print some information to the console.
The `node testRunner.js --clearLogs --runTestsMultipleTimes` command is executing the file `multipleTestExecutionsRunner.js`, which uses `shelljs` to execute the script `npm run test-fail-fast` in a loop and counts the failures and print some information to the console.

### Running a specific test file multiple times

Expand All @@ -250,7 +256,7 @@ Remember that with this approach we are not running the test file from scratch e

To run specific test files from scratch, use the same setup described in this section, and for the script instead run:

> npm run run-tests-multiple-times
> node testRunner.js --runTestsMultipleTimes
This will run specific test files from scratch.

Expand All @@ -273,13 +279,13 @@ If our test is one of the last tests, then we will potentially wait more than 15

To run a test file that has the `fulfillRequirementsToRunAsSingleTestFile` function, simply run the following `npm` command:

> npm run run-single-test-file <filename.js>
> node testRunner.js --clearLogs --runSingleTestFile --testFileName=<filename.js>
For example:

> npm run run-single-test-file 01_02_51-post_wasabi_fed_pubkeys_fork.js
> node testRunner.js --clearLogs --runSingleTestFile --testFileName=01_02_51-post_wasabi_fed_pubkeys_fork.js
The command `run-single-test-file` will execute the file `singleTestFileRunner.js` which has some simple logic:
The command `--runSingleTestFile` will execute the `runSingleTestFile` function declared in file `singleTestFileRunner.js` which has some simple logic:

1 - It will assign the `01_02_51-post_wasabi_fed_pubkeys_fork.js` test file name to the `process.env.INCLUDE_CASES` variable. Since it will be the one in that `INCLUDE_CASES` variable, then only that test file will be run.

Expand All @@ -289,9 +295,15 @@ The test file should have a `fulfillRequirementsToRunAsSingleTestFile` function

Another advantage of this is that it will allow us to understand exactly what each test really needs in order to run, reducing uncertainties.

To indicate a fork name to be used in the `fulfillRequirementsToRunAsSingleTestFile` function, you can specify it as the last argument of the command like this, passing the fork name `fingerroot500`:
To indicate a fork name to be used in the `fulfillRequirementsToRunAsSingleTestFile` function, you can specify a `--forkName=<forkName>` parameter, for example, passing the fork name `fingerroot500`:

> node testRunner.js --runSingleTestFile --testFileName=02_00_01-2wp.js --forkName=fingerroot500
Clearing logs:

> node testRunner.js --clearLogs --runSingleTestFile --testFileName=02_00_01-2wp.js --forkName=fingerroot500
> npm run run-single-test-file 02_00_01-2wp.js fingerroot500
> The parameters can be passed in any order. The `--forkName` param is optional.
This is when the `fulfillRequirementsToRunAsSingleTestFile` function needs a fork name that needs to be dynamically passed. For example, the `2wp.js` file is run multiple times with different forks. We cannot simply hardcode which fork to use or to use the latest, because sometimes we will need to run it with a fork passed dynamically.

Expand Down
2 changes: 0 additions & 2 deletions config/regtest-all-keyfiles.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@ const keysPathResolve = 'node-keys';
const classpath = process.env.POWPEG_NODE_JAR_PATH;
const federatesLogbackPath = path.resolve(__dirname, 'logback');

console.log('log file: ', `${federatesLogbackPath}/logback-fed-1.xml`)

module.exports = {
init: {
mineInitialBitcoin: true,
Expand Down
4 changes: 2 additions & 2 deletions lib/federate-runner.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ let removeDir = require('./utils').removeDir;

var DEFAULT_OPTIONS = {
bitcoinPeer: 'localhost:20123',
command: 'java',
command: process.env.JAVA_BIN_PATH || 'java',
mainClass: 'co.rsk.federate.FederateRunner',
removeDataDirOnStop: true,
runnerStdOut: process.stdout,
Expand Down Expand Up @@ -126,7 +126,7 @@ FederateRunner.prototype.start = function() {

args.push(this.options.mainClass);
args.push(`--regtest`);

console.log('this.options.command: ', this.options.command)
this.process = childProcess.spawn(this.options.command, args, {
cwd: this.dataDir
});
Expand Down
60 changes: 33 additions & 27 deletions multipleTestExecutionsRunner.js
Original file line number Diff line number Diff line change
@@ -1,33 +1,39 @@
require('dotenv').config();

const shell = require('shelljs');

const timesArg = Number(process.argv[2]);

const RUN_ALL_TESTS_THESE_TIMES = timesArg || Number(process.env.RUN_ALL_TESTS_THESE_TIMES) || 1;

console.info(`Will attempt to run tests ${RUN_ALL_TESTS_THESE_TIMES} times.`);

const cleanEnvCommand = "kill $(ps -A | grep -e java -e python -e bitcoind | awk '{print $1}')";
const runTestsMultipleTimes = (params) => {

const times = params.times || '';
const RUN_ALL_TESTS_THESE_TIMES = times || Number(process.env.RUN_ALL_TESTS_THESE_TIMES) || 1;

console.info(`Will attempt to run tests ${RUN_ALL_TESTS_THESE_TIMES} times.`);

const cleanEnvCommand = "kill $(ps -A | grep -e java -e python -e bitcoind | awk '{print $1}')";

const ensureCleanEnv = () => {
console.info('Cleaning environment...');
shell.exec(cleanEnvCommand);
console.info('Environment clean.');
};

let fails = 0;
let attempts = 1;

for(let i = 0; i < RUN_ALL_TESTS_THESE_TIMES; i++) {
console.info(`Running tests ${attempts} out of ${RUN_ALL_TESTS_THESE_TIMES} times.`);
ensureCleanEnv();
if (shell.exec('npm run test-fail-fast').code !== 0) {
fails++;
}
console.info(`Tests have failed ${fails} times so far.`);
console.info(`Tests have passed ${attempts - fails} times so far.`);
attempts++;
}

console.info(`Tests failed ${fails} times out of ${RUN_ALL_TESTS_THESE_TIMES} attempts.`);

const ensureCleanEnv = () => {
console.info('Cleaning environment...');
shell.exec(cleanEnvCommand);
console.info('Environment clean.');
};

let fails = 0;
let attempts = 1;

for(let i = 0; i < RUN_ALL_TESTS_THESE_TIMES; i++) {
console.info(`Running tests ${attempts} out of ${RUN_ALL_TESTS_THESE_TIMES} times.`);
ensureCleanEnv();
if (shell.exec('npm run test-fail-fast').code !== 0) {
fails++;
}
console.info(`Tests have failed ${fails} times so far.`);
console.info(`Tests have passed ${attempts - fails} times so far.`);
attempts++;
}

console.info(`Tests failed ${fails} times out of ${RUN_ALL_TESTS_THESE_TIMES} attempts.`);
module.exports = {
runTestsMultipleTimes
};
4 changes: 1 addition & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,7 @@
"main": "index.js",
"scripts": {
"test": "mocha --timeout 600000",
"test-fail-fast": "mocha -b --timeout 600000",
"run-tests-multiple-times": "node multipleTestExecutionsRunner.js",
"run-single-test-file": "node singleTestFileRunner.js"
"test-fail-fast": "mocha -b --timeout 600000"
},
"author": "",
"license": "GPL3",
Expand Down
54 changes: 34 additions & 20 deletions singleTestFileRunner.js
Original file line number Diff line number Diff line change
@@ -1,32 +1,46 @@
require('dotenv').config();

const shell = require('shelljs');

const testFileNameWithJsExtension = process.argv[2];
// Just to make sure we leave everything as we found it
process.env.RUNNING_SINGLE_TEST_FILE = false;
process.env.FORK_NAME = '';

const forkName = process.argv[3] || '';
const runSingleTestFile = (params) => {

console.log(`Executing test file '${testFileNameWithJsExtension}' in isolation.`);
const { testFileName, forkName = '' } = params;

if(forkName) {
console.log(`Using fork '${forkName}'.`);
}
if(!testFileName) {
console.error('No test file name provided.');
return;
}

// Including this test file to be the only one to be executed
process.env.INCLUDE_CASES = testFileNameWithJsExtension;
if(!testFileName.endsWith('.js')) {
testFileName += '.js';
return;
}

// Setting this flag to make the test file know it is being run in isolation and do the necessary adjustments
process.env.RUNNING_SINGLE_TEST_FILE = true;
console.info(`Executing test file '${testFileName}' in isolation.`);

process.env.FORK_NAME = forkName;
if(forkName) {
console.info(`Using fork '${forkName}'.`);
}

if (shell.exec('npm run test-fail-fast').code !== 0) {
console.error(`Test file '${testFileNameWithJsExtension}' failed to run.`);
} else {
console.log(`Test file '${testFileNameWithJsExtension}' ran successfully.`);
}
// Including this test file to be the only one to be executed
process.env.INCLUDE_CASES = testFileName;

// Just to make sure we leave everything as we found it
process.env.RUNNING_SINGLE_TEST_FILE = false;
// Setting this flag to make the test file know it is being run in isolation and do the necessary adjustments
process.env.RUNNING_SINGLE_TEST_FILE = true;

process.env.FORK_NAME = '';
process.env.FORK_NAME = forkName;

if (shell.exec('npm run test-fail-fast').code !== 0) {
console.error(`Test file '${testFileName}' failed to run.`);
} else {
console.info(`Test file '${testFileName}' ran successfully.`);
}

};

module.exports = {
runSingleTestFile
};
70 changes: 70 additions & 0 deletions testRunner.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
require('dotenv').config();
const fs = require('fs').promises;
const shell = require('shelljs');

const defaultParamsValues = {
clearLogs: false,
runTestsMultipleTimes: false,
times: 1,
runSingleTestFile: false,
testFileName: '',
forkName: '',
};

async function deleteAllFiles(directoryPath) {
try {
await fs.rm(directoryPath, { recursive: true, force: true });
} catch (err) {
console.error(`Error deleting files: ${err.message}`);
}
};

const getParsedParams = () => {
const params = process.argv.filter(param => param.startsWith('--'))
.reduce((params, param) => {
if(param.startsWith('--clearLogs')) {
params.clearLogs = true;
} else if(param.startsWith('--runTestsMultipleTimes')) {
params.runTestsMultipleTimes = true;
} else if(param.startsWith('--times')) {
params.times = param.slice(param.indexOf('=') + 1);
} else if(param.startsWith('--runSingleTestFile')) {
params.runSingleTestFile = true;
} else if(param.startsWith('--testFileName')) {
params.testFileName = param.slice(param.indexOf('=') + 1);
} else if(param.startsWith('--forkName')) {
params.forkName = param.slice(param.indexOf('=') + 1);
}
return params;
}, defaultParamsValues);
return params;
};

const runTests = async () => {

const params = getParsedParams();

if(params.clearLogs) {
console.log('Clearing logs...');
await deleteAllFiles(process.env.LOG_HOME);
}

if(params.runTestsMultipleTimes) {
const { times } = params;
return require('./multipleTestExecutionsRunner').runTestsMultipleTimes({ times });
}

if(params.runSingleTestFile) {
const { testFileName, forkName } = params;
return require('./singleTestFileRunner').runSingleTestFile({ testFileName, forkName });
}

if (shell.exec('npm run test-fail-fast').code !== 0) {
console.error(`Tests failed to run.`);
} else {
console.info(`Tests ran successfully.`);
}

};

runTests();

0 comments on commit ffad403

Please sign in to comment.