Skip to content

Commit

Permalink
Epoch is not resolved from the network when possible
Browse files Browse the repository at this point in the history
  • Loading branch information
fboucquez committed Jun 3, 2021
1 parent 8416e08 commit 1f65bd3
Show file tree
Hide file tree
Showing 13 changed files with 170 additions and 88 deletions.
4 changes: 2 additions & 2 deletions docs/upgradeVotingKeys.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ It upgrades the voting keys and files when required.
Voting file upgrade:
- If the node's current voting file has an end epoch close to the current epoch ("close to expiring") this command creates a new 'private_key_treeX.dat' that continues the current file.
- "Close to expiring" happens when the epoch is in the upper half of the voting file. If the file's epoch length is 720, close to expiring will be 360+.
- The current finalization epoch that defines if the file is close to expiration can be passed as parameter. Otherwise, bootstrap will use the locally known value.
- The current finalization epoch that defines if the file is close to expiration can be passed as parameter. Otherwise, bootstrap will try to resolve it from the network.

When a new voting file is created, bootstrap will advise running the link command again.

Expand Down Expand Up @@ -39,7 +39,7 @@ DESCRIPTION
- "Close to expiring" happens when the epoch is in the upper half of the voting file. If the file's epoch length is
720, close to expiring will be 360+.
- The current finalization epoch that defines if the file is close to expiration can be passed as parameter.
Otherwise, bootstrap will use the locally known value.
Otherwise, bootstrap will try to resolve it from the network.
When a new voting file is created, bootstrap will advise running the link command again.
Expand Down
6 changes: 3 additions & 3 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion presets/mainnet/network.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ votingSetGrouping: 1440
minVotingKeyLifetime: 112
maxVotingKeyLifetime: 360
votingKeyDesiredLifetime: 360
lastKnownNetworkEpoch: 153
lastKnownNetworkEpoch: 158
stepDuration: 5m
maxBlockFutureTime: 300ms
maxAccountRestrictionValues: 100
Expand Down
2 changes: 1 addition & 1 deletion presets/testnet/network.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ initialCurrencyAtomicUnits: 7842928625000000
importanceGrouping: 180
votingSetGrouping: 720
votingKeyDesiredLifetime: 720
lastKnownNetworkEpoch: 187
lastKnownNetworkEpoch: 197
minVotingKeyLifetime: 28
maxVotingKeyLifetime: 720
stepDuration: 4m
Expand Down
9 changes: 6 additions & 3 deletions src/commands/upgradeVotingKeys.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ import { Command, flags } from '@oclif/command';
import { LogType } from '../logger';
import Logger from '../logger/Logger';
import LoggerFactory from '../logger/LoggerFactory';
import { BootstrapUtils, CommandUtils, ConfigLoader, CryptoUtils, VotingService } from '../service';
import { BootstrapUtils, CommandUtils, ConfigLoader, CryptoUtils, RemoteNodeService, VotingService } from '../service';
const logger: Logger = LoggerFactory.getLogger(LogType.System);

export default class UpgradeVotingKeys extends Command {
Expand All @@ -27,7 +27,7 @@ export default class UpgradeVotingKeys extends Command {
Voting file upgrade:
- If the node's current voting file has an end epoch close to the current epoch ("close to expiring") this command creates a new 'private_key_treeX.dat' that continues the current file.
- "Close to expiring" happens when the epoch is in the upper half of the voting file. If the file's epoch length is 720, close to expiring will be 360+.
- The current finalization epoch that defines if the file is close to expiration can be passed as parameter. Otherwise, bootstrap will use the locally known value.
- The current finalization epoch that defines if the file is close to expiration can be passed as parameter. Otherwise, bootstrap will try to resolve it from the network.
When a new voting file is created, bootstrap will advise running the link command again.
Expand Down Expand Up @@ -58,6 +58,9 @@ When a new voting file is created, bootstrap will advise running the link comman
const presetData = configLoader.loadExistingPresetData(target, password);
const addresses = configLoader.loadExistingAddresses(target, password);
const privateKeySecurityMode = CryptoUtils.getPrivateKeySecurityMode(presetData.privateKeySecurityMode);

const finalizationEpoch = flags.finalizationEpoch || (await new RemoteNodeService().resolveCurrentFinalizationEpoch(presetData));

const votingKeyUpgrade = (
await Promise.all(
(presetData.nodes || []).map((nodePreset, index) => {
Expand All @@ -68,7 +71,7 @@ When a new voting file is created, bootstrap will advise running the link comman
return new VotingService({
target,
user: flags.user,
}).run(presetData, nodeAccount, nodePreset, flags.finalizationEpoch, true);
}).run(presetData, nodeAccount, nodePreset, finalizationEpoch, true);
}),
)
).find((f) => f);
Expand Down
70 changes: 4 additions & 66 deletions src/service/AnnounceService.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ import {
AccountInfo,
Address,
AggregateTransaction,
ChainInfo,
Convert,
Deadline,
LockFundsTransaction,
Expand All @@ -29,7 +28,6 @@ import {
NetworkType,
PublicAccount,
RepositoryFactory,
RepositoryFactoryHttp,
SignedTransaction,
Transaction,
TransactionService,
Expand All @@ -41,6 +39,7 @@ import LoggerFactory from '../logger/LoggerFactory';
import { Addresses, ConfigPreset, NodeAccount, NodePreset } from '../model';
import { CommandUtils } from './CommandUtils';
import { KeyName } from './ConfigService';
import { RemoteNodeService } from './RemoteNodeService';

const logger: Logger = LoggerFactory.getLogger(LogType.System);

Expand All @@ -60,13 +59,6 @@ export interface TransactionFactory {
createTransactions(params: TransactionFactoryParams): Promise<Transaction[]>;
}

export interface RepositoryInfo {
repositoryFactory: RepositoryFactory;
restGatewayUrl: string;
generationHash?: string;
chainInfo?: ChainInfo;
}

export class AnnounceService {
private static onProcessListener = () => {
process.on('SIGINT', () => {
Expand Down Expand Up @@ -113,22 +105,10 @@ export class AnnounceService {
logger.info(`There are no transactions to announce...`);
return;
}

const url = providedUrl.replace(/\/$/, '');
let repositoryFactory: RepositoryFactory;
const urls = (useKnownRestGateways && presetData.knownRestGateways) || [];
if (urls.length) {
const repositoryInfo = this.sortByHeight(await this.getKnownNodeRepositoryInfos(urls))[0];
if (!repositoryInfo) {
throw new Error(`No up and running node could be found of out: ${urls.join(', ')}`);
}
repositoryFactory = repositoryInfo.repositoryFactory;
logger.info(`Connecting to node ${repositoryInfo.restGatewayUrl}`);
} else {
repositoryFactory = new RepositoryFactoryHttp(url);
logger.info(`Connecting to node ${url}`);
}

const urls = (useKnownRestGateways && presetData.knownRestGateways) || [url];
const repositoryInfo = await new RemoteNodeService().getBestRepositoryInfo(urls);
const repositoryFactory = repositoryInfo.repositoryFactory;
const networkType = await repositoryFactory.getNetworkType().toPromise();
const transactionRepository = repositoryFactory.createTransactionRepository();
const transactionService = new TransactionService(transactionRepository, repositoryFactory.createReceiptRepository());
Expand Down Expand Up @@ -483,48 +463,6 @@ export class AnnounceService {
}
}

private getKnownNodeRepositoryInfos(knownUrls: string[]): Promise<RepositoryInfo[]> {
logger.info(`Looking for the best node out of: ${knownUrls.join(', ')}`);
return Promise.all(
knownUrls.map(
async (restGatewayUrl): Promise<RepositoryInfo> => {
const repositoryFactory = new RepositoryFactoryHttp(restGatewayUrl);
try {
const generationHash = await repositoryFactory.getGenerationHash().toPromise();
const chainInfo = await repositoryFactory.createChainRepository().getChainInfo().toPromise();
return {
restGatewayUrl,
repositoryFactory,
generationHash,
chainInfo,
};
} catch (e) {
const message = `There has been an error talking to node ${restGatewayUrl}. Error: ${e.message}}`;
logger.warn(message);
return {
restGatewayUrl: restGatewayUrl,
repositoryFactory,
};
}
},
),
);
}

private sortByHeight(repos: RepositoryInfo[]): RepositoryInfo[] {
return repos
.filter((b) => b.chainInfo)
.sort((a, b) => {
if (!a.chainInfo) {
return 1;
}
if (!b.chainInfo) {
return -1;
}
return b.chainInfo.height.compare(a.chainInfo.height);
});
}

private async getBestCosigner(
repositoryFactory: RepositoryFactory,
cosigners: Account[],
Expand Down
20 changes: 17 additions & 3 deletions src/service/ConfigService.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ import { CommandUtils } from './CommandUtils';
import { ConfigLoader } from './ConfigLoader';
import { CryptoUtils } from './CryptoUtils';
import { NemgenService } from './NemgenService';
import { RemoteNodeService } from './RemoteNodeService';
import { ReportService } from './ReportService';
import { RewardProgramService } from './RewardProgramService';
import { VotingService } from './VotingService';
Expand Down Expand Up @@ -69,6 +70,7 @@ export interface ConfigParams {
report: boolean;
reset: boolean;
upgrade: boolean;
offline: boolean;
preset?: Preset;
target: string;
password?: string;
Expand All @@ -89,6 +91,7 @@ export class ConfigService {
public static defaultParams: ConfigParams = {
target: BootstrapUtils.defaultTargetFolder,
report: false,
offline: false,
reset: false,
upgrade: false,
user: BootstrapUtils.CURRENT_USER,
Expand Down Expand Up @@ -244,12 +247,17 @@ export class ConfigService {
}

private async generateNodes(presetData: ConfigPreset, addresses: Addresses): Promise<void> {
const currentFinalizationEpoch = this.params.offline
? presetData.lastKnownNetworkEpoch
: await new RemoteNodeService().resolveCurrentFinalizationEpoch(presetData);
await Promise.all(
(addresses.nodes || []).map(
async (account, index) => await this.generateNodeConfiguration(account, index, presetData, addresses),
async (account, index) =>
await this.generateNodeConfiguration(account, index, presetData, addresses, currentFinalizationEpoch),
),
);
}

private async generateNodeCertificates(presetData: ConfigPreset, addresses: Addresses): Promise<void> {
await Promise.all(
(addresses.nodes || []).map(async (account) => {
Expand Down Expand Up @@ -283,7 +291,13 @@ export class ConfigService {
);
}

private async generateNodeConfiguration(account: NodeAccount, index: number, presetData: ConfigPreset, addresses: Addresses) {
private async generateNodeConfiguration(
account: NodeAccount,
index: number,
presetData: ConfigPreset,
addresses: Addresses,
currentFinalizationEpoch: number | undefined,
) {
const copyFrom = join(this.root, 'config', 'node');
const name = account.name;

Expand Down Expand Up @@ -403,7 +417,7 @@ export class ConfigService {
copyFileSync(peersApiFile, join(join(brokerConfig, 'resources', 'peers-api.json')));
}

await new VotingService(this.params).run(presetData, account, nodePreset, undefined, undefined);
await new VotingService(this.params).run(presetData, account, nodePreset, currentFinalizationEpoch, undefined);
}

private async generateP2PFile(
Expand Down
116 changes: 116 additions & 0 deletions src/service/RemoteNodeService.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
/*
* Copyright 2021 NEM
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import { lookup } from 'dns';
import { ChainInfo, RepositoryFactory, RepositoryFactoryHttp } from 'symbol-sdk';
import { LogType } from '../logger';
import Logger from '../logger/Logger';
import LoggerFactory from '../logger/LoggerFactory';
import { ConfigPreset } from '../model';

const logger: Logger = LoggerFactory.getLogger(LogType.System);

export interface RepositoryInfo {
repositoryFactory: RepositoryFactory;
restGatewayUrl: string;
chainInfo: ChainInfo;
}
export class RemoteNodeService {
public async resolveCurrentFinalizationEpoch(presetData: ConfigPreset): Promise<number> {
const votingNode = presetData.nodes?.find((n) => n.voting);
if (!votingNode) {
return presetData.lastKnownNetworkEpoch;
}
const remoteNodeService = new RemoteNodeService();
if (!(await remoteNodeService.isConnectedToInternet())) {
return presetData.lastKnownNetworkEpoch;
}
return (await remoteNodeService.getBestFinalizationEpoch(presetData.knownRestGateways)) || presetData.lastKnownNetworkEpoch;
}

public async getBestFinalizationEpoch(urls: string[] | undefined): Promise<number | undefined> {
if (!urls || !urls.length) {
return undefined;
}
const repositoryInfo = this.sortByHeight(await this.getKnownNodeRepositoryInfos(urls)).find((i) => i);
const finalizationEpoch = repositoryInfo?.chainInfo.latestFinalizedBlock.finalizationEpoch;
if (finalizationEpoch) {
logger.info(`The current network finalization epoch is ${finalizationEpoch}`);
}
return finalizationEpoch;
}

public async getBestRepositoryInfo(urls: string[]): Promise<RepositoryInfo> {
const repositoryInfo = this.sortByHeight(await this.getKnownNodeRepositoryInfos(urls)).find((i) => i);
if (!repositoryInfo) {
throw new Error(`No up and running node could be found out of: \n - ${urls.join('\n - ')}`);
}
logger.info(`Connecting to node ${repositoryInfo.restGatewayUrl}`);
return repositoryInfo;
}

private sortByHeight(repos: RepositoryInfo[]): RepositoryInfo[] {
return repos
.filter((b) => b.chainInfo)
.sort((a, b) => {
if (!a.chainInfo) {
return 1;
}
if (!b.chainInfo) {
return -1;
}
return b.chainInfo.height.compare(a.chainInfo.height);
});
}

public isConnectedToInternet(): Promise<boolean> {
return new Promise<boolean>((resolve) => {
lookup('google.com', (err) => {
if (err && err.code == 'ENOTFOUND') {
resolve(false);
} else {
resolve(true);
}
});
});
}

private async getKnownNodeRepositoryInfos(urls: string[]): Promise<RepositoryInfo[]> {
logger.info(`Looking for the best node out of: \n - ${urls.join('\n - ')}`);
return (
await Promise.all(
urls.map(
async (restGatewayUrl): Promise<RepositoryInfo | undefined> => {
const repositoryFactory = new RepositoryFactoryHttp(restGatewayUrl);
try {
const chainInfo = await repositoryFactory.createChainRepository().getChainInfo().toPromise();
return {
restGatewayUrl,
repositoryFactory,
chainInfo,
};
} catch (e) {
const message = `There has been an error talking to node ${restGatewayUrl}. Error: ${e.message}}`;
logger.warn(message);
return undefined;
}
},
),
)
)
.filter((i) => i)
.map((i) => i as RepositoryInfo);
}
}
1 change: 0 additions & 1 deletion src/service/VotingService.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,6 @@ export class VotingService {
return newFileCreated;
}
if (!update && currentVotingFiles.length > 0) {
logger.warn('');
logger.warn(
`Voting key files are close to EXPIRATION or have EXPIRED!. Run the 'symbol-bootstrap upgradeVotingKeys' command!`,
);
Expand Down
1 change: 1 addition & 0 deletions src/service/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ export * from './ForgeCertificateService';
export * from './LinkService';
export * from './NemgenService';
export * from './PortService';
export * from './RemoteNodeService';
export * from './ReportService';
export * from './RewardProgramService';
export * from './RunService';
Expand Down
Loading

0 comments on commit 1f65bd3

Please sign in to comment.