Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Neutrino: Sync #1168

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion bin/bcoin
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,10 @@ for arg in "$@"; do
--daemon)
daemon=1
;;
--spv)
--neutrino)
cmd='neutrino'
;;
--spv)
cmd='spvnode'
;;
esac
Expand Down
43 changes: 43 additions & 0 deletions bin/neutrino
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
#!/usr/bin/env node

'use strict';

console.log('Starting bcoin');
process.title = 'bcoin';
const Neutrino = require('../lib/node/neutrino');

const node = new Neutrino({
file: true,
argv: true,
env: true,
logFile: true,
logConsole: true, // todo: remove
logLevel: 'debug', // todo: remove
db: 'leveldb',
memory: false,
workers: true,
loader: require
});

if (!node.config.bool('no-wallet') && !node.has('walletdb')) {
const plugin = require('../lib/wallet/plugin');
node.use(plugin);
}

(async () => {
await node.ensure();
await node.open();
await node.connect();
node.startSync();
})().catch((err) => {
console.error(err.stack);
process.exit(1);
});

process.on('unhandledRejection', (err, promise) => {
throw err;
});

process.on('SIGINT', async () => {
await node.close();
});
1 change: 1 addition & 0 deletions lib/bcoin-browser.js
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ bcoin.node = require('./node');
bcoin.Node = require('./node/node');
bcoin.FullNode = require('./node/fullnode');
bcoin.SPVNode = require('./node/spvnode');
bcoin.Neutrino = require('./node/neutrino');
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@masterchief164 Do we need to add neutrino to bcoin-browser? AFAIK, bcoin-browser is obselete.


// Primitives
bcoin.primitives = require('./primitives');
Expand Down
1 change: 1 addition & 0 deletions lib/bcoin.js
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,7 @@ bcoin.define('node', './node');
bcoin.define('Node', './node/node');
bcoin.define('FullNode', './node/fullnode');
bcoin.define('SPVNode', './node/spvnode');
bcoin.define('Neutrino', './node/neutrino');

// Primitives
bcoin.define('primitives', './primitives');
Expand Down
10 changes: 7 additions & 3 deletions lib/client/node.js
Original file line number Diff line number Diff line change
Expand Up @@ -164,9 +164,13 @@ class NodeClient extends Client {
* @returns {Promise}
*/

getFilter(filter) {
assert(typeof filter === 'string' || typeof filter === 'number');
return this.get(`/filter/${filter}`);
getFilter(block) {
assert(typeof block === 'string' || typeof block === 'number');
return this.get(`/filter/${block}`);
}

getBlockPeer(hash) {
return this.call('get block peer', hash);
}

/**
Expand Down
25 changes: 25 additions & 0 deletions lib/indexer/filterindexer.js
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,31 @@ class FilterIndexer extends Indexer {
this.put(layout.f.encode(hash), gcsFilter.hash());
}

/**
* Save filter
* @param {Hash} blockHash
* @param {BasicFilter} basicFilter
* @param {Hash} filterHeader
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: add blockheight

* @returns {Promise}
*/

async saveFilter(blockHash, blockHeight, basicFilter, filterHeader) {
assert(blockHash);
assert(blockHeight);
assert(basicFilter);
assert(filterHeader);
Comment on lines +97 to +100
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can probably use 1 assertion here

Suggested change
assert(blockHash);
assert(blockHeight);
assert(basicFilter);
assert(filterHeader);
assert(blockHash & blockHeight & basicFilter & filterHeader);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We won't be able to know which value gives out the error.

Copy link
Contributor Author

@manavdesai27 manavdesai27 Aug 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And why would you do a logical and here? What if for example my bits are 1000 and 0001?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing he meant &&? but if you are going to do this you mine as well check each input value type is correct too


const filter = new Filter();
filter.filter = basicFilter.toRaw();
filter.header = filterHeader;
Comment on lines +102 to +104
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe I'm being too picky about code styles but I'd prefer if you pass an options object instead of assigning values to class variables

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that many functions in filterindexer.js is doing this. We should make this change everywhere since we are touching this file anyway.


this.batch = this.db.batch();

await this.blocks.writeFilter(blockHash, filter.toRaw(), this.filterType);
this.put(layout.f.encode(blockHash), basicFilter.hash());
await super.syncHeight(blockHash, blockHeight);
}

/**
* Prune compact filters.
* @private
Expand Down
21 changes: 21 additions & 0 deletions lib/indexer/indexer.js
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ class Indexer extends EventEmitter {
this.blocks = this.options.blocks;
this.chain = this.options.chain;

this.neutrino = this.options.neutrino;

this.closing = false;
this.db = null;
this.batch = null;
Expand Down Expand Up @@ -196,6 +198,13 @@ class Indexer extends EventEmitter {
this.height = 0;
}

async syncHeight(hash, height) {
const meta = new BlockMeta(hash, height);
await this._setTip(meta);
await this.commit();
this.height = height;
}

/**
* Bind to chain events and save listeners for removal on close
* @private
Expand Down Expand Up @@ -292,6 +301,11 @@ class Indexer extends EventEmitter {
*/

async _syncBlock(meta, block, view) {
if (this.neutrino) {
if (!this.batch)
this.start();
return true;
}
// In the case that the next block is being
// connected or the current block disconnected
// use the block and view being passed directly,
Expand Down Expand Up @@ -636,6 +650,8 @@ class IndexOptions {
this.cacheSize = 16 << 20;
this.compression = true;

this.neutrino = false;

if (options)
this.fromOptions(options);
}
Expand Down Expand Up @@ -697,6 +713,11 @@ class IndexOptions {
this.compression = options.compression;
}

if (options.neutrino != null) {
assert(typeof options.neutrino === 'boolean');
this.neutrino = options.neutrino;
}

return this;
}

Expand Down
3 changes: 2 additions & 1 deletion lib/net/netaddress.js
Original file line number Diff line number Diff line change
Expand Up @@ -478,7 +478,8 @@ class NetAddress {
NetAddress.DEFAULT_SERVICES = 0
| common.services.NETWORK
| common.services.WITNESS
| common.services.BLOOM;
| common.services.BLOOM
| common.services.NODE_COMPACT_FILTERS;

/*
* Expose
Expand Down
64 changes: 64 additions & 0 deletions lib/net/peer.js
Original file line number Diff line number Diff line change
Expand Up @@ -1009,6 +1009,12 @@ class Peer extends EventEmitter {
case packetTypes.GETHEADERS:
this.request(packetTypes.HEADERS, timeout * 2);
break;
case packetTypes.GETCFHEADERS:
this.request(packetTypes.CFHEADERS, timeout);
break;
case packetTypes.GETCFILTERS:
this.request(packetTypes.CFILTER, timeout);
break;
case packetTypes.GETDATA:
this.request(packetTypes.DATA, timeout * 2);
break;
Expand Down Expand Up @@ -1449,6 +1455,12 @@ class Peer extends EventEmitter {
if (!(this.services & services.NETWORK))
throw new Error('Peer does not support network services.');

if (this.options.neutrino) {
if (!(this.services & services.NODE_COMPACT_FILTERS)) {
throw new Error('Peer does not support Compact Filters.');
}
}

if (this.options.headers) {
if (this.version < common.HEADERS_VERSION)
throw new Error('Peer does not support getheaders.');
Expand Down Expand Up @@ -1668,6 +1680,11 @@ class Peer extends EventEmitter {
this.compactWitness = packet.version === 2;
}

sendSendHeaders() {
const packet = new packets.SendHeadersPacket();
this.send(packet);
}

/**
* Send `getheaders` to peer. Note that unlike
* `getblocks`, `getheaders` can have a null locator.
Expand Down Expand Up @@ -1745,6 +1762,26 @@ class Peer extends EventEmitter {
this.send(packet);
}

/**
* @param {Number} filterType - `0` = basic
* @param {Number} startHeight - Height to start at.
* @param {Hash} stopHash - Hash to stop at.
* @returns {void}
* @description Send `getcfilters` to peer.
*/
sendGetCFilters(filterType, startHeight, stopHash) {
const packet = new packets.GetCFiltersPacket(
filterType,
startHeight,
stopHash);

this.logger.debug(
'Sending getcfilters (type=%d, startHeight=%d, stopHash=%h).',
filterType, startHeight, stopHash);

this.send(packet);
}

/**
* Send `cfheaders` to peer.
* @param {Number} filterType
Expand All @@ -1767,6 +1804,27 @@ class Peer extends EventEmitter {
this.send(packet);
}

/**
* @param {Number} filterType
* @param {Number} startHeight
* @param {Hash} stopHash
* @returns {void}
* @description Send `getcfheaders` to peer.
*/

sendGetCFHeaders(filterType, startHeight, stopHash) {
const packet = new packets.GetCFHeadersPacket(
filterType,
startHeight,
stopHash);

this.logger.debug(
'Sending getcfheaders (type=%d, start=%h, stop=%h).',
filterType, startHeight, stopHash);

this.send(packet);
}

/**
* send `cfcheckpt` to peer.
* @param {Number} filterType
Expand Down Expand Up @@ -2080,6 +2138,7 @@ class PeerOptions {
this.agent = common.USER_AGENT;
this.noRelay = false;
this.spv = false;
this.neutrino = false;
this.compact = false;
this.headers = false;
this.banScore = common.BAN_SCORE;
Expand Down Expand Up @@ -2143,6 +2202,11 @@ class PeerOptions {
this.spv = options.spv;
}

if (options.neutrino != null) {
assert(typeof options.neutrino === 'boolean');
this.neutrino = options.neutrino;
}

if (options.compact != null) {
assert(typeof options.compact === 'boolean');
this.compact = options.compact;
Expand Down
Loading