Releases: decred/dcrdata
v4.0.0
IMPORTANT NOTES
- This version of dcrdata requires a build of dcrd running JSON RPC server version 5.1.x, such as the dcrd v1.4.0 release.
- When upgrading from dcrdata v3.x, the PostgreSQL database does NOT need to be rebuilt from scratch. However, if upgrading from v2.x.y, a complete DB rebuild is needed.
Summary (work in progress!)
This is the release of dcrdata v4.0.0. Since the last published non-patch release (v3.1.0), the following changes were made:
Explorer pages
- Complete redesign with new page styles, more information, and better organization.
- Politeia Proposal(s) pages.
- Listing aggregate block information on time intervals (week, month, etc.) on Block page.
- Charts may use log or linear y axis scaling.
- Exchange rate monitoring for Decred-to-fiat conversions.
- lots...
HTTP APIs
- Added
/block/.../raw
and/block/.../header/raw
endpoints. These return the hexadecimal encoded serialized bytes of the block or block header. - Added
/agendas
and/agenda/{agendaid}
endpoints. "hex"
JSON field added to several "raw" endpoints (TODO: list endpoints here)"network_name"
added to API Status response- Compression is optional. Use
--compress-api
to enable compression in dcrdata's response handlers. - more...
Nuts and Bolts
The following changes are of most interest to site operators and developers:
- The dcrdata module is now v4.
- New build process. Webpack via npm. See README.md and dev/build.sh to learn more.
- Support for go 1.11.5 and 1.12.1.
- Support for dcrd v1.4.0 (JSON-RPC v5.1.0).
- New config settings (see --help for details):
- API:
insight-limit-rps
,max-api-addrs
,compress-api
- DB:
purge-n-blocks
,fast-sqlite-purge
,pgtimeout
,hidepgconfig
,addr-cache-cap
- Politeia:
proposalsdbfile
,politeiaurl
- Exchange bot:
exchange-monitor
,disable-exchange
,exchange-currency
,ratemaster
,ratecert
- Site config:
mainnet-link
,testnet-link
- API:
- Removed config settings:
mempool
(it is required),dumpallmptix
(was a debug tool) - Updated CONTRIBUTING.md, and moved it to the docs folder.
- Information from dcrd's new
getblockchaininfo
RPC is used in many places. - dcrd's notification sequence for chain reorganization events has changed with RPC v5.0.0, and dcrdata was changed to work with the new sequence. JSON-RPC server versions older than v5.0.0 are not compatible. Use the current dcrd release.
- New "gov" packages (
gov/agendas
andgov/politeia
), and the /proposals page. - New
pubsub
package, including related packages (pubsub/psclient
andpubsub/types
) and a demo client app (pubsub/democlient
).pubsub
's main types arePubSubHub
andWebsocketHub
.PubSubHub
manages the collection and distribution of block chain and mempool data.WebsocketHub
and its event loop manage all websocket client connections.psclient
defines aClient
type that can help writing apps that connect to a dcrdata pub-sub server. mempool
package has expanded functionality as a central hub for mempool data.explorer
andpubsub
now get their mempool updates from themempool
package'sMempoolMonitor
type.explorerUI
andPubSubHub
implement theMempoolDataSaver
interface.- New
exchanges
package that implements an exchange rate data collection bot. - New
dcrrates
package andrateserver
app for running a remote exchange data bot with one ore more dcrdata instances as clients. - New
testutil/apiload
package for load testing the dcrdata APIs. - New
db/cache
package for caching address data. Presently used indcrpg
. - The
chainMonitor
in theblockdata
package replacedBlockConnectedSync
withConnectBlock
. It is no longer necessary to run theBlockConnectedHandler
goroutine. - Added the ability to purge N blocks on startup to assist in recovery.
dcrpg
(dcrdata's PostgreSQL interface) changes:dcrpg
table versions are now 3.10.0 (dcrdata v3.1 had tables at version 3.5.5).ChainDB
queries have a query timeout now, and a cancelablecontext.Context
.- Multiple caches added for address (
db/cache
) and ticket pool data. - When the log level is debug or verbose, the PostgreSQL server (postgres) settings are displayed in a table on dcrdata startup to assist in debugging configuration issues.
- Time in PostgreSQL tables is now stored as
TIMESTAMPTZ
instead ofINT64
. Time fields of Go structs inapi/types
are changed to the newTimeDef
type. Tables:addresses
,vins
,blocks
,transactions
. chainwork
column (TEXT
) ADDED to theblocks
table.extra_data
,merkle_root
,stake_root
, andfinal_state
columns are REMOVED from theblocks
table.- The
agendas
table scheme is completely new. The old table is dropped and recreated from scratch. - New
agenda_votes
table. block_time
column (TIMESTAMPTZ
) ADDED to thevotes
table.- New
testing
table. - dcrdata sets the timezone to UTC at the beginning of each PostgreSQL session.
dcrpg
now usesdcrjson/v2
andrpcclient/v2
modules.
dcrsqlite
changes:- Now exports the main SQLite interface type,
WiredDB
. - New block data cache.
WiredDB
will request a shutdown with a provided function if the "database is locked" error is encountered on a table write operation.UnconfirmedTxnsForAddress
is moved fromdcrsqlite
torpcutils
.- Side chains are now tracked:
hash
is the new primary key. Theis_mainchain
andis_valid
columns are added. Ability to import side chain data from dcrd also added. explorer.New
now takes a config struct,ExplorerConfig
.
- Now exports the main SQLite interface type,
- Dependency changes:
- Updated direct deps: asdine/storm, caarlos0/env, all dcrd modules, dcrwallet/wallet, dgraph-io/badger, didip/tollbooth, go-chi/chi, google/gops, googolleego-socket.io, mattn/go-sqlite3, rs/cors, sirupsen/logrus, golang.org/x/net,
- Added deps: politeia, golang/protobuf, google.golang.org/grpc, jessevdk/go-flags
- Removed deps: btcsuite/go-flags, oleide/lane
Contributors
Summarized by number of commits:
3 Ademu Enyo Anthony
76 buck54321
23 David Hill
1 Jamie Holdstock
160 Jonathan Chappelow
2 Macaulay Davies
28 Migwi Ndung'u
2 Wisdom Arerosuoghene
23 ZeroASIC
1 degeri
25 gozart1
1 jujuhar
394 changed files with 47,963 additions and 19,205 deletions (yes, that is enormous!):
v3.1.0...v4.0.0
v3.1.2
This is patch release 2 of the dcrdata 3.1 series.
Fixes:
- recompute checksum of the github.com/Sereal/Sereal dependency, which had the incorrect checksum as a consequence of a symlink handling bug prior to go 1.11.4. Refs.: golang/go#27093 and golang/go@b86522f (dcrdata: 5ec8c96)
v3.1.1
This is patch release 1 of the dcrdata 3.1 series.
Fixes:
- mempool last block JS age controller fixed (8646462)
- address go-sqlite "database is locked" errors (6a9fed5). This can slow down sqlite operations since it limits DB "connections" to 1. In 4.0, the limit is removed since the go-sqlite version dependency is updated to a version built with correct multi-threading flags.
- resolve a possible panic on shutdown (261048e)
v3.1.0
IMPORTANT NOTES:
- This version of dcrdata requires a build of dcrd running JSON RPC server version 5.0.x, such as the current development version of dcrd v1.4.0.
- When upgrading from v3.0.x, the PostgreSQL database does NOT need to be rebuilt from scratch. However, if upgrading from v2.x.y, a complete DB rebuild is needed.
This is the release of dcrdata v3.1.0. Since the last published non-patch release (v3.0.0), the following changes were made:
-
Build with Go 1.11 modules, defining dcrdata as v3 module. (2f68351, 5f24e74, 789129d, 44406ef, 92ff8a2)
-
Major initial sync performance improvements. (d01def5, 636cfa5)
-
Add netparams package for access to network service port numbers.
-
New version scheme with "appPrerelease" and "appBuild" info to generate version strings like v3.1.0-release or v3.1.1-pre+dev. (cc0e85c) This matches the version scheme now in use by dcrd and dcrwallet. The build syntax for dcrdata is as follows:
go build -ldflags "-X github.com/decred/dcrdata/v3/version.appPreRelease= -X github.com/decred/dcrdata/v3/version.appBuild=`git rev-parse --short HEAD`"
-
Bugs fixed on /agendas page (e54cfa1) and with agendadb (4b074b3)
-
Improvements to /address page charts, ensuring the group-by interval selected is the most meaningful for the current address. (d105024)
-
Add the ability to use environment variables for configuration. (f2903f1)
-
All votes, including side chain votes, are included on the mempool page. Votes on blocks other than the best block are highlighted in red. (93b5788)
-
Votes on the mempool page are given an index according to which ticket is spent. (8a2e2ad and b4ae964)
-
Basic Dockerfiles. (61a38fb)
-
Restore next expected ticket price and bounds, like we used to show it. (ba0f9ee)
-
Support the
/sync
Insight API endpoint. (aac0885) -
Block page now handles "future" blocks with height not yet reached. (71c1d03)
-
Side chain support:
- Queries and
/side
page (b159166, 706b8aa)./side
is a list of side chain blocks with links to them, their parents, and their children, if any. The /block page is updated to indicate if the block is a side chain block, and if it is approved by stakeholders. - Shows on the transaction page a list of blocks containing that transaction. There should be only one block that is both mainchain and stake-validated. Any other blocks will be "Mainchain: false" or "Valid: false" or both. When a transaction is found in multiple blocks, list them all with unambiguous links by hash. Also pass the transaction index via
?ind=
URL parameter so that we can use it on the block page to highlight the transaction. (091b91b, 412dd7e)
- Queries and
-
Retrieve and import side chain blocks from dcrd via the new getchaintips RPC. (254ecab, 7adf9bf, 235ddd8)
-
Stake-invalidated blocks page at
/rejects
(19953f6), and improved stake-invalidation handling. (0dfef89) -
Invalidated transaction support. If a transaction is not known to dcrd, it is because it has been orphaned by a reorg, invalidated via stakeholder disapproval, or both. dcrd will now attempt to get transaction details from the dcrpg database, which may have decoded and recorded the transaction. A message is shown on the transaction page when the transaction is not found in any valid and mainchain block. (2cfeccd)
-
Special handling of genesis block coinbase transaction, which is invalid. (10fccfc) Show an attention symbol in red next to the tx hash. A title (hover text) is shown explaining why it is special. The link points to code (chaincfg/genesis.go).
-
Concurrency-related fixes in explorer package. (3ae6c99)
-
Style invalidated transactions on the block and transactions pages: gray out and change background color of regular transaction table data, including the coinbase, on a block's page. On the blocks list page, indicate invalidated blocks. (8d924bf)
-
Improved error handling on /address page. (700d3c5)
-
Address API no longer returns null for address with no transactions, but an object with an empty slice of transactions. (979872d)
-
Address page and query performance improvements. (3e42cd1, 08147c3)
-
Patch valid_mainchain column of addresses table. (078483d)
-
Fix the address page for addresses with only unconfirmed transactions. (38c51a1)
-
Input/output highlighting on the address and transaction pages. To support the highlighting, a new route for identifying the input or output of interest is added:
/tx/{txid}/{in|out}/{idx}
. The input/output that pairs with the corresponding outputs/inputs in a transaction's view are obtained via queries into the vins table. (60d033d) -
Various sync and startup improvements and fixes. (08f1839, 97ed3bd, 769abd2, 59c6fbd)
-
dcrpg query maintenance/reorganization. (102df3e)
-
Add the ability to show transaction output spending information via ?spends=true on the following API endpoints: /api/tx, /api/tx/decoded, and /api/txs. (e96c9d6)
-
Prototyping new home page at /nexthome. (694eaee)
-
Add a syncing status page that starts up before dcrdata completes syncing with dcrd. (44c8a08, eab8e43)
-
Adjust the config/log/data paths if
appdata
is set. (98b418b) -
Show mining fees on /block page. (3a97e63)
-
Docs and sample config updates. (b97a023, f6d2f42, eec01ac, 28c3d69)
-
Add new page /ticketpricewindows that lists all of the historical ticket price windows. (e8cedde, 7461f90)
-
Replace the shared
quit
channel with acontext.Context
from the "context" standard library. (71812de) -
Show Tx Type in the address page's tx table. (052e344)
-
Overhaul reorg handling for dcrd's JSON RPC v5.0.0 notification sequence. (2fd7d04)
-
Improved support for browsers with JavaScript disabled. (a66fba3)
-
Better address validation. (dde716d)
-
Numerous UI, formatting, and wording tweaks. Many other minor and under-the-hood changes.
16 code contributors (alphabetical): @ademuanthony, @buck54321, @chappjc, @codemaestro64, @degeri, @dmigwi, @gozart1, @itswisdomagain, @jholdstock, @Jujhar, @logicminds, @McEdward, @papacarp, @RogueElement, @s-ben, @SeaLightHouse.
158 changed files with 16,122 additions and 6,485 deletions:
v3.0.0...v3.1.0
v3.0.3 - appdata fix
This is a patch release with one fix since v3.0.2:
When appdata
is set, dcrdata now adjusts the config/log/data paths.
With appdata
set to a non-default value via either a command line switch or the
environment variable, this updates the default config file location to avoid an
error. A missing config file in the default appdata
directory should not have been
an error.
This change also updates the default config file location when a custom appdata
is set so there is no error if the config file is missing. It is only an error if
the config file location is explicitly set and does not exist.
A small optimization is also made such that the API status is not updated
with an invalid height when DB is empty. When first starting with an empty
sqlite DB, do not send a bogus height (4294967295 = uint32(-1)
) on the API
status update channel.
The block height is added to the error message if no block is returned by
appContext.BlockData.GetBestBlockSummary
when performing this lookup in
response to an API status update signal received by appContext.StatusNtfnHandler
.
v3.0.2 - Insight and Stake-invalidation Fixes
This is a patch release with several important fixes. The postgresq DB table format is bumped to 3.5.4, requiring a short automatic upgrade on initial startup of this version.
The following changes were made since v3.0.0:
- Fix Insight API's transaction broadcast response. 2824630
- Patch the valid_mainchain column of the addresses table to accurately reflect stake-invalidation of blocks. 20c3a4d
- Patch the matching_tx_hash column of the addresses table, and keep it updated during stake-invalidation of blocks. fd07df2
- Fix marshaling of vouts array in Insight API's new transaction message. 3f4f4de
- Support multiple dcrd RPC versions including 3.x.x and 4.x.x. 60161ef
- Fix dcrpg tests, allowing "go test -tags mainnettest" to work again. 87d6577
- Remove duplicate "Blocks" item in main menu. 8fbdf47
- Fix address page display where there are only unconfirmed transactions. 7b5cd3c
v3.0.0 - Charts, Merged Debits, Side Chain Tracking, Insight API, testnet3
NOTE: The PostgreSQL database must be rebuilt from scratch (using the dcrdata executable) as the scheme is incompatible with previous versions and an upgrade was not practical.
This is the release of dcrdata v3.0.0. Since the last published release (v2.1.3), the following changes were made:
- New "Merged Debits" transaction filter on an addresses page, which eliminates display of multiple inputs belonging to the same transaction, instead consolidating to a single row for such a transaction. This greatly improves display of the project fund spending activity.
- The addresses table has a new scheme, which supports various improvements to the address web page.
- Side chain tracking, supporting many new upcoming features related to reorganization and stake approval.
- Cool new historical charts for the network, and on the address page
- Initial experimental Insight API support (HTTP via /insight/api, and websocket/socket.io via /insight/socket.io).
- Stake agenda DB table and web page.
- Add block rewards breakdown API endpoint /block/.../subsidy
- Distinct color themes for the networks: mainnet, testnet, simnet
- Go module support (github.com/decred/dcrdata/v3)
- Mempool page improvements with respect to old and/or side chain vote display and styling.
- Bring back the next expected ticket price display on the home page.
- testnet3 support (no testnet2)
- Go 1.11 support, with module build support.
- Add
--no-dev-prefetch
option to disable computing the project fund balance after each new block. - Add
--sync-and-quit
option to immediately quit after syncing to the network best block. - Replace error page with a more informative status page.
- Improved display of both unconfirmed and confirmed transactions on the address page.
- New dcrsqlite tests and testutil package.
- Special handling for dummy sstxchange address
9 code contributors (alphabedical): @chappjc, @codemaestro64, @dmigwi, @gozart1, @JFixby, @papacarp, @RogueElement, @SeaLightHouse, @snadorp.
106 changed files with 9,064 additions and 1,822 deletions:
v2.1.3 - Address Transaction Filters, New Endpoints, Chain Parameters Page, testnet3, Menu Animation
NOTE: Not compatible with the data tables scheme or data directory layout of v1.x, v3.x, OR master
.
NOTE 2: This release supports testnet3, not testnet2. The latest release with testnet2 support is v2.1.0.
This is the release of dcrdata v2.1.3. Since the last published release (v2.0), the following changes were made:
- Filtering address transactions by credit/debit
- Added /api/address/{addr}/totals API endpoint
- Added chain parameters page
- Fresh hamburger menu with animated transitions
- Keyboard navigation
- Ticket return calculations shown on home page
- Initial Insight API framework.
- http server has conservative read/write timeouts
- Pagination to /api/address endpoints
- Added batch transactions request API endpoints /txs and /txs/trimmed
- Begin refactoring JavaScript to use Stimulus.js
- Websocket robustness improvements and bug fixes
- Documentation for Go types used in API responses
- Improved API status endpoint with db block height
- Added coin supply API endpoint /api/supply
- Improve PostgreSQL tuning instructions.
- gops agent and instrumentation options
- Dev fund is now "project fund"
- Miner fees displayed for coinbase transaction
- Performance improvements in stakedb package.
- Ticket pool database migrated from storm to badger.
- Many bug fixes and performance improvements.
14 code contributors: @gozart1 , @samparsky, @McEdward, @girino, @papacarp, @RogueElement, @jrick, @JFixby, @orthomind, @lte13, @timi-ola, @SeaLightHouse, @snadorp, @chappjc.
113 changed files with 9,100 additions and 2,222 deletions.
v2.0 - Frontend Restyling, Mempool Page, Stake Tables, Ticket Luck Score, Backend Overhaul, WebSocket Improvements
NOTE: Not compatible with the data tables scheme or data directory layout of v1.3 OR master
.
This is the release of dcrdata v2.0. Since the last release (v1.3.0), the following changes were made:
- Desktop notifications (optional) for new blocks. Just click the green websocket light at the bottom right and click "allow"
- Development fund "balance" is updated asynchronously to prevent slow block processing.
- "Lite" mode is now the default. To start with PostgreSQL in full mode, use
--pg
. - More robust handling of ticket DB, with better crash recovery.
- Support go1.10, drop go1.8
- Lots more
6 contributors: @gozart1 , @RogueElement , @McEdward , @dajohi , @papacarp , @chappjc, @raedah .
6,722 additions and 1,701 deletions.
v1.3.1...v2.0
Hcash Edition! (v1.3.2.0.0.0.0.0.0.1)
This is the Hcash edition of dcrdata. It works (probably)!