Ethash: Abstract and light cache #1608
Conversation
|
You've been very busy, do you even sleep sometimes lol Little clarification though, when i use |
|
What's that sleep you're talking about? From an operations perspective it often doesn't make sense to run geth and miningcore on the same host. |
|
Also I think miningcore will generate DAGs anyway, it has a future cache which stores the DAG for the next epoch and I don't think it can pick that up from a pregenerated file. While mc does that one of your cores kinda goes to waste. |
You are definitely right about the different architectures, i don't generate enough money yet to just even think about running at that scale when you can afford a server for every single use-cases: Also I never ran so far into DAG generation despite the |
Yeah, hate to say it but MC's Dag always sucked 👎 |
|
Really need some consolidated feedback on this one as I currently lack the infrastructure to give the PR the thorough testing it deserves. |
|
I run miningcore on docker swarm, on a ditributed architecture, and I used the light-cache branch of @jon4hz to deploy my etc-pool. It was a headache to setup a shared volume for the dag between nodes, and the fact is open-etc-pool was easier to deploy as it uses a light cache. With this feature, I can stick with miningcore :) |
|
I haven't had time to give it a try, i'm under with my daily job sadly but i will definitely give it a go as soon as possible :) @Censseo Even without that feature, i could not even think about using |
|
I can understand that but open-etc-pool is written in go 😂 anyway, so far so good, no error from the miningcore part since the launch of the pool, but still no block ^^' |
lol my bad, it's Why are you not testing on ETC Mordor, it will be way more easy to find I have been testing the code on |
Sorry if I wasn't clear, I obsviouly tested on mordor before going to mainet 😂 I hate surprises in PROD haha |
|
I have the ETC up and running. Tested it for 4 hours with 0.25 TH/S to see how it went. Got two blocks with no bugs or errors. The other thing I noticed was the payments were processing properly now as well. |
|
I checked one ethash works but the ping jumps a lot, on my server the delay should be 12ms but with light cache min 18ms - max 35ms |
|
Hmm that's odd. I can't think of anything I changed that would affect general latency. |
|
it's just looking in the miner |
|
It would be interesting how the metrics compare. Only the output from the miner isn't really a reliable indicator because it also contains network latency, etc. |
|
But generally speaking I think it makes sense that hashing takes a little longer. From what I understand, the light cache generates the required data on the fly while the full DAG has everything precalculated. A few ms should be ok imo but not too much 😅. |
|
this is all of course not very important, it still works fine, no need to wait for the generation of the dag. |
|
I had some spare time to run a few benchmarks and the results are quite interesting. First of all some disclosure: that was the first time I did some benchmarks on dotnet, so please take a look and correct me if I did any mistakes. I tested the following things:
The fasted method (by far) was miningcore using the full dag. Miningcore with the light cache and go-etchash were slower, however the miningcore light cache implementation was slightly faster than go-etchash. The table below shows the average time it took to calculate the hash.
Full DotNetBenchmark Results:Miningcore.Tests.Benchmarks.Crypto.EthashBenchmarks-report_full.csv Go benchmark output:Benchmark codeMiningcore DAG: jon4hz@72d6ec9 func BenchmarkLight(b *testing.B) {
testHeight := uint64(60000)
testHash := common.HexToHash("5fc898f16035bf5ac9c6d9077ae1e3d5fc1ecc3c9fd5bee8bb00e810fdacbaa0")
testNonce := uint64(0x50377003e5d830ca)
var ecip1099FBlockClassic uint64 = 11700000
var hasher = New(&ecip1099FBlockClassic, nil)
for i := 0; i < b.N; i++ {
epochLength := calcEpochLength(testHeight, hasher.Light.ecip1099FBlock)
epoch := calcEpoch(testHeight, epochLength)
cache := hasher.Light.getCache(testHeight)
dagSize := datasetSize(epoch)
_, _ = cache.compute(uint64(dagSize), testHash, testNonce)
}
}Now what?As you can see, using the light cache is quite a bit slower than using the full dag. However this light cache implementation is still a bit faster than the implementation most ethereum pools use - at least the ones based on open-ethereum-pool. What's your opinion on this? And last but not least, what do you think about having both options in miningcore? It should be possible to add a config option, so the user can choose if they want to generate the full dag or only use the light cache depending on their needs. |
|
Also tested this library suggested by @Konstantin35 and it's approximately the same as go-etchash. Benchmark codefunc BenchmarkLight(b *testing.B) {
testHeight := uint64(60000)
testHash := testutil.MustDecodeHex("5fc898f16035bf5ac9c6d9077ae1e3d5fc1ecc3c9fd5bee8bb00e810fdacbaa0")
testNonce := uint64(0x50377003e5d830ca)
client := NewEthereum()
for i := 0; i < b.N; i++ {
client.Compute(testHash, testHeight, testNonce)
}
}Result |
|
I think you should just let the user decides if she/he wants to use the |
|
true, it depends of the infrastructure of the pool, some are just doing old good servers with node and pool on the same machine, so it can use the dag generated by the node. |
i went through this code and attempted some of my own tests. I'm confused on how you get geth to generate the dag file without the geth node having to allocate a useless thread to start mining. |
It's because currently in production, i'm still using the old legacy But in my situation, |
when you run geth, to get geth to generate the dag file you have to run |
I just run the following command-line: And my I imagine the line partially responsible for the DAG files are the following: I just point |
|
Starting the geth client with Running 19 different stratums around the world, the latency for a submit hash to the light client, which then has to send it to a light server, which then has to put it on the chain for a 15 second block time seems like a great way to get uncles. I could be wrong. |
Hey, i'm just describing a use case scenario here, i understand perfectly there are countless of different infrastructure out there lol |
Sorry didn't mean for it to come across as an attack. Not my intention at all. Thank you sir for your suggestions... I agree with you both options should be set in the code. |
No worry, it's all good. Have a great week ahead 👍 |
|
Well bad news for me with that PR in |
Hi again,
This PR does not only abstract the ethash algos, it also makes the DAG generation obsolete by only using a light cache only.
I did some research an this is the same behavior as open-ethereum-pool has.
Initially I was a bit concerned the this might have a performance impact, but even the official ethereum wiki mentions that the cache is enough for hash validation.
https://ethereum.org/en/developers/docs/consensus-mechanisms/pow/mining-algorithms/ethash/
The code was tested on mordor.
Related discussion: #1586