Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sniffer Improvements #1258

Merged
merged 2 commits into from
Nov 20, 2024
Merged

Sniffer Improvements #1258

merged 2 commits into from
Nov 20, 2024

Conversation

jbesraa
Copy link
Contributor

@jbesraa jbesraa commented Nov 18, 2024

Partially resolves #1253

@jbesraa jbesraa marked this pull request as ready for review November 18, 2024 09:21
@codecov Codecov
Copy link

codecov bot commented Nov 18, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 19.30%. Comparing base (0520c58) to head (27eec57).
Report is 3 commits behind head on main.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main    #1258   +/-   ##
=======================================
  Coverage   19.30%   19.30%           
=======================================
  Files         164      164           
  Lines       10849    10849           
=======================================
  Hits         2094     2094           
  Misses       8755     8755
Flag Coverage Δ
binary_codec_sv2-coverage 0.00% <ø> (ø)
binary_serde_sv2-coverage 3.65% <ø> (ø)
binary_sv2-coverage 5.46% <ø> (ø)
bip32_derivation-coverage 0.00% <ø> (ø)
buffer_sv2-coverage 25.02% <ø> (ø)
codec_sv2-coverage 0.01% <ø> (ø)
common_messages_sv2-coverage 0.13% <ø> (ø)
const_sv2-coverage 0.00% <ø> (ø)
error_handling-coverage 0.00% <ø> (ø)
framing_sv2-coverage 0.29% <ø> (ø)
jd_client-coverage 0.00% <ø> (ø)
jd_server-coverage 7.79% <ø> (ø)
job_declaration_sv2-coverage 0.00% <ø> (ø)
key-utils-coverage 2.39% <ø> (ø)
mining-coverage 2.51% <ø> (+0.01%) ⬆️
mining_device-coverage 0.00% <ø> (ø)
mining_proxy_sv2-coverage 0.70% <ø> (ø)
noise_sv2-coverage 4.35% <ø> (ø)
pool_sv2-coverage 1.38% <ø> (ø)
protocols 24.72% <ø> (ø)
roles 6.55% <ø> (ø)
roles_logic_sv2-coverage 8.06% <ø> (ø)
sv1-mining-device-coverage 0.00% <ø> (ø)
sv2_ffi-coverage 0.00% <ø> (ø)
template_distribution_sv2-coverage 0.00% <ø> (ø)
translator_sv2-coverage 9.60% <ø> (ø)
utils 25.13% <ø> (ø)
v1-coverage 2.47% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚨 Try these New Features:

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 18, 2024 •
edited
Loading

🐰 Bencher Report

Branch2024-11-18-sniffer-improvements
Testbedsv1
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)		Upper Boundary
nanoseconds (ns)
(Limit %)
client-submit-serialize📈 view plot
🚷 view threshold		6,949.60
(+3.22%)		7,247.78
(95.89%)
client-submit-serialize-deserialize📈 view plot
🚷 view threshold		7,826.20
(+2.22%)		8,059.21
(97.11%)
client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle📈 view plot
🚷 view threshold		8,426.00
(+2.54%)		8,711.49
(96.72%)
client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle📈 view plot
🚷 view threshold		879.03
(+0.44%)		939.73
(93.54%)
client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize📈 view plot
🚷 view threshold		678.27
(+0.73%)		716.19
(94.71%)
client-sv1-authorize-serialize/client-sv1-authorize-serialize📈 view plot
🚷 view threshold		246.88
(-0.56%)		261.23
(94.51%)
client-sv1-get-authorize/client-sv1-get-authorize📈 view plot
🚷 view threshold		158.15
(+0.34%)		163.21
(96.90%)
client-sv1-get-submit📈 view plot
🚷 view threshold		6,730.60
(+3.27%)		7,242.95
(92.93%)
client-sv1-get-subscribe/client-sv1-get-subscribe📈 view plot
🚷 view threshold		288.51
(+3.40%)		305.45
(94.45%)
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle📈 view plot
🚷 view threshold		751.24
(+1.78%)		815.21
(92.15%)
client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize📈 view plot
🚷 view threshold		590.85
(-1.34%)		644.25
(91.71%)
client-sv1-subscribe-serialize/client-sv1-subscribe-serialize📈 view plot
🚷 view threshold		201.78
(-2.16%)		225.26
(89.58%)
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 18, 2024 •
edited
Loading

🐰 Bencher Report

Branch2024-11-18-sniffer-improvements
Testbedsv2
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
estimated cycles
(Result Δ%)		Upper Boundary
estimated cycles
(Limit %)		InstructionsBenchmark Result
instructions
(Result Δ%)		Upper Boundary
instructions
(Limit %)		L1 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		L2 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		RAM AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold		2,069.00
(-1.18%)		2,206.26
(93.78%)		📈 view plot
🚷 view threshold		473.00
(-0.07%)		488.51
(96.82%)		📈 view plot
🚷 view threshold		734.00
(-0.26%)		757.27
(96.93%)		📈 view plot
🚷 view threshold		8.00
(+47.34%)		14.07
(56.86%)		📈 view plot
🚷 view threshold		37.00
(-2.67%)		41.17
(89.87%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold		8,208.00
(+0.05%)		8,285.19
(99.07%)		📈 view plot
🚷 view threshold		2,137.00
(-0.03%)		2,143.08
(99.72%)		📈 view plot
🚷 view threshold		3,158.00
(-0.09%)		3,170.61
(99.60%)		📈 view plot
🚷 view threshold		37.00
(+4.09%)		42.15
(87.79%)		📈 view plot
🚷 view threshold		139.00
(-0.01%)		140.87
(98.68%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold		6,297.00
(+0.28%)		6,391.62
(98.52%)		📈 view plot
🚷 view threshold		1,750.00
(-0.05%)		1,766.87
(99.05%)		📈 view plot
🚷 view threshold		2,547.00
(-0.26%)		2,575.63
(98.89%)		📈 view plot
🚷 view threshold		22.00
(+29.77%)		24.57
(89.54%)		📈 view plot
🚷 view threshold		104.00
(-0.02%)		107.12
(97.09%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold		14,728.00
(-0.01%)		14,959.79
(98.45%)		📈 view plot
🚷 view threshold		4,694.00
(-0.02%)		4,710.87
(99.64%)		📈 view plot
🚷 view threshold		6,753.00
(-0.08%)		6,781.90
(99.57%)		📈 view plot
🚷 view threshold		48.00
(+10.34%)		53.34
(89.99%)		📈 view plot
🚷 view threshold		221.00
(-0.24%)		227.57
(97.11%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚷 view threshold		27,772.00
(+0.87%)		27,794.16
(99.92%)		📈 view plot
🚷 view threshold		10,645.00
(+0.60%)		10,650.04
(99.95%)		📈 view plot
🚷 view threshold		15,507.00
(+0.70%)		15,507.55
(100.00%)		📈 view plot
🚷 view threshold		87.00
(+6.91%)		89.92
(96.76%)		📈 view plot
🚷 view threshold		338.00
(+0.87%)		341.64
(98.93%)
client_sv2_open_channel📈 view plot
🚷 view threshold		4,379.00
(-0.36%)		4,601.79
(95.16%)		📈 view plot
🚷 view threshold		1,461.00
(-0.02%)		1,476.51
(98.95%)		📈 view plot
🚷 view threshold		2,159.00
(-0.04%)		2,184.30
(98.84%)		📈 view plot
🚷 view threshold		10.00
(+12.58%)		17.33
(57.70%)		📈 view plot
🚷 view threshold		62.00
(-0.94%)		68.12
(91.02%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold		14,020.00
(-0.24%)		14,421.41
(97.22%)		📈 view plot
🚷 view threshold		5,064.00
(-0.01%)		5,079.51
(99.69%)		📈 view plot
🚷 view threshold		7,325.00
(-0.01%)		7,351.87
(99.63%)		📈 view plot
🚷 view threshold		37.00
(+5.46%)		43.25
(85.56%)		📈 view plot
🚷 view threshold		186.00
(-0.66%)		197.78
(94.05%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold		22,838.00
(+0.77%)		22,903.67
(99.71%)		📈 view plot
🚷 view threshold		8,040.00
(+0.24%)		8,073.69
(99.58%)		📈 view plot
🚷 view threshold		11,693.00
(+0.21%)		11,752.43
(99.49%)		📈 view plot
🚷 view threshold		80.00
(+7.53%)		85.24
(93.85%)		📈 view plot
🚷 view threshold		307.00
(+1.16%)		311.30
(98.62%)
client_sv2_setup_connection📈 view plot
🚷 view threshold		4,655.00
(-0.62%)		4,772.90
(97.53%)		📈 view plot
🚷 view threshold		1,502.00
(-0.02%)		1,517.51
(98.98%)		📈 view plot
🚷 view threshold		2,275.00
(-0.16%)		2,299.14
(98.95%)		📈 view plot
🚷 view threshold		14.00
(+47.98%)		16.23
(86.23%)		📈 view plot
🚷 view threshold		66.00
(-2.04%)		69.81
(94.55%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold		16,126.00
(-0.27%)		16,453.23
(98.01%)		📈 view plot
🚷 view threshold		5,963.00
(-0.01%)		5,978.51
(99.74%)		📈 view plot
🚷 view threshold		8,661.00
(-0.03%)		8,691.57
(99.65%)		📈 view plot
🚷 view threshold		44.00
(+10.26%)		53.22
(82.68%)		📈 view plot
🚷 view threshold		207.00
(-0.84%)		216.14
(95.77%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold		35,670.00
(+0.33%)		35,754.48
(99.76%)		📈 view plot
🚷 view threshold		14,888.00
(+0.26%)		14,908.34
(99.86%)		📈 view plot
🚷 view threshold		21,880.00
(+0.30%)		21,921.26
(99.81%)		📈 view plot
🚷 view threshold		98.00
(+5.96%)		112.77
(86.90%)		📈 view plot
🚷 view threshold		380.00
(+0.20%)		384.63
(98.80%)
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 18, 2024 •
edited
Loading

🐰 Bencher Report

Branch2024-11-18-sniffer-improvements
Testbedsv2
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)		Upper Boundary
nanoseconds (ns)
(Limit %)
client_sv2_handle_message_common📈 view plot
🚷 view threshold		44.81
(+0.28%)		46.99
(95.36%)
client_sv2_handle_message_mining📈 view plot
🚷 view threshold		73.02
(-2.62%)		84.95
(85.95%)
client_sv2_mining_message_submit_standard📈 view plot
🚷 view threshold		14.76
(+0.56%)		14.89
(99.11%)
client_sv2_mining_message_submit_standard_serialize📈 view plot
🚷 view threshold		261.97
(-1.71%)		317.15
(82.60%)
client_sv2_mining_message_submit_standard_serialize_deserialize📈 view plot
🚷 view threshold		626.23
(+3.52%)		741.10
(84.50%)
client_sv2_open_channel📈 view plot
🚷 view threshold		162.88
(+9.35%)		175.46
(92.83%)
client_sv2_open_channel_serialize📈 view plot
🚷 view threshold		293.09
(+2.47%)		315.94
(92.77%)
client_sv2_open_channel_serialize_deserialize📈 view plot
🚷 view threshold		388.03
(+2.55%)		411.55
(94.29%)
client_sv2_setup_connection📈 view plot
🚷 view threshold		159.42
(-0.95%)		174.10
(91.57%)
client_sv2_setup_connection_serialize📈 view plot
🚷 view threshold		482.32
(+3.34%)		558.32
(86.39%)
client_sv2_setup_connection_serialize_deserialize📈 view plot
🚷 view threshold		1,076.60
(+9.57%)		1,170.84
(91.95%)
🐰 View full continuous benchmarking report in Bencher

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 18, 2024 •
edited
Loading

🐰 Bencher Report

Branch2024-11-18-sniffer-improvements
Testbedsv1
Click to view all benchmark results
BenchmarkEstimated CyclesBenchmark Result
estimated cycles
(Result Δ%)		Upper Boundary
estimated cycles
(Limit %)		InstructionsBenchmark Result
instructions
(Result Δ%)		Upper Boundary
instructions
(Limit %)		L1 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		L2 AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)		RAM AccessesBenchmark Result
accesses
(Result Δ%)		Upper Boundary
accesses
(Limit %)
get_authorize📈 view plot
🚷 view threshold		8,447.00
(-0.19%)		8,767.33
(96.35%)		📈 view plot
🚷 view threshold		3,689.00
(-1.61%)		3,868.04
(95.37%)		📈 view plot
🚷 view threshold		5,157.00
(-1.91%)		5,434.70
(94.89%)		📈 view plot
🚷 view threshold		7.00
(-2.82%)		13.03
(53.71%)		📈 view plot
🚷 view threshold		93.00
(+2.68%)		95.57
(97.31%)
get_submit📈 view plot
🚷 view threshold		95,479.00
(+0.05%)		95,924.51
(99.54%)		📈 view plot
🚷 view threshold		59,346.00
(-0.19%)		59,685.41
(99.43%)		📈 view plot
🚷 view threshold		85,219.00
(-0.22%)		85,754.21
(99.38%)		📈 view plot
🚷 view threshold		43.00
(-4.31%)		63.96
(67.22%)		📈 view plot
🚷 view threshold		287.00
(+2.47%)		289.41
(99.17%)
get_subscribe📈 view plot
🚷 view threshold		8,000.00
(-0.25%)		8,317.77
(96.18%)		📈 view plot
🚷 view threshold		2,765.00
(-2.47%)		2,949.25
(93.75%)		📈 view plot
🚷 view threshold		3,845.00
(-2.96%)		4,129.88
(93.10%)		📈 view plot
🚷 view threshold		12.00
(+0.33%)		20.67
(58.06%)		📈 view plot
🚷 view threshold		117.00
(+2.42%)		119.20
(98.15%)
serialize_authorize📈 view plot
🚷 view threshold		12,172.00
(-0.88%)		12,595.47
(96.64%)		📈 view plot
🚷 view threshold		5,266.00
(-1.03%)		5,436.74
(96.86%)		📈 view plot
🚷 view threshold		7,327.00
(-1.25%)		7,594.59
(96.48%)		📈 view plot
🚷 view threshold		10.00
(+9.40%)		16.27
(61.46%)		📈 view plot
🚷 view threshold		137.00
(-0.41%)		143.02
(95.79%)
serialize_deserialize_authorize📈 view plot
🚷 view threshold		24,404.00
(-1.14%)		25,253.33
(96.64%)		📈 view plot
🚷 view threshold		9,838.00
(-0.54%)		10,038.64
(98.00%)		📈 view plot
🚷 view threshold		13,889.00
(-0.55%)		14,188.85
(97.89%)		📈 view plot
🚷 view threshold		31.00
(-10.55%)		43.85
(70.69%)		📈 view plot
🚷 view threshold		296.00
(-1.78%)		315.49
(93.82%)
serialize_deserialize_handle_authorize📈 view plot
🚷 view threshold		29,976.00
(-1.13%)		30,772.05
(97.41%)		📈 view plot
🚷 view threshold		12,015.00
(-0.56%)		12,209.40
(98.41%)		📈 view plot
🚷 view threshold		17,011.00
(-0.58%)		17,297.47
(98.34%)		📈 view plot
🚷 view threshold		52.00
(-6.40%)		67.61
(76.91%)		📈 view plot
🚷 view threshold		363.00
(-1.76%)		381.76
(95.09%)
serialize_deserialize_handle_submit📈 view plot
🚷 view threshold		126,220.00
(-0.20%)		126,967.05
(99.41%)		📈 view plot
🚷 view threshold		73,199.00
(-0.12%)		73,538.67
(99.54%)		📈 view plot
🚷 view threshold		104,920.00
(-0.14%)		105,467.54
(99.48%)		📈 view plot
🚷 view threshold		102.00
(-4.70%)		133.32
(76.51%)		📈 view plot
🚷 view threshold		594.00
(-0.39%)		605.25
(98.14%)
serialize_deserialize_handle_subscribe📈 view plot
🚷 view threshold		27,626.00
(-0.74%)		28,625.49
(96.51%)		📈 view plot
🚷 view threshold		9,584.00
(-0.67%)		9,765.34
(98.14%)		📈 view plot
🚷 view threshold		13,541.00
(-0.78%)		13,820.11
(97.98%)		📈 view plot
🚷 view threshold		66.00
(+4.03%)		75.89
(86.97%)		📈 view plot
🚷 view threshold		393.00
(-0.81%)		417.05
(94.23%)
serialize_deserialize_submit📈 view plot
🚷 view threshold		115,172.00
(-0.06%)		115,844.10
(99.42%)		📈 view plot
🚷 view threshold		68,059.00
(-0.07%)		68,403.91
(99.50%)		📈 view plot
🚷 view threshold		97,657.00
(-0.08%)		98,226.64
(99.42%)		📈 view plot
🚷 view threshold		59.00
(-5.88%)		78.24
(75.41%)		📈 view plot
🚷 view threshold		492.00
(+0.17%)		498.89
(98.62%)
serialize_deserialize_subscribe📈 view plot
🚷 view threshold		23,080.00
(-0.74%)		24,027.43
(96.06%)		📈 view plot
🚷 view threshold		8,143.00
(-0.75%)		8,321.07
(97.86%)		📈 view plot
🚷 view threshold		11,460.00
(-0.84%)		11,725.94
(97.73%)		📈 view plot
🚷 view threshold		35.00
(-5.90%)		44.22
(79.15%)		📈 view plot
🚷 view threshold		327.00
(-0.56%)		348.40
(93.86%)
serialize_submit📈 view plot
🚷 view threshold		99,833.00
(-0.01%)		100,323.86
(99.51%)		📈 view plot
🚷 view threshold		61,407.00
(-0.16%)		61,721.60
(99.49%)		📈 view plot
🚷 view threshold		88,078.00
(-0.19%)		88,591.02
(99.42%)		📈 view plot
🚷 view threshold		48.00
(+2.49%)		64.68
(74.21%)		📈 view plot
🚷 view threshold		329.00
(+1.33%)		333.90
(98.53%)
serialize_subscribe📈 view plot
🚷 view threshold		11,347.00
(-0.48%)		11,710.12
(96.90%)		📈 view plot
🚷 view threshold		4,118.00
(-1.54%)		4,293.70
(95.91%)		📈 view plot
🚷 view threshold		5,707.00
(-1.93%)		5,984.07
(95.37%)		📈 view plot
🚷 view threshold		15.00
(+16.65%)		20.71
(72.44%)		📈 view plot
🚷 view threshold		159.00
(+0.85%)		163.59
(97.19%)
🐰 View full continuous benchmarking report in Bencher

@plebhash
Copy link
Collaborator

from description of #1253:

  1. Introduce a function to check if a message was exchanged regardless of the order(this is good because sometimes in a SubmitShare scenario, upstream next message can be SubmitShare[STATUS] or it can be something totally different, while it is handling the received share

I don't see this being implemented here.

Is that the reason why this PR says it only partially solves #1253?

will that be done in a follow up PR?

other than that, LGTM

@jbesraa
Copy link
Contributor Author

jbesraa commented Nov 20, 2024

from description of #1253:

  1. Introduce a function to check if a message was exchanged regardless of the order(this is good because sometimes in a SubmitShare scenario, upstream next message can be SubmitShare[STATUS] or it can be something totally different, while it is handling the received share

I don't see this being implemented here.

Is that the reason why this PR says it only partially solves #1253?

will that be done in a follow up PR?

other than that, LGTM

Indeed, will be pushed in a different pr

@plebhash plebhash added the ready-to-be-merged triggers auto rebase bot label Nov 20, 2024
@jbesraa
Add identifier property to Sniffer
9c94856 
..This should make it easier to debug multiple sniffers
by printing the sniffer identifier if a problem occurs during the `drop`
function.
@jbesraa
Add check_on_drop property to Sniffer
27eec57 
..If this flag is set to true, the `Sniffer` **will**
validate that all the downstrea and upstream messages were handled
before dropping, otherwise they are ignored.
@pavlenex pavlenex force-pushed the 2024-11-18-sniffer-improvements branch from e4abf67 to 27eec57 Compare November 20, 2024 14:37
@plebhash plebhash merged commit 40c93c8 into stratum-mining:main Nov 20, 2024
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@plebhash plebhash plebhash approved these changes

Assignees
No one assigned
Labels
ready-to-be-merged triggers auto rebase bot
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Sniffer improvements
2 participants
@jbesraa @plebhash