Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

provider fails to broadcast a bid for orders with mixed count #149

Closed
andy108369 opened this issue Nov 20, 2023 · 4 comments
Closed

provider fails to broadcast a bid for orders with mixed count #149

andy108369 opened this issue Nov 20, 2023 · 4 comments
Assignees
@troian
Labels
P0 repo/node Akash node repo issues

Comments

@andy108369
Copy link
Contributor

andy108369 commented Nov 20, 2023 •
edited
Loading

The provider encounters a capabilities mismatch error when attempting to broadcast a bid for an order request where the count for one service is higher than the count for another service.

Scenarios

"no bid" here means provider received capabilities mismatch error

scenario 1 - bids (2 in total) 🟢

service foo, count 1
service bar, count 1

scenario 2 - no bid (3 in total) 💥

service foo, count 1
service bar, count 2

scenario 3 - bids (4 in total) 🟢

service foo, count 2
service bar, count 2

scenario 4 - no bid (5 in total) 💥

service foo, count 2
service bar, count 3

scenario 5 - bids (6 in total) 🟢

service foo, count 3
service bar, count 3

scenario 6 - bids (8 in total) 🟢

service foo, count 4
service bar, count 4

scenario 7 - no bids (9 in total) 💥

service foo, count 5
service bar, count 4

scenario 8 - bids (10 in total) 🟢

service foo, count 5
service bar, count 5

scenario 9 - no bids (4 in total, but 3 services now) 💥

service foo, count 1
service bar, count 2
service baz, count 1

scenario 10 - no bids (5 in total, 3 services) 💥

service foo, count 1
service bar, count 2
service baz, count 2

scenario 11 - no bids (6 in total, 3 services) 🟢

service foo, count 2
service bar, count 2
service baz, count 2

scenario 12 - bids (3 in total, 3 services) 🟢

service foo, count 1
service bar, count 1
service baz, count 1

scenario 13 - bids (4 in total, 4 services) 🟢

service foo, count 1
service bar, count 1
service baz, count 1
service qux, count 1

scenario 14 - no bids (5 in total, 4 services) 💥

service foo, count 2
service bar, count 1
service baz, count 1
service qux, count 1

scenario 15 - no bids (6 in total, 4 services) 💥

service foo, count 2
service bar, count 2
service baz, count 1
service qux, count 1

scenario 16 - no bids (7 in total, 4 services) 💥

service foo, count 2
service bar, count 2
service baz, count 2
service qux, count 1

scenario 17 - bids (8 in total, 4 services) 🟢

service foo, count 2
service bar, count 2
service baz, count 2
service qux, count 2

SW

provider & client both 0.4.8
akash node 0.28.2

provider log: capabilities mismatch

service foo, count 1
service bar, count 2

I[2023-11-20|18:03:25.755] order detected                               module=bidengine-service cmp=provider order=order/akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
I[2023-11-20|18:03:25.757] group fetched                                module=bidengine-order cmp=provider order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
I[2023-11-20|18:03:25.758] requesting reservation                       module=bidengine-order cmp=provider order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
D[2023-11-20|18:03:25.758] reservation requested                        module=provider-cluster cmp=provider cmp=service cmp=inventory-service order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1 resources[{resource:{id:1,cpu:{units:{val:1000}},memory:{size:{val:1073741824}},storage:[{name:default,size:{val:1073741824}}],gpu:{units:{val:0}},endpoints:[{kind:1,sequence_number:0}]},count:1,price:{denom:uakt,amount:1000000.000000000000000000}},{resource:{id:2,cpu:{units:{val:1000}},memory:{size:{val:1073741824}},storage:[{name:default,size:{val:1073741824}}],gpu:{units:{val:0}},endpoints:[{kind:1,sequence_number:0}]},count:2,price:{denom:uakt,amount:1000000.000000000000000000}}]=(MISSING)
D[2023-11-20|18:03:25.758] reservation count                            module=provider-cluster cmp=provider cmp=service cmp=inventory-service cnt=1
I[2023-11-20|18:03:25.758] Reservation fulfilled                        module=bidengine-order cmp=provider order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
D[2023-11-20|18:03:26.418] submitting fulfillment                       module=bidengine-order cmp=provider order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1 price=9.617504000000000000uakt
E[2023-11-20|18:03:26.422] transaction broadcast failed                 cmp=provider cmp=client/broadcaster err="rpc error: code = Unknown desc = rpc error: code = Unknown desc = failed to execute message; message index: 0: capabilities mismatch [cosmos/[email protected]/baseapp/baseapp.go:781] With gas wanted: '0' and gas used: '62826' : unknown request"
E[2023-11-20|18:03:26.422] unable to broadcast messages                 cmp=provider cmp=client/broadcaster error="rpc error: code = Unknown desc = rpc error: code = Unknown desc = failed to execute message; message index: 0: capabilities mismatch [cosmos/[email protected]/baseapp/baseapp.go:781] With gas wanted: '0' and gas used: '62826' : unknown request"
E[2023-11-20|18:03:26.422] bid failed                                   module=bidengine-order cmp=provider order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1 err="rpc error: code = Unknown desc = rpc error: code = Unknown desc = failed to execute message; message index: 0: capabilities mismatch [cosmos/[email protected]/baseapp/baseapp.go:781] With gas wanted: '0' and gas used: '62826' : unknown request"
I[2023-11-20|18:03:26.422] shutting down                                module=bidengine-order cmp=provider order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
D[2023-11-20|18:03:26.422] unreserving reservation                      module=bidengine-order cmp=provider order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
D[2023-11-20|18:03:26.422] unreserving capacity                         module=provider-cluster cmp=provider cmp=service cmp=inventory-service order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
I[2023-11-20|18:03:26.422] attempting to removing reservation           module=provider-cluster cmp=provider cmp=service cmp=inventory-service order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
I[2023-11-20|18:03:26.422] removing reservation                         module=provider-cluster cmp=provider cmp=service cmp=inventory-service order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
I[2023-11-20|18:03:26.422] unreserve capacity complete                  module=provider-cluster cmp=provider cmp=service cmp=inventory-service order=akash1h24fljt7p0nh82cq0za0uhsct3sfwsfu9w3c9h/13768245/1/1
D[2023-11-20|18:03:26.422] reservation count                            module=provider-cluster cmp=provider cmp=service cmp=inventory-service cnt=0

SDL

version: '2.0'

services:
  foo:
    image: ubuntu:22.04
    expose:
      - port: 80
        as: 80
        to: 
          - global: true

  bar:
    image: ubuntu:22.04
    expose:
      - port: 80
        as: 80
        to:
          - global: true

profiles:
  compute:
    foo:
      resources:
        cpu:
          units: 1
        memory:
          size: 1Gi
        storage:
          size: 1Gi

    bar:
      resources:
        cpu:
          units: 1
        memory:
          size: 1Gi
        storage:
          size: 1Gi


  placement:
    westcoast:
      pricing:
        foo:
          denom: uakt
          amount: 1000000
        bar:
          denom: uakt
          amount: 1000000
          
deployment:
  foo:
    westcoast:
      profile: foo
      count: 1
  bar:
    westcoast:
      profile: bar
      count: 2
@andy108369 andy108369 self-assigned this Nov 20, 2023
@troian
Copy link
Member

troian commented Nov 20, 2023

are those deployments created via cloudmos or cli?

@andy108369
Copy link
Contributor Author

are those deployments created via cloudmos or cli?

CLI - provider-services 0.4.8 (binary from the GH releases page)

@andy108369 andy108369 removed their assignment Nov 21, 2023
@andy108369 andy108369 added repo/node Akash node repo issues repo/provider Akash provider-services repo issues awaiting-triage P1 and removed repo/provider Akash provider-services repo issues labels Nov 21, 2023
@github-project-automation github-project-automation bot moved this to Backlog (not prioritized) in Core Product and Engineering Roadmap Nov 21, 2023
@brewsterdrinkwater brewsterdrinkwater moved this from Backlog (not prioritized) to In Progress (prioritized) in Core Product and Engineering Roadmap Nov 21, 2023
@brewsterdrinkwater
Copy link
Collaborator

brewsterdrinkwater commented Nov 21, 2023 •
edited
Loading

November 21st, 2023:

This requires a network upgrade to Mainnet 9. Upgrade only affects node.

Proposal can be found here: https://www.mintscan.io/akash/proposals/238

@andy108369 andy108369 mentioned this issue Nov 22, 2023
Closed
@brewsterdrinkwater brewsterdrinkwater mentioned this issue Nov 28, 2023
Closed
@brewsterdrinkwater
Copy link
Collaborator

Post Akash Mainnet 9, this has been thoroughly tested, and has been solved.

@brewsterdrinkwater brewsterdrinkwater moved this from In Progress (prioritized) to Released (in Prod) in Core Product and Engineering Roadmap Nov 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@troian troian

Labels
P0 repo/node Akash node repo issues
Projects
Core Product and Engineering Roadmap
Status: Released (in Prod)
Milestone
No milestone
Development

No branches or pull requests
3 participants
@troian @brewsterdrinkwater @andy108369