Subxtpy Milestone 1

[philoniare]

Milestone Delivery Checklist

• The milestone-delivery-template.md has been copied and updated.
• This pull request is being made by the same account as the accepted application.
• I have disclosed any and all sources of reused code in the submitted repositories and have done my due diligence to meet its license requirements.
• In case of acceptance, invoices must be submitted and payments will be transferred to the Polkadot AssetHub and/or fiat account provided in the application.
• The delivery is according to the Guidelines for Milestone Deliverables.

Link to the application pull request: Proposal.

[keeganquigley]

Hi @philoniare thanks for the delivery, I'm evaluating it now.

When I try to run pipx install maturin I get the following error: Not installing to existing directory '/home/ubuntu/.local/pipx/venvs/maturin'. Pass '--force' to force installation.

When I use --force the installation works, but then maturin develop fails:

maturin failed
  Caused by: Couldn't find a virtualenv or conda environment, but you need one to use this command. For maturin to find your virtualenv you need to either set VIRTUAL_ENV (through activate), set CONDA_PREFIX (through conda activate) or have a virtualenv called .venv in the current or any parent folder. See https://virtualenv.pypa.io/en/latest/index.html on how to use virtualenv or use `maturin build` and `pip install <path/to/wheel>` instead.

Do I need to run it from a different path? Or otherwise how can I fix this? Thanks for any help you can provide.

Cheers and have a great weekend.

[keeganquigley]

pinging @philoniare

[philoniare]

@keeganquigley Sorry for the late response, been a bit preoccupied with JAM dev. Thanks for taking the time to review the milestone, which OS are you testing it out on? Actually, it would probably be easier to release it a python package that it can be installed directly without external dependencies. Let me go ahead and do that so that it's easier for you to test it out. Will update the test instructions and let you know soon

[philoniare]

@keeganquigley Here's the published pypi package, you should now be able to install it by simply running:

pip install subxtpy

Then, either run a local node or specify the node url to connect to and then interact with the node runtime similar to the example provided in the README.

Will be sure to keep an eye out on this PR. Thanks for your time

[keeganquigley]

Hi @philoniare apologies for the late reply, I was out of office. So I'm testing this on macOS and have a local node running, but when running pytest from subxtpy dir I got the following:

tests/test_constants_dynamic.py::test_fetch_constant
tests/test_events.py::test_fetch_events
tests/test_runtime_api_call.py::test_fetch_account_nonce
tests/test_storage_fetch_dynamic.py::test_subxt_client_creation
tests/test_storage_fetch_dynamic.py::test_fetch_free_balance
tests/test_storage_iterating_dynamic.py::test_fetch_storage_entries
tests/test_tx_basic.py::test_transfer_balance
  /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/_pytest/python.py:148: PytestUnhandledCoroutineWarning: async def functions are not natively supported and have been skipped.
  You need to install a suitable plugin for your async framework, for example:
    - anyio
    - pytest-asyncio
    - pytest-tornasync
    - pytest-trio
    - pytest-twisted
    warnings.warn(PytestUnhandledCoroutineWarning(msg.format(nodeid)))

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

The tests were all skipped because of this missing plugin. I successfully installed pytest-asyncio but then it gives me this:

Output

/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pytest_asyncio/plugin.py:208: PytestDeprecationWarning: The configuration option "asyncio_default_fixture_loop_scope" is unset.
The event loop scope for asynchronous fixtures will default to the fixture caching scope. Future versions of pytest-asyncio will default the loop scope for asynchronous fixtures to function scope. Set the default fixture loop scope explicitly in order to avoid unexpected behavior in the future. Valid fixture loop scopes are: "function", "class", "module", "package", "session"

  warnings.warn(PytestDeprecationWarning(_DEFAULT_FIXTURE_LOOP_SCOPE_UNSET))
================================================== test session starts ===================================================
platform darwin -- Python 3.11.1, pytest-8.3.2, pluggy-1.5.0
rootdir: /Users/keeganquigley/subxtpy
configfile: pyproject.toml
plugins: asyncio-0.24.0
asyncio: mode=Mode.STRICT, default_loop_scope=None
collected 7 items

tests/test_constants_dynamic.py F                                                                                  [ 14%]
tests/test_events.py F                                                                                             [ 28%]
tests/test_runtime_api_call.py F                                                                                   [ 42%]
tests/test_storage_fetch_dynamic.py FF                                                                             [ 71%]
tests/test_storage_iterating_dynamic.py F                                                                          [ 85%]
tests/test_tx_basic.py F                                                                                           [100%]

======================================================== FAILURES ========================================================
__________________________________________________ test_fetch_constant ___________________________________________________

    @pytest.mark.asyncio
    async def test_fetch_constant():
>       client = await SubxtClient.new()
E       RuntimeError: Rpc error: RPC error: Error when opening the TCP socket: Connection refused (os error 61)

tests/test_constants_dynamic.py:7: RuntimeError
___________________________________________________ test_fetch_events ____________________________________________________

    @pytest.mark.asyncio
    async def test_fetch_events():
>       client = await SubxtClient.new()
E       RuntimeError: Rpc error: RPC error: Error when opening the TCP socket: Connection refused (os error 61)

tests/test_events.py:7: RuntimeError
________________________________________________ test_fetch_account_nonce ________________________________________________

    @pytest.mark.asyncio
    async def test_fetch_account_nonce():
>       client = await SubxtClient.new()
E       RuntimeError: Rpc error: RPC error: Error when opening the TCP socket: Connection refused (os error 61)

tests/test_runtime_api_call.py:7: RuntimeError
_______________________________________________ test_subxt_client_creation _______________________________________________

    @pytest.mark.asyncio
    async def test_subxt_client_creation():
>       client = await SubxtClient.new()
E       RuntimeError: Rpc error: RPC error: Error when opening the TCP socket: Connection refused (os error 61)

tests/test_storage_fetch_dynamic.py:7: RuntimeError
________________________________________________ test_fetch_free_balance _________________________________________________

    @pytest.mark.asyncio
    async def test_fetch_free_balance():
>       client = await SubxtClient.new()
E       RuntimeError: Rpc error: RPC error: Error when opening the TCP socket: Connection refused (os error 61)

tests/test_storage_fetch_dynamic.py:12: RuntimeError
_______________________________________________ test_fetch_storage_entries _______________________________________________

    @pytest.mark.asyncio
    async def test_fetch_storage_entries():
>       client = await SubxtClient.new()
E       RuntimeError: Rpc error: RPC error: Error when opening the TCP socket: Connection refused (os error 61)

tests/test_storage_iterating_dynamic.py:7: RuntimeError
_________________________________________________ test_transfer_balance __________________________________________________

    @pytest.mark.asyncio
    async def test_transfer_balance():
        # Test balance transfer from Alice to Bob
>       client = await SubxtClient.new()
E       RuntimeError: Rpc error: RPC error: Error when opening the TCP socket: Connection refused (os error 61)

tests/test_tx_basic.py:7: RuntimeError
================================================ short test summary info =================================================
FAILED tests/test_constants_dynamic.py::test_fetch_constant - RuntimeError: Rpc error: RPC error: Error when opening the TCP socket: Connection refused (os error 61)
FAILED tests/test_events.py::test_fetch_events - RuntimeError: Rpc error: RPC error: Error when opening the TCP socket: Connection refused (os error 61)
FAILED tests/test_runtime_api_call.py::test_fetch_account_nonce - RuntimeError: Rpc error: RPC error: Error when opening the TCP socket: Connection refused (os error 61)
FAILED tests/test_storage_fetch_dynamic.py::test_subxt_client_creation - RuntimeError: Rpc error: RPC error: Error when opening the TCP socket: Connection refused (os error 61)
FAILED tests/test_storage_fetch_dynamic.py::test_fetch_free_balance - RuntimeError: Rpc error: RPC error: Error when opening the TCP socket: Connection refused (os error 61)
FAILED tests/test_storage_iterating_dynamic.py::test_fetch_storage_entries - RuntimeError: Rpc error: RPC error: Error when opening the TCP socket: Connection refused (os error 61)
FAILED tests/test_tx_basic.py::test_transfer_balance - RuntimeError: Rpc error: RPC error: Error when opening the TCP socket: Connection refused (os error 61)

Am I supposed to have to install the additional plugins? I wonder if this is happening because I'm running the Docker version of the polkadot-sdk-minimal-template.

Speaking of which, the substrate-node-template has now been deprecated in favor of these three template forks, which includes the minimal build linked above. Could you please update the instructions to work with one of these templates instead of the one you have linked (which now gives a 404 error). Thanks!

[philoniare]

Ah I see. Yep, will let you know after I update it.

[philoniare]

@keeganquigley I've pushed some updates to the instructions after trying out the docker. Looks like by default, the docker doesn't expose the ws, so you'll need to run the minimal template image with the following command:

docker run -p 9944:9944 --rm polkadot-sdk-minimal-template --dev --rpc-external

I've also created a requirements.txt file so that you can easily install the dependencies with:

pip install -r requirements.txt

[keeganquigley]

Thanks for the fixes @philoniare much appreciated. Everything is working now and all unit tests are passing. Happy to pass the milestone, and looking forward to seeing it all come together in the next milestone. Here is my final evaluation.

[github-actions]

🪙 Please fill out the invoice form in order to initiate the payment process. Thank you!

[philoniare]

@keeganquigley tysm for your time and support! I believe the library will be most useful by the time next milestone is completed once we have python decorators generate the dynamic runtime callable methods based on node metadata.

[RouvenP]

hi @philoniare we just sent the payments