Added support for loading FPGA for ecosystem version 2.0 and higher.

[b-swat]

Hi.
Added checking the operating system version. If it is version 2.0, then the FPGA is booted via the new boot mechanism (xdevcfg is deprecated and no longer used in Vivado versions newer than 2020).
We also moved the FPGA source code to the RedPitaya repository, otherwise we would have to add a lot of code to your repository in order to be compatible with the current architecture.
FPGA is now pre-built and will be included in ecosystem updates.

[b-swat]

The version with pre-assembled FPGA starts from 316 and higher:
https://downloads.redpitaya.com/downloads/Unify/nightly_builds/
FPGA is not yet available in the release.

[peteasa]

This commit is an interesting one. In my opinion one of the strengths of the pyrpl code is that is allows custom fpga's to be installed easily. Whilst this change definitely works for version 2 fpga installation from the rp-xxxxxx:/opt/redpitaya/fpga/z10_125/pyrpl folder on the device. However this change restricts the use of the pyrpl code to just using the fpga from the eco system.

In my code I copy of the overlay.sh script (called update_fpga.sh) and with that additional script I install the custom fpga from the /opt/pyrpl folder. The custom fpga is installed from python by using the additional init paramters reloadfpga and filename for example:
p = pyrpl.Pyrpl(config = CONFIG, hostname = HOSTNAME, gui = False, reloadfpga = True, filename = 'pyrpl/fpga/red_pitaya.bin')
Associated changes are required to redpitaya.py to make this work. In particular with version 2 fpga I would suggest that if a filename is not provided the flow in this pull request is used and the "default" fpga from rp-xxxxxx:/opt/redpitaya/fpga/z10_125/pyrpl is installed with the normal /opt/redpitaya/sbin/overlay.sh script.

What do you think about this alternative?

[peteasa]

See peteasa@84497a6 for a possible solution to this. the update_fpga.sh file will ultimately contain the necessary shell commands to install the custom pyrpl fpga and the associated device tree. to date I have tested this with the installation of the fpga but not the installation of the update to the device tree.

[b-swat]

See peteasa@84497a6 for a possible solution to this. the update_fpga.sh file will ultimately contain the necessary shell commands to install the custom pyrpl fpga and the associated device tree. to date I have tested this with the installation of the fpga but not the installation of the update to the device tree.

Hi.
I added this pull request in hopes of supporting this repository, but the owner said that he will not support it anymore at all, since he has neither the time nor the desire.

I moved the repository that will be used further with the 2.0 ecosystem to the RedPitaya account
https://github.com/RedPitaya/pyrpl.

I think you know about this already.

Regarding the use of FPGA from users.

1. Device tree compatibility issue. We have added pyrpl FPGA to our working repository as a sub-project. This will eliminate the conflict between devices created by the pyrpl project and the root project for uboot. In the old version of the ecosystem, there was no overlay support and the device tree was always fixed.
   With the new release, these source codes will be available.
2. In the future, users will be able to modify the FPGA source code themselves, since it will be open.
3. Additional bypasses can be added to pyrpl scripts, but now there are other tasks. This will all be left to the users.

[peteasa]

Great to hear from you guys. From your comment I would suggest that the correct commit would be to remove the fpga code from the pyrpl project completely as well as to simplify the redpitaya.py file and the pyrpl so that the option of uploading from the pyrpl project was removed.

So in conclusion I see that there are a few options on the table.
Option A is to remove the fpga code from pyrpl project - thus avoiding confusion and removing complexity of the install process.
Option B is to have two projects with the pyrpl code and that adds to confusion

Perhaps there is a third option where the fpga code from the redpitaya project is included as a sub module of the pyrpl project.

I would be interested to hear any comments on these three options. I suggest that we try to think about this before December and plan to make the appropriate change before January! What do you think @michaelcroquette & @SamuelDeleglise??

[michaelcroquette]

Hello,

Sorry I wanted to test this modification and I didn't have the time yet as this requires a bit more of work.

Hi. I added this pull request in hopes of supporting this repository, but the owner said that he will not support it anymore at all, since he has neither the time nor the desire.

I guess you speak to Leo, I've talked with him and I have now admin rights on the repo and I'm still using Pyrpl so I'm willing to try to maintain it and improve things if I have more time.

I moved the repository that will be used further with the 2.0 ecosystem to the RedPitaya account https://github.com/RedPitaya/pyrpl.

I think you know about this already.

Regarding the use of FPGA from users.

1. Device tree compatibility issue. We have added pyrpl FPGA to our working repository as a sub-project. This will eliminate the conflict between devices created by the pyrpl project and the root project for uboot. In the old version of the ecosystem, there was no overlay support and the device tree was always fixed.
   With the new release, these source codes will be available.
2. In the future, users will be able to modify the FPGA source code themselves, since it will be open.
3. Additional bypasses can be added to pyrpl scripts, but now there are other tasks. This will all be left to the users.

I actually never use the reload FPGA option as I would recompile the FPGA code so I'm not sure which option is the best.

[peteasa]

@michaelcroquette .. I have been discussing this change with as many people that I can find who are active users of custom FPGA code.

One important consideration is that it should be easy to see if the python code and the FPGA code are aligned. This is obvious in the old pyrpl repository because the .v files and the .py files are all in the same repository.

Now and in the future they will not be in the same repository, so it seems sensible to include a "version" number in the FPGA code itself so that any random eco system version can be used to identify on the gui or via the api the version of python code that should be used.

It will be important to be able to clearly see the version information because people frequently swap between fpga versions and custom fpga to check that the changes they are making does not break anything / or to help debug the custom changes.

Note that something similar is already part of the eco system version where a file $FPGAS/$MODEL/$1/git_info.txt stored on the redpitaya run time in /opt/redpitaya has the git commit information about the fpga .bin file. This is hidden from "normal" users so exposing a version number coded into the fpga.bin file would make this a bit more visible.

By the way I am now a custom pyrpl fpga user as well... I have built both the pyrpl version and the RedPitaya version with vivado version 2023.2, a version that I use for other projects. So I can test out my ideas.

[b-swat]

Hi
The version of the FPGA in the ecosystem is displayed in the system information.

[peteasa]

@b-swat good suggestion. This is exactly the type of information that I think should be visible in the pyrpl gui that you see when, on the host, you launch

python3 -m pyrpl example_filename

At the moment there is no such display on the pyrpl Qt gui.

The pyrpl Qt gui could display information about the running fpga. The only difference between your suggestion and mine is that in my case if the fpga itself could have a version number that could be accessed from the pyrpl api then the version of the fpga would be displayed on the pyrpl gui. A custom fpga could be programmed with a tag that the developer recognises and so if they switch from custom fpga to eco system fpga they would easily be able to see which pyrpl fpga they are working with.

With the pyrpl code you may not know but the information about the actual fpga that is being used is rather hidden from the user. At the moment you have to switch on debug logging in the pyrpl init() function to display what is going on when the fpga gets downloaded to the redpitaya board and programmed into the fpga. Or you have to disable the automatic install of the fpga and instead first program the fpga then launch the pyrpl application. I know that some people prefer not to use the reloadfpga option. I my self prefer to use the api

p = Pyrpl(hostname=HOSTNAME,
    gui = False,
    reloadfpga = True, filename = 'mycustom.bit.bin'
    )

[kakitkelvinho]

I was redirected here by @michaelcroquette as I wanted to run pyrpl with the newest python and a red pitaya with the newest OS. For this, I have installed conda and have had to setup the environment with pyqt instead of pyqt4 or pyqt5 as conda did not recognize those, and after that I ran the setup.py develop command, and then when I finally ran python -m pyrpl testconfig I started having errors like:

Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use QT_QPA_PLATFORM=wayland to run on Wayland anyway.
Traceback (most recent call last):
  File "<frozen runpy>", line 189, in _run_module_as_main
  File "<frozen runpy>", line 148, in _get_module_details
  File "<frozen runpy>", line 112, in _get_module_details
  File "/home/ninux/Documents/redpitaya_pyrpl/pyrpl/__init__.py", line 59, in <module>
    global_config = MemoryTree('global_config', source='global_config')
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ninux/Documents/redpitaya_pyrpl/pyrpl/memory.py", line 507, in __init__
    self._savetimer.setInterval(self._loadsavedeadtime*1000)
TypeError: setInterval(self, msec: int): argument 1 has unexpected type 'float'

After fixing this by type casting to int in the memory.py file, I got

Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use QT_QPA_PLATFORM=wayland to run on Wayland anyway.
Traceback (most recent call last):
  File "<frozen runpy>", line 189, in _run_module_as_main
  File "<frozen runpy>", line 148, in _get_module_details
  File "<frozen runpy>", line 112, in _get_module_details
  File "/home/ninux/Documents/redpitaya_pyrpl/pyrpl/__init__.py", line 66, in <module>
    from .redpitaya import RedPitaya
  File "/home/ninux/Documents/redpitaya_pyrpl/pyrpl/redpitaya.py", line 19, in <module>
    from . import redpitaya_client
  File "/home/ninux/Documents/redpitaya_pyrpl/pyrpl/redpitaya_client.py", line 29, in <module>
    from .hardware_modules.dsp import dsp_addr_base, DSP_INPUTS
  File "/home/ninux/Documents/redpitaya_pyrpl/pyrpl/hardware_modules/__init__.py", line 23, in <module>
    from .dsp import DspModule, DSP_INPUTS, all_inputs, all_output_directs, \
  File "/home/ninux/Documents/redpitaya_pyrpl/pyrpl/hardware_modules/dsp.py", line 2, in <module>
    from ..attributes import BoolRegister, SelectProperty, SelectProperty, SelectRegister, IntRegister, FloatRegister, BaseRegister, BoolProperty
  File "/home/ninux/Documents/redpitaya_pyrpl/pyrpl/attributes.py", line 20, in <module>
    from .widgets.attribute_widgets import BoolAttributeWidget, \
  File "/home/ninux/Documents/redpitaya_pyrpl/pyrpl/widgets/attribute_widgets.py", line 681, in <module>
    class DataWidget(pg.GraphicsWindow):
                     ^^^^^^^^^^^^^^^^^
AttributeError: module 'pyqtgraph' has no attribute 'GraphicsWindow'. Did you mean: 'GraphicsView'?

I have also went into this PR / followingI have looked into https://github.com/peteasa/pyrpl_testbench/tree/main but I am unsure what to do after following the installation instructions.

[peteasa]

should all be straight forward. At the moment as you have gathered the python3 support is not fully integrated into https://github.com/pyrpl-fpga/pyrpl. This is ongoing work. However if you need to use python3 now I suggest you pick up the latest from my branch from my clone of this repository. Some of what I have will in the end get merged into this repository. So the steps are basically get / clone https://github.com/peteasa/pyrpl/tree/consolidate_python3 and you are good to go with the python3 pyrpl code. I did create a second repository with some test code that I use to explore the pyrpl features... this is the second repository that you mention https://github.com/peteasa/pyrpl_testbench that is not necessary to use pyrpl.

In my environment I simply import the pyrpl code from the local directory with no need for complex setup etc just a lot of apt installs. What I mean by that is clone https://github.com/peteasa/pyrpl/tree/consolidate_python3. cd pyrpl and run python3 -m pyrpl example_filename.. in that folder you will find the main pyrpl folder that is effectively just being imported in the python3 environment so it just works. So the pyrpl init.py file is in pyrpl/pyrpl/init.py.. you need to be in the directory above the init.py file for it all to work. the command python3 -m pyrpl example_filename picks up the module ie pyrpl/init.py and passes the parameter example_filename to the pyrpl module and up comes the gui provided you have the correct versions of everything.

I use RedPitaya OS 2.0: Latest Beta (2.05-37) Install 2.00-37 - Branch 2024.3 Python is 3.12.3 pyrpl 1.0.0.0 using IPython 8.20.0 numpy 1.26.4 pandas 2.1.4 paramiko 2.12.0 scipy 1.11.4 scp 0.14.5 ruamel.yaml 0.17.21 yaml 6.0.1 qtpy 2.4.1

I suggest that if you have any problems with my consolidate_python3 branch you discuss with me via https://github.com/peteasa/pyrpl_testbench/discussions. From your message above I suspect you are not using my https://github.com/peteasa/pyrpl/tree/consolidate_python3 branch.