Development Ideas

[Ricks-Lab]

I have opened this thread to facilitate any discussions on development ideas and code refinements. Potential topics:

• Recommendations for more Pythonic approach
• Gtk improvements for a nicer gui
• Better way to write to driver files as root
• Plotting capability approach
• Code criticism

[csecht]

When 'Save' is clicked in the PAC Gtk window and the sudo password is needed in the terminal, perhaps have the terminal window automatically come to the front, or perhaps have a notice pop up in the PAC window to prompt for an action in the terminal. As it is now, when the terminal window is behind the PAC window, it is not obvious for new users that terminal is waiting for sudo input.

[Ricks-Lab]

It's a good idea, though I will have to figure out how to not prompt the user after the credentials are entered the first time and still valid. Perhaps I can look at the delay to finish. It it is longer than a few seconds, the tell the user to check the window. I will also need to figure out how to construct a gtk dialog box...

[Ricks-Lab]

@dormiy @zhenyu-repo Any ideas?

I'm struggling with code to open a dialog before executing the batch file and then closing the dialog when complete. Below is code I was using to run the dialog in another thread. In the main thread, I use a dialogWin.close() call to close the dialog. It almost works. It displays the message, runs the bash file but gets error when running dialogWin.close() or hide or destroy. Here are error details:

[xcb] Unknown sequence number while processing queue
[xcb] Most likely this is a multi-threaded client and XInitThreads has not been called
[xcb] Aborting, sorry about that.
python3: ../../src/xcb_io.c:259: poll_for_event: Assertion `!xcb_xlib_threads_sequence_lost' failed.
[xcb] Unknown sequence number while processing queue
[xcb] Most likely this is a multi-threaded client and XInitThreads has not been called
[xcb] Aborting, sorry about that.
python3: ../../src/xcb_io.c:259: poll_for_event: Assertion `!xcb_xlib_threads_sequence_lost' failed.
Aborted (core dumped)

Here is the code:

def run_dialog(dialogWindow):
    dialogWindow.run()
    return


class PACDialog(Gtk.Dialog):
    def __init__(self):
        Gtk.Window.__init__(self, title="Execute PAC")
        message = "Executing bash script.  You may need to enter credentials in original window"
        dialogWindow = Gtk.MessageDialog(self,
                #Gtk.DialogFlags.MODAL | Gtk.DialogFlags.DESTROY_WITH_PARENT,
                Gtk.DialogFlags.DESTROY_WITH_PARENT,
                Gtk.MessageType.INFO,
                Gtk.ButtonsType.NONE,
                message)
        dialogWindow.set_title("Execute PAC")
        dialogWindow.show_all()
        #dialogWindow.run()
        p = multiprocessing.Process(target=run_dialog, args=(dialogWindow,))
        p.start()
        return

[Ricks-Lab]

I found that this code run after batch complete doesn't give any errors, but it looks like the destroy command never makes it to the dialog running in the new thread. The dialog doesn't close and the final join statement is stuck waiting for the process to finish.

   dialogWin.close()
   dialogWin.destroy()
   threadInfo["queue"].close()
   threadInfo["queue"].join_thread()
   threadInfo["p"].join()

[Ricks-Lab]

Maybe this is what I need:
Global Variable or Queue
I will give this a try when I have time.

[zhenyu-repo]

Maybe this is what I need:
Global Variable or Queue
I will give this a try when I have time.

Good luck, I'm new to Gtk, not sure what caused this. I guess you'll get it fixed this weekend 👍

[zhenyu-repo]

One feedback, I notice you have duplicate code blocks in multiple files, suggest moving them in one file/module and encapsulate them as a function. If you have similar design for other features, it would be better to refactor the same way.

    if args.about == True :  
        print(__doc__ )
        print("Author: ", __author__ )
        print("Copyright: ", __copyright__)
        print("Credits: ", __credits__)
        print("License: ", __license__)
        print("Version: ", __version__)
        print("Maintainer: ", __maintainer__)
        print("Status: ", __status__)
        sys.exit(0)

[Ricks-Lab]

@csecht Seems like you gave me quite a challenge with your apparently simple request. I still have not found a solution, but I have cleaned up my test cases into different branches:

• master is now stable version.
• threaded is the threaded message dialog approach - currently segfaults when terminating thread.
• message_box uses a text box at the bottom of the gui - currently doesn't update as I expect.

I may have to let my thoughts brew on this for a while and move on to working on some other things.

[Ricks-Lab]

One feedback, I notice you have duplicate code blocks in multiple files, suggest moving them in one file/module and encapsulate them as a function. If you have similar design for other features, it would be better to refactor the same way.

    if args.about == True :  
        print(__doc__ )
        print("Author: ", __author__ )
        print("Copyright: ", __copyright__)
        print("Credits: ", __credits__)
        print("License: ", __license__)
        print("Version: ", __version__)
        print("Maintainer: ", __maintainer__)
        print("Status: ", __status__)
        sys.exit(0)

Not sure if there is an easy way to do this as each of these metadata items are local to the file where they are defined and could be different in each file, though many could be the same. Not sure if I call a method in another module if it will use metadata as define in the calling module.

[Ricks-Lab]

Reddit saved the day! Solution for the message-box approach found:
PyGTK: How to update/redraw a Widget (GTKLabel)?

I hope I can clean up and release this approach over the weekend.

[Ricks-Lab]

When 'Save' is clicked in the PAC Gtk window and the sudo password is needed in the terminal, perhaps have the terminal window automatically come to the front, or perhaps have a notice pop up in the PAC window to prompt for an action in the terminal. As it is now, when the terminal window is behind the PAC window, it is not obvious for new users that terminal is waiting for sudo input.

@csecht I have completed the implementation of this feature. It is implemented in this branch
message_box Let me know what you think. Thanks for the idea.

[csecht]

@csecht I have completed the implementation of this feature. It is implemented in this branch
message_box Let me know what you think. Thanks for the idea.

Yes, that got my attention with the red highlighted message box at the bottom of the PAC window. Hard to miss! The use of that area for general information and status updates is a good idea too. Cheers.

[Ricks-Lab]

@csecht I am getting prepared to release v2.3.0. In addition to the amdgpu-pac message box, I have implemented checks of card compatibility throughout the reading of card details. Any card that has issues, will be flagged as incompatible and the pac and monitor tools will not use them. amdgpu-ls should still give the details it has for all AMD GPUs. I think this should make the tools much more robust. It would be great if you can give it a try and let me know of any issues. Thanks!

The latest is now on master.

[csecht]

I downloaded v2.3.0 from Master and see that now both -monitor and -pac no longer show the decoded model names in their windows, but instead show the short or display model names, so something broke there with this recent version. The correct preferred model name (Decoded Device ID) is still listed in amdgpu-ls, however (example from -ls);

Decoded Device ID: RX460
Card Model:  Baffin [Radeon RX 460/560D / Pro 450/455/460/555/560] (rev cf)
Short Card Model:  RX 460/560D / Pro 450/455/460/555/560
Display Card Model:  RX 460/560D / Pro 450/455/460/555/560

Also, it would be helpful in README, under the ## amdgpu-pac section, to mention that AUTO/default fan control can be restored by entering a negative integer as the % value. (I've always just used -1).

Everything else seems to work well with my RX 460 / RX 570 host. Looking good!
Cheers.

[csecht]

...BTW, the decoded device IDs were properly displayed in the monitor and pac windows with last version I downloaded from the message_box branch

[Ricks-Lab]

I just found a new problem while testing on a Ubuntu 16.04 install. When Python version is older than 3.6, the code will give a Syntax error before any of my checking gets done, so the program can not inform the user of the Python revision compatibility problem. Interim solution is to create stripped down utility amdgpu-chk using pre3.6 syntax to check, but it would be better if the main utilities could check on their own. Let me know if anyone has ideas on this one.

[zhenyu-repo]

One of the options is: create a shell script as the entry point, move the Python version checking logic to this shell script, if >3.6, then call your original Python script.

[Ricks-Lab]

One of the options is: create a shell script as the entry point, move the Python version checking logic to this shell script, if >3.6, then call your original Python script.

Thanks for the suggestion. I did make a Python2 compliant tool amdgpu-chk, which can do this check even if the user is using python 2, but it would seem kind of cludgy to make this a wrapper for all other scripts. I may just include this new script in the package and reference it in a users guide.

[zhenyu-repo]

If Python2 is not installed, then your amdgpu-chk won’t work, so it would be better to check Python version in shell script. You just need to wrap your main Python script(the entry program), no need to wrap all other scripts, I don’t understand why you need to make it a wrapper for all other scripts.

[Ricks-Lab]

If Python2 is not installed, then your amdgpu-chk won’t work, so it would be better to check Python version in shell script. You just need to wrap your main Python script(the entry program), no need to wrap all other scripts, I don’t understand why you need to make it a wrapper for all other scripts.

I tested the new script in both python 2 and 3 to make sure it works in both. But that is with only the 2.7.12 version of python 2. I made the script as simple as possible, so hopefully that will not be an issue. I haven't pushed this version yet , since the system in now using an old version of git. Don't want to mess up the repository...

[csecht]

With amdgpu-ls and with the --clinfo option, the vBIOS for my RX 570 card running its preloaded 'mining' bios is listed as
vBIOS Version: 113-57045EHD1-M90
From the TechPowerUp GPU database for that card, https://www.techpowerup.com/vgabios/196582/196582, the vBIOS is reported as
VBIOS Version: 015.050.002.001.000000
The vbios reported by amdgpu-ls matches that grouped with the device ID on the TechPowerUp page, while the format of the vbios version from TechPowerUp matches bios listings with a utility like GPU-Z (Windows).
So, one issue is, why does amdgpu-ls list a different vbios?
The other issue is a suggestion for an enhancement: offer an amdgpu option to list a card's full BIOS internals, e.g., as seen on TechPowerUp's GPU database (below). Bios details, the BIOS internals, might help some BOINC crunchers who are comparing cards or considering flashing their card bios. Sorry, but I've no idea where such card data is retrieved from the system.

113-57045EHD1-M90 
D00034 Polaris20 XL A1 GDDR5 128Mx32 4GB 300e/300m (C) 1988-2010, Advanced Micro Devices, Inc. 
ATOMBIOSBK-AMD VER015.050.002.001.000000 5704EHD1.M90 
CCC Overdrive Limits 
  GPU Clock: 2000 MHz 
  Memory Clock: 2250 MHz 
PowerTune Limit: -50% to +50% 
Limits 
  TDP: 120 W 
  TDC Power: 117 A 
  Battery Power: 120 W 
  Small Power Power: 120 W 
  Max. Power Limit: 120 W 
  Max. Temp: 90°C 
GPU Clocks 
  300 MHz, 588 MHz, 952 MHz, 997 MHz 1021 
  MHz, 1052 MHz, 1071 MHz, 1100 MHz 
Memory Clocks 
  300 MHz, 1000 MHz, 1850 MHz 
Temperature Target: 75 °C 
Memory Support 
  4096 MB, GDDR5, Autodetect 
  4096 MB, GDDR5, Elpida EDW4032BABG 
  4096 MB, GDDR5, Hynix H5GC4H24AJR 
Memory Timings (Elpida) 
  tRCDW-tRCDWA-tRCDR-tRCDRA-tRC-tCL-tRFC 
  200 MHz: 0-3-2-3-7-7-12 
  400 MHz: 0-3-5-5-15-8-25 
  800 MHz: 5-5-11-11-31-12-51 
  1000 MHz: 9-9-14-14-39-15-64 
  1250 MHz: 13-13-18-18-50-18-81 
  1375 MHz: 15-15-20-20-55-20-89 
  1425 MHz: 15-15-20-20-55-20-89 
  1500 MHz: 15-15-20-20-55-20-89 
  1625 MHz: 17-17-22-22-60-21-97 
  1750 MHz: 17-17-22-22-60-21-97 
  2000 MHz: 17-17-22-22-60-21-97 
Memory Timings (Hynix) 
  tRCDW-tRCDWA-tRCDR-tRCDRA-tRC-tCL-tRFC 
  400 MHz: 4-4-5-5-18-7-39 
  800 MHz: 7-7-11-11-34-10-79  
  900 MHz: 9-9-12-12-38-11-89 
  1000 MHz: 9-9-13-13-41-12-98 
  1125 MHz: 11-11-15-15-47-13-111 
  1250 MHz: 12-12-17-17-52-15-123 
  1375 MHz: 14-14-19-19-57-17-136 
  1425 MHz: 14-14-19-19-57-17-136 
  1500 MHz: 14-14-19-19-57-17-136 
  1625 MHz: 14-14-20-20-61-18-148 
  1750 MHz: 14-14-20-20-61-18-148 
  2000 MHz: 14-14-20-20-61-18-148```

[Ricks-Lab]

With amdgpu-ls and with the --clinfo option, the vBIOS for my RX 570 card running its preloaded 'mining' bios is listed as
vBIOS Version: 113-57045EHD1-M90
From the TechPowerUp GPU database for that card, https://www.techpowerup.com/vgabios/196582/196582, the vBIOS is reported as
VBIOS Version: 015.050.002.001.000000
The vbios reported by amdgpu-ls matches that grouped with the device ID on the TechPowerUp page, while the format of the vbios version from TechPowerUp matches bios listings with a utility like GPU-Z (Windows).
So, one issue is, why does amdgpu-ls list a different vbios?
The other issue is a suggestion for an enhancement: offer an amdgpu option to list a card's full BIOS internals, e.g., as seen on TechPowerUp's GPU database (below). Bios details, the BIOS internals, might help some BOINC crunchers who are comparing cards or considering flashing their card bios. Sorry, but I've no idea where such card data is retrieved from the system.

It would be a useful feature, but currently, amdgpu-utils only reads the file vbios_version driver file in the card path as given by amdgpu-ls. If there is a linux tool like clinfo to get these details, I can make a call to it just like I do for clinfo.

I probably don't have time for that now, since I have 2 major items I want to work on:

1. I have ordered a Radeon VII, so I really want to implement a way to customize the F/V curve.
2. I would like to build a visualization of the the GPUs by combining Gtk and matplotlib. I have no experience with matplotlib, so it will take some focus.

Let me know if you identify a Linux project that can be leveraged for this capability.

[csecht]

IDEA: Add ability to set fan speed curves over GPU temperature ranges. This is feature of most GPU control utilities that users might find useful in amdgpu-utils. e.g. It is used in the vega-fan-control module at https://github.com/dasunsrule32/radeon-scripts (though I haven't tried it).

[csecht]

IDEA: Add specific discussion of overclocking to USER_GUIDE. 'Overclock' is listed as a topic for amdgpu-utils but it isn't explicitly covered in the docs and may be something of interest to users, particularly Linux gamers.
This site, https://wiki.archlinux.org/index.php/AMDGPU, discusses that as of Linux 4.17 there is a different approach to enable overclocking with amdgpu.ppfeaturemask=0xffffffff. My understanding of this is vague and I don't know whether it's relevant.

[csecht]

Is amdgpu-utils functional with RX 5700-series Navi cards? If not, are there update plans to accommodate those new generation of cards. I'm considering buying one for my E@H host, but won't until I can use amdgpu-utils to optimize it.

[Ricks-Lab]

I imagine it to be consistent with Type 2 cards, like Radeon VII, but I have not tried one.

[csecht]

In one of the other Issues discussions, I mentioned working on a bash script to automate changing task multiplicities for optimal running of Einstein@Home tasks. I have a script running well for the gravitational wave tasks and figure it's time to code it for Python. (The script is not really ready for distribution yet because I'm waiting to learn VRAM requirements for the full range of task classes in the Spotlight data series. Once that info is available, conditional statements will need to be optimized.)

I'm only at the stage of viewing the beginner level YouTube 'learn Python' videos, but can see that I will want to recycle some of the environment checking and GPU reporting functions from gpu-utils. The whole object-oriented programming concept and structure is still a bit baffling to me. I'm guessing that cutting and pasting code to suit my purposes will not be straight forward. Are there any general bits of advice you might have for re-purposing parts of gpu-utils code?

BTW, Rick, I took up your suggestion of using an IDE and am quite happy with PyCharm, though I'm only using it mainly as a text editor at this stage.

[Ricks-Lab]

@csecht Great to hear that you are getting into Python! As you get more familiar with the syntax, PyCharm will guide you to PEP8 style compliant code, but don't worry about it yet. I have gone through multiple rounds of refactoring of my projects, each time being more PEP8 compliant and more 'Pythonic'. For me, taking on the use of dictionaries was the first big challenge. Kind of used like structures in C. Don't avoid dictionaries, the earlier you embrace them the better. Using classes is a different way of thinking about programming when compared to scripting languages. On my later projects, including GPU utils, I started with classes (in GPUmodules) and built the rest of the code around it. Each utility has a very simple main that is basically an outline, with all of the work done in class methods.

GPU utils has an open source license (GPLv3) feel free to re-use any code you like. Typically when I get an idea from other code I put a link to it in a comment and when I reuse code, I give credit. But wouldn't want you to get too bogged down with details as you are learning, so consider this comment as permission to reuse any code in GPU utils as you need.

I will be less active on GitHub as I am recovering from surgery, but when I am back, let me know if you have any questions as you make progress on your project.

[csecht]

Well, I did it! I just posted an alpha version of a Python utility for counting and analyzing boinc-client tasks.
https://github.com/csecht/CountBOINCtasks
I went with a "simple" counting project rather than the automated task multiple project. Even at that, there is MUCH I've yet to figure out with maintaining a GitHub repo and integrating PyCharm and GitHub (let alone working with Python).
Thanks for the inspiration. When you get a chance, please have a look and comment?

[Ricks-Lab]

Well, I did it! I just posted an alpha version of a Python utility for counting and analyzing boinc-client tasks.
https://github.com/csecht/CountBOINCtasks
I went with a "simple" counting project rather than the automated task multiple project. Even at that, there is MUCH I've yet to figure out with maintaining a GitHub repo and integrating PyCharm and GitHub (let alone working with Python).
Thanks for the inspiration. When you get a chance, please have a look and comment?

Great progress! I passed on GitHub integration in PyCharm. It is a good idea to become very proficient in git command line first. I suggest checking your code with pylint3. If you have any questions better suited for email, feel free to use the email associated with my gpg key 0xE8496782C98B8839, which is posted to keyserver.ubuntu.com.