Rendering animation through director is not using multiple threads

[sdemarre]

I did the following:

• $ git clone https://github.com/edyoung/gnofract4d
• $ cd gnofract4d
• $ ./setup.py build
• $ ./gnofract4d
• find some nice starting point. top indicates some python3 process taking up > 600% cpu (8 core machine), 1920x1080, renders very quickly.
• tools->director->add->from current fractal
• find some other nice point
• tools->director->add->from current fractal
• configure resulting video file
• render ===> top shows the same python3 process taking up < 80% cpu (and takes forever to render a small 25 frames animation at 640x480)

I would expect it to use all cores as during normal rendering. Could it be that the process is blocking due to some io ? Images are stored in /tmp, which is on an ssd and should be able to handle the io.

[sdemarre]

perhaps also worth mentioning:
$ git describe
V_3_14-289-g220966b
$ uname -a
Linux hostname 4.13.0-26-generic 29~16.04.2-Ubuntu SMP Tue Jan 9 22:00:44 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

[cjmayo]

I've had a go at speeding up and improving the video generation in #45. Do give it a try.

[wasdmco]

@cjmayo I rebuilt a new version from your a812937 commit, but as far as I can see I still have single threaded rendering of animation frames. If I can help in anyway to investigate why or how this is happening, feel free to let me know and I'll see what I can do.

[cjmayo]

Thanks for testing quickly. I probably set expectations a bit high - it is significantly faster for me but only because it is using one thread more effectively. In other words the Python threading was slowing things down but it was always coded to use one thread.

For you own use you could try:

--- a/fract4dgui/PNGGen.py
+++ b/fract4dgui/PNGGen.py
@@ -136,6 +136,7 @@ class GenerationManager:
         self.current = gtkfractal.HighResolution(
             compiler,
             int(self.anim.get_width()), int(self.anim.get_height()))
+        self.current.set_nthreads(self.anim.userConfig.getint("general", "threads"))
 
         self.current.connect('status-changed', self.onStatusChanged)
         self.current.connect('progress-changed', self.onProgressChanged)

But I find that although it works some of the time it does also lead to segfaults and other crashes so I won't be recommending it.

So, unfortunately I think #45 is as far as we can go for now. It would likely take a major investigation of the code to work out why this is crashing.

[wasdmco]

Cool, I will apply your suggested diff and do some testing.
I did notice those segfaults you're talking about (even without applying your suggested diff), and I have noticed that they happen more quickly when rendering to a very small size (e.g. 320x240, which seems to be the smallest allowed by the input). But, even when rendering at high resolution, it will crash at some random point in time. Some race condition ? I have not been able to localize what part of the software is segfaulting. It can't be python, right ? Should be on the c/c++ side of things.

[wasdmco]

I confirm your observation that it crashes very quickly when calling set_nthreads.

[cjmayo]

It's good to have confirmation of the faults - at least it isn't my hardware. Yes I wouldn't expect it to be the Python but it could be something the Python isn't doing right in using the C/C++ extensions, even the GTK/GLib ones.

[cjmayo]

I've tried the set_nthreads patch again on current Git master and haven't had any segfaults. Looks like the fixes to the extension have helped. Maybe a bit early to make this change though, do still see some crashes when running GitHub Actions.