[Info req] how does gintro differs from gtk-rs approach?

[arkanoid87]

Hello!

lately I've been playing with some of the approaches out there for working with GObjects in Nim:

• manually wrapping C API
• automatic wrapping C API with futhark
• use gintro
• write Vala/Genie code and compile it to C source code plus Headers, wrap it automatically with futhark and use the Vala/Genie public functions from Nim

Here comes my question: how does gintro approach differs from gtk-rs? https://gtk-rs.org/gir/book/tutorial/introduction.html

Both projects seems to start from GObject Introspection files, and both end up with type-safe, memory-safe API for target language.

Why starting from .gi/.typelib files instead of using valalib or .vapi file, which already solve the bridge from low to high language level? What am I missing from the global picture?

I'm just looking for an expert opinion on this domain.

Thanks

[StefanSalewski]

Well, I started with gintro in early 2016, after realizing that my low level bindings, now called oldgtk3, are mostly useless. I created the NEd editor with the oldgtk3 bindings, it was very ugly; writing that in plain C would have been easier. That time we had not many Nim bindings, I think only the gtk2 ones from Aporia. Actually, I came to Nim, because I used GTK3 for my old Ruby CAD/PCB tool, which I intended to port to Nim, and I initially thought that the Nim core devs or others would provide fine high level GTK3 bindings soon. But that was not the case, and after realizing that my oldgtk3 bindings made not much sense, I started with high level bindings. Actually two others started with high level bindings, a Mr Hass from Crystal team, and someone from Nim, who retired after a few weeks. That time I asked a few questions at Nim and GTK forum about bindings generation. From Nim there was no feedback, and from GTK side I got from Mr Bassi the hint that typically gobject-introspection is used for bindings generation. No one told me how difficult it would be, I would have never done it if I would have known before. I considered starting with Python bindings, but of course for an interpreted, dynamically typed language all is very different. I think that time, early 2016, Vala and Rust bindings were in a very early state, and I had no contacts to Vala or Rust devs that time. What I was told from GTK side was, that for languages with GC one should use the gobject-introspection toggle references, which resulted in the design with Nim proxy objects for the GTK ones with the 00 prefix. All that was very hard, a lot of try and error, no real help from any side. Use of the gobject-introsection library gave a lot of issues. Later Bassi suggested, parsing the GIR XML files directly instead use of Gobject-Introspection, but that advice came too late. Today, as the Rust bindings may be the best ones, I would start with that I guess. Indeed, three or four years ago, we discussed hiring some smart rust dev together with a smart GTK one, to let us create a new bindings set. I was willing to donate up to 1000 Dollar for that, and with maybe 49 others paying the same amount, collecting 50k Dollar would be an option. I have never investigated Vala that much, I think it is dead now. Rust has no GC, so its bindings are different from Nims. Actually, I have inspected some Rust GTK apps, my impression was that it was looking ugly.

But well, I think that no one is still using Gintro, and from other languages there are very few people that really use GTK for new projects. I look from time to time in the Gnome forum, or for new GTK projects at GitHub. Trere is really not much activity. GTK is more in maintenance mode now. I had some hope when GTK4 came out, but it has not improved the situation. I even started the GTK4 book, but again, interest was nearly zero. And I assume you know in what a bad state Nim is now. I have still the CAD/PCB project at projecthttps://github.com/StefanSalewski/SDT which basically works not bad, so finishing it in Nim would make some sense. But after all that unfriendly behaviour from the main "Nim devs" my motivation for any Nim work is gone. I try from time to time to work on that project, but with no real success. Even my motivation to finish the book is very low. I set up its homepage at https://nimprogrammingbook.com/, and did most of the grammar fixes. But I think before printing it on paper at Amazon or elsewhere, I should do a final proof read.

I think I will not any longer work on Gintro. 1600 hours have been really enough work for zero users. What is missing most, and maybe is the reason why you actually asked, is true gobject support, e.g. the possible of creating new gobjects. For that, one would have to make all the C GTK macros available to Nim. Should be possible, but would be some work, and I personally do not need that.

You mentioned Futhark from Mr. Munch: I talked to him, but he said that it can only generate low level bindings. So mostly useless, but for special tasks maybe an option, I recommended Futhark to someone interested in gstreamer. GStreamer is very low level by design, so low level bindings would be OK.

If you really should intend to create a new Nim GTK bindings set: I am not sure if I can kept my offer to support it with up to 1000 Dollar. I donated already 500 Dollar to the Nim project two years ago, and actually I am not really using Nim anymore. And I think we all have to admit that a native GUI is always more fun. Rust people are trying to create a native Rust GUI, they have the people, the motivation, and the money, so maybe they will succeed. That would be a reason to port my tool to Rust. But for the next 12 months I will observe Mojo as well. I am still skeptical about that project, their advertising with AI and fully Python compatible is a bit strange, but I have to admit that it may be good marketing, because that is what people desire. And when Mojo gets a fine GUI, I may port to Mojo instead of Rust. For next winter, it would make some sense to continue with Nim. I know, but when the fun is gone, it is hard to do any unpaid work. And there have been rumors, that they blocked gintro for latest Nim 2.0 by disabling some tuple unpacking support. That was working for 8 years, then it was removed, making a fresh gintro install impossible. I have no idea if they fixed it, I have still an old install on my box, which is enough for me. I will not fix gintro for that, even when it should be very easy.

[StefanSalewski]

Well, perhaps you have already retired. But I just saw a recent Rust code snippet on Gnome forum: https://discourse.gnome.org/t/gtk4-rust-failed-to-create-signal/16433/1 Seems that the Rust GTK code is as ugly as it was some years ago:

factory.connect_bind(move |_factory, item| {
            let item = item.downcast_ref::<gtk::ListItem>().unwrap();
            let reserve = item.item().and_downcast::<Reserve>().unwrap();
    
            let child = item.child().and_downcast::<Row>().unwrap();
            child.set_reserve(&reserve);

            let obj = self.obj();

            child.connect_clicked(move |btn| {
                obj.emit_by_name::<()>("reserve-pressed", &[&btn.id()]);
            });
});

[arkanoid87]

Thanks Mr Salewski for the in-depth answer.

I did know just part of the story, apparently. Your words are punching my trust on Nim project , yet I consider your work as a standing column in Nim community.

From my point of view, your gintro project is present in some of my small gstreamer and gtk based projects, and I've used your generated Nim files from GIR files as "reference" to understand "how pros dealt with that problem". Thank you for this, and for your book which I used to invite some people to Nim, even friends unexperienced in programming in general.

I don't intend to create new GObject high-level bindings for Nim, I just want to understand how statically compilable bindings can be generated, and understand the current state of the art in this matter. At this point, apart from Nim, Rust and Vala I don't know which other tools are up to the task, but while i agree that Rust GIR-based binding generator seems more complete, Vala devs told me that their project is live and kicking, and this old blog post from Mr. Bassi is no more reflecting the state of the Vala project, and Genie is actually dead.

[arkanoid87]

I also just got some feedback:

• from GStreamer dev: "The Rust bindings are likely the best maintained and most widely used bindings there are at this point"
• from Vala dev: "well, I don't know Nim, but really I would recommend just working in the Nim bindings if they already exist, and not put much work into translating the Vala AST to Nim"

to sum up:

1. Rust bindings seems the way to go, but I strongly dislike the messy syntax of Rust in general, and the one wrapping GObject is even more ugly.
2. Your work generates the best looking GObject code I know. Almost like Vala, but with a hugely better programming language surrounding it, but is a one-man-vs-the-world task, and I feel that there's no enough traction for it.

[StefanSalewski]

Rust bindings seems the way to go, but I strongly dislike the messy syntax of Rust in general, and the one wrapping GObject is even more ugly.

Yes, I fully agree. Rust has a GTK team of bright devs, one of them is Mr. Droege, who gave me some tips for the Nim bindings. But Rust use no GC, and Rust GTK app code looks worse than Python GTK code.

I forget to mention the other Nim GTK project, I think called ownkettle by Can Lehman. He is a bright and very active guy. His bindings are declarative. I don't know if someone is using them for serious projects, but for toy projects they should work, see https://github.com/can-lehmann/owlkettle

[arkanoid87]

I also used owlkettle in the past.
https://github.com/can-lehmann/owlkettle/blob/main/owlkettle/gtk.nim seems cimporting C API and then creating a DSL on top of it. I think there is not GIR involved here, and there's no "automated process" also.

[arkanoid87]

Again from gstreamer dev:

There are two layers to the Rust bindings, one is ffi bindings which is basically a 1:1 mapping to the C API and unsafe {} in Rust terms, and then there's higher-level bindings on top which feel Rust-y, and a lot of that is manually implemented
just like for python there's gst-python which has syntactic sugar and pythonic APIs on top of the g-i bindings

so fro Rust the first pass is generated reading GIRs, while higher level is manually written.
This basically means that higher level bindings cannot be automatically generated without tons of quirks, probably.

AFAIK, Futhark is capable of autogenerating the (unsafe) C API bindings automatically for any C library (excluding some macros). I've tested this with GLib and Gst and it seems to work.
From there higher level bindings should be created manually (or maybe with GIR?)

[arkanoid87]

Here is just an example how Vala + (Nim + Futhark) can be used to get clean syntax and easy interop between Nim and the GObject world.

[StefanSalewski]

This basically means that higher level bindings cannot be automatically generated without tons of quirks, probably.

Well, I struggled with all that for years. The GTK advertising, that GTK with its Gobject-Introspection allows easy bindings generation for other programming languages is a large exaggeration. The bindings that are OK are the Python and Rust ones. Maybe JavaScript bindings are also not in a too bad shape. Actually, all the bindings are mostly used only for toy projects. The few remaining GTK devs mostly use C. And with the new GTK4 widgets GTK has become even more C centric. A lot of the new stuff is not fully supported by Python. You may consult my GTK4 book, the sections that I added last, maybe 18 months ago, are about some of the new GTK4 containers. I can not remember details, but for one widget or container type I spent 70 hours to get it working. There is not much support from GTK devs, Mr. Bassi and Mr. Larsen are the two with deep knowledge. Larsen is generally not available, and Bassi has to answer 90% of the forum questions and so is often unfriendly. Larsen was hired by RedHat, but Bassi may be again unemployed, so I can somehow understand when he is unfriendly. Furthermore, GTK has a lot of bugs, often not reported, because so few people are using it. GTK documentation has improved a bit, but often is not sufficient and unclear. Gobject-introspection is a mess created as a weekend project decades ago, never really finished. Some libs like cairo are nearly unsupported by gobject-introspection, others like gstreamer or libnice are very different from the core GTK libs and work on a very low level. For the split of bindings in low and high level part: Yes, I considered that myself, but then early concluded that it would have no real benefit. For Gintro, 97% of the high level bindings are autogenerated. But the remainding 3% are still a lot of work. Gintro has currently about 10k procs, and 2k data types. 3% of 10k is 300. And that are the difficult ones with GList and Treeview and all that. And from the other 97%, I bet that perhaps 3% have bugs, so again 300 procs to test and fix. You may ask the Rust, Python or JS devs how much time they have investigated in their bindings. I guess a few thousand hours total, and that are really smart people. Generally, GTK is a very difficult GUI. I started learning it in 2007 with the book of A. Krause, and still know only a tiny part of it. When using with other languages, there is always the problem that it is difficult to extend, improve, or fix widgets directly from that language, or to create real new native widgets, that can be then used from other languages and C as well. So you see, people have reasons why they create no new software with GTK. And then there is the missing support for mobile, Android and IPhone. And generally, using a GUI native to the used language is more fun, as it is then possible to easily extend the GUI elements. Gintro works OK, but for example improving the GtkSourceView widgets for a Nim text editor is very difficult. We could support creating new Widgets from Nim, yes, when we make all the C macro stuff available from Nim. Expand the nested C macros, port to Nim templates or Nim macros, create a nice API, write a nice documentation. Should be possible, for me, as I am no expert, maybe 200 upto 300 hours. For an expert maybe only 100 hours. The project of Can Lehmann may be interesting as well, but there we have to learn one more layer on top, and when something does not work, we may get even more problems. I think Mr. Munch had also a GTK project, I thing genUI?. And I think there are even more. Low level bindings with Futhark are still an option. Picheta created Aporia with his low level GTK2 bindings. But GTK2 was much easier, and Aporia code was ugly. My NEd is created with oldgtk3 bindings, creation was difficult, and code is ugly as well. About your idea of Nim bindings based on the Vala ones? I am unsure. How shall all the memory management work? For widgrts, we have always the problem, that we may add widgets to a GTK container, and later get the widget back. So we have temporary no direct reference to that widget. For that GTK devs suggested the use of the toggle references, which works for Gintro. The disadvantage is, that we need proxy objects, which creates some overhead.

When you really like GTK, and want to create a larger new project, you might consider using Rust, Vala, or maybe even plain C. Python may be an option when performance is not critical. For JS I am skeptical. Personally, I would not start today with Nim or GTK again. But for me it is a bit late now, I have the CAD/PCB project, spent more than 6000 hours of unpaid work into Nim in the last 9 years, and some work learning GTK as well:

$ cat nimwork 
oldgtk3     600
gintro      >1600
nim book    1200 + 300
gtk4 book   300
cdt         500
rtree       350
hull of disks 250
chess       400
ned         250
sdt EDA     >1000

600+1600+1500+300+500+350+250+400+250+1000 = 6750 hours. 

Well, at least I should start learning a new language in next winter, Rust, Mojo, or Julia are candidates.

[arkanoid87]

I can feel delusion from your words, and is quite understandable giving the amount of work spent on it. But who is to blame here? Is it GTK giving false promises about GObject being an higher level architecture where bindings are easily doable, or is it Nim fighting against your work somehow?

I don't want to step into personal feelings here, I have very positive opinion about your contributions. I started programming in Nim thanks to your positive answer to me about adding Gst into gintro, years ago.