State of the RGB SDK

[PastaJ36]

It’s great to see that there’s so much activity here, but I feel bad that I haven’t been able to keep up with the developments. There’s some major things lacking and overall I’m not happy with the structure of the RGB SDK. I wanted to share some thoughts here and start a discussion about how we can make it better.

This post is just about the RGB SDK, there is also an Analog SDK post here. If you want to know why we split the Analog and RGB SDK, please read this gist.

What is wrong with the current SDK?

I think it’s easiest to start with some things I think are problematic with the current SDK.

1. It’s too hard to use, a developer needs to define their own set of functions and load the DLL.
2. There’s no good feedback if there’s something wrong.
3. It’s not clear if the developer should add the SDK to its project or if the user installs it.
4. There’s no good way to update the SDK, user nor application side.
5. If you create something cool, how do you share it?
6. It uses the same interface as the Wootility, so it can potentially conflict.

What are some options we can do?

From talking to you guys I’ve learned some options, current and new. I want to list some and talk about how they could address the issues. I’ll refer to the current setup as “1-DLL”.

1. Improve 1-DLL and “install” (add) it developer side

This is the current way. With just a DLL it seems impossible to update automatically. Interfacing and feedback could be improved. Sharing would require sharing full applications.

2. Improve 1-DLL and “install” (add) it user side

We attempted to add this to the Wootility, but we didn’t push it through. The Wootility can install the DLL and also update the DLL. Interfacing would be a little bit harder, since you need to find the installed DLL first. API changes will be very hard, because developer will be responsible for checking versions. Shares also requires sharing full application.

3. Split SDK in two: logic and interface

The user install the logic part and the developer adds just the interface part to its application.

The interface part wouldn’t have to be updates regularly and if it did it can take care of API changes. This also allow defined interfaces per language for easiest interfacing.

The logic part will be maintained by the Wootility and thus easily installed and updated. Sharing would still application dependant

4. Use the Wootility itself as the core

This feels like the end-game for the RGB SDK. It will make it easy to add different interfacing options: DLL, sockets, REST. It also removed potential conflicts. Another big plus is that we can add other configuration options, like changing keyboard mappings. This will morph the RGB SDK in a Wooting Config SDK.

Sharing could be done via a plugin system.

What to do?

Here is where you come in. I would love to get feedback on what you want to see from the RGB SDK and how you think would be the best way to achieve it.

I think one of the things that need to happens for sure is to not have the application add any logic, just the interfacing.

When it comes to the Wootility: even though I like the option, in its current shape it would require close sourcing parts of the SDK. We don’t want to take a step back. If we go this route we’ll open source the whole USB configuration part. It’s already a separate project, so it shouldn’t cause too much workload.

I’m most excited about the possibility of growing the RGB SDK into a Wooting Config SDK to allow full control for your applications.

That’s it for now. It’s a big post, but I feel it’s also a big decision to make.

[BigBrainAFK]

tbh i feel like the wootility should be the center. this would solve a few problems like auto updating and managing what applications have access to rgb/analog. if we split the sdks from the wootility again i feel like it will result in either extra work to manage multiple sdk accesses by different threads or the previously mentioned problem of updating dlls. the wootility would also enable more ways to communicate with the sdk like IPC, REST, etc.

[GottZ]

i'd suggest first decoupling the sdk so the dll is in system32 on windows (for example)
iteratively it would be easy to extend on that and turn it into a interface without the developer even noticing.

another thing.. turning this into a client / service / plugin endgame infrastructure sure is the desired outcome.

for now just look at how aurora does the razer / logitech / ... wrappers to get a inspiration.
why reinvent the wheel.

it would also help with allowing multi wooting support or restricting a wooting from the public sdk.
it would also allow us to disable the sdk client for anything besides a choosen client and pipe the app interactions through, for example, aurora.

one use case: disable xinput and digital input and only use a choosen wooting as macro keyboard with the analog and rgb sdk.

@diogotr7

[BigBrainAFK]

@GottZ i dont get why youd want to out it into system32 thats where drivers should go and not random shit from a small program. it should stay confined to normal userspace stuff and not system folders

[GottZ]

just a thought. i agree. system32 might not be the solution. especially since the sdk could simply retrive the path to the wootility through registry and load the dll from the correct location dynamically

[simon-wh]

@GottZ For the DLL, could just have it in a wootility directory and add it to the path maybe.

Overall, having a shared process that everything goes through is ideal, however, I'm not sure if it being specifically Wootility is the best idea. The main issue I have is the that it's a configuration utility and shouldn't really be running all the time, especially with chromium/electron memory overhead.
One possible solution could be to have a separate process dedicated to running in the background that handles all RGB/analog requests and handles shared access. Wootility could run through the same interface and make sure it's kept updated. Using the same interface DLL strategy for developers to interact with the shared service.

EDIT: Just remembered why the analog and RGB portions are split, so ignore having them both in the shared process

Another note, if Wootility was using the same interface as all 3rd party stuff it would have the added benefit of implicitly giving devs more/similar control as wootility has and ensures that the api is somewhat fully working as it'd be extensively used through Wootility

[Rocky04]

I think too that the long term goal should be just using a service of the Wootility like on a REST interface from it.

In the mean time I also prefer that the core RGB stuff has to be splitted from the configuration stuff of the wootility. So to create a third SDK for configuration of the keyboard, this SDK can include the core RGB SDK maybe so it extends it with the special Wooting stuff to change profiles and so on.

[GottZ]

https://docs.microsoft.com/en-us/windows/desktop/com/com-objects-and-interfaces
https://docs.microsoft.com/de-de/windows/desktop/ipc/named-pipes
https://blogs.msdn.microsoft.com/commandline/2017/12/19/af_unix-comes-to-windows/

https://www.freedesktop.org/wiki/Software/dbus/
!nice! https://github.com/bus1
http://man7.org/linux/man-pages/man7/epoll.7.html
http://man7.org/linux/man-pages/man5/proc.5.html
http://man7.org/linux/man-pages/man7/mq_overview.7.html
http://man7.org/linux/man-pages/man7/unix.7.html

https://www.freebsd.org/cgi/man.cgi?query=kqueue&sektion=2

https://stackoverflow.com/a/2893078/1519836

the wootility should also just be a client to the sdk daemon. thus, just create a service that implements a few of the links i just provided above this line.

in terms of running the service properly:

https://support.microsoft.com/en-us/help/251192/how-to-create-a-windows-service-by-using-sc-exe
(also possible through the registry)
https://www.freedesktop.org/software/systemd/man/systemd.unit.html
freebsd.. they don't need automation. they write their unit files themselves and even compile stuff.
https://en.wikipedia.org/wiki/Launchd

the sdk and wootility should simply connect to a service that is created as listed above.

oh and no worries.. we are a community. we love to help without financial rewards.

[PastaJ36]

Thanks for the discussion. I agree that the Wootility being the center is a bad idea resource wise and it should ideally just function as a client. The SDK daeomon (wooting-config-daemon?) can be responsible for all configuration. Should also have a plugin system, mostly for custom RGB effects and such. It could even take full control of RGB effects, because doing it on the keyboard is too resource intensive.

Main concern for me is cross platform compatibility. REST would be most compatible, but in a few tests I did pretty slow. Named pipes are faster I asume?

In line with the Wootility we could make it with Node js. We'll be able to reuse most Wootility usb code and out of the box cross platform:
https://nodejs.org/api/net.html#net_ipc_support
http://riaevangelist.github.io/node-ipc/

[PastaJ36]

Another addition: I think if we do this right we can set an example for open source keyboard firmwares like QMK. They're in need of standardised configuration.

[cactysman]

It will make it easy to add different interfacing options: DLL, sockets, REST. It also removed potential conflicts. Another big plus is that we can add other configuration options, like changing keyboard mappings. This will morph the RGB SDK in a Wooting Config SDK.

For my part I think having two SDKs like we do right now is kind of troublesome. Simple information retrievement like a possible feature like this would have to be implemented in both SDKs.

Either unifying them or at least having both the RGB SDK and the Analog SDK based on the same base code for stuff like this is the best way to solve this in my opinion.

Main concern for me is cross platform compatibility. REST would be most compatible, but in a few tests I did pretty slow. Named pipes are faster I asume?

I'd like to throw in that Docker uses REST for their API, too, may give a few helpful insights and ideas.

[GottZ]

ipc & pipes (also includes several api's like direct play from microsoft) > virtual files > af unix sockets > sockets > files

either comes with different penalties. as further you go to the right, as easier it is for a beginner to interact with it.
as further you go to the left, as faster it will be.

same with structures.
filling a buffer / a struct is faster than serialization / dynamic objects / json

true.. json is fine these days though I'd advice bson

[PastaJ36]

By now the Analog SDK has been updated to tackle similar issues it has. As for the RGB SDK we have updated it to include the latest PR to make sure it works with the Wooting one and two. We have started making steps to have RGB control in the Wootility with the latest Chroma Connect update (in beta as of now). The plan is to eventually integrate the RGB SDK with that background running service, but we don't have a timeline for that yet.