https://belmoussaoui.com/blog/atom.xml

Damage areas across the VirtIO space

2023-08-15

In the last few months, I have been trying to improve the default UI shipped by QEMU. As you might not know, QEMU ships with various UI backends: GTK, SDL, Cocoa and recently a DBus one.

I first started trying to port the GTK 3 backend to GTK 4 but faced some issues where I couldn't use GtkApplication as it starts its own GMainLoop which interferes with some god knows what internal GMainLoop started by QEMU itself. My intention was not to only do a simple port but also to see how we could optimize the rendering path as well.

At that time, I also learned that Christian Hergert started working on libmks, a new client-side C library of the DBus backend as he has the intention of using it in GNOME Builder. Marc-André Lureau, one of the upstream QEMU maintainers, is also working on something similar, with a larger scope and using Rust called RDW, a Remote Desktop Widget to rule them all.

Damage areas

From a rendering perspective, the major difference between libmks and RDW at that time is that libmks used a custom GdkPaintable doing a tiled rendering of a GLTexture imported from a DMABuf as GTK had no API to set the damaged region of a GLTexture and it was doing a pointer comparison for a GSK_TEXTURE_NODE render nodes, hence the usage of a tiled rendering. RDW on the other hand was using a plain GtkGLArea.

Christian Hergert also shared with Javier Martinez Canillas and me about his findings while working on libmks:

So now that we have fixed the VirtIO GPU driver and QEMU, we should be able to get proper damage reporting information from the guest, right??? Well, that is what Javier & I were hoping for. A few days into debugging what was going on without much luck, I met Robert Mader during Linux App Summit and discussed the issue with him and he mentioned a certain deny list of drivers from using the atomic KMS API in Mutter. Interesting.

The Monday after LAS, I shared the info with Javier and we found out the root cause of our pain. As the atomic KMS API has no properties for setting the cursor hotspot, which is useful for the virtualized use case, it was put on a deny list until such properties are added to the kernel. Javier also found that Zack Rusin from VMware is already on it, he submitted a patch for the kernel and had a local branch for Mutter as well. Albert Esteve tested the Kernel/Mutter patches and confirmed they were good. He also wrote an IGT test, which will hopefully help get the patch merged soon.

Around that time, Benjamin Otte was working on adding a new builder-like API to simplify the creation of a GdkGLTexture, with a very neat feature, allowing to set the exact damage region instead of doing a pointer comparison. I offered to test the API and ported libmks to use it.

Other improvements

Sergio Lopez Pascual worked on adding multi-touch support to virtio-input/QEMU which I then exposed on the QEMU DBus interface & implemented the libmks side of it.

For the future

  • We still don't have all the required features to make libmks a drop-in replacement for Spice usage in GNOME Boxes like USB redirection, Clipboard sharing. I have WIP branches for both, one day I will finish them.
  • Porting GNOME Boxes to GTK 4
  • The Kernel / Mutter patches from Zack being merged

Once all these pieces land, you will hopefully be able to enjoy a slightly better experience when running a virtualized OS that properly propagates the damaged regions.

For now, you can try libmks either by using the debugging tool shipped with the library or by using this very small client I wrote on top of it, Snowglobe.

Acknowledgement

As you can tell, a bunch of components and people were involved in this few months trip. I would like to thank everyone for taking the time to explain basic things to me as well as helping out to get this done! You can now enjoy a well deserved screen recording showing the implication of these changes