What’s all this new-fangled DBus rubbish?

The 1990s called…

DBus has been one of the larger changes that swept through Linux in the last 20 years or so. I mostly (with the exception of a small amount of whinging) ignored it. Usually I’ve encountered it when it hasn’t worked or has got in the way with something or other. Naturally when things worked, I was generally unaware of it.

It’s basically:

  • Async RPC system with call, response, exception and asynchronus push.
  • Authentication (useful for talking to daemons running as root)
  • Introspection (the set of methods, arguments etc are exposed via DBus)
  • Central point for services to register objects and methods: you open a dbus connection as opposed to a random socket

Various tools exist to send and receive messages, e.g. python libraries, shell commands via ‘dbus-send’ and so on. I’ve nothing against RPC in general, but it replaces essentially shell scripts triggered by the kernel etc with programs exchanging messages. You can do more with the latter (though often that isn’t necessary), but it replaces easily discoverable scripts with reading documentation, something the programming community is not well known for and as such it has a rather higher barrier to entry.

It has also has been used to replace simply libraries with relatively complex IPC requiring complex setup. But I won’t blame a table saw for injuries it causes through misuse. Anyway this post is a nearly linear stream of me learning DBus with the mistakes and confusion removed.

The tools

I will be relying on the following:

  • python dbus module import dbus
  • The basic dbus commandline program: dbus-send
  • QT’s qdbus since it provides in many cases a nicer interface and introspection
  • Gnome’s gdbus for more verbose introspection

The basics

I’m interested in manipulating the system, so I will be working with the system bus. There’s usually a system bus (for talking to OS related daemons) and a session bus (for all the programs in a logged in session). You can make more if you like but no one does.

In order to make an RPC call, you need:

  1. A program to talk to, e.g. NetworkManager (this is called the bus name) and usually has a name like org.freedesktop.NetworkManager
  2. An object in the program. Each program exports a tree of objects, rooted at / and separated with forward slashes. Something like: /org/freedesktop/NetworkManager. Freedesktop likes redundant naming because it’s redundant and repeats things redundantly. They could equally well have exported /.
  3. The interface and method name. Just like any OO system each object can present zero or more interfaces with methods.

We can examine this. First, qdbus will show the names present on the system bus. For example, on my system (I don’t know what the numbers prefixed with colons are) I have:

qdbus --system  | grep -v :
 org.freedesktop.systemd1
 org.freedesktop.resolve1
 fi.epitest.hostap.WPASupplicant
 fi.w1.wpa_supplicant1
 org.freedesktop.NetworkManager
 org.gnome.DisplayManager
 org.freedesktop.ColorManager
 org.freedesktop.Avahi
 org.bluez
 org.freedesktop.UPower
 org.freedesktop.Accounts
 org.freedesktop.login1
 org.freedesktop.RealtimeKit1
 org.freedesktop.UDisks2
 org.freedesktop.ModemManager1
 org.freedesktop.bolt
 org.freedesktop.PackageKit
 org.freedesktop.PolicyKit1
org.freedesktop.DBus

There’s a few one might recognise, Udisks for hotplug disks, NetworkManager for wifi control etc, ModemManager1 because this machine actually has a real actual physical 56k modem, and a few other well established ones like systemd. You can go further and query what’s on the bus. For example, I can query systemd using gdbus, which as you may recall is the verbose, detailed one. I’m going to query the root object to see what’s there:

$gdbus introspect  --system --dest org.freedesktop.systemd1   --object-path / 
node / {
  interface org.freedesktop.DBus.Peer {
    methods:
      Ping();
      GetMachineId(out s machine_uuid);
    signals:
    properties:
  };
  interface org.freedesktop.DBus.Introspectable {
    methods:
      Introspect(out s data);
    signals:
    properties:
  };
  interface org.freedesktop.DBus.Properties {
    methods:
      Get(in  s interface,
          in  s property,
          out v value);
      GetAll(in  s interface,
             out a{sv} properties);
      Set(in  s interface,
          in  s property,
          in  v value);
    signals:
      PropertiesChanged(s interface,
                        a{sv} changed_properties,
                        as invalidated_properties);
    properties:
  };
  node org {
  };
};

If you’ve ever read IDLs of any sort this will look vaguely familiar. Systemd is exporting an object which presents three interfaces:

  • org.freedesktop.DBus.Peer
  • org.freedesktop.DBus.Introspectable
  • org.freedesktop.DBus.Properties

and a subobject, org. Since gdbus doesn’t recurse by default, that’s all that’s displayed. The first of those objects has two methods neither of which have any arguments, which makes them easy to call. So I shall ping dbus:

$dbus-send --print-reply --system --dest=org.freedesktop.systemd1 / org.freedesktop.DBus.Peer.GetMachineId
method return time=1632074720.055870 sender=:1.0 -> destination=:1.3006 serial=54374 reply_serial=2
   string "73f867af39124a3583c288e620019332"

and it responds. See how the bus name (dest), path (/) and interface.object (org.freedesktop.DBus.Peer.GerMachineId) are used. I can examine another common one. when given a bus, but no object, qdbus will recurse showing the object tree:

$qdbus --literal --system org.freedesktop.NetworkManager 
/
/org
/org/freedesktop
/org/freedesktop/NetworkManager
/org/freedesktop/NetworkManager/DnsManager
/org/freedesktop/NetworkManager/DHCP4Config
/org/freedesktop/NetworkManager/DHCP4Config/70
/org/freedesktop/NetworkManager/ActiveConnection
/org/freedesktop/NetworkManager/ActiveConnection/2
/org/freedesktop/NetworkManager/ActiveConnection/72
/org/freedesktop/NetworkManager/AccessPoint
/org/freedesktop/NetworkManager/AccessPoint/2686
/org/freedesktop/NetworkManager/Devices
/org/freedesktop/NetworkManager/Devices/3
/org/freedesktop/NetworkManager/Devices/2
/org/freedesktop/NetworkManager/Devices/1
/org/freedesktop/NetworkManager/Devices/25
/org/freedesktop/NetworkManager/Devices/4
/org/freedesktop/NetworkManager/AgentManager
/org/freedesktop/NetworkManager/Settings
/org/freedesktop/NetworkManager/Settings/8
/org/freedesktop/NetworkManager/Settings/7
/org/freedesktop/NetworkManager/Settings/6
/org/freedesktop/NetworkManager/Settings/5
/org/freedesktop/NetworkManager/Settings/4
/org/freedesktop/NetworkManager/Settings/3
/org/freedesktop/NetworkManager/Settings/2
/org/freedesktop/NetworkManager/Settings/1
/org/freedesktop/NetworkManager/Settings/24
/org/freedesktop/NetworkManager/Settings/9
/org/freedesktop/NetworkManager/IP6Config
/org/freedesktop/NetworkManager/IP6Config/3
/org/freedesktop/NetworkManager/IP6Config/204
/org/freedesktop/NetworkManager/IP6Config/203
/org/freedesktop/NetworkManager/IP6Config/6
/org/freedesktop/NetworkManager/IP4Config
/org/freedesktop/NetworkManager/IP4Config/3
/org/freedesktop/NetworkManager/IP4Config/204
/org/freedesktop/NetworkManager/IP4Config/203
/org/freedesktop/NetworkManager/IP4Config/6

For reference, gdbus (non recursive; recursive is too verbose for this blog post) gives:

$gdbus introspect  --system --dest org.freedesktop.NetworkManager   --object-path /
node / {
  node org {
  };
};

The root node isn’t very interesting, it just has the child org and nothing else. qdbus will also give a more compact method view, for example, I can query one of the devices:

$qdbus --literal --system org.freedesktop.NetworkManager /org/freedesktop/NetworkManager/Devices/3 
method QDBusVariant org.freedesktop.DBus.Properties.Get(QString interface_name, QString property_name)
method QVariantMap org.freedesktop.DBus.Properties.GetAll(QString interface_name)
signal void org.freedesktop.DBus.Properties.PropertiesChanged(QString interface_name, QVariantMap changed_properties, QStringList invalidated_properties)
method void org.freedesktop.DBus.Properties.Set(QString interface_name, QString property_name, QDBusVariant value)
method QString org.freedesktop.DBus.Introspectable.Introspect()
method QString org.freedesktop.DBus.Peer.GetMachineId()
method void org.freedesktop.DBus.Peer.Ping()
property read QDBusObjectPath org.freedesktop.NetworkManager.Device.ActiveConnection
property readwrite bool org.freedesktop.NetworkManager.Device.Autoconnect
property read QList<QDBusObjectPath> org.freedesktop.NetworkManager.Device.AvailableConnections
property read uint org.freedesktop.NetworkManager.Device.Capabilities
property read uint org.freedesktop.NetworkManager.Device.DeviceType
property read QDBusObjectPath org.freedesktop.NetworkManager.Device.Dhcp4Config
property read QDBusObjectPath org.freedesktop.NetworkManager.Device.Dhcp6Config
property read QString org.freedesktop.NetworkManager.Device.Driver
property read QString org.freedesktop.NetworkManager.Device.DriverVersion
property read bool org.freedesktop.NetworkManager.Device.FirmwareMissing
property read QString org.freedesktop.NetworkManager.Device.FirmwareVersion
property read QString org.freedesktop.NetworkManager.Device.Interface
property read uint org.freedesktop.NetworkManager.Device.Ip4Address
property read QDBusObjectPath org.freedesktop.NetworkManager.Device.Ip4Config
property read QDBusObjectPath org.freedesktop.NetworkManager.Device.Ip6Config
property read QString org.freedesktop.NetworkManager.Device.IpInterface
property read QDBusRawType::aa{sv} org.freedesktop.NetworkManager.Device.LldpNeighbors
property readwrite bool org.freedesktop.NetworkManager.Device.Managed
property read uint org.freedesktop.NetworkManager.Device.Metered
property read uint org.freedesktop.NetworkManager.Device.Mtu
property read bool org.freedesktop.NetworkManager.Device.NmPluginMissing
property read QString org.freedesktop.NetworkManager.Device.PhysicalPortId
property read bool org.freedesktop.NetworkManager.Device.Real
property read uint org.freedesktop.NetworkManager.Device.State
property read QDBusRawType::(uu) org.freedesktop.NetworkManager.Device.StateReason
property read QString org.freedesktop.NetworkManager.Device.Udi
method void org.freedesktop.NetworkManager.Device.Delete()
method void org.freedesktop.NetworkManager.Device.Disconnect()
method QDBusRawType::a{sa{sv}} org.freedesktop.NetworkManager.Device.GetAppliedConnection(uint flags, qulonglong& version_id)
method void org.freedesktop.NetworkManager.Device.Reapply(QDBusRawType::a{sa{sv}} connection, qulonglong version_id, uint flags)
signal void org.freedesktop.NetworkManager.Device.StateChanged(uint new_state, uint old_state, uint reason)
property read QList<QDBusObjectPath> org.freedesktop.NetworkManager.Device.Wireless.AccessPoints
property read QDBusObjectPath org.freedesktop.NetworkManager.Device.Wireless.ActiveAccessPoint
property read uint org.freedesktop.NetworkManager.Device.Wireless.Bitrate
property read QString org.freedesktop.NetworkManager.Device.Wireless.HwAddress
property read uint org.freedesktop.NetworkManager.Device.Wireless.Mode
property read QString org.freedesktop.NetworkManager.Device.Wireless.PermHwAddress
property read uint org.freedesktop.NetworkManager.Device.Wireless.WirelessCapabilities
signal void org.freedesktop.NetworkManager.Device.Wireless.AccessPointAdded(QDBusObjectPath access_point)
signal void org.freedesktop.NetworkManager.Device.Wireless.AccessPointRemoved(QDBusObjectPath access_point)
method QList<QDBusObjectPath> org.freedesktop.NetworkManager.Device.Wireless.GetAccessPoints()
method QList<QDBusObjectPath> org.freedesktop.NetworkManager.Device.Wireless.GetAllAccessPoints()
signal void org.freedesktop.NetworkManager.Device.Wireless.PropertiesChanged(QVariantMap properties)
method void org.freedesktop.NetworkManager.Device.Wireless.RequestScan(QVariantMap options)
property readwrite uint org.freedesktop.NetworkManager.Device.Statistics.RefreshRateMs
property read qulonglong org.freedesktop.NetworkManager.Device.Statistics.RxBytes
property read qulonglong org.freedesktop.NetworkManager.Device.Statistics.TxBytes
signal void org.freedesktop.NetworkManager.Device.Statistics.PropertiesChanged(QVariantMap properties)

Yikes! There’s a lot there. Actually gdbus has the nice thing where it also queries properties for you. I recommend trying that. Anyway if you carefully read through, you can see the GetAccessPoints method. If I call it, I get a list of accesspoints:

$dbus-send --print-reply --system --dest=org.freedesktop.NetworkManager /org/freedesktop/NetworkManager/Devices/3 org.freedesktop.NetworkManager.Device.Wireless.GetAccessPoints
method return time=1632075788.446527 sender=:1.12 -> destination=:1.3041 serial=431030 reply_serial=2
   array [
      object path "/org/freedesktop/NetworkManager/AccessPoint/2686"
      object path "/org/freedesktop/NetworkManager/AccessPoint/2699"
      object path "/org/freedesktop/NetworkManager/AccessPoint/2700"
      object path "/org/freedesktop/NetworkManager/AccessPoint/2701"
      object path "/org/freedesktop/NetworkManager/AccessPoint/2702"
      object path "/org/freedesktop/NetworkManager/AccessPoint/2703"
      object path "/org/freedesktop/NetworkManager/AccessPoint/2704"
      object path "/org/freedesktop/NetworkManager/AccessPoint/2706"
      object path "/org/freedesktop/NetworkManager/AccessPoint/2708"
      object path "/org/freedesktop/NetworkManager/AccessPoint/2710"
      object path "/org/freedesktop/NetworkManager/AccessPoint/2711"
      object path "/org/freedesktop/NetworkManager/AccessPoint/2712"
      object path "/org/freedesktop/NetworkManager/AccessPoint/2713"
      object path "/org/freedesktop/NetworkManager/AccessPoint/2714"
      object path "/org/freedesktop/NetworkManager/AccessPoint/2715"
      object path "/org/freedesktop/NetworkManager/AccessPoint/2716"
      object path "/org/freedesktop/NetworkManager/AccessPoint/2717"
   ]

This is not very shell friendly, but I can persist. For example, if I examine one of the APs, I get:

$gdbus introspect  --system --dest org.freedesktop.NetworkManager   --object-path /org/freedesktop/NetworkManager/AccessPoint/2714
node /org/freedesktop/NetworkManager/AccessPoint/2714 {
  interface org.freedesktop.DBus.Properties {
    methods:
      Get(in  s interface_name,
          in  s property_name,
          out v value);
      GetAll(in  s interface_name,
             out a{sv} properties);
      Set(in  s interface_name,
          in  s property_name,
          in  v value);
    signals:
      PropertiesChanged(s interface_name,
                        a{sv} changed_properties,
                        as invalidated_properties);
    properties:
  };
  interface org.freedesktop.DBus.Introspectable {
    methods:
      Introspect(out s xml_data);
    signals:
    properties:
  };
  interface org.freedesktop.DBus.Peer {
    methods:
      Ping();
      GetMachineId(out s machine_uuid);
    signals:
    properties:
  };
  interface org.freedesktop.NetworkManager.AccessPoint {
    methods:
    signals:
      PropertiesChanged(a{sv} properties);
    properties:
      readonly u Flags = 0;
      readonly u WpaFlags = 0;
      readonly u RsnFlags = 0;
      readonly ay Ssid = [0x48, 0x50, 0x2d, 0x50, 0x72, 0x69, 0x6e, 0x74, 0x2d, 0x33, 0x31, 0x2d, 0x4f, 0x66, 0x66, 0x69, 0x63, 0x65, 0x6a, 0x65, 0x74, 0x20, 0x36, 0x36, 0x30, 0x30];
      readonly u Frequency = 2412;
      readonly s HwAddress = 'xx:xx:xx:xx:xx:xx';
      readonly u Mode = 2;
      readonly u MaxBitrate = 54000;
      readonly y Strength = 0x31;
      readonly i LastSeen = 2602413;
  };
};

That looks interesting. The ssid, which is a byte array is a property. The way to read properties is with the property get method which is present, so you have to call that. Note it takes two arguments, both strings, one which is the interface name, the other being the method, so you have to specify those. And querying AP number 2714 gives:

 $dbus-send --print-reply --system --dest=org.freedesktop.NetworkManager /org/freedesktop/NetworkManager/AccessPoint/2714 org.freedesktop.DBus.Properties.Get string:org.freedesktop.NetworkManager.AccessPoint string:Ssid
method return time=1632076552.174223 sender=:1.12 -> destination=:1.3082 serial=431816 reply_serial=2
   variant       array of bytes "HP-Print-31-Officejet 6600"

Oh look, one of my neighbours has one of the worse printers ever made. I had one of those printers.

Python provides a reasonable library for doing such things. Here’s some code which iterates over all available network interfaces and prints the SSID of whichever access points it finds:

import dbus
bus = dbus.SystemBus()

obj = bus.get_object('org.freedesktop.NetworkManager', '/org/freedesktop/NetworkManager')
network_manager = dbus.Interface(obj, 'org.freedesktop.NetworkManager')

#Iterate over all devices
for device in network_manager.GetDevices():

    #Get the wireless interface for each device
    obj = bus.get_object('org.freedesktop.NetworkManager', device)
    wlan =  dbus.Interface(obj, 'org.freedesktop.NetworkManager.Device.Wireless')
    
    #Note we don't get an error until we attempt to use the interface
    #I suspect there is a better way
    try:
        for ap_path in  wlan.GetAccessPoints():
            # Read the SSID property
            obj =  bus.get_object('org.freedesktop.NetworkManager', ap_path)
            ap_props =   dbus.Interface(obj, 'org.freedesktop.DBus.Properties')
            ssid = ap_props.Get('org.freedesktop.NetworkManager.AccessPoint', 'Ssid')
            print(''.join([str(v) for v in ssid]))

    except dbus.exceptions.DBusException as e:
        pass

So that’s it for the basics. There’s a whole introspection API as well which I believe returns the structure as XML.

But to call methods, introspection isn’t needed.

Writing a service

Writing a service is pretty easy in Python. Here’s an example where dbus doesn’t steal the entire main loop. Note that this runs in the sesson bus, not the system one, because of security.

from gi.repository import GLib
import time

class Service(dbus.service.Object):
   def __init__(self):
      #Register dbus with GLib's main loop
      dbus.mainloop.glib.DBusGMainLoop(set_as_default=True)
      
      #Bus name 
      bus_name = dbus.service.BusName("com.hello.helloworld", dbus.SessionBus())

      #Object to export
      dbus.service.Object.__init__(self, bus_name, "/")

      self._count=0

   #Register two methods to the one interface
   @dbus.service.method("com.hello.helloworld.Message", in_signature='', out_signature='s')
   def get_message(self):
      self._count+=1
      return "Hello, world " + str(self._count)
    
   @dbus.service.method("com.hello.helloworld.Message", in_signature='i')
   def set_counter(self, i):
      self._count = i
    
   #A way of polling the main loop so that GLib doesn't 
   #steal the program's main loop
   def poll(self):
      loop = GLib.MainLoop()
      def quit():
          loop.quit()
      GLib.idle_add(quit)
      loop.run()

#Instantiate the service
service = Service()

#Poll it
while True:
    service.poll()
    time.sleep(.1)

And it works:

~ $qdbus --session com.hello.helloworld / get_message
Hello, world 1
~ $qdbus --session com.hello.helloworld / get_message
Hello, world 2
~ $qdbus --session com.hello.helloworld / get_message
Hello, world 3
~ $qdbus --session com.hello.helloworld / get_message
Hello, world 4
~ $qdbus --session com.hello.helloworld / get_message
Hello, world 5
~ $qdbus --session com.hello.helloworld / set_counter 0

~ $qdbus --session com.hello.helloworld / get_message
Hello, world 1
~ $qdbus --session com.hello.helloworld / get_message
Hello, world 2

qdbus is often a lot less verbose to use!

Writing a system service

If you try to run with the system bus, it will fail, even if you run as root. This is because of the security policy in place. The policies allow non-root daemons to run as system services, and no one special cased it to make root ones always allowed, which is fine.

The policies are in /etc/dbus-1/system.d/ and they’re sort of understandable, but only sort of. It is documented: essentially it’s default deny with allow/deny rules applied top to bottom based on a matching scheme. In order to run the service as root, the following file works:

<!DOCTYPE busconfig PUBLIC
 "-//freedesktop//DTD D-BUS Bus Configuration 1.0//EN"
 "http://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd">
<busconfig>

  <!-- Only root can own the service -->
  <policy user="root">
    <allow own="com.hello.helloworld"/>
  </policy>

  <policy context="default">
    <allow send_destination="com.hello.helloworld"/>
  </policy>
</busconfig>

essentially this allows only root to own the service, but allows anyone (the default context) to send messages. One the file is created, you then need to reload DBus which can be done with SIGHUP or with systemctl reload debus on systemd based systems.

Starting a system service with systemd

I could do it manually, but perhaps I should bend to the winds of change.

This is simple, for this rather basic service. First, add #!/usr/bin/env python3 to the first line of the script and make it executable. Then copy it to /opt/helloservice/service.py. Then add a very simple unit file to /etc/systemd/system/hello.service:

[Unit]
Description=Hello world service

[Service]
ExecStart=/opt/helloservice/service.py

[Install]
WantedBy=multi-user.target

Is it me or is it weird that the old Windows 3 style .INI files have become a perverse sort of standard?

The future..

Anyway, now a simple:

sudo systemctl daemon-reload
sudo systemctl start hello

starts the service. And to enable it on boot:

sudo systemctl enable hello

which apparently symlinks it in a directory.

And that’s it

A new system service, running as root controllable by a user. The purpose is to have some NeoPixels on an Raspberry Pi controlled as a user (the pixels can only run as root due to the need to access low level hardware).