Custom Protocol

The fwupd project already supports a huge number of flashing protocols, everything from standardized protocols like NMVe, ATA, DFU and also a large number of vendor-specfic protocols like logitech_hidpp, synaptics_prometheus and wacom_raw.

Most vendors are using a protocol that fwupd already supports, and thus only need to upload firmware to the LVFS. In the case applying for an account is all that is required.

Note

If using DFU, please also implement the DFU runtime interface – this allows fwupd to automatically switch your device into bootloader mode without having to draw some artwork and write some translated text to explain how the user should persuade the device to enter update mode.

The easiest time to add support for updating hardware using the LVFS is during the project prototype phase. There are several things you can do that makes writing a fwupd plugin much easier.

In the case where the device protocol is a non-compatible variant or a completely custom protocol then a new fwupd plugin will be required. If you have to use a custom protocol, there are a few things that are important to consider.

The fwupd daemon needs to be able to enumerate the device without the user noticing, which means LEDs should not blink or cause the screen to flicker. Disconnecting a kernel driver, changing to bootloader mode or any other method of getting the device firmware version is not acceptable. This means the device needs to expose the current firmware version on the runtime interface, for instance using USB descriptors or PCI revision fields.

For composite devices (e.g. docks) it is much better to provide an interface to query the internal device topology rather than hardcoding it in the plugin or in a quirk file. For instance, fwupd could query the root device that would respond that it is acting as a I²C bridge to a HDMI chip with address 0xBE, rather than hardcoding it for a specific dock model. Querying the information allows the plugin author to write a generic plugin that means future devices can be upgraded without waiting for new fwupd versions to be included in popular Linux distributions and ChromeOS.

Warning

Plan and test for what happens when the user disconnects the USB cable, runs out of battery, or removes the mains plug when the new firmware is being flashed.

If the device remains in bootloader mode, is there a unique VID/PID that can be used to choose the correct firmware file to flash the device back to a functional runtime mode?

Many vendors just use the ISV-provided reference bootloader (which is fine), but fwupd does not know which runtime image to recover with if the ISV-allocated generic VID/PIDs are being used. If it is not possible to modify the bootloader VID/PID, then it may be possible to read a block of NVRAM at a harcoded offset to identify the proper firmware to install.

When updating hardware it is important to provide feedback to the user so that they know the process has not hung. Updating firmware is intimidating to many users and so it is important to provide information about what is being done to the hardware, for instance erasing, writing and verifying. It is also a very good idea to provide percentage completion, so for an operation that is going to take 10 seconds it is better to write 1024 blocks of 16kB with percentage updates after each block rather than one block of 16Mb with just a bouncing progressbar.

Note

It is not possible to upload executable flasher code as part of the cabinet archive – only the payload is allowed.

We will not accecpt non-free executables, static libraries or “shim” layers in fwupd. The only way a custom protocol can be supported is by contributing a LGPLv2+ plugin upstream.

Some vendors will have the experience to build a plugin themselves, and some vendors may wish to use a consulting company that has the required experience.

Depending on a new library

Please do not use model-specific or vendor-specific libraries to update or enumerate the hardware. Unless the library is already shipped by default on Ubuntu LTS and RHEL 8 then it is going to be exceptionally hard to use this library in fwupd. As fwupd is in main for Ubuntu then any library it depends on must also be part of main, which means Canonical has to officially support it. They obviously do not want to do this for vendor-specific libraries that have only existed for a few years with no API or ABI guarantees or long term stable branches.

Similarly for Red Hat; any new library or binary needs to have a Red Hat maintainer who will support it for over 10 years (!) and is willing to do the security due diligence and code review required to be included as a core package in RHEL. Getting approval for a new package is a huge amount of work and takes months.

As fwupd is running as root, any external library it depends on must be audited by several security teams, and have a proven security plan in place. Google also needs to review any new dependencies as fwupd is also being used heavily in ChromeOS now, and they take OS image size and security very seriously.

In this situation it is completely okay to include parts of the open source library (assuming the code can be licensed as LGPLv2+) in the fwupd plugin. Including them in fwupd plugins also allows us to use the fwupd helper functionalily, for instance replacing memcpy() with fu_memcpy_safe() and using high level abstractions for reading and writing to sysfs files or an ioctl. Many plugins already do this, for instance the colorhug plugin does not use libcolorhug, nvme plugin does not use the nvme command line tool and the emmc plugin itself defines EXT_CSD_FFU rather than depending on mmc-utils.

Building fwupd

These instructons below can either be used by the silicon vendor, or the consulting company to debug existing and new plugins. Sometimes new hardware is only supported in the development version of fwupd which may not even be available as a Snap or Flatpak yet.

Prerequisites

• A PC with Linux (preferably the latest version of Fedora) installed bare metal (i.e. not in VirtualBox or VMWare)
• Working access to the internet
• A user account (we’ll use emily as the example here) with administrator permissions

$ cd ~
$ sudo dnf -y builddep fwupd
$ git clone https://github.com/fwupd/fwupd.git
$ cd fwupd
$ OS=fedora ./contrib/ci/generate_dependencies.py | xargs sudo dnf install -y
$ mkdir build && cd build
$ meson ../ -Dsystemd_root_prefix=/tmp -Dudevdir=/tmp --prefix=/home/emily/.root
$ ninja
$ ninja install
$ ./src/fwupdtool --verbose get-devices

Using fwupdtool

The fwupd project is split into three main components:

• fwupd: The binary that’s running in the background, as root
• fwupdmgr: The client tool that end-users use to interact with the running fwupd binary, as a normal user
• fwupdtool: The debugging tool developers use to find problems and to run new code, as root

The fwupdtool binary does most of the things that fwupdmgr does, but without talking to the system fwupd instance. It is a lot easier to run fwupdtool with just one plugin (e.g. --plugin-whitelist vli) than running the daemon and all the plugins. You might have to wait 5 seconds and then read thousands of lines of debugging to see the printf() you added in a new plugin with the daemon, but with ./fwupdtool --plugin-whitelist vli --verbose get-devices it’ll be in a few lines, and instant.

Google actually decided to use fwupdtool in ChromeOS rather than having a daemon process running all the time which is why it used to be a not-installed-by-default debug-tool and now is installed into /usr/libexec/fwupd/ and has translations.

To get the list of devices from one specific plugin I would do:

sudo /usr/libexec/fwupd/fwupdtool --plugin-whitelist vli get-devices --verbose
this outputs lots of text onto the console like:
10:51:49:0584 FuMain               Lenovo ThinkPad WS Dock
 DeviceId:             73ef80b60058b4f18549921520bfd94eaf18710a
 Guid:                 dd1f77bd-88ef-5293-9e34-1fe5ce187658 <- USB\VID_17EF&PID_305A&REV_5011
 Guid:                 1c09a12d-e58a-5b4d-84af-ee3eb4c3c68b <- USB\VID_17EF&PID_305A
 Guid:                 6201fecc-1641-51f6-a6d2-38a06d5476bf <- VLI_USBHUB\SPI_C220
 Guid:                 c9caa540-6e27-5d40-a322-47eaeef84df0 <- USB\VID_17EF&PID_305A&SPI_C220&REV_5011
 Guid:                 cfa1e12c-4eb9-5338-8b23-02acc5423ccb <- USB\VID_17EF&PID_305A&SPI_C220
 Summary:              USB 3.x Hub
 Plugin:               vli
 Protocol:             com.vli.usbhub
 Flags:                updatable|registered|can-verify|can-verify-image
 Vendor:               LENOVO
 VendorId:             USB:0x17EF
 Version:              50.11
 VersionFormat:        bcd
 Icon:                 audio-card
 InstallDuration:      10
 Created:              2019-12-20

We could then install the raw firmware blob (i.e. not the cabinet archive with metadata) on the device using:

sudo /usr/libexec/fwupd/fwupdtool --verbose --plugin-whitelist vli \
 install-blob /home/emily/the-firmware.bin 73ef80b60058b4f18549921520bfd94eaf18710a

Firmware Parsing

You can also parse the raw .bin files using fwupdtool which has access to all the available firmware parsers built into all plugins. For example:

sudo ./src/fwupdtool firmware-parse /home/emily/VL105_APP6_8C_09_08_06_20190815.bin
Choose a firmware type:
0. Cancel
1. conexant
2. 8bitdo
3. synaprom
4. rmi
5. wacom
6. vli-pd
7. raw
8. altos
9. srec
10. ihex
11. vli-usbhub
12. vli-usbhub-pd
12<enter>
FuVliUsbhubPdFirmware:
Version:                 140.9.8.6
ChipId:                  VL105
VID:                     0x2109
PID:                     0x105
 FuFirmwareImage:
 Data:                  0xc000

Using fwupdmgr

You can perform the end-to-end tests using a local version of fwupd by first calling ninja install to get the new plugin installed. Then you’ll need two terminals open. In the first do:

./src/fwupd --verbose

and in the second you can do:

./src/fwupdmgr install VL105.cab

This will send the firmware archive from the locally built fwupdmgr to the locally built daemon using a file descriptor, which will call the new plugin code with the firmware blob in the archive. The daemon terminal will also show lots of useful debugging during this process.