How to run fwupd tests with Moblab

This howto shows how to run the fwupd test suites on Moblab to verify the firmware updatability of peripherals in ChromeOS.


fwupd is a system daemon that allows an OS to update the firmware of a wide array of peripherals. ChromeOS relies on it to perform the firmware updates of compatible and updatable devices.

Moblab contains a set of fwupd test suites to test the basic firmware-related operations that fwupd can perform on a device. The purpose of these tests is, primarily, to validate the correct firmware updatability of new peripherals so that they comply with the WWCB Certification and to check the consistency and correctness of these operations across different ChromeOS versions and firmware releases.

Before you begin

To run the fwupd tests you’ll need a working Moblab setup with at least one DUT (a Chromebook or Chromebox) running a ChromeOS test image.

How to get a Partner Domain account

In order to have access to ChromeOS test releases you’ll need a Partner Domain account. To ask for an account, send a request to


Besides the Partner Domain account, you should also get access to a CPCon account, a Service account and a GCS bucket tied to the CPCon account.

How to ask for access to ChromeOS images for specific boards

Depending on the Chromebooks and Chromeboxes that you use for the tests, you’ll need to have explicit access to the ChromeOS test images for each specific model or board type so you can download them from the ChromeOS Partners Portal.

To request access, create an issue in the Partner Issue Tracker using Component “ChromeOS Public Tracker > Services > Infra > Moblab” (1038089) and template “Build Permission Access Request” and fill in the template details.

How to get a Moblab

Reference: Moblab Instruction Manual section “II - Requirements”

Moblab is based on a Google Chromebox (Wukong or Wyvern). The distributor of Moblab is CREATE TOGETHER TECHNOLOGY Co. Ltd. To inquire a quotation, reach out to:

Please note, Moblab is not supported in mainland China, nor has it been certified by China CCC.

Required hardware

  • One or more DUTs (Chromebook or Chromebox)

  • Network equipment as described in section “II - Requirements” of the Moblab Instruction Manual, at least two Ethernet cables and a USB to Ethernet dongle. Follow the instructions in section “II.2 - Now that I have all the required hardware, how do I connect them?” in the Moblab Instruction Manual to setup the test lab

  • A USB flash drive larger than 8GB

  • An hdmi or displayport monitor, a keyboard and a mouse

  • If the DUT is a Chromebox: an additional monitor, keyboard and mouse

Required software

Chrome Recovery Utility extension installed in Chrome.

Initial DUT (Chromebook) setup

Main references:

In order to use a Chromebook or Chromebox for Moblab tests it needs to be running a ChromeOS test release. Before continuing, make sure you have a partner domain account with access to the ChromeOS Partners Portal and to the ChromeOS images for the specific Chromebook/Chromebox board types you want to test.

Flash a ChromeOS test Image

First, start by flashing a ChromeOS test image into a USB flash drive:

  1. Go to the ChromeOS Partners Portal <> and log in with your partner domain account and click the “Image Files” tab.

  2. Select the board of the Chromebook or Chromebox to test and TEST_IMAGE_ARCHIVE in the “Image Type” drop-down menu, and click “Search” to list all the ChromeOS test images for that board. You can refine the search by entering a specific “Release/Milestone” and/or a specific “Version/prefix”.

  3. Download a recent test image (dev or stable channels recommended) and decompress it with:

    $ tar -Jxvf <release_file.tar.xz>
  4. Click the extension extension-icon button in Chrome, then select Chrome Recovery Utility

  5. Select the gear icon gear-icon in the window. Next, click Use local image.

  6. Select the chromium_test_image.bin file extracted in step 3

  7. Plug in the USB flash drive and select it as the media to use. Click Continue and then Create now. Wait until the image is completely written to the USB drive.

  8. Once complete, Select Done then unplug the USB flash drive


Install test image

Next, to install the image in the Chromebook or Chromebox, follow these steps:

  1. Put the device into Developer Mode with the following procedure:

    1. For Chromebooks, Hold Esc + Refresh refresh-icon and press the Power button. For Chromeboxes, engage the small Reset pinhole with a paperclip, hit Power and continue engaging Reset for 2 seconds. This will put the device into Recovery Mode and it should show a screen similar to this:


      Or this, depending on the model:



      On some Chromebooks the combination to hold is Esc + Full screen fullscreen-icon instead.

    2. In the Recovery Mode screen, press Ctrl + D, followed by Enter to enter Developer Mode


      For other devices without keyboards (such as tablets) follow these instructions to enter Recovery Mode and Developer Mode

    3. Wait until the process is done and the Developer Mode warning screen appears

  2. Once the device is in Developer Mode, it will show the warning screen above every time it boots. It’ll start ChromeOS after 30 seconds or if you press Ctrl + D. Start ChromeOS and wait for it to show the welcome screen

  3. Go to virtual terminal 2 to access a command line prompt by pressing:

    [ Ctrl ] [ Alt ] [ → ]

    where the [ → ] key is the right-arrow key just above the number 3 on the keyboard. If the keyboard doesn’t have this key, use the key in the F2 position. Then log in with user: root

  4. Enable USB boot with the following commands:

    $ sudo crossystem dev_boot_usb=1
    $ sudo crossystem dev_boot_signed_only=0
  5. Now reboot and wait for the Developer Mode warning screen to appear, plug in the USB flash drive and press Ctrl + U to boot the ChromeOS test image from the USB drive.

    Wait for ChromeOS to start

  6. Once ChromeOS is running, go to virtual terminal 2 again and log in with user: root and password: test0000. Then install the test image in the hard disk with the following command:

    $ /usr/sbin/chromeos-install

    and follow the instructions

  7. Once installation has completed, reboot the device (shutdown -h now) and remove the USB flash drive

Moblab setup

Follow the instructions in the Moblab Introduction & User Manual to configure the Moblab, connect the DUT (Chromebook or Chromebox) to it and enroll it. The end result must be something like this, where the “Manage DUTs” tab shows an enrolled DUT with a “Ready” Status:


Other considerations and requirements

About the peripheral status

The peripherals to test must be in working order. If they are meant to be updated wirelessly, they must have sufficient battery level to ensure the firmware update process can be completed successfully. They must also be supported by fwupd and they must have at least a firmware release included in the ChromeOS-specific fwupd remotes, which are defined in the latest .ebuild file in

About the fwupd version and peripheral support

The fwupd version might be different from one ChromeOS version to another, so a device that is supported by fwupd in a newer ChromeOS version might not be supported in an older one.

About DUT provisioning in Moblab

Each Moblab test run will start by provisioning the DUT, that is, updating its ChromeOS version to the one specified by the tester and checking that it can run the tests properly. This has some side effects:

  • The DUT will constantly check for an Ethernet link and will reboot automatically if it doesn’t detect one after a few seconds.

  • All Bluetooth device pairings will be wiped out

A DUT can be also provisioned on demand by selecting the DUT in the “Manage DUTs” tab and clicking on the “Provision DUTs” button, then selecting the milestone and build to use:


Test cases


  • The FWUPD tests are available only on ChromeOS R113-15382.0.0 and later.

  • Tests of firmware updates/downgrades/installs over Bluetooth links are not properly supported at the moment

The procedure to run the tests is the same for all the test cases except for the parameters they take. To start a test, follow these steps:

  1. Make sure the DUT that will run the test is running, connected to the Moblab network and that it shows up in the “Manage DUTs” tab in Moblab

  2. Enroll the DUT for tests by selecting it in the “Manage DUTs” tab and then clicking “Enroll Selected”

  3. Make a note of the DUT IP address, as it’ll need to be specified later as one of the test parameters

  4. Connect the peripheral to the DUT and power it on

  5. Go to the “Run Suite” tab and select “FWUPD” in the top menu

  6. Select the model and build target of the selected DUT, and then the ChromeOS milestone and build to run the test on. Important: only ChromeOS R113-15382.0.0 and later. The tests won’t be started for any version older than that.

  7. Select the IP address of the DUT that will run the test


The rest of the steps depends on the test case to run:

Update a device firmware to the latest release

This test case will try to update a device firmware to the latest release available in the fwupd remotes. To run it:

  1. Select the fwupd_update suite

  2. Select the device you want to test

  3. Click on “Run Suite”



If there aren’t any new firmware releases available for the device, the test won’t proceed and will be marked as an error.

Downgrade a device firmware to the previous release

This test case will try to downgrade a device firmware to the previous release found in the fwupd remotes. To run it:

  1. Select the fwupd_downgrade suite

  2. Select the device you want to test

  3. Click on “Run Suite”


    If there aren’t any previous firmware releases available for the device, the test won’t proceed and will be marked as an error.

Install a firmware version

This test allows the user to flash any available firmware release into a device, regardless of the current version running on it. To run it:

  1. Select the fwupd_install_version suite

  2. Select the device you want to test

  3. Input the release version you want to install. Note that the version must be specified in the format defined by the hardware vendor (as a number pair, triplet, hexadecimal number, etc.). The Device info at the bottom shows the Current FW version with the expected format.

  4. Click on “Run Suite”


Install a firmware file

This test allows the user to flash a specific firmware file into a device. The file can be provided either as a URL if it’s in a remote server, or through an external drive connected to the DUT.

  • If the file is provided in an external drive, a USB flash drive is recommended. Format it as FAT32 and set FWUPDTESTS as its label. Then copy the firmware file to it and plug it to the DUT.

  • If the file is provided as a URL, it must be accessible for the DUT to download.

To run the test:

  1. Select the fwupd_install_file suite

  2. Select the device you want to test

  3. Enter the complete URL of the firmware file if it’s a remote file or the file name if it’s a file provided through a USB flash drive

  4. Click on “Run Suite”


How to verify the test results

Once a test has started, it’ll show up in the “View Jobs” tab. You can enable the “Auto Refresh” toggle switch to keep the job list updated as the job status progress:


When the test finishes, its status will change to “COMPLETE”, the logs will be stored locally and eventually uploaded to a Google Storage bucket. A cloud icon in the “Logs” column means that the logs have been uploaded to Google Storage and are no longer stored locally.

Note that for each test run there’ll be two entries in the jobs table: one representing the test suite (control.fwupd_update in the image above) and another representing the test proper (fwupd_FirmwareUpdate in the image).

Once the job has finished you can check the results. If the logs haven’t been uploaded yet, you can check them directly by clicking on the “Logs” icon for the test job (not the suite). From there, navigate to the “status.log” file:


The “status.log” file shows a summary of the test result:

INFO ---- ---- kernel=5.15.94-16358-gae97cc7d22a2 timestamp=1677059177 localtime=Feb 22 09:46:17
START ---- ---- timestamp=1677059182 localtime=Feb 22 09:46:22
START fwupd_FirmwareUpdate fwupd_FirmwareUpdate timestamp=1677059182 localtime=Feb 22 09:46:22
ERROR fwupd_FirmwareUpdate fwupd_FirmwareUpdate timestamp=1677059183 localtime=Feb 22 09:46:23        No FW releases found for 4a69bff2d096b361b6e8a070a012728aff92538e (Unifying Receiver)
END ERROR fwupd_FirmwareUpdate fwupd_FirmwareUpdate timestamp=1677059183        localtime=Feb 22 09:46:23
END GOOD ---- ---- timestamp=1677059183 localtime=Feb 22 09:46:23

In this case it shows that the FirmwareUpdate test failed because the selected device doesn’t have any FW releases available.

This other case shows a successful run:

INFO ---- ----        kernel=5.15.94-16358-gae97cc7d22a2 timestamp=1677060168 localtime=Feb 22 10:02:48
START ---- ---- timestamp=1677060173        localtime=Feb 22 10:02:53
START fwupd_FirmwareUpdate fwupd_FirmwareUpdate timestamp=1677060173 localtime=Feb 22 10:02:53
GOOD fwupd_FirmwareUpdate fwupd_FirmwareUpdate timestamp=1677060174 localtime=Feb 22 10:02:54        completed successfully
END GOOD fwupd_FirmwareUpdate        fwupd_FirmwareUpdate timestamp=1677060174 localtime=Feb 22 10:02:54
END GOOD ---- ---- timestamp=1677060174 localtime=Feb 22 10:02:54

When the logs are uploaded to Google Cloud Storage, you can check them through CPCon, accessing with your partner domain account and clicking on the “Autotest View” option. From there you can see all the test suites run on every Chromebook type and ChromeOS milestone:


Clicking on any of the test suite results will lead you to a detailed summary of that suite. All the different test runs for that particular Chromebook type and ChromeOS version will show up listed either as a “Non-Passed test” or as a “Passing test”. The most recent test run will be the last log listed:


Clicking on any of the logs will direct you to the Google Storage bucket directory containing the logs for that test run:


From there you can download the “status.log” file. The directory and file structure is the same as in the locally stored logs.

How to get debug information

If the “status.log” report doesn’t give enough information about a failed test, you can also download the “debug/client.0.DEBUG” file, which contains the full log including debug messages. Additionally, the “sysinfo/messages” file contains the full system log that can also be useful to investigate a bug.

For instance, this “install file” test failed with this message in the status.log file:

FAIL fwupd_FirmwareInstallFile fwupd_FirmwareInstallFile timestamp=1678199270
localtime=Mar 07 14:27:50 Command <CACHE_DIRECTORY='/var/cache/fwupd' fwupdmgr
local-install --json --allow-older --allow-reinstall b1f4e48ebea67bc554a73407241469583497b8e6> failed, rc=1,
Command returned non-zero exit status

Which doesn’t tell the reason. Checking the “debug/client.0.DEBUG” file shows the whole fwupdmgr output:

* Command:
      CACHE_DIRECTORY='/var/cache/fwupd' fwupdmgr local-install --json --allow-
      older --allow-reinstall
   Exit status: 1
   Duration: 0.028001070022583008

    "Error" : {
      "Domain" : "FwupdError",
      "Code" : 8,
      "Message" : "No supported devices found"

This means that the file used for the update isn’t compatible with the specified device.


How to find the board type of a Chromebook or Chromebox?

In the Chromebook/Chromebox, open Chrome and enter chrome://version in the URL bar. In the info screen that will appear, the board type and variant will show up in the Platform and Customization ID fields, respectively:


How to log into the DUT through SSH?

Reference: How to SSH to DUT without a password

In some scenarios it could be useful or needed to run certain console commands in the DUT or to retrieve data from it that can only be accessed through a terminal interface. If the DUT is running a test ChromeOS release, it’ll have an SSH daemon running so you can connect to it remotely.


  • A Linux PC with an SSH client installed. If running Ubuntu, it can be installed with the following command: sudo apt install ssh

  • The Linux host must be able to ping the DUT. The easiest way to achieve this is to connect the DUT and the Linux PC to the same local network. If the DUT is connected to a Moblab through a wired connection you can also connect to the Linux PC using a wireless connection

Assuming the host you’re connecting from is running Linux, in order to connect to the DUT you’ll need to download the ChromeOS test keys and configure your ssh client properly following these steps:

  1. Download the SSH keys from this link and copy them to ~/.ssh in the Linux host

  2. Set the correct file permissions for the private key:

    chmod 0600 ~/.ssh/testing_rsa
  3. Get the IP address of the DUT. If the DUT is connected to multiple networks, we need the IP address of the NIC that’s connected to the Linux PC network. To list the available connections and their IP addresses go to virtual terminal 2 in the DUT ([ Ctrl ] [ Alt ] [ ]) and type: ip -4 -br a

    localhost ~ # ip -br -4 a
    lo               UNKNOWN
    wlan0            UP   
    arc_ns0@if2      UP   
    arc_ns1@if2      UP   
    eth0             UP   

    In this example, if the DUT is connected to the Linux PC network through a wireless link, then we can check that the PC can ping the DUT at

  4. Add the following to ~/.ssh/config:

    Host dut
      HostName $IP_ADDRESS
      User root
      CheckHostIP no
      StrictHostKeyChecking no
      IdentityFile ~/.ssh/testing_rsa
      ControlMaster auto
      ControlPersist 3600

    Where $IP_ADDRESS is the IP address of the DUT

How to check the list of peripherals detected by fwupd?

If, for debugging purposes, you need to check the current list of peripherals that fwupd is detecting, you can do so by running this command in a DUT terminal:

fwupdmgr get-devices --json

How to stop the DUT from rebooting automatically

If the DUT was provisioned (updated) using Moblab, it will check for an Ethernet link and it will reboot if it doesn’t find one. Make sure to keep the DUT connected to the Moblab network using an Ethernet link.

When you are done testing with the Chromebook, reflash it with a recovery image to prevent it from restarting continuously when not connected to a network through the Ethernet port.

How to send debug information

In case something goes wrong when launching or running a test, you can get the complete Moblab logs for debug through the Mobmonitor menu:


Then you can either download the logs or send them to your Cloud Storage bucket:


The logs will be compressed as a .tgz file.

Alternatively, if there’s an issue with Moblab you can report it by issuing a buganizer ticket using this template and filling in the details.