Updating firmware on devices is traditionally difficult for users on Linux systems.
Not knowing exact hardware details, where to look for updates or how to run the
Windows-specific flashing tools makes it almost impossible to update firmware on
devices.
As a result, “broken” hardware is being returned to the vendor and customer
systems are left in an insecure state even when updates have been released that
fix the specific issues. Linux as the OS is now mainstream and vendors need to
support these customers.
The LVFS is a secure web service that can be used by OEM’s to upload firmware
archives and can also be used by users to securely download metadata about
available updates and optionally, the updates themselves.
Millions of customer devices are being updated every month thanks to the LVFS!
Linux users have traditionally had problems with keeping hardware up to date
with firmware updates.
There are three components to this problem:
They do not know what exact hardware they have installed, the current
firmware version, or even if the devices support being upgraded at all.
They do not know where to look for updates; often searching the various
vendor websites is an exercise in frustration and as a result most users do not bother.
Windows-specific flashing tools do not work on Linux; a significant number of
Linux users keep a Windows virtual machine for essential business-critical
software that is not available on Linux.
This will not work for firmware update utilities that require low level hardware access.
The fwupd project can query supported hardware for the current firmware versions
and also deploy new firmware versions to devices, but requires metadata from the
LVFS to know the details about available updates.
It also requires vendors to prepare the firmware with the required metadata and
to use a standardized deployment framework e.g. DFU or UEFI UpdateCapsule.
Using the information from from higher level software centers can show the user the update
description in their own language and offer the update to be installed using
just three clicks of the mouse.
Security updates are handled in the same way as other OS updates meaning it is
just one mechanism for the user to understand.
The LVFS supplies the data in a secure format, allowing the fwupd project to
install the update safely.
Existing approaches have been OEM specific which meant that a large amount of
engineering effort was required, making this approach only financially viable
for enterprise use-cases.
There are a significant number of legal problems with the redistribution of firmware,
and we have been working with vendors finding acceptable methods of redistribution
whilst ensuring confidentially throughout the process. Being backed by a large
Linux vendor with heterogeneous support for many vendors and platforms puts the
LVFS in exactly the right place to build this kind of shared infrastructure.
The architecture is built into three layers: a presentation layer, a mechanism
and a data-provider and each can be replaced as required as they all use standard
protocols.
GNOME Software is an application store
designed to make installing, removing and updating both easy and beautiful.
It is available for Linux and used by millions of people on the following
distributions:
RHEL and CentOS 7.4 or newer
Fedora 22 or newer
Ubuntu 16.04 (Xenial) or newer
Debian 9 (Stretch) or newer
openSUSE 15.0 or newer
Arch from 2017-06-13
For most desktop systems, at session start-up the metadata XML and detached
signatures are checked for a specified age, and if required newer files are
automatically downloaded from the LVFS and pushed into fwupd over D-Bus.
When the update list is required we query the fwupd daemon over D-Bus for any
pending updates.
If there are updates that need applying then they are downloaded and the user is
notified and the update details are shown in the specified language.
The user has to explicitly agree to the firmware update action before the update
is performed.
This project provides a system-activated daemon fwupd with a D-Bus interface
that can be used by unprivileged clients.
Clients can perform system wide upgrades and downgrades according to a security
policy, which uses PolicyKit to negotiate for authorization if required.
The command line tool fwupdmgr can be used to administer headless clients on
the command line over SSH or using a management framework like Red Hat Satellite
or Dell CCM.
The daemon parses metadata in AppStream
format from the LVFS along with a detached GPG or PKCS#7 signature.
The .cab archives which must contain at least a .metainfo.xml file and a detached
GPG/PKCS#7 signature of the firmware payload.
Other files are permitted in the archive which allows the same deliverable to be
used for the Windows Update system.
Internally fwupd creates a device with a unique ID, and then a number of
GUIDs are assigned to the device by the plugin.
It is these GUIDs specified in the update metadata file that are used to match
a firmware file to a device.
Although it is usually the responsibility of the system vendor to generate a new
GUID if the hardware requires a different firmware file, we can match an update
that only applies to specific versions of hardware using
CHID GUIDs.
Adding more plugins to fwupd is of course possible, but where possible vendors should use
the existing code and for instance add an ESRT data table when building the system firmware.
When the user agrees to a UEFI firmware update the firmware is unpacked into the
EFI System Partition, several UEFI keys are set and the system reboots.
On reboot the fwupd.efi binary is run before the bootloader is started and
the firmware UpdateCapsule UEFI runtime source is called.
For most devices (e.g. USB, Thunderbolt, Synaptics, etc.) the update is
performed without requiring a reboot.
The LVFS provides an OEM-facing website that requires a username and password
to access the secure console.
There is no charge to vendors for the hosting or distribution of content,
although there are some terms of service to vendors distributing content.
This service should only be used to distribute firmware that is flashed onto
non-volatile memory.
It is not designed for firmware that has to be uploaded to devices every time
the device is used.
When .cab firmware files are submitted the following actions are performed:
The update metadata in the archive is checked.
The firmware capsule is signed with our GPG key or PKCS#7 certificate.
Clients do not verify the signatures in the catalog file as this is
for Windows Update only
The new cab file is repacked. Only required files are included in the cabinet
file, typically making the download size much smaller
The metadata is added to our database.
Many ODMs are distinct and decoupled from the OEM, and in most cases the ODM is
allowed to upload new firmware but not make it available for users.
For this use case, users on the LVFS can have different attributes, for example:
Unprivileged users that can upload files to the testing target
Read only access to all analytics data for a specific vendor
Quality assurance users that can modify all firmware uploaded to a specific vendor
Trusted users that can move files to the testing or stable target, and can
move files from testing to stable
Manager users that can add new users to an existing vendor
The LVFS has grown to be an essential part of the Linux ecosystem used by
over one hundred vendors, 15 of which are multi-billion dollar companies.
The LVFS is a mature service providing important functionality for Linux users.
Various vendors are working on custom plugins for fwupd as they either cannot
retrofit older hardware with the ESRT data table, or because they want more
control over the low level flashing protocol.
We certainly would encourage any new vendors wanting to use the LVFS and fwupd
to use a well-known standard like DFU or UEFI UpdateCapsule with ESRT as it
means there is no application code to write.
From a system administrators point of view, it will also soon be possible to get
notified of updates and perform upgrades using the Cockpit framework as well as
the usual client tools.
The Dell Repository Manager
allows you to update the firmware on various models of Dell enterprise hardware.
There are several software (e.g. the SSU and SBUU) and hardware elements
specific to Dell (e.g., the LCC or USC) and most of the stack is proprietary.
Microsoft provides a service called Windows Update
which takes driver updates from vendors, optionally performs some quality
control on the update, signs the firmware and then hosts the firmware on a CDN.
The entire stack is proprietary and for Microsoft Windows only.
There is no charge to vendors for opening an account or for distribution of content.
You can start the process by opening a ticket
with as much information you have, or just with questions or for more details.
The domain used for email address assigned to this vendor, e.g. @realtek.com,@realtek.com.tw
The update protocol are you using, and if it is already supported in fwupd
Legal permission that you have the required permission to upload to the LVFS. There is an
example document which
can be modified, signed, and uploaded as an attachment to the GitLab issue.
We can create a vendor account without this, but the account will not be able to push firmware to
the public remotes until this document is provided.
The VendorID for all hardware uploaded by this vendor (from fwupdmgrget-devices e.g. USB:0x046D)
The reverse DNS AppStream ID namespace prefix you plan to use for all uploaded firmware, e.g. com.hp
The URL to use for any possible security incident response (PSIRT), e.g. https://www.vendor.com/security
An assigned “vendor manager” that can create new accounts on the LVFS in the future, and be the primary point of contact
If you going to be acting as an ODM or IHV to another vendor, e.g. uploading firmware on their behalf
If you are acting as an ODM or IHV to another vendor:
Which OEM(s) will you be uploading for?
Do you have a contact person for the OEM? If so, who?
Will you be QAing the update and pushing to stable yourselves, or letting the OEM do this?
Note
If you wish for the ticket to remain private (only viewable by the LVFS administrators)
you must mark it as confidential as otherwise the ticket is viewable by public users:
Note
Vendors who can upload firmware updates are in a privileged position where files
can be installed on end-user systems without authentication.
This means we have to do careful checks on new requests, which may take a few
days to complete.
On the LVFS there are several classes of user that can be created.
By default users are created as upload only which means they can only view
firmware uploaded by themselves.
Users can be promoted to QA users by the vendor manager so that they can see
(and optionally modify) other firmware in their vendor group.
QA users are typically the people that push firmware to the testing and stable
remotes.
There can be multiple vendor groups for large OEMs, for instance an OEM might
want a storage vendor group that is isolated from the BIOS team.
Alternatively, vendors can use Azure to manage users on the LVFS.
Contact the LVFS administrator for more details if you would like to use this.
The vendor manager can add users to an existing vendor group.
If the vendor manager has additional privileges (e.g. the permission to push to stable)
then these can also be set for the new user.
New users have to match the username domain glob, so if the value for the vendor
is @realtek.com,@realtek.com.tw then dave@realtek.com.tw could be added by
the vendor manager – but dave@gmail.com would be forbidden.
Vendor groups are created initially as untrusted which means no users can
promote firmware to testing and stable.
Once a valid firmware has been uploaded correctly and been approved by someone
in the the LVFS admin team we will unlock the user account to the trusted
state which allows users to promote firmware to the public remotes.
Note
In most cases we also need some kind of legal document that shows us that
the firmware is legally allowed to be redistributed by the LVFS.
For instance, something like this is usually required:
<vendor> is either the sole copyright owner of all uploaded firmware,
or has permission from the relevant copyright owner(s) to upload files to
Linux Vendor Firmware Service Project a Series of LF Projects, LLC (known as the “LVFS”)
for distribution to end users.
<vendor> gives the LVFS explicit permission to redistribute the
unmodified firmware binary however required without additional restrictions,
and permits the LVFS service to analyze the firmware package for any purpose.
<signature>, <date>, <title>
Some firmware may contain binary code that has been deemed subject to some kind
of export control.
The exact meaning of export control has been defined in various places,
including Export Administration Regulations (EAR), and International Traffic
in Arms Regulations (ITAR).
Code capable of strong encryption like AES, RSA or 3DES may be subject to
export control and it may be forbidden to distribute to users located in specific
embargoed countries like Cuba, Iran, North Korea, Sudan or Syria.
Note
Although there is a specific and notable export exception for “software updates”
in EAR, it should of course be the decision of the legal team of the OEM to
make the decision themselves.
The list of countries is usually specified per-vendor which means it is applied
for all firmware in the vendor account.
It can also be specified per-firmware, which might be useful where just one
specific firmware is explicitly covered under export control, for instance
for a model only designed to be sold to the US military.
This can be specified in the metadata block for the firmware component:
Only LVFS admin team and vendor manager can edit the vendor export control list.
It is specified according to ISO3166, which would typically be CU,IR,KP,SD,SY
for most large vendors.
Note
Like all other services hosting files, the LVFS uses GeoIP data to identify
which country the user is downloading files from.
This is not a perfect science, and although the assigned list of IP blocks is
updated daily some false positives and false negatives can occur.
Legal teams of vendors sometimes request that we make the end user agree to a
license agreement or legal declaration before deploying the update.
There are several reasons why have chosen to not support EULAs:
The majority of updates applied in the enterprise are done “unattended” and
also done at scale with thousands of devices.
Forcing the end-user to do any interactive action makes these automated or
“headless” updates impossible.
Allowing other users to “pre-accept” the end-user license agreement isn’t
what this legal mechanism was designed for – for example is it legally binding
if the junior sysadmin accepts the agreement on the end-users behalf?
Or does it have to be accepted by someone from the destination legal team with
the authority to do so – which needs to be recorded for audit purposes.
Vendors often want to use a “generic” boilerplate legal agreement that controls
how the user is allowed to use the hardware using overly broad language that is
either not applicable to the device, totally confusing to the end user, or by
adding restrictions on an already purchased product.
Vendors often want to show a EULA so that if broken firmware gets deployed then
it becomes the users fault for attempting the upgrade action and the vendor
cannot be considered responsible in any way.
This isn’t fair to customers – risky or untested updates should never be
pushed to millions of end users.
LVFS is used all over the world, and users might not even understand the
language the EULA is written in.
Legal jurisdictions also differ between the nations of this world, and the EULA
might not be legally binding or permissible.
We’ve been asked to add support for EULAs a few times and the answer has always been no.
The almost-universal consensus from the community was that allowing EULAs is a terrible
idea that would be a slippery slope, encouraging vendors to take the easy option and
show pages of overly restrictive boilerplate legalese for each update.
If your legal department disagrees, please let them know that every vendor shipping
firmware on the LVFS has agreed that a EULA was not actually required.
Note
The UI can show the release notes and an optional update message, but it is purely
advisory and the user is free to ignore or suppress it – by disabling the
condition in the source code or even patching the binary executable.
The front-end client (e.g. GNOME Software or Google Chrome) also has no
requirement to implement showing either.
This UI was not designed for EULA text and should not be used in this way.
We typically only allow the silicon vendor, the ODM or the OEM to upload firmware
for hardware, and only if that entity has legal permission to upload the file to
the LVFS.
The security model for fwupd relies on standardized registries like USB and PCI,
along with immutable DMI information to ensure that only the correct vendors
can ship firmware for their own hardware, and nothing else.
This strict rule breaks down where the OEM responsible for the hardware considers
the device end-of-life and so will no longer receive updates (even for
critical security issues).
There may also be a situation where there exists an alternate (not provided by
the vendor) free software re-implementation of the proprietary firmware, which
may be desired for licensing reasons.
In these situations we allow another legal entity to also upload firmware for the
hardware, but with a few restrictions:
The user must manually and explicitly opt-in to the new firmware stream,
perhaps using fwupdmgrswitch-branch, with a suitable warning that there
is no vendor support available and that the hardware warranty is now invalid.
This means that the alternate firmware must set the device branch
appropriately without any additional configuration.
The alternate firmware must not ship with any code, binaries or generated
assets from the original hardware vendor (perhaps including trademarks) unless
written permission is provided in writing by the appropriate vendor.
Some real world examples might be providing a Open Source BCM57xx GPL firmware
for Broadcom network hardware, or providing a coreboot system firmware for a
long-EOLed Lenovo X220 ThinkPad.
In this instance, the LVFS may be the legal entity distributing the firmware,
which is actually provided by a trusted contributor who has permissions to upload
and hardware to test the update.
In other cases another legal entity (like coreboot itself) or an individual
trusted contributor may be considered the distributor.
In all cases the specifics should be discussed with the LVFS maintainers,
as should any concerns by licensors or existing distributors.
Note
It is insanity to throw a perfectly working machine into landfill just because
it’s considered EOL by the original hardware vendor and no longer receiving
security updates.
If we can help provide alternate safe firmware, these machines then provide
inexpensive access for education and employment for those otherwise unable to
afford devices.
The .metainfo.xml file describes the device and firmware and
is extra metadata added to the firmware archive by the OEM or ODM.
The file is XML format, and uses a subset of the
AppStream component specification.
An example metainfo.xml file looks like this:
<?xml version="1.0" encoding="UTF-8"?><!-- Copyright 2018 Richard Hughes <richard@hughsie.com> --><componenttype="firmware"><id>com.hughski.ColorHugALS.firmware</id><name>ColorHugALS</name><name_variant_suffix>BlackFridaySpecialEdition</name_variant_suffix><summary>FirmwarefortheHughskiColorHugAmbientLightSensor</summary><description><p>UpdatingthefirmwareonyourColorHugALSdeviceimprovesperformanceand
addsnewfeatures.
</p></description><provides><firmwaretype="flashed">84f40464-9272-4ef7-9399-cd95f12da696</firmware></provides><urltype="homepage">http://www.hughski.com/</url><metadata_license>CC0-1.0</metadata_license><project_license>proprietary</project_license><releases><releaseurgency="high"version="3.0.2"date="2017-02-09"install_duration="120"><checksumfilename="my-custom-name.bin"target="content"/><description><p>Thisstablereleasefixesthefollowingbugs:</p><ul><li>FixthereturncodefromGetHardwareVersion</li><li>ScaletheoutputofTakeReadingRawbythedatasheetvalues</li></ul></description><issues><issuetype="cve">CVE-2016-12345</issue><issuetype="cve">CVE-2017-54321</issue><issuetype="dell">DSA-2020-321</issue><issuetype="intel">INTEL-SA-54321</issue><issuetype="intel">INTEL-TA-12345</issue><issuetype="lenovo">LEN-28775</issue><issuetype="vince">257161</issue></issues></release></releases><!-- we can optionally restrict this update to specific fwupd versions, or even previous firmware or bootloader versions --><requires><idcompare="ge"version="0.8.0">org.freedesktop.fwupd</id><firmwarecompare="ge"version="0.1.2"/><firmwarecompare="ge"version="0.3.4">bootloader</firmware></requires><custom><valuekey="LVFS::VersionFormat">example</value><valuekey="LVFS::UpdateProtocol">org.acme.example</value></custom><!-- these keywords are optional and are used for searching --><keywords><keyword>bios</keyword><keyword>dfu</keyword></keywords></component>
GUID means ‘Globally Unique Identifier’ and is a 128-bit integer number used
to identify a device.
GUIDs are often formatted as strings such as 84f40464-9272-4ef7-9399-cd95f12da696.
Another name for GUID is UUID (‘Universally Unique Identifier’) and the two
terms can be used interchangeably.
When using GUIDs on the LVFS they should always be lowercase.
In fwupd the GUID is generated from the DeviceInstanceId strings,
so for a single USB device the GUIDs would be generated like this:
You also can use the online generator to manually
convert Instance IDs to GUIDs.
Having multiple GUIDs for each device allows the vendor to choose the GUID for what
should match; to match on the vendor+product+revision you’d choose the first one,
and the vendor+device you would use the second.
We only really use the third GUID for fixing a vendor name, or other very
broad quirks that apply to all USB devices from a specific vendor.
In the case for PCI devices and other technologies like NVMe, you can dump
the GUIDs generated by fwupd using this tool:
The AppStream <id> has to be unique for each device firmware stream as it
used to combine separate <release> tags in the .metainfo.xml files
into the metadata catalog that is downloaded by end users.
Choosing the correct AppStream ID is thus very important for correct operation
of the front end tools.
Firstly, the AppStream ID should have a lowercase prefix that matches the
reverse-DNS name of your vendor, similar to Java.
For instance, appropriate prefixes would be com.lenovo… or org.hughski….
The ID should also contain the model type, and perhaps also the module that is
being updated if there are (or will be) multiple updates for the same hardware.
For instance, we would build the ID further into org.hughski.ColorHug2.BIOS….
The ID should always have a suffix of .firmware, which means the finished
AppStream ID for this hardware would be org.hughski.ColorHug2.BIOS.firmware
Note
The ID has to be totally specific to the GUIDs used to match the device.
For hardware that uses a different firmware stream it is important that the
AppStream ID does not match existing firmware with the same ID.
The LVFS will warn you if you try to upload firmware with the same ID and
different sets of GUIDs.
Including the mode name is just convention; you can use the partial GUID appended
if this helps, e.g. com.hughski.ColorHug84f40464.firmware
Warning
Never include forward or backwards slashes in the ID.
By telling the LVFS the firmware category to use for the component the front end can
correctly translate the update type in the UI.
Also for this reason, .metainfo.xml files should not
include the words ME, EC, BIOS, Firmware, Device or Update
in the component name and they will be removed if included.
The component category can be set as part of the metainfo.xml file
or set from the LVFS web console.
Most users will want to include the extra metadata to make the upload process
quicker for QA engineers.
To do this, add this to the metainfo file:
The LVFS needs to know what protocol is being used to flash the device.
The protocol value is used to provide information about the security of the
firmware update to end users.
The update protocol can be set as part of the metainfo.xml file
or set from the LVFS web console.
Most users will want to include the extra metadata to make the upload process
quicker for engineers.
To do this, add this to the metainfo file:
Some update protocols just transport the image to the target device and make no
guarantee of the signing requirements. Such “generic” protocols include NVMe, ATA,
DFU and many others.
The device integrity mechanism can be set as part of the metainfo.xml file or set
from the LVFS web console.
signed : The firmware payload is verified on-device the payload using strong
cryptography such as RSA, AES or ECC.
It is usually not possible to modify or flash custom firmware not provided by
the vendor.
unsigned: The firmware payload is unsigned and it is possible to modify and
flash custom firmware.
Some hardware returns the version number as a string such as
1.23.4567, and this is easily handled as a semantic version.
In other cases we are not so lucky, and the hardware returns a uint16_t or uint32_t
with no extra metadata about how it should be formatted.
This lack of specification precision means that different vendors have chosen
to convert the large integer number to various different forms.
The latest allowed values for LVFS::VersionFormat can be found
on the LVFS.
To override the default of unknown vendors should ship extra metadata
in the metainfo.xml file:
If the version format is unspecified, and cannot be derived from the
LVFS::UpdateProtocol then a warning will be shown during upload
and the firmware cannot be moved to stable until this is resolved.
Various security teams also want us to always show the device firmware version
with the correct format, even if an update is not available.
This may be for audit reasons, or just so customers know the version of the
firmware compared to release notes written for another operating system.
For instance, if the vendor release notes says the firmware should be any
version above 39.0.45.x (formatted as a quad) and the user is running
39.0.11522 (formatted as a triplet) it is not clear to the user what to do.
To change from the default triplet version format we can set
a fwupd quirk on the hardware device.
For instance, changing the UEFI
Lenovo ME device
to use the intel-me format.
Quirk files can be added upstream for future fwupd versions, or simply copied to
/usr/share/fwupd/quirks.d.
The fwupd daemon will detect the new file and refresh devices as required.
Some device flags can be populated from the firmware metadata, rather than the more traditional
way of setting in a per-plugin .quirk file or in the plugin code itself.
This allows device flags to be pushed from the server to the client.
For instance, on some hardware, the UEFI UpdateCapsule process would fail to deploy because there
was no HDMI/DP display attached.
Firmware can opt-in to this new requirement by setting a device flag that gets copied to the local
fwupd device.
If the client is new enough, then any firmware that opts-in then the user will be warned before
the update is scheduled that a display must be connected to continue – which is a much better user
experience than it failing after the user has rebooted to deploy the update.
To use this, the internal device flag can be populated from the firmware metadata:
The exact flags that are allowed for each protocol are restricted, and the allowed values for
LVFS::DeviceFlags (along with fwupd requirements) can be found via the LVFS.
Note
Only fwupd versions greater or equal to 1.9.1 are able to copy device flags from the metadata,
and only devices with the FU_DEVICE_INTERNAL_FLAG_MD_SET_FLAGS flag set.
If you require the device flag to be set for a successful update then you should also have the
correct fwupd version requirement to ensure the flag get copied. e.g.
When the user requests updates for a specific device, all the GUIDs provided by
the device will be match against any of the GUIDs in the metadata.
To limit these matches using a variety or requirements the <requires> tag
can be used.
For instance, the update can be conditional on the firmware version of another
device, or on the kernel version of the installed system.
Requirements can use different methods to compare version numbers.
Using fwupd >= 1.9.10 the uploader can also deny updates to a specific Computer Hardware IDs:
<!-- only newer versions of fwupd understand 'not_hardware' requirements --><requires><idcompare="ge"version="1.9.10">org.freedesktop.fwupd</id><not_hardware>6de5d951-d755-576b-bd09-c5cf66b27234|27234951-d755-576b-bd09-c5cf66b27234</not_hardware></requires>
CHIDs can also be added or removed in the LVFS web UI, but only before
the firmware is published to stable channel.
Newer versions of fwupd can restrict updates on one device depending on
the version of firmware on another device.
This is most useful when requiring a minimum EC controller version before
updating a system firmware, or when a modem firmware needs a specific fix
for the baseband firmware:
Newer versions of fwupd can restrict updates on one device depending if another
firmware GUID exists on the system of any version.
This is similar to the CHID method above but uses the GUID of the firmware,
not a hardware ID.
This can be used to ensure that a specific embedded controller is detected
for a specific system firmware update, for example.
For composite devices such as docks you might want to restrict the child
device with respect to the parent, for instance requiring the parent to
have greater than a specific bootloader version number.
The other useful thing to use this for is checking if the parent has a specific
GUID (of any version) which allows us to match against the common VID&PID
instance IDs. This would allow us to restrict a generic child device update to
a specific OEM vendor parent.
Depth is specified as 1 to match the parent device and
2 to match the grandparent device:
Newer versions of fwupd can understand an OR requirement using a |
separator between the listed GUIDs.
<!-- only newer versions of fwupd understand parent OR requirements --><requires><idcompare="ge"version="1.8.9">org.freedesktop.fwupd</id><firmwaredepth="1">12345678-1234-1234-1234-123456789012|6de5d951-d755-576b-bd09-c5cf66b27234</firmware></requires>
Composite devices can also specify that a device sibling has to exist, optionally
with a specific version. To do this, specify the depth as 0:
<!-- only newer versions of fwupd understand the 'depth' property --><requires><idcompare="ge"version="1.6.1">org.freedesktop.fwupd</id><firmwaredepth="0">12345678-1234-1234-1234-123456789012</firmware></requires>
Composite devices can also restrict the parent device with respect to the child.
This is useful when a generic parent device has vendor-specific child devices attached.
To do this, specify the depth as -1 to match any child device.
<!-- only newer versions of fwupd understand the negative 'depth' property --><requires><idcompare="ge"version="1.9.7">org.freedesktop.fwupd</id><firmwaredepth="-1">12345678-1234-1234-1234-123456789012</firmware></requires>
Versions of fwupd >=1.4.5 can restrict updates depending on the features the
client can provide. For instance, if the tools are being run in non-interactive
mode then it may not be possible to ask the user to perform a manual action.
Some devices may need to show the user some text or an image of how to
manually detach the firmware from runtime mode to bootloader mode.
Other firmware may require showing the user a message or image on how to reset
the hardware when the firmware update has completed.
This specific post-update message functionality is only available for specific protocols and
implemented in some versions of fwupd and GNOME Software.
Showing the user some instructions to reboot the hardware.¶
This action can be performed with one or two metadata keys set in the
.metainfo.xml file, or chosen using the LVFS component editor.
…
<custom><valuekey="LVFS::UpdateMessage">Pleaseturnthedeviceoffandbackonagainfortheupdatetocomplete</value><valuekey="LVFS::UpdateImage">https://people.freedesktop.org/~hughsient/temp/unifying-power.png</value></custom>
…
<!-- only newer versions of fwupd understand 'client' requirements --><requires><idcompare="ge"version="1.4.5">org.freedesktop.fwupd</id><client>update-action</client></requires>
…
Note
You can include either the UpdateImage or <image> PNG file in the
cabinet archive rather than uploading it. In this case use a URL with a file:// prefix,
e.g. file://unifying-power.png.
The image will still be mirrored onto the LVFS CDN at upload time, as there
is no ability for GUI clients to read binary files from inside the cabinet
archive. This means that internet access will still be required when deploying
firmware if the image is specified.
All fwupd versions understand “hard” requirements; those that cannot be ignored.
Some vendors may want to add some suggested requirements, which can be ignored
by the end user if required.
Typically this would be done using the --force command line option.
A good example here would be from a server OEM who release a set of updates
once per quarter.
It is expected that the end user only updates from the previous n-3 quarter
releases (and not older releases), and it is also vendor recomendation that updates
are installed in a specific order.
In the field, however, admins may want to override this, either due to
security policy requirements or to work around specific hardware issues.
…
<!-- only newer versions of fwupd understand 'recommends' requirements --><requires><idcompare="ge"version="1.6.2">org.freedesktop.fwupd</id></requires><recommends><firmwarecompare="ge"version="0.1.2">6de5d951-d755-576b-bd09-c5cf66b27234</firmware></recommends>
…
If you’d rather not have users downloading the .cab archive directly you
can opt to hide the direct download links in the LVFS search results.
To do this, add this to the metainfo file:
<!-- most OEMs do not need to do this... --><custom><valuekey="LVFS::InhibitDownload"/></custom>
The LVFS administrator can configure the policy for all firmware owned by
the vendor to be blocked from download in embargoed or otherwise
sanctioned countries.
The blocked ISO 3166 country codes can also be specified in the firmware
itself, using the LVFS::BannedCountryCodes metadata key.
If a vendor is distributing firmware which contains GPL licensed parts (for
example the Linux kernel) then they must include a source URL for the GPL
licensed parts in the releases section in the metainfo file.
This should point to the release-specific source code that can be used to
rebuild the binary from the code, for instance:
GPL firmware without source information can not be moved to testing or stable.
You can also edit or add the source URL in the existing Update Details
section in the component view:
Tags can be used to identify a vendor-specific keyword to the component release, for example
identifying components included in a specific service pack or combined update.
Users with fwupd version >= 1.7.3 can install multiple firmware files using the tag value.
For example setting HostBkc=vendor-factory-2021q1 in /etc/fwupd/daemon.conf`andthendoing``fwupdmgrsync-bkc will install all firmwares with that matching tag.
The icon show in GUI fwupd clients is normally set by the plugin automatically.
In some cases the plugin may not know the appropriate icon until firmware has
been uploaded to the LVFS.
For this cosmetic purpose the firmware uploader can specify the stock icon in
the metainfo.xml file which gets put in the the AppStream metadata and used
by the graphical clients. In most cases specifying the icon is not required.
To manually override the icon to one of the stock values, use this:
A vendor can build a single .cab archive with multiple firmware files with
different .metainfo.xml files describing them.
This allows a single file to be used to update either multiple devices, or
a single composite device.
An example of a composite device would be a Dell dock, where electrically there
are various components connected using USB, but physically it looks like one
piece of hardware. Wacom does the same for the various Intuit tablets.
Some tools such as gnome-software may be unable to show more than one update
description for the single .cab file. The LVFS also needs to know how to sort
the components inside the firmware when showing the logged in user.
To solve this, assign the firmware components a priority, where higher numbers
are better. For example main.metainfo.xml:
The priority can also influence the composite install order.
Client-side, fwupd will check the explicit device order (e.g. Flags=install-parent-first) and
then fallback to the component priority where higher numbers are installed first.
This can be used to ensure that releases are always installed in a specific order.
If the priority is specified on the component it is used for all releases, but if per-release
control of the composite device is needed, it can instead be set on the release node, e.g.
<releases><releasepriority="5">...
</release></releases><requires><!-- only newer versions of fwupd understand per-release priorities --><idcompare="ge"version="1.9.10">org.freedesktop.fwupd</id></requires>
In the case where the update order is different between releases the component or device should
probably also have Flags=install-all-releases to ensure that every version is installed with a
predictable release order.
There are currently quite strict limits on the format of the release
description included in the description part of the metainfo,
or edited on the LVFS.
For instance, OEMs are not allowed to include links within the text and have
to adhere to a limiting style guide.
As a workaround, all firmware can now specify an additional url:
This should point to a website page or PDF description of the specific
release.
This would allow vendors to provide more information about specific CVEs or
provide more technical information mentioned in the update details.
Whist the update details should still be considered the “primary” method to
convey information about the firmware release, the URL may be useful for larger
OEMs with existing contractual requirements.
It is important to set the urgency of the release to the correct value as this
may influence how the client notifies the user. For instance, critical updates
may cause a daily session notification to the user, but low priority updates
might only be visible when the user manually visits the software center.
Value
Meaning
low
Low importance
medium
Medium importance, e.g. optional update
high
High importance, e.g. recommended update
critical
Critical importance, e.g. urgent or security issue
In some circumstances we may need to ask the user to perform an action to
manually put the device into a special firmware-update mode.
We can achieve this using a translatable update caption and an optional line-art
image:
Showing the user some instructions before updating firmware.¶
To achieve this the firmware needs to declare the public location of the image
in the metainfo file:
Vendors can include an extra .metainfo.xml file with the
<componenttype="generic"> to supply information used by the
LVFS to identify the top-level device.
This is only useful when there is no obvious existing high-priority
component that can be used for display.
This would be useful for a dock to have the title WonderDock2
rather than showing a seemingly random sub-component of it on the
public pages.
An example generic.metainfo.xml file would look like this:
<?xml version="1.0" encoding="UTF-8"?><!-- Copyright 2020 Richard Hughes <richard@hughsie.com> --><componentpriority="9"type="generic"><id>com.hughsie.WonderDock.firmware</id><name>WonderDock</name><summary>FirmwarefortheACMEWonderDock</summary><urltype="homepage">http://www.hughsie.com/</url><metadata_license>CC0-1.0</metadata_license></component>
A generic component can also be created for composite firmware manually
on the LVFS for firmware that has already been uploaded.
Use a short device name, e.g. “ThinkPad P52s” or “ColorHug 2”.
Use a UTF-8 character (e.g. ™ or ®) rather than (R) if required
Don’t include the vendor name
Note
If a component matches two components with the same GUID, please use a forward slash to delimit
each model and specifier, for instance, you SHOULD do this:
<name>ThinkPadT580/ThinkPadP52s</name>
…and NOT do any of these:
<name>ThinkPadT580/P52s</name>
<name>FC30,NES30</name>
<name>ColorHug2&ColorHug2.1</name>.
To avoid specifying multiple components in the <name> you could also have one firmware
payload in the cabinet archive referenced by two different .metainfo.xml files – each with
a different DMI CHID, parent or child requirement.
Try to avoid explaining the implementation details of the fix, e.g.
“Ensure accurate color profile creation with high screen brightness.”
rather than
“Fix overflow in counter when over 500 Lux detected.”
Do not use overly technical descriptions when simpler text would suffice, e.g. use
“Fix LED color during system start up.” rather than
“Fix LED color during POST.”
Try to describe fixed bugs and new features from the point of view of the user
and how it affects them
For security or important updates also include the effect of not applying the update, e.g.
“Disk corruption resulting in possible data loss may occur until this update is installed.”
The release tag may be optional or required based on component category and
vendor policy. If provided it can used to show a vendor-specific text
identifier that is different from the version number.
The tag may be unique only to the model, or be unique for the entire vendor.
This attribute should not be used if the tag is not used to identify the
specific firmware on the vendor homepage.
Depending on vendor policy, the release tag may be displayed with the header
External release Software ID or Machine Type Model.
The .cab archive format was chosen to match the format expected
by Windows Update.
This allows vendors to produce one deliverable that can be submitted to the LVFS
for signing and then to Microsoft Update, or the other way around.
Signatures from one process will not overwrite signatures from another.
It is recommended you name the archive with the vendor, device and version
number, e.g. hughski-colorhug-als-1.2.3.cab and is suggested that
the files inside the cab file have the
same basename, for example:
When building archives on Windows you will need to use the makecab.exe
program. This works slightly different to gcab in that it needs
a manifest to be created of all the files that are included.
To create the manifest create a file called config.txt with the
following contents:
If you forget the .OPTIONEXPLICIT in the manifest then the size of
the archive is limited to 1.38Mb.
If you try including a firmware with a size greater than this you will see
Invalidfolderindex when trying to use fwupdmgr as the archive is not valid.
The upload process repacks the uploaded archive into a new cabinet file
and signs the firmware image using a detached GPG or PKCS#7 signature
so client tools can be sure the firmware actually originated from the LVFS.
Any existing Windows Update signatures are also copied into the new
archive although are not used on Linux.
The signed archive is prefixed with the hash of the uploaded file to avoid
clashes with other uploaded files and to make the download location non-predictable.
Normally firmware is uploaded to a private remote.
This firmware is available to only the user that uploaded it, and any QA users
in the vendor group. It is not visible to end-users, other vendors or to fwupd
running locally.
Firmware can be moved to a so-called embargo remote that means that is included
in the private metadata catalog that is available for any users in the same
vendor group.
It is not available to any other vendors (even vendors acting as ODM or OEM) and
is also not available to the public.
Once the firmware is moved to testing it is available to the general public,
typically a few thousand users who have opted-in to testing pre-release firmware.
Then the firmware can be moved to stable which makes it available to tens of
millions of public users.
The affiliates feature on the LVFS may be interesting to larger OEMs, as it
allows users working for other ODMs to upload firmware on the OEMs behalf.
First, some nomenclature:
OEM: Original Equipment Manufacturer, the user-known company name on
the outside of the device, e.g. Sony, Panasonic, etc.
ODM: Original Device Manufacturer, typically making parts for one or more
OEMs, e.g. Foxconn, Compal, etc.
There are some OEMs where the ODM is the entity responsible for uploading the
firmware to the LVFS.
The per-device QA is typically done by the OEM, rather than the ODM, although
it can be both.
Allowing the ODM to log in as the OEM is not good design from a security,
privacy or audit point of view.
The LVFS administrator can mark other vendors as affiliates of other vendors.
This gives the ODM permission to upload firmware that is owned by the OEM to
the LVFS, and that appears in the OEM embargo metadata.
The OEM QA team is also able to edit the update description, move the firmware
to testing and stable (or delete it entirely) as required.
The ODM vendor account also doesn’t have to appear in the search results or the
vendor list, making it hidden to all users except the OEM.
This also means if an ODM like Foxconn builds firmware for two different OEMs,
they also have to specify which vendor should own the firmware at upload time.
This is achieved with a simple selection widget on the upload page, but is only
shown if affiliations have been set up.
The ODM is able to manage their user accounts directly, either using local
accounts with passwords, or ODM-specific OAuth which is the preferred choice as
it means there is only one place to manage credentials.
In some instances it is better to upload firmware by the ODM vendor to the ODM
group, rather than the affiliated OEM. This would let anyone in the ODM QA group
modify the update, for instance changing the update description or performing
an end-to-end test.
Once the firmware has been tested, it can be moved to the OEM account,
although it can only be moved back by the OEM as the ownership has been
transferred.
You can automate the upload of firmware from a build pipeline by creating a user
token. This can then be used to upload firmware for that user using a script, e.g.
When a firmware format is set in the metainfo.xml file
various tests are performed on the firmware by the LVFS.
This includes checking file headers, magic numbers or CRCs for the chosen
update protocol.
The update protocol can be changed on the LVFS website, and the correct tests
will be run automatically.
Firmware that has unresolved test failures cannot be pushed to the
testing or stable remotes.
For some tests the failure can be waived by a QA user.
All update binaries, and shards contained within are scanned for strings which
may indicate a problem with the firmware. Example strings are:
DONOTSHIP
ToBeDefinedByO.E.M
Although these can be waived by a QA user, firmware should not be uploaded that
have this text.
Additionally, the blocklist plugin will search for other information that may
add a component claim. For instance the computrace claim will be added to any
firmware shipping the official Computrace agent, and it will be visible to users
when viewing the component information.
All UEFI updates are decompressed, and if a processor microcode is found then it
is compared with older firmware versions that have been uploaded to the LVFS.
If the microcode has been downgraded then the test will fail, although the failure
can be waived by a QA user.
Any EFI shards are loaded and will have their PE signatures checked.
If any certificate is out of date, or otherwise invalid a test failure will appear.
This failure can be waived by a QA user.
Once the firmware is in an embargo remote anyone in the vendor group can then
download the vendor-embargo.conf from the LVFS metadata page
and install it locally on their Linux system.
Warning
The vendor-embargo.conf file should never be emailed to anyone not in your
vendor group.
If you want to allow access to an ODM or OEM this can be done by transferring
the ownership of the firmware.
To use the embargo remote:
Download a new version of vendor-embargo.conf from the LVFS
Put your LVFS email in the Username= field in vendor-embargo.conf
Install it to /etc/fwupd/remotes.d if using a distribution build of fwupd, or /var/snap/fwupd/common/var/lib/fwupd/remotes.d if using the snap build
Use fwupdmgrrefresh to download the new metadata
Warning
Do not rename the vendor-embargo.conf to lvfs.conf – both are required
Do not manually modify the existing lvfs.conf file
You should use a fwupd versions newer than 1.7.9 – use sudoaptremovefwupd then
sudosnapinstallfwupd to get a newer fwupd on Ubuntu LTS.
After waiting a few minutes for the LVFS to regenerate the vendor group metadata,
the user can do fwupdmgrrefresh to get the new metadata which includes
the new firmware release.
Once the new metadata is available on the local system the device can be updated
either using fwupdmgrupdate or using GNOME Software.
Note
You can force GNOME Software to update the metadata catalog using the refresh
button in the left hand side of the header bar in the Updates panel.
You should only move stable firmware to testing and stable after completing an
end-to-end test with the embargo remote.
Warning
It can take a few hours to regenerate the testing and stable remotes
and up to 24 hours for users to download the new metadata catalog.
Most vendors see a large spike in downloads the day after they move a firmware
to stable, and then a steady decay the days after.
If you’ve moved the firmware to embargo, waited for the remote to regenerate,
and then done fwupdmgrrefresh and still do not have any update available you
can check for the new release in the downloaded metadata using vim:
After each update the fwupdmgr client tools allow the end user to submit a “report”
which is used by the firmware owner to validate the firmware deployment is correct.
Any failures can be analyzed and patterns found and the metadata can be fixed.
For instance, the failures might indicate that the required fwupd version needs
to be raised to a higher value, or that the update requires a specific bootloader
version.
Part of the anonymous report also includes the device checksum which can be used
to verify the firmware is being deployed correctly.
All users can submit reports, and there is no way to verify the report has not
been modified by the end user before submission.
This means reports should not be used for trust, and the information only used
when statistically significant.
There is provision in fwupd 1.2.6 and newer to actually sign the report contents
using a per-machine certificate.
This allows the LVFS to verify the report has not been modified after being signed,
and also means the LVFS now knows what user submitted the report.
If the LVFS knows which user (and thus which vendor) owns which certificate the
report can then be used for trusted operations.
For instance, setting the “golden” device checksums for the update, or verifying
that the firmware was indeed tested on specific hardware.
To do this, the user must add at the certificate from each machine used for testing:
The user can then upload reports to the LVFS in a trusted way by signing the report:
Additionally, the signed report can be made available to 3rd parties,
typically as part of a hardware certification program.
Note
Sharing the URL allows a user (possibly outside your organization)
to read the specific report details but not modify the firmware or
report in any way.
In some automated tested scenarios the image on the DUT is ephemeral and manually uploading the
host-generated signing key to the LVFS is not appropriate.
In this case, allow the user to upload a report using Basic authentication so that the LVFS can
treat it as “signed” by the report uploader.
Then on a different (internet-connected) machine, log into the LVFS, navigate to the specific
firmware page and click the Reports tab.
From there the /media/YOUR_USBDISK/*.fwupdreport files can be uploaded:
Note
The LVFS uses the current logged in user to assign the test user and vendor group.
Firmware uploaded to the LVFS is scanned, and attributes about the update are added
automatically.
Some claims may be positive, for instance if hardware supports verification.
Negative claims are also added, for instance if verification checksums are missing.
Informational neutral claims are also added, which are not positive or negative,
but may be a consideration for the user, e.g. if Computrace is included.
Including the Shell.efi in a firmware update can create additional supply chain
security risks.
From the UEFI shell it is very easy to downgrade processor microcode or to abuse
the existing update process.
It also makes attacking SMI handlers much easier, e.g. ThinkPwn.
The EFI shell allows direct RW access to memory using mm command, which by
itself defeats SecureBoot and everything else that’s security is based on memory
not being being attacker-controlled.
Processor microcode can be thought of runtime firmware for the CPU processor itself.
It maps “high level” x86 instructions to hardware micro-opcodes that are specific
to the processor.
Microcode is supplied as an encrypted blob by CPU vendors like Intel and AMD
and cannot be modified in any way by the end user.
Only microcode signed by the processor vendor can be loaded onto the CPU.
In some cases, the processor vendor will issue a new microcode to address an issue,
which may be security sensitive.
This has been done many times in the past, e.g. to fix or mitigate the Spectre,
Meltdown and Foreshadow security issues.
In some cases microcode updates are even done to increase performance for a
specific workload.
If a firmware is tagged as _containing old microcode it doesn’t always mean
that there is an unpatched security issue.
Some microcode is vendor-specific, so for instance Lenovo might create an update
on the LVFS that updates the version of microcode of CPUID 0x906ec from 0xd2 to 0xd3.
Although Dell might be using the same processor, the motherboard hardware is not
affected and no update will be prepared.
When a computer equipped with Computrace is reported stolen, the firmware agent
attempts to notify the monitoring center, allowing the Absolute Theft Recovery Team to
forensically mine the computer using a variety of procedures including key
captures, registry scanning, file scanning, geolocation, and other investigative
techniques to determine who has the computer and how it is being used.
Absolute then works with local law enforcement agencies to recover the computer.
Due to the way the agent works, it’s often seen as a “legitimate” firmware implant,
which may be a consideration when purchasing hardware.
The Computrace agent is nonfunctional under Linux and only works when using
Microsoft Windows XP and newer.
The related LoJax UEFI rootkit hijacks the Computrace agent for malicious puposes.
No production firmware should include the EDK Debug Agent as it allows the end
user to trivially disable host protections like BootGuard, and potentially also
allows unauthenticated access to SMM, which is the most secure layer in the machine.
Every time the PC powers on, HP Sure Start automatically validates the integrity
of the BIOS code to help ensure that the PC is safeguarded from malicious attacks.
Once the PC is operational, runtime intrusion detection constantly monitors memory.
In the case of an attack, the PC can self-heal using an isolated “golden copy”
of the BIOS in less than a minute.
HP Sure Start is a hardware technology available only on some HP hardware.
BIOS guard helps ensure that firmware malware stays out of the BIOS by blocking
all software based attempts to modify protected BIOS without the platform
manufacturer’s authorization.
Typically, this is implemented by blocking SMM writes to the SPI flash chip.
Intel Boot Guard is a technology introduced by Intel in the 4th Intel Core
generation (Haswell) to verify the boot process.
This is accomplished by flashing the public key of the BIOS signature into the
write-once field programmable fuses of the CPU itself, typically during the
manufacturing process.
In this way it has the public key of the BIOS and it can verify the correct
signature of the firmware during every subsequent boot.
Once enabled by the manufacturer, Intel Boot Guard cannot be disabled.
Firmware can either be signed or unsigned.
Signed in this context means the binary code has been either signed or encrypted
using private-public asymmetric key cryptography.
It does not include firmware protected with weak symmetric methods such as XTEA as
the private key would need to be stored on the device itself, which is insecure.
It also does not include firmware “protected” with checksums like CRC32.
Devices supporting signed firmware can only be updated by the original OEM
and alternate “homebrew” or malicious firmware cannot be written.
When devices are flashed with new firmware the device will normally self-check that
the data has been written correctly.
Some devices just write new data to an SPI flash chip and hope for the best.
When devices are flashed with new firmware the device will normally verify that
the data has been written correctly.
Devices supporting verified firmware either allow the host to read back the written
firmware at a later time, or will return a internally-calculated checksum.
This allows users to verify that devices have not been tampered with, which may
even be a concern before first use due to supply chain attacks.
For UEFI firmware, although the firmware capsule is signed by the OEM or ODM,
software can’t reliably read the SPI EEPROM from userspace.
The UEFI firmware does provide a hash of the firmware, or more specifically,
a hash derived from the stored firmware event log.
A final hash of all the TPM firmware events is stored in the TPM chip as PCR0.
To list the various PCRs on the running system you can use
cat/sys/class/tpm/tpm0/pcrs for TPMs using protocol 1.2, or
tpm2_listpcrs for TPMs using protocol 2.0.
The PCR0 can be included in the vendor-supplied firmware.metainfo.xml in the
cabinet archive:
Multiple golden device checksums are possible for each system depending on the
specific set up options.
For instance, enabling or disabling Intel TXT would change the system PCR0
checksum.
The device checksums can also be set using the admin console of the LVFS:
Adding PCR0 checksums to a component for attestation¶
The LVFS only allows OEMs, ODMs and silicon vendors to upload firmware.
Some OEMs allow the ODM to QA firmware on their behalf and for this reason there
are strictly controlled “affiliate relationships” defined on the LVFS.
Furthermore, the AppStream prefix is checked on upload, to prevent the vendor
trying to replace or inpersonate another vendors legitimate firmware.
This namespacing keeps the OEMs firewalled from each other.
Client side there is another check which verifies the uploader of the firmware
has the matching set of restrictions for the USB or PCI-assigned vendor ID.
For instance, Hughski Limited can only deploy firmware onto devices with
VendorId=USB:0x273F and so even if the LVFS account for this company was hacked
they could not update firmware from Logitech or Wacom.
All firmware licensed with a GPL-like license must include links to the exact
source release used to build the firmware update.
This claim is only shown for firmware that requires a source URL, although can
be included even for non-open-source firmware if required.
All firmware uploaded to the LVFS gets scanned by the ClamAV security scanner.
Additionally, when the firmware is no longer embargoed and available to the
public it is uploaded to VirusTotal for further anaysis.
Most UEFI firmware images uploaded to LVFS are scanned by the Binarly FwHunt community scanner
to check for publicly disclosed security issues. Security issues still under vendor embargo are
not detected.
Any potential issues detected are visible to the OEM vendor and uploader, but are not shown to end
users. When a firmware image has a detectable issue, the exact details will not be displayed here.
Firmware is scanned with the latest set of public rules at upload time, and may be scanned again at
a later date when new rules become available.
Please contact Binarly if you would like more details about FwHunt technology.
Some devices can be marked as “end-of-life” as they are no longer supported by
by the original OEM.
These are unlikely to get updates to fix critical security problems.
All firmware uploaded to the LVFS gets scanned for both CoSWID data embedded in
the SBOM section of the COFF binaries, but also uSWID external metadata.
For instance, there may be embedded CoSWID metadata in 75 PE files, where each
EFI binary contributes information to the composite package SBoM.
This is possible as we can include the CoSWID metadata in the PE files at build
time, generating accurate data automatically.
Sometimes it is not possible is in embed the CoSWID metadata directly into a
proprietary or vendor-specific section, e.g. AMD microcode or Intel FSP.
For these binary blobs it’s expected that the IVH or the system integrator will
generate some external metadata about the non-free blob and include it in the
system image somehow.
This might be in an FV section for an EFI image, the DT for an ARM image, or
just appended as raw data in a free section in the ROM file.
By allowing fwupd to phone home after attempting a firmware update,
it allows the hardware vendor that uploaded firmware to know there are
problems straight away, rather than waiting for frustrated users to file bugs.
The report contains information that identifies the machine and
old/new firmware versions, and in the event of an error, enough debug
information to actually be useful.
It obviously involves sending the user’s IP address to the server too.
We have to be exceptionally careful with users’ privacy and trust.
We cannot just enable automated collection, and this document outlines what
we implemented for fwupd >= 1.0.4.
This functionality should be acceptable to even the most paranoid of users.
The fwupd daemon stores the result of each attempted update in a local SQLite
database.
In the event there is a firmware update that has been attempted, we now ask the
user if they would like to upload this information to the LVFS.
Using GNOME this would just be a slider in the control center privacy panel,
although this feature is currently unimplemented.
If the user is using the fwupdmgr tool this is what it shows:
This means vendors using the LVFS know the approximate number of successes and
failures, and can add different tests to existing QA tests accordingly.
This allows the LVFS to automatically pause the specific firmware deployment if > 1%
of the reports come back with failures.
Some key points:
We do not share the IP address with the vendor, and it is not even saved in
the database
The MachineId is a salted hash of the machine /etc/machine-id
The LVFS does not store reports for firmware that it did not sign itself,
i.e. locally built firmware archives will be ignored and not logged
The user can disable the reporting functionality in all applications by
editing /etc/fwupd/remotes.d/*.conf
Known issues are problems we know about, and that can be triaged automatically
on the LVFS.
Of course, firmware updates should not ever fail, but in the real world they do,
Of all the failures logged on the LVFS, 95% fall into about 3 or 4 different
failure causes, and if we know hundreds of people are hitting an issue we
already understand we can provide them with some help.
A good example here is the user not being on AC power when rebooting, which
causes a failure, albeit transient and non-fatal.
Another example is if the user tries to do the update with an incorrect system
configuration, for instance a missing /boot/efi partition.
The URL for the user to click on is the result of a rule engine being included
in the LVFS.
Users on the LVFS with the appropriate permissions can also create and view
rules for firmware owned by just their vendor group:
The fwupd project already supports a huge number of flashing protocols,
everything from standardized protocols like NMVe, ATA, DFU and also a large number
of vendor-specfic protocols like logitech_hidpp, synaptics_prometheus and wacom_raw.
Most vendors are using a protocol that fwupd already supports, and thus only
need to upload firmware to the LVFS. In the case applying for an account is all
that is required.
Note
If using DFU, please also implement the DFU runtime interface – this allows
fwupd to automatically switch your device into bootloader mode without having to
draw some artwork and write some translated text to explain how the user should
persuade the device to enter update mode.
The easiest time to add support for updating hardware using the LVFS is during
the project prototype phase.
There are several things you can do that makes writing a fwupd plugin much easier.
In the case where the device protocol is a non-compatible variant or a completely
custom protocol then a new fwupd plugin will be required.
If you have to use a custom protocol, there are a few things that are important
to consider.
The fwupd daemon needs to be able to enumerate the device without the user noticing,
which means LEDs should not blink or cause the screen to flicker.
Disconnecting a kernel driver, changing to bootloader mode or any other method
of getting the device firmware version is not acceptable.
This means the device needs to expose the current firmware version on the runtime
interface, for instance using USB descriptors or PCI revision fields.
For composite devices (e.g. docks) it is much better to provide an interface to
query the internal device topology rather than hardcoding it in the plugin or in
a quirk file.
For instance, fwupd could query the root device that would respond that it is
acting as a I²C bridge to a HDMI chip with address 0xBE, rather than hardcoding
it for a specific dock model.
Querying the information allows the plugin author to write a generic plugin that
means future devices can be upgraded without waiting for new fwupd versions to
be included in popular Linux distributions and ChromeOS.
Warning
Plan and test for what happens when the user
disconnects the USB cable, runs out of battery, or removes the mains plug when
the new firmware is being flashed.
If the device remains in bootloader mode, is there a unique VID/PID that can
be used to choose the correct firmware file to flash the device back to
a functional runtime mode?
Many vendors just use the ISV-provided reference bootloader (which is fine),
but fwupd does not know which runtime image to recover with if the ISV-allocated
generic VID/PIDs are being used.
If it is not possible to modify the bootloader VID/PID, then it may be possible
to read a block of NVRAM at a hardcoded offset to identify the proper firmware
to install.
When updating hardware it is important to provide feedback to the user so that
they know the process has not hung.
Updating firmware is intimidating to many users and so it is important to provide
information about what is being done to the hardware, for instance erasing,
writing and verifying.
It is also a very good idea to provide percentage completion, so for an operation
that is going to take 10 seconds it is better to write 1024 blocks of 16kB with
percentage updates after each block rather than one block of 16Mb with just a
bouncing progressbar.
Note
It is not possible to upload executable flasher code as part of the cabinet
archive – only the payload is allowed.
We will not accecpt non-free executables, static libraries or “shim” layers
in fwupd. The only way a custom protocol can be supported is by contributing
a LGPL-2.1-or-later plugin upstream.
Some vendors will have the experience to build a plugin themselves, and some vendors
may wish to use a consulting company
that has the required experience.
The plugin code in fwupd is Open Source (LGPL-2.1-or-later), licensed in a way that makes
it possible for another vendor or an end-user to read and modify the code.
Some companies have initially said that an open source plugin would be
impossible due to concerns about either trade secrets, security or both.
Let’s look at the trade secret or intellectual property concern by answering
some questions:
How many firmware updater binaries were sold last year?
Is the update protocol significantly more complicated than read version
number, switch to bootloader, erase blocks, write blocks, read back
blocks to verify, switch to runtime?
Can a user dump the USB communication using a $20 capture tool and replay the
recording to update a different device?
Is the shared secret token sent unencrypted as part of the update protocol?
The fact is that most OEM vendors sell physical devices to consumers and both
the hosting of updates on a company webserver and the development of the update
client itself is typically a cost-centre, and not a revenue stream.
The fwupd project supports more than 80 different update protocols, and most of
them use exactly the same design; there may be differences in required header
format or CRC32 polynomials, but 95% of “secret vendor protocols” are almost
exactly the same from a high-level design perspective – and thus do not
constitute valuable intellectual property.
Another important consideration is that the fwupd plugin doesn’t need to know
everything about the hardware – for instance, in the Synaptics MST plugin we
know the offset in the configuration header of the current firmware version,
but the rest of the header is unspecified and still secret.
In the PixArt plugin we know the offset of the 32 bit AA.BB.CC.DD version
number, but the rest of the file is treated as a binary blob that is just sent
to the device in small sections.
Another concern from vendors is that as an open source project, anyone can edit,
modify, or even sabotage code that communicates with their device.
Of course, the maintainers of the fwupd project will review and check carefully
every proposed change.
Most plugins also have tests that emulate that specific device firmware update –
that get verified for each and every proposed change.
Any plugins that require stronger “ownership” requirements can also add an
Owners section in the per-plugin README.md file that will be used to notify
responsible users that validation or functional re-testing is required before a
change is merged.
Some vendors are also unwilling to agree to an open source plugin to fwupd as
the payload will be discovered as unsigned or communication is unencrypted,
and attackers will know how to attack the devices.
Unfortunately, it’s very easy to scan a firmware payload for signatures or even
to calculate the entropy.
It’s even possible to perform a manual bitswap in something like the USB
descriptor to know if the payload signature is being verified correctly.
Security through obscurity has never been acceptable, and hackers are more than
capable of dumping an unencrypted firmware update process and then modifying
the code to inject malware.
An attacker does not care about having source code through legitimate means and
does not need permission.
The answer is either to properly implement firmware signing, or to just be okay
that the current device has unsigned firmware.
It is completely acceptable for devices to not implement firmware signing for
example with OpenHardware devices, or devices where the user is encouraged to
build custom firmware.
The LVFS only allows vendors to update their own hardware (e.g. Wacom can only
update devices with USB vendor ID of 0x056A) and so it is not possible for
another vendor to automatically update firmware on your hardware, even if the
payload is unencrypted or unsigned.
On a similar note, some vendors will not want to open the update protocol as it
would allow consumers to flash the “pro” version of a firmware to the “basic”
device to software-unlock features that are usually only available in a more
expensive model.
Whilst this is a concern if the two devices have the same signing key (they
should have different keys) or if they have no signing requirement at all –
it’s very easy to fool even official non-free binary updater programs just by
changing how the Windows/Linux kernel reports either the device USB VID/PID
or USB HID descriptor.
From real world experience, the number of users that are going to disable the
device checks in an open source project (a risky procedure) is going to be a
staggeringly small number of people compared to the number of people that will
be able to update the correct hardware with the latest firmware.
Turning this around completely, many vendors who have added new fwupd plugins
have found that the RMA (return merchandise authorization) rate for hardware
has actually reduced significantly.
This is because the newer firmware fixes a bug, for instance a USB-4 dock not
working with their existing DisplayPort monitor – where it does work with the
latest dock firmware applied.
Similarly, users may choose the consumer device or peripheral exactly because
it has a fwupd support, and because firmware updates can be managed at scale
using Windows, Linux, macOS or ChromeOS.
Please do not use model-specific or vendor-specific libraries to update or
enumerate the hardware.
Unless the library is already shipped by default on Ubuntu LTS and RHEL 8 then
it is going to be exceptionally hard to use this library in fwupd.
As fwupd is in main for Ubuntu then any library it depends on must also be
part of main, which means Canonical has to officially support it.
They obviously do not want to do this for vendor-specific libraries that have
only existed for a few years with no API or ABI guarantees or long term stable branches.
Similarly for Red Hat; any new library or binary needs to have a Red Hat maintainer
who will support it for over 10 years (!) and is willing to do the security due
diligence and code review required to be included as a core package in RHEL.
Getting approval for a new package is a huge amount of work and takes months.
As fwupd is running as root, any external library it depends on must be audited
by several security teams, and have a proven security plan in place.
Google also needs to review any new dependencies as fwupd is also being used
heavily in ChromeOS now, and they take OS image size and security very seriously.
In this situation it is completely okay to include parts of the open source
library (assuming the code can be licensed as LGPL-2.1-or-later) in the fwupd plugin.
Including them in fwupd plugins also allows us to use the fwupd helper functionalily,
for instance replacing memcpy() with fu_memcpy_safe() and using high level
abstractions for reading and writing to sysfs files or an ioctl.
Many plugins already do this, for instance the colorhug plugin does not use
libcolorhug, nvme plugin does not use the nvme command line tool and
the emmc plugin itself defines EXT_CSD_FFU rather than depending on mmc-utils.
There are many layers of security in the LVFS and fwupd design, including restricted account modes,
2 factor authentication, and server side AppStream namespaces.
The most powerful one is the so-called vendor-id that the vendors cannot assign themselves,
and is assigned by a member of the LVFS admin team when creating the vendor account on the LVFS.
The way this works is that all firmware from the vendor is tagged with a requirement like
USB:0x056A which matches the USB consortium vendor assigned ID.
Client side, the vendor-id from the signed metadata is checked against the physical device
and the firmware is updated only if the ID matches.
This ensures that malicious or careless users on the LVFS can never ship firmware updates for other
vendors hardware.
All vendors on the LVFS are now locked down with this mechanism.
Some vendors have to use IDs that they do not own, a good example here is for a DFU device like
the 8BitDo controllers.
In runtime mode they use the USB-assigned 8BitDo VID, but in bootloader mode they use a generic VID
which is assigned to the chip supplier as they are using the reference bootloader.
This is obviously fine, and both vendor IDs are assigned to 8BitDo on the LVFS for this reason.
Another example is where Lenovo is responsible for updating Lenovo-specific NVMe firmware, but
where the NVMe vendor is not using the Lenovo PCI ID.
All devices exported by fwupd must have at least one vendor ID, mostly automatically added as the
vast majority derive from either FuUsbDevice or FuUdevDevice.
The vendor IDs can be dispayed using fwupdmgrget-devices.
Capsule updates are a popular way to distribute firmware updates.
As the ESRT convays no vendor ownership information, we use the platform DMI data.
For instance Lenovo is only able to update Lenovo hardware with DMI:Lenovo.
We hold personal data about vendors, administrators, clients and other
individuals for a variety of purposes.
This policy sets out how we seek to protect personal data and ensure that
administrators understand the rules governing their use of personal data to
which they have access in the course of their work.
In particular, this policy requires that the Data Protection Officer (DPO) be
consulted before any significant new data processing activity is initiated to
ensure that relevant compliance steps are addressed.
As the Data Protection Officer, Richard Hughes.
has overall responsibility for the day-to-day implementation of this policy.
The DPO is registered with the Information Commissioner’s Office (ICO) in the
United Kingdom as a registered data controller.
We must process personal data fairly and lawfully in accordance with individuals’ rights.
This generally means that we should not process personal data unless the
individual whose details we are processing has consented to this happening,
or where such collection is unavoidable and/or considered pragmatic in the
context, e.g. logging the number of downloads of a particular file.
We do not consider an IP address to represent a single user (due to NAT or VPN use),
and as such metadata requests are not considered personal data using the draft GDPR guidelines.
We will ensure that any personal data we process is accurate, adequate,
relevant and not excessive, given the purpose for which it was obtained.
We will not process personal data obtained for one purpose for any unconnected
purpose unless the individual concerned has agreed to this or would otherwise
reasonably expect this.
Individuals may ask that we correct inaccurate personal data relating to them.
If you believe that information is inaccurate you should inform the DPO.
You must take reasonable steps to ensure that personal data we hold about
hardware vendors is accurate and updated as required.
For example, if your personal circumstances change, please update them using
the profile pages or inform the Data Protection Officer.
We keep personal data secure against loss or misuse.
Where other organizations process personal data as a service on our behalf,
the DPO will establish what, if any, additional specific data security
arrangements need to be implemented in contracts with those third party
organizations.
We must retain personal data for no longer than is necessary.
What is necessary will depend on the circumstances of each case, taking into
account the reasons that the personal data was obtained, but should be
determined in a manner consistent with our data retention guidelines.
Anonymized user data (e.g. metadata requests) will be kept for a maximum of
5 years which allows us to project future service requirements and provide
usage graphs to the vendor.
There are restrictions on international transfers of personal data.
We do not transfer personal data anywhere outside the EU without the approval
of the Data Protection Officer, unless required to do so by law.
Please note that under the Data Protection Act 1998, individuals are entitled,
subject to certain exceptions, to request access to information held about them.
On receiving a subject access request, we will refer that request immediately
to the DPO. We may ask you to help us comply with those requests.
Please also contact the Data Protection Officer if you would like to correct
or request information that we hold about you.
There are also restrictions on the information to which you are entitled under
applicable law.
Being transparent and providing accessible information to individuals about how
we will use their personal data is important for our project.
The following are details on how we collect data and what we will do with it:
What: Machine ID (hashed), failure string and checksum of failing file,
OS distribution name and version.
Why collected: Allows the hardware vendor to assess if the firmware update
is working on real hardware.
Where stored: AWS hosted PostgreSQL database in Oregon, USA region.
When copied: Full backups weekly, with daily snapshots both to AWS backup.
Who has access: The hardware vendor (filtered by the QA group), Linux
Foundation Infrastructure Team and the DPO.
Wiped: When the firmware is deleted.
We will ensure any use of personal data is justified using at least one of
the conditions for processing and this had been specifically documented above.
The data that we collect is subject to active consent by the data subject.
This consent can be revoked at any time.
Revoking consent to use data ends any vendor relationship with the LVFS.
Upon request, a data subject should have the right to receive a copy of their
data in a structured format, typically an SQL export.
These requests should be processed within one month, provided there is no
undue burden and it does not compromise the privacy of other individuals.
A data subject may also request that their data is transferred directly to
another system. This is available for free.
A vendor may request that any information held on them is deleted or removed,
and any third parties who process or use that data must also comply with the request.
An erasure request can only be refused if an exemption applies.
Privacy by design is an approach to projects that promote privacy and data
protection compliance from the start.
The DPO will be responsible for conducting Privacy Impact Assessments and
ensuring that all changes commence with a privacy plan.
When relevant, and when it does not have a negative impact on the data subject,
privacy settings will be set to the most private by default.
Regular data audits to manage and mitigate risks will inform the data register.
This contains information on what data is held, where it is stored,
how it is used, who is responsible and any further regulations or retention
timescales that may be relevant.
Everyone who actively uses the LVFS must observe this policy.
The DPO has overall responsibility for this policy.
They will monitor it regularly to make sure it is being adhered to.
We take compliance with this policy very seriously.
Failure to comply puts both you and us at risk.
The importance of this policy means that failure to comply with any requirement
may lead to disciplinary action under our procedures.
If you have any questions or concerns about anything in this policy,
do not hesitate to contact the DPO.
The LVFS is a public webservice designed to allow
OEMs and ODMs to upload firmware easily, and for it to be distributed securely
to tens of millions of end users. For some people, this simply does not work
for various reasons:
They don’t trust the LVFS team, fwupd.org, GPG, certain OEMs or the CDN we use
They don’t want thousands of computers on an internal network downloading all
the files over and over again
The internal secure network has no internet connectivity
For these cases there are a few different ways to keep your hardware updated,
in order of simplicity:
If the OS is shipped as an image, you can just install the .cab files into
/usr/share/fwupd/remotes.d/vendor/firmware and then enable vendor-directory.conf
with fwupdmgrenable-remotevendor-directory.
Then once you have disabled the public LVFS using fwupdmgrdisable-remotelvfs,
running fwupdmgr will use only the cabinet archives you deploy in your
immutable image.
Of course, you’re deploying a larger image because you might have several unused
firmware files included for each image, but this is how Google Chrome OS is using
fwupd.
You can use Pulp to mirror the entire public
contents of the LVFS (but never private or embargoed firmware).
Create a repo pointing to PULP_MANIFEST
and then sync that on a regular basis to download the metadata and firmware.
The contents will not change any more frequently than every 4 hours, so please
use a polling interval of at least that.
Note, this command can be run safely multiple times with different --filter-tag values on the
same destination directory; the superset of files will be downloaded.
By exporting the entire LVFS (including the metadata, and metadata signature)
you can still delay the deployment of firmware.
Using the approved firmware list the client can filter out firmware that has not
been tested by your organization without creating and signing a custom remote.
Again, use PULP_MANIFEST to create a big directory holding all the firmware
(currently ~50GB, but growing), and keep it synced.
Create a NFS or Samba share and export it to clients. Map the folder on each client,
and then create a myprivateshare.conf file in /etc/fwupd/remotes.d:
Download the .cab files that match your hardware and then install them
on the target hardware via Ansible or
Puppet using fwupdmgrinstallfoo.cab. You can also
use fwupdagentget-devices to get the existing firmware versions of all
hardware in a format you can parse from scripts.
The local.py
script allows you to create the metadata for a directory of .cab files.
Note
If you want to use signed metadata then please use
jcat-toolfirmware.xml.gz.jcatfirmware.xml.gzCERTIFICATEPRIVATE_KEY.
You will need to create a custom certificate
and you’ll also need to distribute the the PKCS#7 certificate on all clients
that are going to use the remote.
The LVFS is a free software Python 3 Flask application and an instance can be set up
internally if required. You have to configure much more this way, including
generating your own GPG and PKCS#7 keys, uploading your own firmware and setting
up users and groups on the server.
Doing all this has a few advantages, namely:
You can upload each firmware file and QA it, only pushing it to stable when ready
You don’t ship firmware which you didn’t upload
You can control the staged deployment, e.g. only allowing the same update to
be deployed to 1000 servers per day
You can see failure reports from clients, to verify if the deployment is going well
You can see nice graphs about how many updates are being deployed across your organization
However, running a secure LVFS instance is a lot of work as PostgreSQL has to be
used as a database, Redis has to also be set up as a queue manager, and Celery
is used to manage the worker queues.
Although minor versions of the LVFS can be upgraded easily, you should review all
the commits to lvfs-website to ensure that any manual migration is also performed.
We want to make it easy for ODMs and OEMs to choose components that already have fwupd plugin support.
This will do a few things:
The onus is pushed onto the IHV to maintain the plugin not the OEM, ODM or Linux distributor (e.g. Red Hat)
The ODM and OEM will prefer components that do not require any software development work to pass
the Works With ChromeBook (WWCB) and Red Hat Enterprise Linux (RHEL) hardware certifications
Having a fwupd plugin will be seen as a commercial advantage for the IHV
There are two versions of the fwupd friendly firmware certification, one for devices that will
only accept signed firmware (signed-payload) and another for insecure hardware that does not
implement cryptographic signing (unsigned-payload).
Either is fine from a fwupd plugin point-of-view, but some OEMs will have a policy that forces them
to choose hardware that cannot be altered by the end user.
Note
Consumer devices end-users buy from the store are not suitable for fwupd friendly firmware,
and already have device pages on the LVFS.
To register a device for fwupd friendly firmware the original silicon vendor must have an existing
LVFS vendor account, and also provide:
Device model, e.g. CX2098X
One line summary, e.g. USB3.2Gen14-porthubcontroller
Link to the device page, e.g. https://www.kinet-ic.com/ktm50x0/ (optional)
Link to the upstream fwupd plugin that handles this device type, e.g. https://github.com/fwupd/fwupd/tree/main/plugins/synaptics-cxaudio
Device firmware certification level, e.g. always signed, optionally signed, or unsigned
Hash and signature algorithm used for firmware signing, e.g. SHA256+RSA2048, or n/a
To add a device to the certification page,
please send an email to the
mailing list
with the required details.
On meeting the requirements, the entry will be added and then the vendor is allowed use the signed or
unsigned fwupd friendly firmware logo as required.
Vendors should not have a “generic” fwupd friendly firmware assigned to them, as a vendor may
have multiple devices with different update protocols. e.g. Synaptics has cxaudio, mst,
prometheus and cape protocols, each with a different fwupd plugin.
Vendors using the fwupd friendly firmware logo mark will make it easier for product creators to
support firmware updates for Linux users.
The burden of development moves earlier to the IHVs rather than later to the OEMs.
OEMs can verify the fwupd friendly firmware certification and compare hardware using the public
pages on the LVFS.
Many thanks should also go to the UEFI SBOM Sub Team for all thier support in the creation of this document.
This specification document may be subsumed by a future UEFI specification or best practice document, but was published here to provide a reference specification in the interim.
The purpose of this document is to present a set of guidelines and best practices for vendors of firmware to provide Software Bill of Materials (SBoM) information to their clients and customers, to aid in vulnerability detection and license management.
Note: The keywords “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document should be interpreted as described in RFC 2119.
This document assumes a working knowledge of terminology related to firmware, and of software concepts such as “libraries” and “compilers”.
The terms defined in this glossary may appear in italics as a reminder that they are being used as defined here.
Readers may be expecting to see terms like “IBV” (Independent BIOS Vendor), “ODM” (Original Design Manufacturer), “IFV” (Independent Firmware Vendor) and “OEM” (Original Equipment Manufacturer), but this document mostly avoids those terms.
This is because those entities may, at any given moment and in any given commercial arrangement, be acting as component vendors, firmware vendors or platform vendors in the context of this document.
SBoM: Software Bill of Materials.
A formal document which can be used to articulate what components are contained within a binary deliverable, and who is responsible for each part.
Component: any identifiable, discrete element of a firmware, including but not limited to any item that can be removed from, replaced in or added to a file volume or archive.
This includes, but is not limited to, PE files, PEIMs, CPU microcodes, CMSE/PSP, FSP/AGESA, EC and OptionROMs – but SHOULD NOT include encryption keys or source code references.
Each component may be provided as a precompiled binary by a component vendor to a firmware vendor, or it may be built from an independent source code tree by the firmware vendor.
Component SBoM: an SBoM for a single component.
Component Vendor: a party responsible for directly supplying a component for use by a firmware vendor in a firmware image.
Firmware: a complete firmware image, which typically comprises multiple components.
Firmware SBoM: an SBoM that represents all the components present in a single firmware and which could be generated in full or in part by combining component SBoMs.
Firmware Vendor: a party responsible for building firmware, for use by the platform vendor.
Platform SBoM: an SBoM that represents all the components in use on a real-world device.
This may be equivalent to the firmware SBoM for single system firmware deployed on a device, or be a superset that includes metadata for multiple firmware (e.g. separate firmware for the system and for an attached touchpad or camera device).
Platform Vendor: the party responsible for supplying a combined platform firmware image, typically comprising multiple firmware, for use on end-user hardware.
Source Code: Text written in a program language (for example, C, assembly or Rust) that is compiled into binary object files and is not included verbatim in the firmware image.
Due to the increasing number of high-profile supply chain attacks, it has become more important to record information about critical software such as system and peripheral firmware.
For US companies, Executive Order 14028 “Improving the Nation’s Cybersecurity” and the Cyber Trust Mark now make providing an SBoM with this information a legal obligation for many companies.
It has traditionally been difficult to build firmware or platform SBoMs for systems due to the involvement of three separate entities: the Firmware Vendor that produces the bulk of the source code, the ODM (Original Design Manufacturer) that compiles it with other additional code and adds additional binaries, and the OEM (Original Equipment Manufacturer) that may add their own extensions and then distributes the firmware.
Most consumer laptop and desktop devices also have many other firmware blobs of firmware supplied for factory burn-in, e.g. fingerprint reader, SD card reader, touchpad, PCI retimer, Synaptics MST, Intel Thunderbolt, and many more – and these might not have any communication channel to the system firmware at all.
End-users do not buy “firmware” and any firmware deliverable will normally be included in a larger OEM per-device platform SBoM.
At the same time, we also need to provide access to the runtime “current firmware SBoM” so that we can use newer technologies such as VEX to automatically identify systems that require security fixes.
This document explains why SBoM metadata for all constituent components should be embedded in all firmware, what should be included in it, and how it should be used as part of a larger platform SBoM that is useful to end-users.
When we talk about “embedding the SBoM”, we refer to the general idea of having SBoM metadata for all components in a given firmware included into the firmware image itself, either by providing a firmware SBoM or just by ensuring all components are represented in multiple component SBoMs.
Traditionally there has been pressure to keep firmware images as small as possible to minimize SPI storage space and to minimize the cost of the Hardware Bill of Materials.
While this is a noble aim, sacrificing a few hundred bytes of space for an embedded SBoM has several advantages:
The SBoM does not need to be verified against a binary deliverable, it can be assumed to be “part of” the existing source artefact itself.
Vendors at any link in the supply chain that don’t care about or understand SBoMs do not “strip” the SBoM information.
The component SBoMs and/or firmware SBoMs from all the factory burn-in firmware images can be combined into one generated public platform SBoM that can be used for contractual or compliance reasons, without the need to request component or firmware SBoMs separately from each component vendor and firmware vendor.
Build-time automated embedding as part of CI/CD is recommended as part of the US Cyber Trust Mark initiative.
Some firmware build systems require the firmware blob and definition files to be put in a predefined place to generate a new firmware binary, which means non-embedded SBoM metadata may get out-of-sync with the blob.
If the SBoM is not embedded as a build artifact, a firmware engineer could rebuild the firmware capsule and forget to also regenerate or replace the SBoM in the new archive because it is a separate process that is hard to verify was done.
If the SBoM is part of the image itself and automatically constructed as part of the deliverable, then it is impossible to forget.
Sending the capsule or manually dumped ROM image to a QA engineer means they can know with almost complete certainty what blobs the image was built with.
Embedding the SBoM makes doing the “right” thing easy and doing the “wrong” thing hard.
All component vendorsSHOULD embed an SBoM in the component image, formatted as described below.
They MAY also create a more detailed detached SBoM (for instance referencing internal issues or source code filenames) that MAY be provided to the firmware vendor under NDA.
Firmware vendorsSHOULD ensure embedded SBoM metadata is included for every PE binary and all additional components included in the firmware formatted as described below.
This MUST be done by:
Including the SBoM for each component in a “defragmented” firmware SBoM created at build time, OR
Ensuring that each component contains embedded SBoM metadata, OR
Doing both of the above.
Component and firmware SBoMs SHOULD NOT reference any code or blobs which are not actually present, or which have been disabled in the system.
Firmware and component vendorsMUST use the DTMF coSWID binary format with CBOR encoding when directly embedding SBoM sections in firmware.
This format was chosen due to the small compiled size of data compared to SPDX (YAML or JSON) and SWID (XML), because the specification is freely available and because it can act as a superset format to both SPDX and CycloneDX.
Most components in a typical firmware are compiled from source code and linked into PE binaries.
These can be considered components whose vendor is the firmware vendor.
The firmware vendorSHOULD ensure that the SBoM metadata is automatically built and verified at compile time and then added to the PE binary (in the .sbom COFF section), placed directly in the “defragmented” firmware SBoM (see below), or both.
If for any reason this is not done automatically at compile time, the firmware vendor still MUST ensure the SBoM is included in the binary .sbom COFF section or the “defragmented” firmware SBoM, as required above.
For Tianocore/EDK2 firmware, there is an example showing how to supplement the information in the .inf file with per-component and per-platform overrides.
More specific recommendations on how to include additional artifacts into the .sbom section have not been made as this will be heavily influenced by the existing proprietary build system and tools used to build the image.
In the case where there is no natural place to store the component SBoM, it SHOULD be included as a per-volume metadata section. In this case it MUST include a uSWID magic header, as described in Components that are not Portable Executables (PE) below.
Firmware vendors do not have to compile all the PE binaries in the EFI volume from source code.
They may get pre-compiled and pre-signed binaries from third-party component vendors.
Component vendorsSHOULD include the coSWID SBoM metadata for these components in a .sbomCOFF section which can be easily included at link time.
These binaries MUST NOT use the magic header of uSWID (described below) as the PE header can be parsed for the correct offset of the section.
An additional benefit of including the SBoM in a COFF section is that it is verified by the existing Authenticode digital signature.
If a firmware vendor uses a PE binary which does not have this embedded SBoM metadata, the firmware vendorMUST ensure SBoM metadata for the binary is present in a “defragmented” firmware SBoM, as described below.
Components that are not Portable Executables (PE)¶
When embedding SBoM metadata into any binary that is not a Portable Executable (PE), the component vendorMUST use the discoverable uSWID header so that software can easily discover the embedded SBoM.
The 25-byte uSWID header is listed below:
The header length MAY be increased for alignment reasons (e.g. to 0x100 bytes), and in this case the additional header padding MUST be NUL bytes.
The uSWID payload SHOULD be compressed with either zlib or LZMA, and a firmware image containing the binary SHOULDpass validation using uswid, for example:
$ uswid --load firmware.bin --validate
Found USWID header at offset: 0x18000
Validation problems:
dd4bbe2e40ba component: No software name (uSWID >= v0.4.7)
dd4bbe2e40ba entity: Invalid regid http://www.hughsie.com, should be DNS name hughsie.com (uSWID >= v0.4.7)
dd4bbe2e40ba entity: No entity marked as TagCreator (uSWID >= v0.4.7)
dd4bbe2e40ba payload: No SHA256 hash in FSPS (uSWID >= v0.4.7)
dd4bbe2e40ba link: Has no LICENSE (uSWID >= v0.4.7)
dd4bbe2e40ba link: Has no COMPILER (uSWID >= v0.4.7)
Although there are many tools for the distribution of the firmware SBoM to end-users, fewer tools exist to embed SBoMs into binary blobs, or to extract and merge SBoM components to build a firmware SBoM or platform SBoM. The python-uswid project is one such tool.
A firmware image can contain a “defragmented” top-level firmware SBoM with a uSWID header, produced at build time. If each component in the image has uSWID metadata, coSWID data in PE/COFF .sbom sections and/or file volumes with uSWID metadata, the firmware vendorMAY omit this firmware SBoM. If not, the firmware vendorMUST include it.
If the firmware SBoM is present:
It MUST contain all component SBoMs present in the image.
This requirement is to ensure that tools do not need to combine and deduplicate component SBoMs with the firmware SBoM to provide all available information.
It SHOULD be compressed.
The components MAY also have component SBoMs as described in this document, to allow them to be analyzed in isolation.
The purpose of an SBoM is to tell the end-user what components make up the software deliverable, and to give them information on where it was retrieved from or built. The questions end-users need to be able to answer are “what version of OpenSSL is included, and where did it come from” and “do I trust all the companies contributing code and binaries to this image”. Answering the what and who in a standardized way also allows us to use other specifications such as VEX.
In this section we use the term “SBoM component” to refer to a single ingredient within an SBoM (in a coSWID SBoM, this is a single tag).
Each SBoM component SHOULD describe either:
A single component, as defined in the glossary, or
An individually identifiable part of a component that has security and/or licensing implications, for example an image loading library used by a PE binary, or
Something that has security and/or licensing implications and was used to produce a component, but is not present in the component itself, for example a compiler used to produce a PE binary, or
Any kind of defined logical component, for example “optional features” or “value add” options that may be matched from a VEX file (see below).
Each componentMUST be represented by an SBoM component in its component SBoM, or the firmware SBoM if the component does not have its own SBoM (see the Embedding the SBoM section above for possible scenarios).
Libraries, compilers etc. SHOULD be represented by SBoM components (see the Component Relationships section below for more on this).
Thus, a component SBoM or firmware SBoMMUST contain at least one tag, and MAY contain more.
For components or relationships that cannot currently be disclosed for legal reasons, vendors MAY use the literal text REDACTED in place of the correct string value.
This is intended as a temporary measure while contracts or NDAs are renegotiated.
Any SBoM components with REDACTED text MAY be marked as incomplete and MUST fail validation.
MUST have an identifier in the form of a GUID.
See the Identifier section below for more details.
MUST have a non-zero length descriptive name, e.g. “CryptoDxe”, and SHOULD NOT include a file extension as this is already included in the SWID payload section.
MUST have at least one entity entry and SHOULD have more than one, if more than one legal entity is involved in its creation, maintenance and/or distribution.
One entity MUST have the tag-creator role.
One entity MUST have the software-creator role, and it MAY be the same entity as the one specified in tag-creator.
See the Vendor Entity section below for details.
In specifying entity roles, vendors SHOULD be careful not to make business relationships public that are not already in the public domain.
MUST have a version, which SHOULD be a semantic version like 1.2.3.
MUST have a file hash that is generated from all the source files, if it is a binary built from source code or other constituent parts. This MUST be either a SHA-1 or SHA-256 hash.
This is what uSWID calls a “colloquial version.”
SHOULD have a revision control tree hash which MUST be either a SHA-1 or SHA-256 hash (e.g. the output from gitdescribe), if it is a binary built from source code under revision control.
This is what uSWID calls an “edition.”
MAY or MUST include one or more link entries expressing relationship(s) to another SBoM component. See the Component Relationships section below for details, including when link entries are REQUIRED and when they are OPTIONAL.
The file hash SHOULD include the hashes of the source code files used to construct the binary, such as .c and .h files.
Any library statically-linked with the PE binary SHOULD be included as an additional SBoM component.
In some cases, the most obvious identifier to use for the SBoM component is already in a GUID form – for instance using the UEFI GUID defined in an official specification or reference implementation.
In other cases, like GCC (where there is no GUID defined), vendors MUST use a swid: prefix to generate a GUID that is linked within the object.
Using a GUID is deliberate because it can obscure internal references, and can be encoded as a 128-bit number in coSWID.
An “entity” describes a party responsible for the creation, maintenance, and/or distribution of a firmware or component.
An entity can perform one or more roles (e.g. creator, maintainer and distributor), and multiple entities (even with the same role) can be defined for each component.
For instance, Intel FSP is created by Intel, maintained by Intel, and distributed by Intel.
A modified DXE might originally be created by Intel in EDK2, but then be modified and maintained by AMI and distributed by Lenovo.
In this case, the component for the FSP would have only one entity entry, but the component for the DXE would have three entity entries.
For each entity entry:
The name MUST be the legal or common-use name of the open-source project, the component vendor, the firmware vendor, or the platform vendor.
The registration ID MUST be the DNS name of the named legal entity, or the DNS name of the upstream project URL in the case of open-source projects.
SBoM component links are used to supply additional information about how components relate to each other.
They also include any required licensing information, statically linked libraries and links to additional resources.
Libraries that may be matched from a VEX file (for instance, where a third-party library has previously security vulnerabilities) SHOULD be included as a component, but other internal libraries MAY be omitted.
SBoM components MAY use multiple links, even of the same relationship type.
SBoM components representing open-source software MUST include one or more license link(s) indicating all licenses that apply.
The URL for each license link MUST be the SPDX license URL, e.g.: https://spdx.org/licenses/LGPL-2.1-or-later.html
The license relationship type MUST be used.
All open-source code SHOULD be identified with its own SBoM component to allow verification of license compliance.
SBoM components representing non-open-source software SHOULD include one or more license link(s) indicating all licenses that apply.
The URL for each license link MUST be a public webpage with the full text of the proprietary license.
The license relationship type MUST be used.
SBoM components representing compiled binaries SHOULD reference SBoM components representing the compiler and linker used to build the binary where possible.
The see-also relationship type MUST be used, and the swid-prefixed URL MUST be an existing component identifier defined in the component or firmware SBoM.
SBoM components representing compiled binaries SHOULD reference SBoM components representing libraries that are linked into the binary and that may be referenced in VEX documents (see below).
The requires relationship type MUST be used, and the swid-prefixed URL MUST point to an existing component in the SBoM.
SBoM components MAY include a link specifying the source URL where they can be downloaded. e.g. https://github.com/intel/FSP/AmberLakeFspBinPkg
The installationmedia relationship type MUST be used.
The figure below shows the possible flows of SBoM information from the component vendor(s), firmware vendor(s) and/or platform vendor to the end-user.
VEX data (see below) is used to notify the end user about security issues of components referenced by the SBoM.
Depending on existing business relationships, the firmware vendor (the ODM) may take on some of the responsibilities of the platform vendor (the OEM) or the component vendor (the IBV).
Dumping the SPI contents using an external SPI programmer or OS interface allows the end-user to extract a “current” firmware SBoM.
This allows analyzing the image without having access to a public SBoM provided by the platform vendor or a vendor neutral firmware provider like the Linux Vendor Firmware Service (“LVFS”).
To comply with Executive Order 14028, OEM vendors SHOULD also publish either the SPDX or CycloneDX SBoM export as a downloadable file on the public device webpage.
The SHA-256 checksum of the generated SBoM SHOULD be used as the unique collection ID for the component and firmware SBoMs.
This enables the SBoM to be found using a search engine even if the original OEM has been renamed or the device HTML URI has been modified.
Vulnerability Exploitability eXchange (VEX) allows a component vendor to assert the status of a specific vulnerability in a particular firmware.
VEX can have any of the following “status” values for each component:
Not affected: No remediation is required regarding this vulnerability.
Affected: Actions are recommended to remediate or address this vulnerability.
Fixed: Represents that these product versions contain a fix for the vulnerability.
Under Investigation: It is not yet known whether these product versions are affected by the vulnerability.
Only the entity with the source code tree and the config files used to build it (usually the IBV or ODM) has all the information required to know whether a given EFI binary is affected by a specific vulnerability.
If our aim is to find out if a specific firmware is vulnerable to a specific security issue, there are only three ways to solve this without access to a complete SBoM:
The end-user asks the component vendor, who finds the firmware version, checks out the source code for that revision, then looks for affected code, and replies with the answer.
The component vendor proactively passes detailed vulnerability status and remediation info to the immediate downstream supply chain partner, who then in turn proactively passes this down to each customer.
The component vendor shares the code and the config to the customer and assumes the customer can work it out themselves.
We consider these ways to be clearly unsatisfactory.
Therefore, both component vendors and platform vendorsSHOULD upload the SBoM to a trusted neutral entity, allowing multiple customers and end-users to query the information.
The neutral entity MAY also process additional trusted VEX data directly from component vendors, which allows firmware to automatically be marked as affected or not affected without direct involvement of the firmware vendor.
Vendors writing VEX rules MUST use the same identifier as used in the SBoM.
VEX product IDs are specified using PURL, and the GUID MUST be used as the component name.
Where a semantic version is required it MAY also be specified.
Further details about using Vulnerability Exploitability eXchange (VEX) standards such as OpenVEX with embedded firmware SBoMs will be provided in the future.
With these sets of recommendations we feel sure that the resulting firmware SBoM will be useful to security teams and end-users alike.
This would greatly benefit the entire firmware ecosystem and make the global supply chain measurably safer.
This document strongly encourages vendors to embed the SBoM metadata into the respective binaries, but there are two situations where externally referenced SBoM metadata would be allowed:
Where the binary is loaded onto critically space-constrained devices, for example microcode that is loaded into the processor itself.
Where only later newer versions of the component have embedded SBoM metadata, and backwards compatibility is required with older revisions.
In these cases, the component vendorMUST provide “detached metadata” from the same source (or in the same archive file) as is used to distribute the immutable blob.
As the SBoM metadata is detached, vendors MUST ensure that the files do not get “out of sync” and are updated at the same time in the firmware source tree.
Detached metadata MUSTalways contain the SHA256 hash value of the binary as evidence to allow validation and MAY be signed using a detached signature if the archive is not already signed.
The public key SHOULD be distributed on a keyserver or company website for verification.
Some vendors have expressed concerns about “wasted” space from including the SBoM data in the binary image.
For source components such as CPU microcode, a single component and vendor entity would use an additional ~350 bytes (zlib compressed coSWID), compared to 48kB for the average EFI binary and 25kb for a typical vendor BGRT “splash” logo.
The uswid command can automatically generate a complete “worst case” platform SBoM with 1,000 plausible components.
This SBoM requires an additional 140kB of SPI flash space (uncompressed coSWID), or 60kB when compressed with LZMA.
For reference, the average free space in an Intel Flash ROM BIOS partition is 5.26Mb, where “free space” is defined as a greater than 100KiB stream of consecutive 0xFF’s after the first detected EFI file volume.
Adding the SBoM as embedded metadata would use 1.1% of the available free space.
Other firmware ecosystems such as Coreboot also now include SBoM generation as part of the monolithic image.
The ACPI SBOM ACPI table may be used in the future to return the coSWID formatted binary SBoM data from any device exporting an ACPI callable interface.
Further details will be provided when the SBOM table has been implemented.
If the platform allows direct access to the system SPI device, then the entire firmware image can be dumped to a local file and analyzed by tools such as uswid.
The embedded SBoM SHOULD be converted it into one or more SBoM export formats before publication.
This can be achieved easily using tools such as uswid.
For example, this can be used to produce two JSON files in CycloneDX and SPDX formats from the platform image:
The embedded SBoM MAY be signed, and MAY also be included in the firmware checksum.
If the firmware component is signed then the SBoM SHOULD be included in to the signature.
The signing step is optional because a malicious silicon provider can typically do much worse things (e.g. adding or replacing a DXE binary) than modify the SBoM metadata.
When firmware is uploaded to the LVFS it automatically extracts all available SBoM metadata and generates a HTML page with SPDX, SWID and CycloneDX download links that can be used for compliance purposes.
The LVFS MAY allow vendors to upload firmware or platform SBoMs without uploading the firmware binary.
Other services like Windows Update may offer this service in the future.
The VEX “trusted neutral entity” MAY also be the LVFS, even for firmware updates not distributed by the LVFS.
Uploading VEX data requires vendors to register for a LVFS vendor account which is available at no cost.
By default the filesystem is mounted in ReadOnly mode. For testing purposes we
have to do some changes in configuration, meaning we have to disable rootfs
verification.
Switch to the linux console on ChromeBook by pressing: “Ctrl + Alt + →” ( ‘→’
or ‘F2’ on top row).
You should see the login prompt asking for login.
Use username “chronos” for login, password is not needed.
Allow fwupd access to HID devices for ChromeOS versions prior to v107¶
ChromeOS versions prior to version 107 have an issue with managing the HID devices
like mice or keyboard. In this case we need to avoid this restriction.
Switch to linux console on ChromeBook by pressing: “Ctrl + Alt + →”
You should see the login prompt asking for login.
Use username “chronos” for login, no password should be asked.
For composite devices, both LVFS and fwupd allow the use of a single CAB
file. In this case we have to prepare and pack several XML files with metadata,
one file for each firmware.
In the case of ColorHug, only one metadata file is requested:
firmware.metainfo.xml
The name of the files doesn’t matter – the only requirement is the extension .metainfo.xml.
Both metadata XML above contains the minimal amount of data. The most
interesting tags are:
<id> – the name of AppStream unique identifier for the device.
Vendor must choose the unique string in reverse-DNS style and this ID must
contain the device name and .firmware suffix
<name> – this is the short name of the device
<name_variant_suffix> – for composite devices, this is added to short name
<firmware> – the GUID in this tag is extremely important! It helps fwupd
to recognize the updatable device. See the output from fwupdmgrget-devices
for devices GUIDs.
It is allowed to use several GUIDs here if the same update file fits for several devices.
<value key=”LVFS::UpdateProtocol”> – here should be used the name of
the protocol supported by the colorhug plugin (see
README.md for
actual protocols supported).
urgency=”low” – the urgency field has no effect on fwupd itself. This is
the hint for UI frontends how to notify users, Gnome Software center for
instance. At the moment Chrome OS has very limited UI support for device
updates. The upstream is expecting
following values:
Value
Meaning
low
Low importance
medium
Medium importance, e.g. optional
update
high
High importance, e.g. recommended
update
critical
Critical importance, e.g. urgent
or security issue
The ColorHug-1.2.6.cab will be created containing 2 files: 1 metadata XML
and 1 firmware binary.
The generated file will contain only the minimal amount of the metadata.
No additional information for firmwares validation, checksums or signatures are
included at this step!
To test the generated file you need to copy it onto a ChromeBook device.
The simplest method is to copy it via ssh from the build host.
Please substitute the IP address from the example below with the IP address of
your device, and use the password you’ve set during ChromeBook setup
(test0000 in the example):
Alternatively you may want to use other methods for accessing the CAB file from
ChromeBook device, for instance: own HTTP server, network share, USB mass
storage and others.
The most interesting here are device GUID(s) and device flags: Updatable
meaning that the device updates are supported by fwupd manager, and
Supportedonremoteserver flag shows there are firmwares available on the
LVFS site.
As mentioned above, the development variant of locally generated CAB file was
not digitally signed, so it is not possible to install it on Chrome OS based
devices with the fwupdmgr tool due security reasons:
By default Chromium OS has a local vendors repository enabled (see
/etc/fwupd/remotes.d/vendor-directory.conf), so any CAB file placed into the
local directory /usr/share/fwupd/remotes.d/vendor/firmware will be automatically
detected and could be used for the device upgrade or downgrade:
There are 4 LVFS
remotes
available for the CAB file uploading:
private – should be the initial remote for uploading, the CAB file with
FW uploaded to private could be accessible only via direct link
embargo – remote with non-public catalog of FWs available for Vendor
only; used during development and QA testing. The remote may be added to the
Chrome OS just like a common remote, so all fwupd functionality is
available for testing.
Should be used for testing by Vendor before giving access to FW to end-users.
testing – this remote is generally available for end users, encouraged
enough to deal with the potential risk of the freshest FW version.
stable – the main remote with released FW considered as good enough for
the mass market.
This remote is enabled by default on Chrome OS and Linux systems with
installed fwupd.
The private remote doesn’t provide the catalog with uploaded files.
Instead you have to go to the CAB file you are interested in and get its
URL for the direct download and install.
Choose the needed firmware from private repo (“Firmware” ->
“State::Private”) and press the button details.
In the “Details” tab you will see the “Overview” block with the name of the
uploaded firmware. The right click (or long press) will give you the URL of
the signed CAB file.
Gain access to the Chrome OS device terminal as described in the
Access to ChromeBook section.
Via terminal install the CAB file from the LVFS by copied URL:
The embargoed remote provides the catalog of available firmwares but it is
restricted to vendor, so only the vendor is able to use it or share for 3-rd
parties.
This is the one-time action needed for enabling the Embargo remote on the
testing Chrome OS system.
Check on Chrome OS device if the remote is not available and not
enabled already:
# fwupdmgr get-remotes
If there is the remote named Embargoedforcollabora or similar and it
is enabled (check the key Enabled:true), probably the remote is
configured already and you may proceed with Install the FW CAB file from the
embargo remote .
On LVFS site find the section “Metadata” – it would contain links to the
recent versions of metadata for Embargo, Testing and Stable repositories.
However we are interesting in the configuration file for the Embargo remote
of your company (collabora-embargo.conf in the example)
Download the configuration file for the embargoed remote by clicking the
name.
Please keep in mind – this is the private file accessible for authorized
users only!
You need to copy the downloaded file to the ChromeBook with any
method available for you (mass-media device, ssh and others).
To copy the file from the workstation with ssh, for instance:
scpcollabora-embargo.confroot@192.168.1.115:~/
Please use the correct file name and IP address of the target device!
On Chrome OS device you need to copy the file to configuration directory for
fwupd daemon:
The Testing remote provides the catalog of firmwares for public access, however
only users explicitly enabled this remote will have access to published FWs.
LVFS side is similar to Move the FW CAB file to embargoed remote , but the
target is “Testing remote” and no need to download the file with remote
configuration.
To enable the testing remote, use text editor to edit configuration file to
replace Enabled=false by Enabled=true in
/etc/fwupd/remotes.d/lvfs-testing.conf or simply run the command below:
# sed -i -e "s/^Enabled=false/Enabled=true/" /etc/fwupd/remotes.d/lvfs-testing.conf
The Stable remote makes the FW available for all users of fwupd over the
World running on different operating systems (primarily Linux).
For the Chrome OS this repository isn’t enabled by default, so it is needed to
enable Stable remote explicitly. This makes the stable remote similar to the
testing one for the Chrome OS.
Google’s team is keeping its own mirror of LVFS, so the FW from the LVFS Stable
remote will be published for Chrome OS users only after some time.
This is the one-time action needed for enabling the stable remote for the Chrome
OS system.
By default the stable remote is already configured on Chrome OS but not enabled.
Check on Chrome OS device if the remote named LinuxVendorFirmwareService
is not enabled already:
# fwupdmgr get-remotes
Check the key Enabled: and if it is set to true please proceed
with
Enable the stable remote, use text editor to edit configuration file to
replace Enabled=false by Enabled=true in
/etc/fwupd/remotes.d/lvfs.conf or simply run the command below:
# sed -i -e "s/^Enabled=false/Enabled=true/" /etc/fwupd/remotes.d/lvfs.conf
After each update the fwupdmgr client tools allow the end user to submit a
“report” which is used by the firmware owner to validate the firmware deployment
is correct. Any failures can be analyzed and patterns found and the metadata can
be fixed. For instance, the failures might indicate that the required fwupd
version needs to be raised to a higher value, or that the update requires a
specific bootloader version.
It is expected that only F/W having signed reports would be automatically copied
into the LVFS mirror for Chrome OS.
To do this, the user must either upload the certificate from each machine used
for testing, or hardcode a user token.
Uploading a signed report on ChromeOS is slightly different than on regular Linux
distributions as /etc is immutable and cannot be changed.
By creating a file /var/lib/fwupd/remotes.d/lvfs.conf in a mutable directory
the fwupd daemon will reload the information and disregard the immutable
/etc/fwupd/remotes.d/lvfs.conf contents.
To use a host certificate rather than hardcoding Username and Password:
Go to the “Profile Settings”
Upload the fwupd certificate
Warning
The metadata must now be downloaded from the embargo remote using
fwupdmgrrefresh otherwise the message hasnoRemoteID will be
seen using fwupdmgrreport-history--verbose
The user can then upload reports to the LVFS in a trusted way by signing the
report:
$fwupdmgrupdate# or fwupdmgr install foo.cab# …reboot if required…
Since fwupd version 1.8.11 it is possible to record the firmware update of
the USB devices. This can be used for a failing update to allow the plugin
developer to replay the update for debugging purposes .
Enable device emulation support, use text editor to edit configuration file
to replace AllowEmulation=false by AllowEmulation=true in
/etc/fwupd/daemon.conf or simply run the command below:
# sed -i -e "s/^ AllowEmulation =false/ AllowEmulation =true/" /etc/fwupd/daemon.conf
Restart the fwupd daemon:
# restart fwupd
fwupdstart/running,process20199
Copy the “Device ID:” or “GUID” value from the target device, for example:
Disable device emulation support, use text editor to edit configuration file
to replace AllowEmulation=true by AllowEmulation=false in
/etc/fwupd/daemon.conf or simply run the command below:
# sed -i -e "s/^ AllowEmulation =true/ AllowEmulation =false/" /etc/fwupd/daemon.conf
Restart the fwupd daemon:
# restart fwupd
fwupdstart/running,process20199
The emulation file can now be sent to the plugin developer.
Enable device emulation support, use text editor to edit configuration file
to replace AllowEmulation=false by AllowEmulation=true in
/etc/fwupd/daemon.conf or simply run the command below:
# sed -i -e "s/^ AllowEmulation =false/ AllowEmulation =true/" /etc/fwupd/daemon.conf
Disable device emulation support, use text editor to edit configuration file
to replace AllowEmulation=true by AllowEmulation=false in
/etc/fwupd/daemon.conf or simply run the command below:
# sed -i -e "s/^ AllowEmulation =true/ AllowEmulation =false/"
/etc/fwupd/daemon.conf
If you want to update all supported devices attached to the ChromeBook, just run
fwupdmgrupdate to apply any FW update available from the enabled remote:
If several versions of the FW are available from enabled remote(s) it is
possible to perform downgrade to any available version lower than the current by
selection via the proposed menu. If only the single version for downgrade is
available you will need only to confirm the operation:
For tests in this section please use CAB files listed in section List of FWs
used in this doc or prepare own CAB
files and upload it to appropriate remote as described in LVFS section.
Most of the test cases are the same for all the HW, only CAB files and versions
are different. To unify the test cases for all devices, some variables are
defined and used in test cases for any device:
OLDCAB – URL or path to the CAB file of the previous version
NEWCAB – URL or path to the CAB file of the target version
For instance for ColorHug it is required to define URL to CAB files with
export command prior the test:
The example above sets the OLDCAB variable to the URL of the CAB file with FW
version 1.2.5, and the NEWCAB to the URL of the CAB file with FW version
1.2.6.
fwupd is a system daemon that allows an OS
to update the firmware of a wide array of peripherals. ChromeOS relies on it to
perform the firmware updates of compatible and updatable devices.
Moblab contains a set
of fwupd test suites to test the basic firmware-related operations that fwupd
can perform on a device. The purpose of these tests is, primarily, to validate
the correct firmware updatability of new peripherals so that they comply with
the WWCB Certification
and to check the consistency and correctness of these operations across
different ChromeOS versions and firmware releases.
In order to have access to ChromeOS test releases you’ll need a Partner Domain
account. To ask for an account, send a request to cros-pd-owners@google.com.
Note
Besides the Partner Domain account, you should also get access to a
CPCon account, a Service
account and a GCS bucket tied to the CPCon account.
How to ask for access to ChromeOS images for specific boards¶
Depending on the Chromebooks and Chromeboxes that you use for the tests, you’ll
need to have explicit access to the ChromeOS test images for each specific model
or board type so you can download them from the ChromeOS Partners Portal.
To request access, create an issue in the Partner Issue Tracker
using Component “ChromeOS Public Tracker > Services > Infra > Moblab”
(1038089) and template “Build Permission Access Request” and fill in the
template details.
Moblab is based on a Google Chromebox (Wukong or Wyvern). The distributor
of Moblab is CREATE TOGETHER TECHNOLOGY Co. Ltd. To inquire a quotation, reach
out to:
Network equipment as described in section “II - Requirements” of the Moblab
Instruction Manual,
at least two Ethernet cables and a USB to Ethernet dongle. Follow the
instructions in section “II.2 - Now that I have all the required hardware, how
do I connect them?” in the Moblab Instruction Manual to setup the test lab
A USB flash drive larger than 8GB
An hdmi or displayport monitor, a keyboard and a mouse
If the DUT is a Chromebox: an additional monitor, keyboard and mouse
In order to use a Chromebook or Chromebox for Moblab tests it needs to be
running a ChromeOS test release. Before continuing, make sure you have a partner
domain account with access to the ChromeOS Partners Portal
and to the ChromeOS images for the specific Chromebook/Chromebox board types you
want to test.
First, start by flashing a ChromeOS test image into a USB flash drive:
Go to the ChromeOS Partners Portal <https://www.google.com/chromeos/partner/fe/#release>
and log in with your partner domain account and click the “Image Files” tab.
Select the board of the Chromebook or Chromebox to test and
TEST_IMAGE_ARCHIVE in the “Image Type” drop-down menu, and click “Search” to
list all the ChromeOS test images for that board. You can refine the search
by entering a specific “Release/Milestone” and/or a specific “Version/prefix”.
Download a recent test image (dev or stable channels recommended) and
decompress it with:
$ tar -Jxvf <release_file.tar.xz>
Click the extension button in Chrome, then select Chrome
Recovery Utility
Select the gear icon in the window. Next, click Use local
image.
Select the chromium_test_image.bin file extracted in step 3
Plug in the USB flash drive and select it as the media to use. Click
Continue and then Create now. Wait until the image is completely
written to the USB drive.
Once complete, Select Done then unplug the USB flash drive
Next, to install the image in the Chromebook or Chromebox, follow these steps:
Put the device into Developer Mode
with the following procedure:
For Chromebooks, Hold Esc+Refresh and press the
Power button. For Chromeboxes, engage the small Reset pinhole with
a paperclip, hit Power and continue engaging Reset for 2 seconds.
This will put the device into Recovery Mode and it should show a
screen similar to this:
Or this, depending on the model:
Note
On some Chromebooks the combination to hold is Esc+Fullscreen instead.
In the Recovery Mode screen, press Ctrl+D, followed by Enter to
enter Developer Mode
Note
For other devices without keyboards (such as tablets) follow
these instructions
to enter Recovery Mode and Developer Mode
Wait until the process is done and the Developer Mode warning screen appears
Once the device is in Developer Mode, it will show the warning screen above
every time it boots. It’ll start ChromeOS after 30 seconds or if you press
Ctrl+D. Start ChromeOS and wait for it to show the welcome screen
Go to virtual terminal 2 to access a command line prompt by pressing:
[ Ctrl ] [ Alt ] [ → ]
where the [ → ] key is the right-arrow key just above the number 3 on the
keyboard. If the keyboard doesn’t have this key, use the key in the F2
position. Then log in with user: root
Now reboot and wait for the Developer Mode warning screen to appear, plug in
the USB flash drive and press Ctrl+U to boot the ChromeOS test image
from the USB drive.
Wait for ChromeOS to start
Once ChromeOS is running, go to virtual terminal 2 again and log in with
user: root and password: test0000. Then install the test image in the
hard disk with the following command:
$ /usr/sbin/chromeos-install
and follow the instructions
Once installation has completed, reboot the device (shutdown-hnow) and
remove the USB flash drive
Follow the instructions in the Moblab Introduction & User Manual
to configure the Moblab, connect the DUT (Chromebook or Chromebox) to it and
enroll it. The end result must be something like this, where the “Manage DUTs”
tab shows an enrolled DUT with a “Ready” Status:
The peripherals to test must be in working order. If they are meant to be
updated wirelessly, they must have sufficient battery level to ensure the
firmware update process can be completed successfully. They must also be
supported by fwupd and they must have at least a firmware release included in
the ChromeOS-specific fwupd remotes, which are defined in the latest .ebuild
file in https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/refs/heads/main/sys-firmware/fwupd-peripherals/.
The fwupd version might be different from one ChromeOS version to another, so a
device that is supported by fwupd in a newer ChromeOS version might not be
supported in an older one.
Each Moblab test run will start by provisioning the DUT, that is, updating its
ChromeOS version to the one specified by the tester and checking that it can run
the tests properly. This has some side effects:
The DUT will constantly check for an Ethernet link and will reboot
automatically if it doesn’t detect one after a few seconds.
All Bluetooth device pairings will be wiped out
A DUT can be also provisioned on demand by selecting the DUT in the “Manage
DUTs” tab and clicking on the “Provision DUTs” button, then selecting the
milestone and build to use:
The FWUPD tests are available only on ChromeOS R113-15382.0.0 and later.
Tests of firmware updates/downgrades/installs over Bluetooth links are not
properly supported at the moment
The procedure to run the tests is the same for all the test cases except for the
parameters they take. To start a test, follow these steps:
Make sure the DUT that will run the test is running, connected to the Moblab
network and that it shows up in the “Manage DUTs” tab in Moblab
Enroll the DUT for tests by selecting it in the “Manage DUTs” tab and then
clicking “Enroll Selected”
Make a note of the DUT IP address, as it’ll need to be specified later as one
of the test parameters
Connect the peripheral to the DUT and power it on
Go to the “Run Suite” tab and select “FWUPD” in the top menu
Select the model and build target of the selected DUT, and then the ChromeOS
milestone and build to run the test on. Important: only ChromeOS
R113-15382.0.0 and later. The tests won’t be started for any version older
than that.
Select the IP address of the DUT that will run the test
The rest of the steps depends on the test case to run:¶
This test allows the user to flash any available firmware release into a device,
regardless of the current version running on it. To run it:
Select the fwupd_install_version suite
Select the device you want to test
Input the release version you want to install. Note that the version must be
specified in the format defined by the hardware vendor (as a number pair,
triplet, hexadecimal number, etc.). The Device info at the bottom shows
the Current FW version with the expected format.
This test allows the user to flash a specific firmware file into a device. The
file can be provided either as a URL if it’s in a remote server, or through an
external drive connected to the DUT.
If the file is provided in an external drive, a USB flash drive is
recommended. Format it as FAT32 and set FWUPDTESTS as its label. Then copy the
firmware file to it and plug it to the DUT.
If the file is provided as a URL, it must be accessible for the DUT to
download.
To run the test:
Select the fwupd_install_file suite
Select the device you want to test
Enter the complete URL of the firmware file if it’s a remote file or the file
name if it’s a file provided through a USB flash drive
Once a test has started, it’ll show up in the “View Jobs” tab. You can enable
the “Auto Refresh” toggle switch to keep the job list updated as the job status
progress:
When the test finishes, its status will change to “COMPLETE”, the logs will be
stored locally and eventually uploaded to a Google Storage bucket. A cloud icon
in the “Logs” column means that the logs have been uploaded to Google Storage
and are no longer stored locally.
Note that for each test run there’ll be two entries in the jobs table: one
representing the test suite (control.fwupd_update in the image above) and
another representing the test proper (fwupd_FirmwareUpdate in the image).
Once the job has finished you can check the results. If the logs haven’t been
uploaded yet, you can check them directly by clicking on the “Logs” icon for the
test job (not the suite). From there, navigate to the “status.log” file:
The “status.log” file shows a summary of the test result:
When the logs are uploaded to Google Cloud Storage, you can check them through
CPCon, accessing with your
partner domain account and clicking on the “Autotest View” option. From there
you can see all the test suites run on every Chromebook type and ChromeOS
milestone:
Clicking on any of the test suite results will lead you to a detailed summary of
that suite. All the different test runs for that particular Chromebook type and
ChromeOS version will show up listed either as a “Non-Passed test” or as a
“Passing test”. The most recent test run will be the last log listed:
Clicking on any of the logs will direct you to the Google Storage bucket
directory containing the logs for that test run:
From there you can download the “status.log” file. The directory and file
structure is the same as in the locally stored logs.
If the “status.log” report doesn’t give enough information about a failed
test, you can also download the “debug/client.0.DEBUG” file, which contains
the full log including debug messages. Additionally, the “sysinfo/messages”
file contains the full system log that can also be useful to investigate a bug.
For instance, this “install file” test failed with this message in the status.log file:
How to find the board type of a Chromebook or Chromebox?¶
In the Chromebook/Chromebox, open Chrome and enter chrome://version in the
URL bar. In the info screen that will appear, the board type and variant will
show up in the Platform and Customization ID fields, respectively:
In some scenarios it could be useful or needed to run certain console commands
in the DUT or to retrieve data from it that can only be accessed through a
terminal interface. If the DUT is running a test ChromeOS release, it’ll have an
SSH daemon running so you can connect to it remotely.
Requirements:
A Linux PC with an SSH client installed. If running Ubuntu, it can be
installed with the following command: sudo apt install ssh
The Linux host must be able to ping the DUT. The easiest way to achieve this
is to connect the DUT and the Linux PC to the same local network. If the DUT
is connected to a Moblab through a wired connection you can also connect to
the Linux PC using a wireless connection
Assuming the host you’re connecting from is running Linux, in order to connect
to the DUT you’ll need to download the ChromeOS test keys and configure your ssh
client properly following these steps:
Download the SSH keys from this link
and copy them to ~/.ssh in the Linux host
Set the correct file permissions for the private key:
chmod0600~/.ssh/testing_rsa
Get the IP address of the DUT. If the DUT is connected to multiple networks,
we need the IP address of the NIC that’s connected to the Linux PC network.
To list the available connections and their IP addresses go to virtual
terminal 2 in the DUT ([Ctrl][Alt][→]) and type: ip-4-bra
localhost~# ip -br -4 aloUNKNOWN127.0.0.1/8wlan0UP192.168.1.137/24arc_ns0@if2UP100.115.92.129/30arc_ns1@if2UP100.115.92.133/30eth0UP192.168.231.25/24
In this example, if the DUT is connected to the Linux PC network through a
wireless link, then we can check that the PC can ping the DUT at
192.168.1.137.
Add the following to ~/.ssh/config:
Host dut
HostName $IP_ADDRESS
User root
CheckHostIP no
StrictHostKeyChecking no
IdentityFile ~/.ssh/testing_rsa
ControlMaster auto
ControlPersist 3600
Where $IP_ADDRESS is the IP address of the DUT
How to check the list of peripherals detected by fwupd?¶
If, for debugging purposes, you need to check the current list of peripherals
that fwupd is detecting, you can do so by running this command in a DUT
terminal:
If the DUT was provisioned (updated) using Moblab, it will check for an Ethernet
link and it will reboot if it doesn’t find one. Make sure to keep the DUT
connected to the Moblab network using an Ethernet link.
When you are done testing with the Chromebook, reflash it with a recovery image
to prevent it from restarting continuously when not connected to a network
through the Ethernet port.
and press Power.
button in Chrome, then select Chrome
Recovery Utility
in the window. Next, click Use local
image.
instead.
