Bob’s product#

Bob works for Super Security Solutions Corp.

His newest product will be a networking device that brings security monitoring features to networks.
The device will have no graphical user interface.
The device will receive Over-the-air updates.
No network interfaces are needed for interaction.
The device will need to meet additional security standards.

The following steps will be needed to turn his project into a product.

Create a project layer

First of all you will need to create a project layer.

You will need the following

a git server to host the repository (e.g. Github, Gitlab, your in-house git)
an empty repository (e.g. meta-myproduct)

Now you can create the project layer, as described by Create your project layer.

Create a project distro

Now it is time to create a custom distro.

Adjust the highlighted lines

The highlighted lines need to be configured according to your needs

For that create a file with the following content under <path/to/project-layer>/conf/distro/my-distro.conf.

DISTRO = "my-distro" # change the name according to your preferences
DISTRO_NAME = "My Distro"  # change the name according to your preferences
DISTRO_BASEVERSION = "1.2.3" # you can change the version according to your preferences
DISTRO_VERSION = "${DISTRO_BASEVERSION}"

SDK_VENDOR = "-mydistroosdk"
SDK_VERSION = "${DISTRO_VERSION}"
SDK_NAME = "${DISTRO}-${TCLIBC}-${SDKMACHINE}-${IMAGE_BASENAME}-${TUNE_PKGARCH}-${MACHINE}"
SDKPATHINSTALL = "/opt/mydistro/${SDK_VERSION}/${MACHINE}"

TARGET_VENDOR = "-mydistro"  # you can change the version according to your preferences

DISTROOVERRIDES .= ":my-distro" # needs to match setting from DISTRO

# add the DISTRO_FEATURES you need here, see below for more details
DISTRO_FEATURES = ""

TCLIBCAPPEND = ""

require conf/distro/include/no-static-libs.inc
require conf/distro/include/yocto-uninative.inc
require conf/distro/include/security_flags.inc
require conf/distro/include/simplecore-distro/preferred_versions.inc

INHERIT += "uninative"

DISTRO_FEATURES

Please select the DISTRO_FEATURES you require and add them to above’s config file at DISTRO_FEATURES. Possible options are

alsa - Include ALSA audio support
bluetooth - bluetooth support.
directfb - DirectFB (direct framebuffer graphics) support.
ipsec - IPSec support.
ipv6 - IPv6 support.
keyboard - keyboard support .
ldconfig - support for ldconfig and ld.so.conf on the target.
nfs - NFS client support (for mounting NFS exports on device).
opengl - Open Graphics Library, used for rendering two and three-dimensional graphics.
pci - PCI bus support.
pcmcia - PCMCIA/CompactFlash support.
ppp - PPP dialup support (also required for modem connections).
smbfs - SMB networks client support (for mounting Samba/Microsoft Windows shares on device).
systemd - support for SystemD.
usbgadget - USB Gadget Device support (for USB networking/serial/storage).
usbhost - USB Host support (allows to connect external keyboard, mouse, storage, network etc).
usrmerge - Merges the /bin, /sbin, /lib, and /lib64 directories into their respective counterparts in the /usr directory to provide better package and application compatibility.
wayland - the Wayland display server protocol, required for graphical applications.
virtualization - Support for container and other virtualization techniques.
wifi - WiFi support.

Only select what you need

Please select only the mandatory options. In case you are unsure omit the item

Now you can select your custom distribution config by setting in local.conf

DISTRO = "my-distro"

For DISTRO_FEATURES Bob chooses ipv6 systemd

Create a build manifest

To define what layers and settings your product image will require, it is needed to define a so called manifest.

These are files that define repositories and their used revision in a reproducible way.

First you will need to create a new repository on your git server.

There are different tools available, but we would recommend

repo
kas

repo is the reference tool used by Google Android to manage different repositories.

It uses xml files to configure what repository is checkout with what revision.

Create a manifest

Create a file named default.xml in your manifest repository.

You can create such a file with the help of Scotty.

Just run

$ scotty export --mode repo

to create a good starting point for your project

Create a config template

As repo only specializes in managing git repositories, you will need to create a bitbake template to configure the Yocto build.

You can create such a files with the help of Scotty.

Just run

$ scotty export --mode local.conf

# and

$ scotty export --mode bblayers.conf

to create a good starting point for your project

Build

$ repo init -u git://<URL to your git server hosting your project layer>.git
$ repo sync
$ TEMPLATECONF=sources/meta-my-project-layer/template/my-template source sources/poky/oe-init-build-env

now you can start your bitbake build with

$ bitbake my-image

kas is a widely known tool to manage bitbake layers and build configuration.

Create a manifest

Everything is controlled by a single yaml file.

You can create such a file with the help of Scotty.

Just run

$ scotty export --mode kas

to create a good starting point for your project

Build

$ kas build <yaml config file>

Create a license policy

Almost all the time it is advised to create a license policy for your product. This will prevent accidentally using software that isn’t licensed under terms you cannot agree on in your product.

A typical example is that, if you want to use secure boot or measured boot, it is not possible to use any e.g. GPL-3.0-or-later licensed software in your product.

Define a policy

You can define the policy on a DISTRO or on an image level. We recommend to use it at a DISTRO level.

To automatically filter out incompatible licenses add

INCOMPATIBLE_LICENSE += ""

to your distro conf.

The following licenses are known to cause issues, so we recommend to add them to the INCOMPATIBLE_LICENSE license list

INCOMPATIBLE_LICENSE += "\
    AGPL-3.0 \
    CC-BY-NC-1.0 \
    CC-BY-NC-2.0 \
    CC-BY-NC-2.5 \
    CC-BY-NC-3.0 \
    CC-BY-NC-ND-1.0 \
    CC-BY-NC-ND-2.0 \
    CC-BY-NC-ND-2.5 \
    CC-BY-NC-ND-3.0 \
    CC-BY-NC-SA-1.0 \
    CC-BY-NC-SA-2.0 \
    CC-BY-NC-SA-2.5 \
    CDDL-1.0 \
    EPL-1.0 \
    EPL-2.0 \
    EUPL-1.0 \
    EUPL-1.1 \
    GPL-1.0 \
    GPL-3.0 \
    GPL-3.0-with-autoconf-exception \
    LGPL-3.0 \
    MPL-1.1 \
    MS-PL \
    MS-RL \
"

Now the build is automatically terminated if a piece of software with any of above’s licenses is tried to be installed into your image

What to do if it breaks your build

In case your build breaks, you have the following options

see if you can remove the need for mentioned component.
- e.g. use POSIX shell instead of bash
- e.g. replace the dependency with a permissively licensed variant
- e.g. ask the author for a commercial license
see if the need can be deactivated by e.g. PACKAGECONFIG
and of course you can accept the license terms and remove the license from INCOMPATIBLE_LICENSE

Application recipes

For each of your product application can services you’ll need to create recipes. Please see Create a recipe on how to do that.

After following the guide Bob has created these recipes

secure-backend_git.bb
secure-monitor_git.bb

Application services

For each of your recipe (if they don’t ship systemd services on their own), you will need to create SystemD Service Units.

We recommend to create a new recipe for each of your applications/services.

create service unit

Under <path/to/project-layer>/recipes-project/<service name>/files/<service name>.service create a file with the following content

[Unit]
# A brief description of your service
Description=My custom service

[Service]
# the following line defines what to run
ExecStart=/usr/bin/my-service

[Install]
WantedBy=multi-user.target

create service recipe

Under <path/to/project-layer>/recipes-project/<service name>/<service name>_1.0.bb add the following content.

SUMMARY = "${BPN} systemd service"
LICENSE = "CLOSED"

SRC_URI = "file://${BPN}.service"

inherit allarch

S = "${WORKDIR}"

do_install() {
    install -d ${D}${systemd_system_unitdir}
    install -m 0644 ${WORKDIR}/${BPN}.service ${D}${systemd_system_unitdir}
}

FILES:${PN} += "${systemd_system_unitdir}"
RDEPENDS:${PN} += "<name of the application this service is for>"

e.g. if the application recipe is called backend the recipe needs to look like that

SUMMARY = "${BPN} systemd service"
LICENSE = "CLOSED"

SRC_URI = "file://${BPN}.service"

inherit allarch

S = "${WORKDIR}"

do_install() {
    install -d ${D}${systemd_system_unitdir}
    install -m 0644 ${WORKDIR}/${BPN}.service ${D}${systemd_system_unitdir}
}

FILES:${PN} += "${systemd_system_unitdir}"
RDEPENDS:${PN} += "backend"

For Bob’s application he will need to create

secure-backend-service
secure-monitor

Defining the image

Time to create your project specific image, create a recipe in your project specific layer. E.g. at <path/to/project-layer>/recipes-core/images/my-image.bb

SUMMARY = "My project image"

LICENSE = "CLOSED"

inherit core-image

# Install additional packages/recipe by adding to the
# following line
IMAGE_INSTALL += ""

To add the right recipes for this image Bob adds

IMAGE_INSTALL += "\
   secure-backend-service \
   secure-monitor \
"

to his image

Trim down of the U-Boot config

To reduce the used u-boot configuration you will need to run the following commands from a bitbake environment

$ bitbake u-boot-imx-msc -c do_menuconfig

Now remove the options you are sure you don’t need. Hit ? to get an explanation on each item. When done save the configuration.

Now run

$ bitbake u-boot-imx-msc -c do_diffconfig

Save the output to a new file under <path/to/project-layer>/recipes-bsp/u-boot-imx-msc/files/my-project.cfg

and finally create a recipe append at <path/to/project-layer>/recipes-bsp/u-boot-imx-msc/u-boot-imx-msc_%.bbappend with the following content

FILESEXTRAPATHS:prepend := "${THISDIR}/${BPN}:"
SRC_URI:append = " file://my-project.cfg"

Trim down of the Linux kernel config

To reduce the used kernel configuration you will need to run the following commands from a bitbake environment

$ bitbake linux-imx-msc -c do_menuconfig

Now remove the options you are sure you don’t need. Hit ? to get an explanation on each item. When done save the configuration.

Now run

$ bitbake linux-imx-msc -c do_diffconfig

Save the output to a new file under <path/to/project-layer>/recipes-kernel/linux/files/my-project.cfg

and finally create a recipe append at <path/to/project-layer>/recipes-kernel/linux/linux-imx-msc_%.bbappend with the following content

FILESEXTRAPATHS:prepend := "${THISDIR}/${BPN}:"
SRC_URI:append = " file://my-project.cfg"

Integrate OTA

There are several OTA solutions in the market, which offer you an off-the-shelf experience.

Here an overview of a few of the options

Mender
foundries.io
Rauc

Mender is one of the most known platforms to provide over-the-air updates. It also will give you to possibility to self-host update servers.

foundries.io is a well known platform for over-the-air updates. It will add your applications on top on a well maintained BSP coming from foundries.

RAUC offers you a robust over-the-air update solution.

Secure Boot

Integration of secure boot and techniques beyond that, is sometimes tricky. If your product requires secure boot and/or other techniques, we recommend to talk to our experts.

Additional hardening of the product

When you want to secure your product further you might want to consider looking into the following topics

adding read-only-rootfs to DISTRO_FEATURES turn your image read-only
removing debug-tweaks from DISTRO_FEATURES
adding a password for the root account on the device
removing unwanted services, like ssh

the following video might give you some more pointers what to look into

Setup testing

As you more and more shape your project into a product, we recommend that you consider adding automated testing to your CI/CD pipelines.

From experience we recommend to have a look at

labgrid
or LAVA
alternatively you can consider the commercial Pluma offering

Update/Upgrade policy

When turning a project into a product you also need to think about how and when to update/upgrade your software.

Upgrade of layers

You will need to update the used revisions of your manifest on a regular basis.

It might be best to create CI automation for that to probe for updates to the selected branches on a regular basis.

Upgrade of your project specific recipes

You will also need to probe on a regular basis if there are changes to the recipes in your project specific layer.

Yocto provides a tools called devtool for that purpose.

First you will need to identify the recipe in your layer. From a sourced bitbake environment run

$ bitbake-layers show-recipes -l <your layer name> -r 2>/dev/null \
    | grep -v skipped | grep -v WARNING | grep -v NOTE | grep -v Loading \
    | grep -v Loaded | cut -d'=' -f2
# this will return something like
my-custom-recipe
my-application

now you can use on each of them devtool check-upgrade-status

for item in $(bitbake-layers show-recipes -l <your layer name> -r 2>/dev/null \
    | grep -v skipped | grep -v WARNING | grep -v NOTE | grep -v Loading \
    | grep -v Loaded | cut -d'=' -f2); do
    devtool check-upgrade-status $item
done

in case of an available update it will give you information like

INFO: my-recipe    1.0  new commits  None fdc6e54294649a30557d2f09bb88f371c8ac0769

with the last information on that line being the new SRCREV.

Now you can do the update of the recipe.

Define a release process

You will also need to setup a process to create a release, which includes

setting the new release version in your distro configuration (DISTRO_BASEVERSION)
create a release with this patched information
saving the artifacts from the build

Depending on your product additional steps might be required.

Talk to our experts

In case you need further support

Storage for SBOMs and copy-left artifacts

As your product will most certainly contain Copyleft licensed code, you will need to make these parts of the code available to users that request it.

So one of the tasks will be to create a webserver, that will host archives created by the build.

Enable archive.bbclass

Add the following line to your distro config

INHERIT += "archiver"

Copy archives after each build

After each (release) build you will have to copy the files from

$ cp -r build/tmp/deploy/sources/* /path/to/your/webservers/www-root/

if the webserver is a remote machine you may consider using rsync for that.

You will likely be also required to provide Software Bills of Material (or short SBOM)

Enable create-spdx.bbclass

Add the following line to your distro config

INHERIT += "create-spdx"
SPDX_INCLUDE_SOURCES = "1"
SPDX_UUID_NAMESPACE = "<URL of your webserver>/spdx"
SPDX_ORG = "<Your company name>"

Copy SBOMs after each build

After each (release) build you will have to copy the files from

$ cp -r build/tmp/deploy/spdx/* /path/to/your/webservers/www-root/spdx/

if the webserver is a remote machine you may consider using rsync for that.

You may also want to have a look at the following webinar, which provides further steps and recommendations

Talk to our experts

In case you need further support