path: root/2020-10-13.md

---
title: WTF is Linux
author: Rémi Nicole <remi.nicole@smile.fr>
date: 2020-10-13
slide-level: 2
aspectratio: 169

theme: metropolis
colortheme: owl
beameroption: "show notes on second screen=right"

toc: true
highlightstyle: breezedark
lang: en-US

bibliography: ../bibliography.bib
---

# Makefile primer

## Goal

Make is a tool to create files


::: notes

A Makefile is a file that will direct Make on how to create these files

:::

## Concepts---Rules[^rule-intro]

Target
: Usually the name of a file/directory that this rule creates

Prerequisites
: The files/directories that are needed to produce the target

Recipe
: The commands to execute in order to create the target

```make
my_executable: source_1.c source_2.c
	gcc source_1.c source_2.c -o my_executable
```

[^rule-intro]: <https://www.gnu.org/software/make/manual/make.html#Rules>

::: notes

- The target can also be an action, but it is rarer
- If one of the prerequisite changes, Make figures out that the target needs to
  be created again

:::

## Concepts---Variables[^variables]

Very similar to shell variables, but use `$(VAR)` to expand it.

```make
MY_SOURCES = source_1.c source_2.c

my_executable: $(MY_SOURCES)
	gcc $(MY_SOURCES) -o my_executable
```

[^variables]: <https://www.gnu.org/software/make/manual/make.html#Using-Variables>

## Concepts---Implicit Variables[^implicit-variables]

Some variables are set implicitly by Make, for example:

`$(CC)`{.no_minted}
: The C compiler

`$(CXX)`{.no_minted} / `$(CPP)`{.no_minted}
: The C++ compiler

`$(RM)`{.no_minted}
: The `rm -f` command

`$(MAKE)`{.no_minted}
: The `make` command

[^implicit-variables]: <https://www.gnu.org/software/make/manual/make.html#Implicit-Variables>

::: notes

Yes, you can call "make" inside of a Makefile

:::

## Concepts--Automatic Variables[^automatic-variables]

Some variables are set automatically per-rule, for example:

`$@`{.no_minted}
: The target of the current rule

`$<`{.no_minted}
: The first prerequisite of the current rule

`$^`{.no_minted}
: All prerequisites of the current rule

...

[^automatic-variables]: <https://www.gnu.org/software/make/manual/make.html#Automatic-Variables>

## Our example redone

```make
MY_SOURCES = source_1.c source_2.c

my_executable: $(MY_SOURCES)
	$(CC) $^ -o $@
```

## Relation with Buildroot

Buildroot uses Makefiles to:

- Figure out which packages to (re-)compile
- How to compile them
- Create the outputs (images, root filesystem, etc.) in general.

But it is heavily abstracted

# Setting up Buildroot for our modifications

## For additional packages

- We create the directory `package/<company>`
- We put our additional packages in it
- We create the `my_company.mk` and `Config.in` files[^adding-project-specific-packages]
- And include the new `Config.in` in `package/Config.in`

 [^adding-project-specific-packages]:
 	<https://buildroot.org/downloads/manual/manual.html#customize-packages>

## Packages Makefile

```makefile
include $(sort $(wildcard package/my_company/*/*.mk))
```

## Packages configuration file

In `package/my_company/Config.in`:

```kconfig
source "package/my_company/my_package1/Config.in"
source "package/my_company/my_package2/Config.in"
source "package/my_company/.../Config.in"
# ...
```

## Packages configuration file inclusion

In `package/Config.in`:

```kconfig
# ---8<--- Rest of the configuration

# We add our own packages in our own submenu
menu "My Company packages"
        # We just created this file
        source "package/my_company/Config.in"
endmenu

endmenu # <-- Was here before, it's the "Target packages" submenu
```

## For everything else

- We create the directory `board/<company>/<board>`
- We put our additional files, patches, configurations, scripts, etc. in it

::: notes

In our case, the board would be `raspberrypi2`.

:::

# Adding single files

## Overlays

A file hierarchy from the root directory that is going to get copied on top of
the built system

Can be used for:

- Startup scripts
- Program configuration files
- Files containing data (like sound)

::: notes

However, if a startup script, configuration file, or data file is tied to
a given package, it is usually better to install it with the package, instead
of using an overlay.

Overlays are more meant for files that are not tied to any specific package.

:::

## How to add an overlay

- Create the directory `board/<company>/<board>/rootfs_overlay`
- Add your file in that directory, as if it were the root directory
- Configure Buildroot (`make menuconfig`) and set
  `BR2_ROOTFS_OVERLAY`[^overlay-config] to
  `board/<company>/<board>/rootfs-overlay`.

[^overlay-config]:
	Under "System configuration", "Root filesystem overlay directories"

# Adding packages

## Packages---Structure

- Create the directory `package/<company>/<my-package>/`
- Create the file `package/<company>/<my-package>/Config.in`[^config-files]
- Create the file `package/<company>/<my-package>/<my-package>.mk`[^mk-files]

[^config-files]: <https://buildroot.org/downloads/manual/manual.html#_config_files>

[^mk-files]: <https://buildroot.org/downloads/manual/manual.html#_the_literal_mk_literal_file>

<!-- _____ -->

::: notes

- The `Config.in` will allow Buildroot to add an option to install your
  package. This will also specify your dependencies (what is needed to build or
  run your package).
- The `<my-package>.mk` is a Makefile which instructs how to compile and
  install your package.

:::

## Packages---Configuration

Example configuration:

```kconfig
config BR2_PACKAGE_DUMMY_PACKAGE
        bool "dummy package"
        default n
        depends on BR2_PACKAGE_BUSYBOX || BR2_PACKAGE_BASH
        help
          A dummy package for demonstration purposes
```

## Packages---Makefile

Example Makefile configuration:

```makefile
define DUMMY_PACKAGE_INSTALL_TARGET_CMDS
        $(INSTALL) -m 0755 -D \
                $(DUMMY_PACKAGE_PKGDIR)/dummy_script.sh \
                $(TARGET_DIR)/usr/bin
endef

$(eval $(generic-package))
```

::: notes

- This "package" just installs a script that is in the same directory of the
`.mk` file
- It is important that all variables 

:::

## Packages---More generic Makefile{.allowframebreaks}

```makefile
LIBFOO_VERSION = 1.0
LIBFOO_SOURCE = libfoo-$(LIBFOO_VERSION).tar.gz
LIBFOO_SITE = http://www.foosoftware.org/download
LIBFOO_LICENSE = GPL-3.0+
LIBFOO_CONFIG_SCRIPTS = libfoo-config
LIBFOO_DEPENDENCIES = host-libaaa libbbb

# ---8<---
```

\framebreak

If your package uses Makefiles to be built:

```makefile
# ---8<---

define LIBFOO_BUILD_CMDS
    $(MAKE) $(TARGET_CONFIGURE_OPTS) -C $(@D) all
endef

# ---8<---
```

`$(@D)`{.makefile} is the directory containing the downloaded sources.

\framebreak

```makefile
define LIBFOO_INSTALL_TARGET_CMDS
    $(MAKE) \
        $(TARGET_CONFIGURE_OPTS) PREFIX=$(TARGET_DIR)/usr \
        -C $(@D) \
        install
    # Or...
    $(INSTALL) -D -m 0755 $(@D)/libfoo.so* $(TARGET_DIR)/usr/lib
    $(INSTALL) -d -m 0755 $(TARGET_DIR)/etc/foo.d
endef

$(eval $(generic-packages))
```

::: notes

- In this instances, our package is called "libfoo"
- `TARGET_CONFIGURE_OPTS` sets options like the compiler to use, the
  compilation flags and other things

:::

## Packages---Notes

- This was for a "generic package"
- For package using common build frameworks (e.g. python, autotools, cmake,
  etc.), some things are already done for you[^mk-files]
- Don't forget to include the package's `Config.in` in
  `package/my_company/Config.in`

# Adding a service

## Buildroot's Busybox init

Every script or executable that is named like `/etc/init.d/S<something>` is
going to be started on boot

Two solutions then:

- Add a script to the overlay (if not tied to one of your own package)
- Install it with your package in the `<PACKAGE>_INSTALL_TARGET_CMDS`

# Adding users

## Using the board directory

- Create the file `board/<company>/<board>/users.txt`[^adding-users]
- Fill it with lines using the "makeusers" syntax[^makeusers-syntax]
- Set `BR2_ROOTFS_USERS_TABLES`[^users-conf] to `board/<company>/<board>/users.tx`

[^adding-users]: <https://buildroot.org/downloads/manual/manual.html#customize-users>

[^makeusers-syntax]: <https://buildroot.org/downloads/manual/manual.html#makeuser-syntax>

[^users-conf]: Under "System Configuration", "Path to the users tables"

## Using the board directory---Example

```kconfig
#  ,---------------------- name
#  |        ,------------- group      
#  |        |      ,------ password
#  |        |      |
# vvv    vvvvvv    v
  foo -1 libfoo -1 * - - - LibFoo daemon
#     ^^        ^^         ^^^^^^^^^^^^^
#      |         |               |
#      |         |               '--- comment
#      |         '------------------- GID
#      '----------------------------- UID
```

::: notes

- If UID or GID is "-1", buildroot will select an unused number
- If password is "\*", then there is no password you cannot login as this user
  (but you can still execute programs as this user)

:::

## In a package

Inside the `.mk` file.

```makefile
define LIBFOO_USERS
    foo -1 libfoo -1 * - - - LibFoo daemon
endef
```

# Saving the configuration

## Why it is important

- Right now, every modification to the configuration (added packages, overlay,
  etc.) is only in the build directory
- We want to save it so that people cloning our modified Buildroot can have the
  same configuration
- Please often verify that your modifications are saved by deleting your
  `output` directory, or downloading and compiling or project again.

::: notes

- This is necessary for me (the reviewer) since I want to build exactly what you
  have.

:::

## How to save the Buildroot configuration

Once you have done your configuration of Buildroot (`make menuconfig`), execute
these commands:

- Make sure the `BR2_DEFCONFIG` option[^defconfig-option] is set to
  `configs/<my_company_board>_defconfig`
- Execute `make savedefconfig` to save it

And commit your changes

[^defconfig-option]: Under "Build options", "Location to save buildroot config"

# References