add file-system article + small fixes

1 files changed, 327 insertions, 0 deletions

diff --git a/file-system.md b/file-system.md
new file mode 100644
index 0000000..acd7ab5
--- /dev/null
+++ b/file-system.md
@@ -0,0 +1,327 @@
+% The File System
+
+---
+subtitle: WTF is Linux
+---
+
+[back](index.md)
+
+
+## Purpose
+
+If we look at storage devices as a way to store a bunch of ones and zeroes, we
+can see that this is not the most humane way of storing data. At the hardware
+level, there is no concept of file or directories, so we need a way of
+organizing these ones and zeroes in such a way that we can tell "such and such
+series of bytes corresponds to the `/home/user/file.txt` content".
+
+This is exactly what a file system is. At the core, a file system is a way of
+organizing data (we can this a "format", since it's a way of "formatting"
+data).
+
+Once we have some storage formatted for a known file system, we rely on the
+kernel to be able to read this file system.
+
+Then, once we know the kernel is able to read the file system, we want to tell
+the kernel where we would like to see it in our directory tree. This is called "[mounting](#mounts)" a file system.
+
+
+## Storage devices
+
+Before we talk about the file system properly, let's talk about different kinds
+of storage devices.
+
+We said in the introduction that "storage devices are a way to store a bunch of
+ones and zeroes", but different kinds of devices have different specificities.
+
+For example, with a Hard Disk Drive (HDD), you will want to read close data in
+quick succession, else you would have to wait for the next rotation of the
+disk.
+
+In Flash Memory, used in USB Flash drives and often used in embedded systems,
+you cannot set a single 0 to a 1, but instead have to erase a whole "sector" to
+set every bit in that sector back to a 1.
+
+While these specificities might have an impact in high end computing, or bare
+metal embedded systems (i.e. without a kernel), in our case, we thankfully
+don't have to care about any of that.
+
+Every particularity of these devices is abstracted away by "device drivers"
+implemented in the kernel, leaving us with a much simpler interface, like
+`read` and `write` system calls.
+
+
+## Some known file systems
+
+Just as there are multiple ways of storing an image in a file (jpeg, png, bmp,
+etc.), there are multiple file system formats.
+
+### FAT32
+
+FAT32 is probably the most "compatible" file system around.
+
+USB Flash Drives usually comes pre-formatted as FAT32, and when plug a USB
+Flash Drive into something (computer, router, gaming console, etc.), you can be
+pretty sure it'll work.
+
+It is not without drawbacks though: it is prone to fragmentation, can't handle
+files bigger than 4GB, and doesn't have many features that more modern file
+systems have (e.g. error recovery).
+
+### ext4
+
+ext4 is the de facto standard file system on GNU/Linux: when you install a
+GNU/Linux distribution, you will most likely end up with partitions formatted
+using ext4.
+
+### NTFS
+
+NTFS is the standard file system used by Windows computers. To my knowledge, it
+is not much used elsewhere, except to have Windows compatible external drives,
+and support files bigger than 4GB (so FAT32 is not a valid choice).
+
+
+## Enabling file systems in the kernel
+
+As an end user of a normal GNU/Linux distributions, there is usually no need to
+enable the support for any file system, as the most common ones are enabled by
+default.
+
+As an embedded system developer however, this is useful to know how to enable,
+disable file systems: you already know which file system(s) you're going to
+use, so you can only enable those, and disable the rest.
+
+Disabling unused file systems have two main benefits:
+
+- Less code in the kernel means a smaller kernel, and "size matters" is
+  actually true in the embedded world
+- Less code in the kernel also means less attack surface, so less potential
+  vulnerabilities in the kernel
+
+To enable or disable a file system in the kernel, simply go to your
+[configuration](kernel.md#configuring-the-kernel), look up your file system in
+the "File Systems" menu, and enable or disable it.
+
+## Usual format of a file system
+
+Because a file system stores more that just the content of files, it needs
+space for this "meta data". Different file systems do this differently, and
+have different needs, so let's quickly look at a few examples:
+
+TODO: reference source of image
+
+![FAT32 format](./res/fat32.jpg)
+
+The FAT32 way of doing things is the simplest: it simply has a table of files
+and directories at the beginning, and space afterwards for the content of
+files.
+
+## Partitioning schemes
+
+We've seen how a file system works, but we're missing a crucial component of
+the storage story: partitions.
+
+While having an ext4 file system take up a full hard drive is possible, this is
+usually not done in practice.
+
+In a lot of cases, we want *several* file systems in a single hard drive. An
+embedded example is having two system partitions, and if booting to one
+partitions fails, fall back on booting on the other (in case a system upgrade
+broke something, for example).
+
+Like everything else in computer science, you have several ways of doing it: in
+a similar to file systems, partitioning schemes are a way of formatting ones
+and zeroes, but instead of storing files, it stores file systems.
+
+The two most common partitioning schemes are GPT (the most modern one) and MBR
+(which you'll often find on older systems).
+
+TODO: reference source of image
+
+![GPT format](./res/GUID-Partition-Table-Scheme.svg)
+
+
+Like file format, you'll find sections dedicated to the meta data of the
+partitions (i.e. name of the partitions, id of the partition, etc.), and
+a whole space (partition data) to put your file system.
+
+## Block devices
+
+
+Now that we know how partitions and file systems work, let's see how that
+integrates with the GNU/Linux ecosystem.
+
+With the "everything is a file" philosophy of Linux, we can safely expect to
+see our disks and partitions represented as files. These special kind of files
+are called "block devices", and can be found in the `/dev` directory.
+
+Depending on how your disk is connected to your computer, the block device is
+going to be named differently in `/dev`.
+
+On most systems, hard disk drives and SSDs are connected using an
+[SCSI](https://en.wikipedia.org/wiki/SCSI) connection, so you will find your
+drives under `/dev/sda`, `/dev/sda1`, `/dev/sdb`, etc.
+
+The naming scheme goes like this:
+
+```
+       ,----- Represents the whole first SCSI disk the kernel found
+       |
+       v
+/dev/sda
+/dev/sda1 <-- Represents the first  partition of the first disk
+/dev/sda2 <-- Represents the second partition of the first disk
+
+       ,----- Represents the whole second SCSI disk the kernel found
+       |
+       v
+/dev/sdb
+/dev/sdb1 <-- Represents the first  partition of the second disk
+/dev/sdb2 <-- Represents the second partition of the second disk
+```
+
+So, if we wanted to read directly from the file `/dev/sda`, you would get
+exactly the bytes that are stored in your first storage device (and you could
+parse yourself the GPT or MBR format).
+
+But this is not what we usually want, we want to access the partitions inside
+the disk. For this, the kernel provides us with the files `/dev/sda1`,
+`/dev/sda2`, etc.
+
+In a similar fashion, if you were to read directly from the file `/dev/sda2`,
+you would get the bytes that are stored in the second partition of the first
+disk (and you could parse yourself the format ext4, fat32, etc.)
+
+Again, we do not usually want to access the bytes inside of the partition, but
+we want to access the files stored inside of the file system of the partition!
+To that end, what we do is [mount](#mounts) the file system.
+
+Accessing the raw bytes of the disk or partition is often useful, however. It
+can be used to list all the partitions in a disk, resize partitions, or get
+some information of a given file system.
+
+To read or modify partitions, you can use the `parted` command-line tool, or
+`gparted` graphical program:
+
+```bash
+user@host:~$ sudo parted -l
+
+Model: Thumb Drive (scsi)
+Disk /dev/sdb: 4041MB
+Sector size (logical/physical): 512B/512B
+Partition Table: gpt
+
+Number  Start   End     Size     File system  Name        Flags
+ 1      17.4kB  1000MB  1000MB                first
+ 2      1000MB  4040MB  3040MB                second
+```
+
+![GParted screenshot](./res/GParted_1.0_screenshot.png)
+
+To read or modify file systems, well it depends on the file system format you
+want to access.
+
+To access file systems in the ext4 format, you can use programs from the
+[e2fsprogs project](https://en.wikipedia.org/wiki/E2fsprogs).
+
+To access file systems in the fat32 format, you can use programs from the
+[dosfstools project](https://directory.fsf.org/wiki/Dosfstools).
+
+TODO: create a file system
+
+## Mounts
+
+"Mounting" a file system is the action of representing the files of a partition
+or disk into the directory hierarchy.
+
+For example, if the `/dev/sda1` file system is mounted, files of the first
+partition of the first disk will appear, and reading, writing, or changing the
+permissions of these files will instruct the kernel to read or modify the bytes
+stored in that first partition of the first disk (in the file data if
+reading/writing to the file, and in the file meta-data if modifying
+permissions, in our example).
+
+To mount a file system, we can use the `mount` command (being root is needed):
+
+```bash
+#                      ,----------- What file system to mount
+#                      |            (disk or partition)
+#                      |
+#                      |      ,---- Where to mount it (a directory)
+#                      |      |
+#                  vvvvvvvvv vvvv
+root@host:~$ mount /dev/sda1 /mnt
+```
+
+With this command, the kernel will try to auto detect the type of file system
+(ext4, fat, etc.), and provide you with the files inside that file system under
+the given directory (in this example: `/mnt`).
+
+The directory where a file system is mounted is called a "mount point".
+
+You can also specify mount options, with the `-o` or `--options` command-line
+argument. Common options include:
+
+- `ro`: read-only
+- `rw`: read-write
+- `defaults`: several common options
+- `sync`: when a file is read or written, do it directly on the hardware
+- `async`: when a file is read or written, do it asynchronously
+
+For more information, see the man page `man 8 mount`.
+
+
+## Special file systems
+
+One consequence of the "everything is a file" philosophy of Linux, is that not
+every file has to be tied to a storage device.
+
+For example, or files `/dev/sda`, ... that represent or hard drives and
+partitions don't have to be stored in a physical location. It's way better to
+create these files on boot according to the present hardware, and forget them
+when shutting down the machine.
+
+An often used file system like that is the `tmpfs` special file system. Like
+its name suggests, it is a temporary file system. Every file and directory in
+it is stored on memory only, and disappears on shutdown.
+
+Another commonly used temporary file system is the `devtmpfs` special file
+system. It is much like `tmpfs`, but instead of being created empty, all the
+block devices (among other things) like `sda`, `sda1`, ... are automatically
+created inside it.
+
+In that sense, the `/dev` directory really is just a standard mount point for
+the `devtmpfs` file system.
+
+Finally, two really useful file systems are the `sysfs` and `proc` special file
+systems.
+
+The `sysfs` is there to export information directly from the kernel. It can be
+related to hardware devices, drivers, etc. It is normally mounted under the
+`/sys` directory.
+
+The `proc` is mainly there to provide information about currently running
+processes. For example, doing `cat /proc/<PID>/cmdline` will get you the
+command-line arguments of the running process with the given `<PID>`. It is
+normally mounted under the `/proc` directory.
+
+There are also a lot of configuration options under the `/proc/sys` directory.
+
+## Further reading
+
+- "Design of the FAT file system" on Wikipedia
+: <https://en.wikipedia.org/wiki/Design_of_the_FAT_file_system>
+- ext4 on Wikipedia
+: <https://en.wikipedia.org/wiki/Ext4>
+- GPT on Wikipedia
+: <https://en.wikipedia.org/wiki/GUID_Partition_Table>
+- MBR on Wikipedia
+: <https://en.wikipedia.org/wiki/Master_boot_record>
+- Using the `parted` command line tool on LinuxJourney
+: <https://linuxjourney.com/lesson/disk-partitioning>
+- Manual of the `mount` command
+: <https://linux.die.net/man/8/mount>
+- Manual of `sysfs` file system
+: <https://man7.org/linux/man-pages/man5/sysfs.5.html>
+- Manual of `proc` file system
+: <https://man7.org/linux/man-pages/man5/proc.5.html>