Embedded Linux System with Custom Character Driver and Ioctl-Enabled TCP Service

A full-stack embedded Linux system built using Yocto (Kirkstone), targeting QEMU AArch64, integrating a custom Linux kernel character device driver, a multi-threaded TCP socket server, and a userspace-to-kernel ioctl interface.

---

System Architecture

                    Host Machine
┌──────────────────────────────────────────────────────────┐
│  bitbake / Yocto build system                            │
│  netcat / sockettest.sh / drivertest.sh                  │
│        │                                                 │
│        ├── TCP :9000 ─────────────────────────────────┐  │
│        └── SSH :10022 (QEMU port-forwarded) ────────┐ │  │
└─────────────────────────────────────────────────────┼─┼──┘
                                                      │ │
                            ┌─────────────────────────▼─▼───────────────────────────┐
                            │         QEMU (qemuarm64) — Yocto Linux Image          │
                            │                                                       │
                            │   aesdsocket (userspace daemon, port 9000)            │
                            │       └── /dev/aesdchar  (USE_AESD_CHAR_DEVICE=1)     │
                            │              └── aesd-char-driver (kernel module)     │
                            │                      └── circular buffer (kernel)     │
                            └───────────────────────────────────────────────────────┘

---

Components

1. Yocto Layer (meta-aesd)

Custom Yocto layer providing:

• Kernel module recipe: aesd-char-module — builds and installs aesd-char-driver as a loadable kernel module
• Userspace application recipe: aesd-assignments — builds and installs aesdsocket as a daemon with sysvinit integration
• Image recipe: core-image-aesd — produces a complete bootable image for qemuarm64

Build the full image:

./build.sh          # sets up Yocto environment and runs bitbake

---

2. aesdsocket — Userspace TCP Server

Threading Diagram

Host Machine
┌──────────────────────────────────────────────────────────────┐
│  nc / sockettest.sh                                          │
│  connection 1 ──┐                                            │
│  connection 2 ──┤── TCP :9000                                │
│  connection N ──┘                                            │
└──────────────────────────────┬───────────────────────────────┘
                               │ QEMU port forwarding
                               ▼
┌──────────────────────────────────────────────────────────────┐
│                    aesdsocket (daemon)                       │
│                                                              │
│   main thread                                                │
│   └── accept loop                                            │
│         ├── connection 1 → pthread_create → worker thread 1  │
│         │                                    └── recv/write  │
│         ├── connection 2 → pthread_create → worker thread 2  │
│         │                                    └── recv/write  │
│         └── connection N → pthread_create → worker thread N  │
│                                              └── recv/write  │
│                                    │                         │
│                              file_mutex                      │
│                                    │                         │
│                             /dev/aesdchar                    │
│                                    │                         │
└────────────────────────────────────┼─────────────────────────┘
                                     │
                               kernel space
                                     │
                    ┌────────────────▼───────────────┐
                    │     aesd-char-driver           │
                    │     circular buffer (10 max)   │
                    │     mutex-protected ops        │
                    └────────────────────────────────┘

A multi-threaded TCP daemon that:

• Listens on port 9000
• Handles concurrent clients using pthread, one thread per connection
• Accepts newline-delimited data packets per connection
• Stores received data in a backend (file or kernel device)
• Returns the full accumulated backend content to the client after each packet

Supports two storage backends, selected at compile time:

Mode	Backend	Flag
Device mode (default)	/dev/aesdchar	USE_AESD_CHAR_DEVICE=1
File mode	/var/tmp/aesdsocketdata	USE_AESD_CHAR_DEVICE=0

In device mode:

• Timestamp printing is disabled
• The /dev/aesdchar node is not removed on exit
• File descriptor is opened per-connection (not at startup)

Ioctl Command Interface

When a client sends AESDCHAR_IOCSEEKTO:X,Y (newline-terminated) over the socket:

• The string is not written to the device
• X (write command index) and Y (byte offset within that command) are parsed
• ioctl(AESDCHAR_IOCSEEKTO) is issued to /dev/aesdchar on the same file descriptor
• The driver repositions the read offset to entry X, byte Y
• The device content from that offset is read back and returned to the client
• The same fd is used for both ioctl and read — closing/reopening would lose the offset

Example (from sockettest.sh):

# After writing swrite1..swrite10 to the device:

echo "AESDCHAR_IOCSEEKTO:0,2" | nc localhost 9000
# Returns: rite1\nswrite2\n...swrite10\n   (entry 0, skipping first 2 bytes "sw")

echo "AESDCHAR_IOCSEEKTO:8,6" | nc localhost 9000
# Returns: 9\nswrite10\n                   (entry 8 "swrite9\n", skipping "swrite")

Daemon Mode

Run with -d to daemonize:

aesdsocket -d

Double-forks, creates new session, redirects stdio to /dev/null, writes PID to /tmp/aesdsocket.pid. Handles SIGINT and SIGTERM gracefully — joins all worker threads before exit.

---

3. aesd-char-driver — Kernel Character Device Driver

A Linux kernel module (aesd-char-module) implementing /dev/aesdchar with:

• Circular buffer storing up to 10 write entries (dynamically allocated per write)
• Mutex-protected read/write/ioctl operations
• llseek support via generic_file_llseek_size for absolute byte positioning
• ioctl command AESDCHAR_IOCSEEKTO for two-level seek: entry index + intra-entry offset

The driver appends each newline-terminated write as a new circular buffer entry. Reads stream content sequentially from the current file offset.

Ioctl Definition

struct aesd_seekto {
    uint32_t write_cmd;         // circular buffer entry index (0-based)
    uint32_t write_cmd_offset;  // byte offset within that entry
};
#define AESDCHAR_IOCSEEKTO _IOW('k', 1, struct aesd_seekto)

---

Concurrency Model

• One pthread worker thread per accepted TCP connection
• file_mutex serializes all device read/write/ioctl operations across threads
• list_mutex protects the TAILQ-based thread tracking list
• Worker threads are tracked via a TAILQ linked list; completed threads are joined and freed during the accept loop
• On shutdown, all client sockets are shutdown(SHUT_RDWR) to unblock blocked recv() calls before joining

---

Testing

Tests run automatically via the Yocto CI pipeline using assignment-autotest:

./full-test.sh

Driver Test (drivertest.sh)

Directly writes to /dev/aesdchar and reads back using dd skip=N (byte-level seek):

# Writes write1..write10 to device, then:
dd if=/dev/aesdchar skip=2 bs=1   # skips first 2 bytes → "ite1\nwrite2\n..."
dd if=/dev/aesdchar skip=61 bs=1  # skips to entry 9    → "9\nwrite10\n"

Socket Test (sockettest.sh)

Tests the full stack end-to-end over TCP using nc:

1. Sends swrite1 through swrite10 as separate connections, verifying cumulative readback after each write
2. Sends AESDCHAR_IOCSEEKTO:0,2 — expects readback from entry 0, byte 2
3. Sends AESDCHAR_IOCSEEKTO:8,6 — expects readback from entry 8, byte 6

Each nc invocation is a separate TCP connection and thus a separate worker thread with its own file descriptor.

---

Build Configuration

Variable	Default	Description
USE_AESD_CHAR_DEVICE	1	1 = use /dev/aesdchar, 0 = use /var/tmp/aesdsocketdata
MACHINE	qemuarm64	Yocto target machine
DISTRO	poky	Yocto distro (Kirkstone 4.0.x)
Kernel version	5.15.x-yocto-standard	Linux kernel for qemuarm64

---

Shutdown Behavior

On SIGINT or SIGTERM:

1. Accept loop exits
2. All active client sockets are shut down to unblock worker threads
3. All worker threads are joined
4. Listening socket is closed
5. /dev/aesdchar node is preserved (device mode)
6. /var/tmp/aesdsocketdata is removed (file mode only)
7. PID file is removed (daemon mode)
8. Syslog: Caught signal, exiting

---

Limitations

• Single global circular buffer (10 entries max); oldest entry is overwritten when full
• No persistent storage across reboot or module reload
• No authentication or encryption on the TCP interface
• Ioctl only supports AESDCHAR_IOCSEEKTO; no arbitrary seek beyond that

---

Key Technical Concepts

• Linux kernel character driver with file_operations (read, write, llseek, ioctl)
• Circular buffer with dynamic per-entry allocation in kernel space
• Userspace ioctl via ioctl(2) syscall with custom command encoding (_IOW)
• Multi-threaded TCP server with per-connection worker threads
• Compile-time backend switching via preprocessor flag
• POSIX signal-safe shutdown with sig_atomic_t and sigaction
• Yocto recipe authoring: module.bbclass, update-rc.d, EXTRA_OEMAKE, SRCREV pinning