[Tracking] Linux-compatible network control plane and userspace DHCP migration

[fslongjin]

Background

DragonOS currently starts an unconditional in-kernel DHCP worker during boot. The worker selects the first eth* interface, obtains a one-shot lease, and writes the address and default route directly into smoltcp state.

This differs from the normal Linux responsibility split: the kernel provides network devices, sockets, rtnetlink, and compatibility ioctls, while userspace owns DHCP, static addressing, DNS updates, retries, and multi-interface policy.

The problem is broader than moving one DHCP function:

• the in-kernel DHCP worker can race with an external userspace network manager;
• rtnetlink mutations do not yet share one request context, capability check, and global RTNL serialization boundary;
• address and route state have multiple writable views, so different mutation paths can update only part of the observable state;
• BusyBox udhcpc still needs a complete SIOCGIFHWADDR, UDP SO_BINDTODEVICE, and AF_PACKET path;
• removing the kernel worker before those prerequisites are ready could leave normal images without networking.

This issue tracks a staged migration: complete the Linux-compatible mechanisms first, land a dormant userspace DHCP service, switch ownership atomically in one PR, and then incrementally converge the network control-plane state.

Confirmed design decisions

• Normal images will ultimately use userspace DHCP by default.
• Externally managed mode will not start DHCP; an external userspace manager owns interface configuration.
• rtnetlink is the primary link/address/route/neighbor control ABI.
• Legacy ioctls are compatibility adapters and must reuse the same mutation core.
• All control-plane mutations use shared authorization, locking, validation, commit, rollback, and notification rules.
• A partially compatible ip=dhcp implementation is out of scope. Linux IP_PNP should only be implemented for a real early-root networking requirement.
• Cube has already booted successfully in the target environment. This issue does not include a Cube harness, Cube Agent testing, or Cube E2E testing.

Removing Cube E2E from this roadmap does not relax the generic ABI requirements. Linux-compatible capabilities, errno values, ACKs, dumps, concurrency behavior, and control-plane/data-plane consistency remain required.

Goals

After this issue is complete:

1. normal boot no longer runs in-kernel DHCP;
2. userspace owns DHCP acquire, renew, rebind, deconfigure, and DNS updates;
3. one interface has exactly one policy owner at a time;
4. rtnetlink mutations follow Linux 6.6.139 authorization and RTNL serialization semantics;
5. BusyBox udhcpc completes its lifecycle through standard Linux ABIs;
6. address, route, neighbor, and link state have explicit authoritative control-plane ownership;
7. every implementation PR builds, boots, tests, and reverts independently after its declared dependencies have merged.

Non-goals

• Cube startup, Agent, container, or E2E testing;
• a DragonOS- or Cube-specific set-IP syscall/ioctl/netlink family;
• NetworkManager, systemd-networkd, or a full distribution network manager;
• DHCPv6, a userspace IPv6 RA manager, policy routing, or multipath routing;
• Linux IP_PNP / NFS-root early networking;
• a one-shot rewrite of the network stack;
• timing delays, repeated address deletion, ignored ACKs, or other workarounds.

Target architecture

flowchart TB
    U[Userspace policy<br/>udhcpc / static / external manager]
    N[NETLINK_ROUTE adapter]
    I[socket ioctl compatibility adapter]
    C[RtnlRequestContext<br/>cred + netns + portid + origin]
    A[Capability checks]
    L[Global RTNL mutex]
    O[Typed link / address / route / neighbor operations]
    S[Authoritative control state]
    D[smoltcp / Router / driver projections]

    U --> N
    U --> I
    N --> C
    I --> C
    C --> A --> L --> O --> S --> D

Loading

Mutation order:

parse/copy userspace input
  -> snapshot sender credentials and target netns
  -> authorize
  -> acquire global RTNL
  -> prepare/validate/reserve
  -> commit authoritative state and data-plane projections
  -> enqueue ordered notifications after successful commit
  -> release RTNL
  -> return ACK/errno

Required lock order:

RTNL_MUTEX
  -> netns device/topology state
    -> interface control state
      -> smoltcp/data-plane locks
        -> netlink notification queue (enqueue only; no control-plane callback)

PR independence rules

“Independent PR” means:

• a PR only depends on explicitly listed PRs that are already merged;
• it does not rely on unpublished stacked commits;
• the branch builds and boots after that PR is merged;
• behavior changes and their dunitest coverage are in the same PR;
• the PR can be reverted without requiring a later PR;
• PR-01 through PR-07 do not change default network ownership;
• PR-07 lands userspace DHCP in a dormant state;
• PR-08 is the only PR that changes the default boot policy;
• unexpected dependencies require an update to this tracking issue before implementation scope expands.

Dependency graph

flowchart LR
    P1[PR-01<br/>RTNL context and authorization]
    P2[PR-02<br/>Global RTNL serialization]
    P3[PR-03<br/>Network query ioctls]
    P4[PR-04<br/>UDP SO_BINDTODEVICE]
    P5[PR-05<br/>AF_PACKET DHCP prerequisites]
    P6[PR-06<br/>Centralize address mutations]
    P7[PR-07<br/>Dormant userspace DHCP]
    P8[PR-08<br/>Atomic DHCP cutover]
    P9[PR-09<br/>Authoritative route state]
    P10[PR-10<br/>Authoritative neighbor state]
    P11[PR-11<br/>Transactional link mutations]

    P1 --> P2
    P2 --> P6
    P3 --> P7
    P4 --> P7
    P5 --> P7
    P6 --> P7
    P7 --> P8
    P2 --> P9
    P6 --> P9
    P2 --> P10
    P2 --> P11

Loading

PR-03, PR-04, and PR-05 may proceed in parallel. PR-09, PR-10, and PR-11 may proceed independently after their declared foundations merge. PR-08 is the only default-behavior switch.

Tracking checklist

PR-01: RTNL request context and mutation authorization

Suggested title: fix(net): enforce capabilities for rtnetlink mutations

• Snapshot sender Cred, target NetNamespace, port ID, and request origin at the route-netlink entry.
• Pass an explicit RtnlRequestContext through handlers.
• Check CAP_NET_ADMIN in the target netns owning user namespace for mutation requests.
• Keep GET/dump authorization message-specific and Linux-compatible.
• Match Linux error precedence, including EPERM versus object lookup errors.
• Add QEMU-run dunitest coverage for permissions and namespace isolation.

Dependency: none. Default DHCP behavior remains unchanged.

PR-02: Global RTNL serialization and ACK/notification boundaries

Suggested title: refactor(net): serialize network control mutations with RTNL

• Add one global RTNL_MUTEX.
• Replace the link-local RTNL_LINK_LOCK.
• Serialize link/address/route/neighbor mutations in one domain.
• Keep parsing and authorization before the lock.
• Commit state and enqueue notifications under the lock; send ACK after unlock.
• Add concurrent rename/address/route/neighbor dunitests and rollback checks.
• Ensure interrupt and data-plane hot paths never acquire RTNL in reverse order.

Dependency: PR-01. Functional behavior should remain equivalent apart from corrected ordering.

PR-03: Network-device query ioctls required by udhcpc

Suggested title: feat(net): support Linux network device query ioctls

• Implement SIOCGIFHWADDR.
• Add the required SIOCGIFFLAGS and SIOCGIFMTU query semantics.
• Cover ifreq layout, name termination, missing devices, address family, and user-copy failures.
• Add socket_ioctl_netdev_query dunitest coverage.
• Remove the need for MAC fallback in existing AF_PACKET tests.

Dependency: none. This PR does not mutate network configuration.

PR-04: Linux-compatible UDP SO_BINDTODEVICE

Suggested title: feat(net): implement SO_BINDTODEVICE for UDP sockets

• Store the bound interface state in UDP sockets.
• Enforce the binding in send, receive, and local interface selection.
• Match Linux 6.6.139 behavior for initial bind, rebind, empty-string unbind, missing device, name length, and capability checks.
• Handle rename/delete and netns isolation.
• Reuse common socket-option logic rather than duplicating raw-socket semantics.
• Add QEMU-run dunitests for data-path filtering and lifecycle behavior.

Dependency: none. UDP behavior without this option remains unchanged.

PR-05: AF_PACKET DHCP-client prerequisites

Suggested title: test(net): cover AF_PACKET DHCP client socket semantics

• Cover PF_PACKET/SOCK_DGRAM/ETH_P_IP.
• Cover sockaddr_ll bind, DHCP broadcast frames, and PACKET_AUXDATA.
• Fix any discovered implementation gap in the same PR.
• Keep the test generic; do not add Cube or a DHCP server to normal unit-test CI.
• Avoid fixed interface indices and hard-coded MAC addresses.

Dependency: none. Startup networking remains unchanged.

PR-06: Centralize address mutations and connected-route synchronization

Suggested title: refactor(net): centralize interface address mutations

• Add typed address add/delete/replace operations.
• Route rtnetlink and internal callers through the same mutation path.
• Remove the mismatch between IfaceCommon::update_ip_addrs() and manual sync_router_ip_addrs().
• Commit or roll back addresses and connected routes in one RTNL transaction.
• Cover multiple addresses, duplicate operations, dump consistency, and rollback.
• Keep the existing kernel DHCP functional until PR-08.

Dependency: PR-02. Default DHCP ownership remains unchanged.

PR-07: Dormant userspace DHCP service

Suggested title: feat(user): add dormant userspace DHCP service

• Add a DragonOS-specific BusyBox udhcpc lease script.
• Use explicit /bin/busybox ip; do not depend on legacy ifconfig or route.
• Make bound, renew, and deconfig idempotent.
• Store per-interface lease state under /run.
• Update DNS configuration atomically.
• Model dhcp, static, and external/unmanaged modes.
• Do not connect the service to default boot in this PR.
• Validate acquire/renew/rebind/deconfigure on a dedicated second interface or another topology not managed by kernel DHCP.
• Verify failure cleanup and no-server behavior in QEMU.

Dependencies: PR-03, PR-04, PR-05, and PR-06.

PR-08: Atomic cutover to userspace DHCP

Suggested title: refactor(init): move default DHCP policy to userspace

• Enable the PR-07 service by default for normal images.
• Remove the unconditional net_init() DHCP call from the initial kernel thread.
• Remove or isolate the old kernel DHCP worker.
• Do not start DHCP in static or external/unmanaged mode.
• Resolve ownership of fixed versus lease-provided DNS configuration.
• Validate normal DHCP boot, renew, no-server boot, static mode, and unmanaged mode in QEMU.
• Prove that only one policy owner exists for each interface.
• Keep this PR independently revertible to the previous default behavior.

Dependency: PR-07. Enabling userspace DHCP and disabling kernel DHCP must remain in this single PR. Cube E2E is not an acceptance criterion.

PR-09: Authoritative route state and precise route identity

Suggested title: refactor(net): make route state authoritative

• Define route identity using family, destination/source, table, priority, protocol, scope, type, output interface, gateway/next-hop, and relevant flags.
• Generate dump, lookup, and data-plane projections from one authoritative state.
• Implement Linux-compatible NLM_F_CREATE/EXCL/REPLACE.
• Delete exact matching routes instead of over-deleting by destination.
• Preserve the PR-06 connected-route transaction boundary.
• Cover same-prefix routes with different metrics/gateways, rollback, and netns isolation.

Dependencies: PR-02 and PR-06. It may proceed independently of PR-08.

PR-10: Authoritative neighbor state

Suggested title: refactor(net): centralize rtnetlink neighbor state

• Define authoritative neighbor entries.
• Unify rtnetlink add/delete/dump and ARP/ND projections.
• Match Linux behavior for duplicate, replace, state, flags, missing devices, and netns.
• Cover IPv4 and IPv6 static neighbors.
• Do not expand the full dynamic NUD state machine without a separately reviewed requirement.

Dependency: PR-02. It may proceed independently of PR-08 and PR-09.

PR-11: Transactional link mutations and ioctl reuse

Suggested title: refactor(net): unify link mutations across rtnetlink and ioctl

• Centralize rename, MTU, administrative flags, and NOARP mutations.
• Validate all attributes before committing a multi-attribute request.
• Add mutation ioctls such as SIOCSIFFLAGS and SIOCSIFMTU only where compatibility requires them.
• Make ioctl handlers parse ifreq and call the same control core.
• Cover permission failures, name conflicts, invalid MTU, device disappearance, concurrency, and rollback.
• Do not add legacy route ioctls without a separate requirement.

Dependency: PR-02. It may proceed independently of PR-08, PR-09, and PR-10.

Validation required for every PR

Each PR description must document:

• the corresponding Linux 6.6.139 source paths and semantics;
• the pre-change DragonOS behavior or why direct reproduction was impractical;
• the affected DragonOS modules and architectural ownership;
• capability, errno, locking, lifetime, rollback, and edge-case handling;
• updated/new dunitest coverage;
• successful make kernel;
• execution of the focused dunitest inside a DragonOS QEMU guest;
• real QEMU boot/data-plane validation when startup or DHCP behavior changes;
• declared merged dependencies and single-PR revert behavior.

After implementation, re-review the result against Linux 6.6.139. If that review finds a semantic mismatch, architectural problem, missing edge case, concurrency/lifetime risk, or workaround, return to design before continuing.

Logical milestones

M1: Userspace DHCP migration

• PR-01 RTNL context and authorization
• PR-02 global RTNL serialization
• PR-03 network-device query ioctls
• PR-04 UDP SO_BINDTODEVICE
• PR-05 AF_PACKET DHCP prerequisites
• PR-06 centralized address mutations
• PR-07 dormant userspace DHCP
• PR-08 atomic DHCP cutover

M2: Control-state convergence

• PR-09 authoritative route state
• PR-10 authoritative neighbor state
• PR-11 transactional link mutations and ioctl reuse

These are logical groupings for this issue; they do not require pre-existing GitHub milestone objects.

Completion criteria

• PR-01 through PR-11 are merged, or an approved replacement/cancellation is recorded here.
• Normal DHCP, static, and external/unmanaged boot modes have reproducible validation.
• A normal image has only a userspace DHCP policy owner.
• rtnetlink mutations share request context, authorization, and RTNL serialization.
• ABI handlers no longer maintain inconsistent address/route/neighbor/link copies directly.
• User-visible behavior has dunitest coverage; startup/DHCP changes have real QEMU validation.
• Documentation reflects the final implementation.
• No Cube-specific API, dual DHCP ownership, timing workaround, or ignored error is introduced.

Deferred follow-ups

Open separate issues if product requirements arise for:

• Linux IP_PNP / ip= / NFS-root early networking;
• DHCPv6 and userspace IPv6 RA management;
• multipath, policy routing, or advanced rtnetlink attributes;
• full dynamic neighbor/NUD expansion;
• Cube-specific E2E regression after a Cube/Agent integration change.