Skip to content

Commit 9ecfc83

Browse files
authored
feat(console): reach a maintenance node with no DHCP server (#385)
* feat(console): reach a maintenance node with no DHCP server With no DHCP server the kernel's ip=dhcp attempt gives up and leaves the interfaces down with no address, so a maintenance node had nothing to connect to and the console's Address line was empty. Maintenance and reprovision mode now run a link-local fallback: while no interface holds a reachable IPv4 address, it brings Ethernet links up and gives the first one with carrier an RFC 3927 address in 169.254.0.0/16, derived from its MAC so it is stable across boots. Only one interface gets one, so the node never holds two connected routes for the prefix. The console uses the new ReachableAddrs: the routable addresses when there are any, else the IPv4 link-local ones. IPv6 link-local addresses stay ignored, since the node configures no IPv6. The maintenance listener already binds 0.0.0.0, and the maintenance certificate names only localhost and is reached with --insecure, so no SAN change is needed. Closes #381 Signed-off-by: Bugs5382 <12115015+Bugs5382@users.noreply.github.com> * docs(management-trust): say how each laptop OS gets a link-local address Signed-off-by: Bugs5382 <12115015+Bugs5382@users.noreply.github.com> --------- Signed-off-by: Bugs5382 <12115015+Bugs5382@users.noreply.github.com> Co-authored-by: Bugs5382 <12115015+Bugs5382@users.noreply.github.com>
1 parent a4b9bae commit 9ecfc83

11 files changed

Lines changed: 619 additions & 11 deletions

File tree

‎docs/management-trust.md‎

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,25 @@ certificate; reach it with `--insecure`. It has no configured address, only the
9393
one DHCP gave it, so its console shows that address under **Address** and the
9494
certificate's **Mgmt SHA-256**.
9595
96+
With no DHCP server on the link (a laptop cabled straight to the node, for
97+
example), the node assigns itself an IPv4 link-local address (RFC 3927) in
98+
169.254.0.0/16 on the first interface with a cable in, and the console shows
99+
that address instead. The address is derived from the interface's MAC, so the
100+
node comes back on the same one every boot. A routable address always takes
101+
priority: the link-local one is only assigned, and only shown, when the node
102+
has no other IPv4 address. Give the laptop a 169.254.x.x address too (macOS
103+
and Windows assign one on their own when DHCP gets no answer; on Linux set the
104+
connection's IPv4 method to link-local, or add one with `ip addr add`) and
105+
reach the node at the console's address:
106+
107+
```sh
108+
cryptosctl --endpoint 169.254.x.x:443 --insecure config apply -f node.yaml
109+
```
110+
111+
The maintenance certificate names only `localhost`, whatever the address, so
112+
`--insecure` is what reaches it on a DHCP address and on a link-local one alike;
113+
the check is the console's **Mgmt SHA-256**, as below.
114+
96115
> [!CAUTION]
97116
> Compare the console's **Mgmt SHA-256** with the fingerprint the Fleet
98117
> Manager's adoption preview (or `openssl s_client`) shows before you adopt the
@@ -120,7 +139,8 @@ same form as on the serving dashboard:
120139
| Intermediate or issuing CA waiting for its parent | **Awaiting parent certificate** | Fetch trust, then get the CSR signed |
121140
122141
Only the maintenance screen also shows an **Address** line: the node's IPv4
123-
addresses, one per line. An installed node uses the address in its config.
142+
addresses, one per line, or its self-assigned 169.254.x.x address when it has
143+
no other. An installed node uses the address in its config.
124144
125145
The two screens an adoption compares against say which check they are: under
126146
the fingerprint, the maintenance screen shows **check 1/2: compare with the

‎internal/console/dashboard.go‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -50,8 +50,9 @@ func fleetLabel(s FleetState) string {
5050
// network or crypto identifiers, with two exceptions: MgmtFingerprint, the
5151
// SHA-256 of the public management certificate, which exists so an operator
5252
// can verify a pin against the node's own screen, and MgmtAddrs, which only
53-
// the maintenance screen shows, because a node with no config has a DHCP
54-
// address the operator cannot know in advance.
53+
// the maintenance screen shows, because a node with no config has a DHCP (or,
54+
// with no DHCP server, a self-assigned link-local) address the operator cannot
55+
// know in advance.
5556
type View struct {
5657
RootCN string
5758
Issuer string

‎internal/console/mgmtcert.go‎

Lines changed: 22 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -92,10 +92,26 @@ func readManagementCert(path string) *x509.Certificate {
9292
return cert
9393
}
9494

95-
// ManagementAddrs returns, in order, the addresses in addrs an operator can
96-
// reach the management listener on: IPv4 only, since the node configures no
97-
// IPv6, and neither loopback nor link-local.
95+
// ManagementAddrs returns, in order, the routable addresses in addrs an
96+
// operator can reach the management listener on: IPv4 only, since the node
97+
// configures no IPv6, and neither loopback nor link-local.
9898
func ManagementAddrs(addrs []net.Addr) []string {
99+
return ipv4Addrs(addrs, func(ip net.IP) bool { return !ip.IsLoopback() && !ip.IsLinkLocalUnicast() })
100+
}
101+
102+
// ReachableAddrs returns ManagementAddrs when there is at least one, else the
103+
// IPv4 link-local (169.254.0.0/16) addresses in addrs. A maintenance node on
104+
// a network with no DHCP server has only the link-local address it assigned
105+
// itself, and a laptop cabled straight to it reaches the listener there. IPv6
106+
// link-local addresses are never returned: the node configures no IPv6.
107+
func ReachableAddrs(addrs []net.Addr) []string {
108+
if out := ManagementAddrs(addrs); len(out) > 0 {
109+
return out
110+
}
111+
return ipv4Addrs(addrs, net.IP.IsLinkLocalUnicast)
112+
}
113+
114+
func ipv4Addrs(addrs []net.Addr, keep func(net.IP) bool) []string {
99115
var out []string
100116
for _, a := range addrs {
101117
var ip net.IP
@@ -106,20 +122,20 @@ func ManagementAddrs(addrs []net.Addr) []string {
106122
ip = v.IP
107123
}
108124
ip4 := ip.To4()
109-
if ip4 == nil || ip4.IsLoopback() || ip4.IsLinkLocalUnicast() {
125+
if ip4 == nil || !keep(ip4) {
110126
continue
111127
}
112128
out = append(out, ip4.String())
113129
}
114130
return out
115131
}
116132

117-
// LocalManagementAddrs returns ManagementAddrs for this host's interfaces, or
133+
// LocalManagementAddrs returns ReachableAddrs for this host's interfaces, or
118134
// nil when they cannot be read.
119135
func LocalManagementAddrs() []string {
120136
addrs, err := net.InterfaceAddrs()
121137
if err != nil {
122138
return nil
123139
}
124-
return ManagementAddrs(addrs)
140+
return ReachableAddrs(addrs)
125141
}

‎internal/console/mgmtcert_test.go‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -246,6 +246,69 @@ func TestManagementAddrsKeepsReachableIPv4(t *testing.T) {
246246
}
247247
}
248248

249+
func ipNet(cidr string) net.Addr {
250+
ip, n, err := net.ParseCIDR(cidr)
251+
if err != nil {
252+
panic(err)
253+
}
254+
return &net.IPNet{IP: ip, Mask: n.Mask}
255+
}
256+
257+
// With no DHCP server the node's only IPv4 address is the link-local one it
258+
// assigned itself, and that is what the console shows. A routable address
259+
// always wins, loopback is never shown, and IPv6 (the node configures none,
260+
// so any fe80:: address is the kernel's own) is ignored either way.
261+
func TestReachableAddrsFallsBackToLinkLocal(t *testing.T) {
262+
cases := []struct {
263+
name string
264+
addrs []net.Addr
265+
want string
266+
}{
267+
{"routable present", []net.Addr{ipNet("127.0.0.1/8"), ipNet("169.254.10.1/16"), ipNet("192.0.2.10/24"), ipNet("fe80::1/64")}, "192.0.2.10"},
268+
{"only link-local", []net.Addr{ipNet("127.0.0.1/8"), ipNet("169.254.10.1/16"), ipNet("fe80::1/64")}, "169.254.10.1"},
269+
{"two link-local in order", []net.Addr{ipNet("169.254.10.1/16"), &net.IPAddr{IP: net.ParseIP("169.254.20.2")}}, "169.254.10.1,169.254.20.2"},
270+
{"only loopback", []net.Addr{ipNet("127.0.0.1/8")}, ""},
271+
{"only IPv6 link-local", []net.Addr{ipNet("127.0.0.1/8"), ipNet("fe80::1/64")}, ""},
272+
{"none", nil, ""},
273+
}
274+
for _, tc := range cases {
275+
if got := strings.Join(console.ReachableAddrs(tc.addrs), ","); got != tc.want {
276+
t.Errorf("%s: ReachableAddrs = %q, want %q", tc.name, got, tc.want)
277+
}
278+
}
279+
}
280+
281+
func TestRenderDashboardMaintenanceAddressLineForEachAddressCase(t *testing.T) {
282+
fp := console.Fingerprint([]byte("maintenance cert"))
283+
cases := []struct {
284+
name string
285+
addrs []net.Addr
286+
want string
287+
}{
288+
{"routable present", []net.Addr{ipNet("169.254.10.1/16"), ipNet("192.0.2.10/24")}, "Address 192.0.2.10"},
289+
{"only link-local", []net.Addr{ipNet("127.0.0.1/8"), ipNet("169.254.10.1/16"), ipNet("fe80::1/64")}, "Address 169.254.10.1"},
290+
{"only loopback", []net.Addr{ipNet("127.0.0.1/8")}, ""},
291+
{"only IPv6 link-local", []net.Addr{ipNet("fe80::1/64")}, ""},
292+
}
293+
for _, tc := range cases {
294+
v := maintenanceView(fp, console.ReachableAddrs(tc.addrs)...)
295+
var got string
296+
for _, ln := range screenLines(console.RenderDashboard(v, 64, 24)) {
297+
ln = strings.Join(strings.Fields(strings.Trim(ln, "| ")), " ")
298+
if strings.HasPrefix(ln, "Address") {
299+
got = ln
300+
}
301+
}
302+
if got != tc.want {
303+
t.Errorf("%s: Address line = %q, want %q", tc.name, got, tc.want)
304+
}
305+
compact := stripSGR(console.RenderDashboard(v, 30, 10))
306+
if hasIP := strings.Contains(compact, "IP "); hasIP != (tc.want != "") {
307+
t.Errorf("%s: compact IP line shown = %v, want %v:\n%s", tc.name, hasIP, tc.want != "", compact)
308+
}
309+
}
310+
}
311+
249312
// awaitingCeremonyView is an installed Root node that has not run its
250313
// ceremony yet.
251314
func awaitingCeremonyView(fp string) console.View {

‎internal/init/linklocal.go‎

Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
package init
2+
3+
/*
4+
Copyright The CryptOS Authors.
5+
6+
Licensed under the Apache License, Version 2.0 (the "License");
7+
you may not use this file except in compliance with the License.
8+
You may obtain a copy of the License at
9+
10+
http://www.apache.org/licenses/LICENSE-2.0
11+
12+
Unless required by applicable law or agreed to in writing, software
13+
distributed under the License is distributed on an "AS IS" BASIS,
14+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
See the License for the specific language governing permissions and
16+
limitations under the License.
17+
*/
18+
19+
import (
20+
"context"
21+
"crypto/sha256"
22+
"encoding/binary"
23+
"errors"
24+
"log"
25+
"net"
26+
"net/netip"
27+
"time"
28+
29+
"github.com/CryptOS-PKI/cryptos-node/internal/console"
30+
"github.com/CryptOS-PKI/cryptos-node/internal/init/netlink"
31+
)
32+
33+
// linkLocalInterval is how often a maintenance node checks whether it needs a
34+
// self-assigned address. A laptop can be cabled in at any time after boot.
35+
const linkLocalInterval = 2 * time.Second
36+
37+
// nicState is the part of an interface the link-local fallback reads.
38+
type nicState struct {
39+
Name string
40+
MAC net.HardwareAddr
41+
Up bool
42+
Carrier bool
43+
Loopback bool
44+
Addrs []net.Addr
45+
}
46+
47+
// linkLocalDeps are the host operations the fallback needs, injectable so the
48+
// selection logic is tested without touching the host's interfaces.
49+
type linkLocalDeps struct {
50+
nics func() ([]nicState, error)
51+
linkUp func(name string) error
52+
addAddr func(name string, p netip.Prefix) error
53+
}
54+
55+
// linkLocalAddr returns the IPv4 link-local address (RFC 3927) a node with
56+
// this MAC assigns itself: drawn from 169.254.1.0 to 169.254.254.255 and seeded
57+
// from the MAC, as RFC 3927 section 2.1 recommends, so a node comes back on the
58+
// same address every boot and the operator can keep using it.
59+
func linkLocalAddr(mac net.HardwareAddr) netip.Addr {
60+
sum := sha256.Sum256(mac)
61+
n := binary.BigEndian.Uint16(sum[:2]) % (254 * 256)
62+
return netip.AddrFrom4([4]byte{169, 254, byte(1 + n/256), byte(n % 256)})
63+
}
64+
65+
// step makes one pass. When no interface has a reachable IPv4 address (the
66+
// kernel's ip=dhcp got no answer, and nothing has been assigned yet), it brings
67+
// down Ethernet links up so their carrier shows, and gives the first one with
68+
// carrier its link-local address in 169.254.0.0/16. Only one interface gets one,
69+
// so the node never holds two connected routes for the same prefix. It returns
70+
// the prefix it assigned and the interface, or the zero prefix when it did
71+
// nothing.
72+
//
73+
// No ARP probe is sent before claiming the address: the case this serves is a
74+
// laptop cabled straight to the node, where the only other host on the link
75+
// picks its own address the same way, and a clash is a 1 in 65024 chance that
76+
// the operator sees at once and fixes by replugging.
77+
func (d linkLocalDeps) step() (netip.Prefix, string, error) {
78+
nics, err := d.nics()
79+
if err != nil {
80+
return netip.Prefix{}, "", err
81+
}
82+
var all []net.Addr
83+
for _, n := range nics {
84+
all = append(all, n.Addrs...)
85+
}
86+
if len(console.ReachableAddrs(all)) > 0 {
87+
return netip.Prefix{}, "", nil
88+
}
89+
var errs []error
90+
for _, n := range nics {
91+
if n.Loopback || len(n.MAC) != 6 {
92+
continue
93+
}
94+
if !n.Up {
95+
if err := d.linkUp(n.Name); err != nil {
96+
errs = append(errs, err)
97+
}
98+
continue
99+
}
100+
if !n.Carrier {
101+
continue
102+
}
103+
p := netip.PrefixFrom(linkLocalAddr(n.MAC), 16)
104+
if err := d.addAddr(n.Name, p); err != nil {
105+
return netip.Prefix{}, "", errors.Join(append(errs, err)...)
106+
}
107+
return p, n.Name, errors.Join(errs...)
108+
}
109+
return netip.Prefix{}, "", errors.Join(errs...)
110+
}
111+
112+
// runLinkLocalFallback runs step every interval until ctx ends, so a maintenance
113+
// node with no DHCP server stays reachable on a self-assigned address. The
114+
// console picks the address up on its next refresh (console.ReachableAddrs).
115+
func runLinkLocalFallback(ctx context.Context, d linkLocalDeps, every time.Duration) {
116+
t := time.NewTicker(every)
117+
defer t.Stop()
118+
var lastErr string
119+
for {
120+
p, nic, err := d.step()
121+
if p.IsValid() {
122+
log.Printf("maintenance: no DHCP address; assigned link-local %s to %s", p, nic)
123+
}
124+
msg := ""
125+
if err != nil {
126+
msg = err.Error()
127+
}
128+
if msg != "" && msg != lastErr {
129+
log.Printf("maintenance: link-local fallback: %v", err)
130+
}
131+
lastErr = msg
132+
select {
133+
case <-ctx.Done():
134+
return
135+
case <-t.C:
136+
}
137+
}
138+
}
139+
140+
// startLinkLocalFallback starts runLinkLocalFallback on this host's interfaces
141+
// and returns the func that stops it.
142+
func startLinkLocalFallback(ctx context.Context) func() {
143+
ctx, cancel := context.WithCancel(ctx)
144+
go runLinkLocalFallback(ctx, linkLocalDeps{
145+
nics: localNICs,
146+
linkUp: netlink.SetLinkUp,
147+
addAddr: netlink.AddAddress,
148+
}, linkLocalInterval)
149+
return cancel
150+
}
151+
152+
func localNICs() ([]nicState, error) {
153+
ifs, err := net.Interfaces()
154+
if err != nil {
155+
return nil, err
156+
}
157+
out := make([]nicState, 0, len(ifs))
158+
for _, ifc := range ifs {
159+
addrs, err := ifc.Addrs()
160+
if err != nil {
161+
return nil, err
162+
}
163+
out = append(out, nicState{
164+
Name: ifc.Name,
165+
MAC: ifc.HardwareAddr,
166+
Up: ifc.Flags&net.FlagUp != 0,
167+
Carrier: ifc.Flags&net.FlagRunning != 0,
168+
Loopback: ifc.Flags&net.FlagLoopback != 0,
169+
Addrs: addrs,
170+
})
171+
}
172+
return out, nil
173+
}

0 commit comments

Comments
 (0)