What happened
RegisterFromNodeAnnotations creates a printedLog map and passes it to every register() call:
printedLog := map[string]bool{}
for {
...
s.register(labelSelector, printedLog)
}
register() adds a node to this map the first time it logs the node:
if printedLog[val.Name] {
klog.V(5).InfoS("Node device updated", ...)
} else {
klog.InfoS("Node device added", ...)
printedLog[val.Name] = true
}
The problem is that entries are never removed.
onDelNode already cleans up other per-node state when a node is deleted, but it cannot clean printedLog because the map is local to RegisterFromNodeAnnotations.
Why this is a problem
printedLog keeps growing for the lifetime of the scheduler. Each node name that is ever registered remains in the map.
- If a node is deleted and later recreated with the same name, the scheduler treats it as already seen and logs
Node device updated at V(5) instead of the expected info-level Node device added.
This can make it harder for operators to see that a replacement node has registered.
Expected behavior
printedLog should be cleaned up when a node is deleted, just like the other per-node state.
How to reproduce
- Run
hami-scheduler with the default log verbosity.
- Register a GPU node and confirm that
Node device added is logged.
- Delete the node.
- Recreate a node with the same name.
- When it registers again,
Node device added is not logged. Instead, it is logged as Node device updated at V(5).
Suggested fix
Move printedLog into the scheduler state and clean it up when the node is removed:
- Add
printedLog map[string]bool to the scheduler and initialize it in NewScheduler().
- Remove the
printedLog parameter from register().
- Delete the node from
s.printedLog in cleanupNodeUsage(), which is already called by onDelNode.
This keeps the change limited to pkg/scheduler/scheduler.go and reuses the existing per-node cleanup path.
Testing
Add a regression test covering:
node added → node deleted → same node name added again
The test should pass with -race.
Scope
Only pkg/scheduler/scheduler.go and pkg/scheduler/scheduler_test.go are affected. No API, annotation, or configuration changes are needed.
I'm happy to take this if the approach looks reasonable.
What happened
RegisterFromNodeAnnotationscreates aprintedLogmap and passes it to everyregister()call:register()adds a node to this map the first time it logs the node:The problem is that entries are never removed.
onDelNodealready cleans up other per-node state when a node is deleted, but it cannot cleanprintedLogbecause the map is local toRegisterFromNodeAnnotations.Why this is a problem
printedLogkeeps growing for the lifetime of the scheduler. Each node name that is ever registered remains in the map.Node device updatedatV(5)instead of the expected info-levelNode device added.This can make it harder for operators to see that a replacement node has registered.
Expected behavior
printedLogshould be cleaned up when a node is deleted, just like the other per-node state.How to reproduce
hami-schedulerwith the default log verbosity.Node device addedis logged.Node device addedis not logged. Instead, it is logged asNode device updatedatV(5).Suggested fix
Move
printedLoginto the scheduler state and clean it up when the node is removed:printedLog map[string]boolto the scheduler and initialize it inNewScheduler().printedLogparameter fromregister().s.printedLogincleanupNodeUsage(), which is already called byonDelNode.This keeps the change limited to
pkg/scheduler/scheduler.goand reuses the existing per-node cleanup path.Testing
Add a regression test covering:
node added → node deleted → same node name added againThe test should pass with
-race.Scope
Only
pkg/scheduler/scheduler.goandpkg/scheduler/scheduler_test.goare affected. No API, annotation, or configuration changes are needed.I'm happy to take this if the approach looks reasonable.