You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
inventory-agent: deliver and schedule it on the node #635
inventory-agent is built and released but nothing installs it, configures it, or
runs it. It is the only inventory component with no deployment path at all, which
means the aggregator, viewer and inspector have nothing to aggregate.
Why it is a host process, not a workload
designs/inventory.md says "The agent runs as root on each node", and lists
deployment manifests for the aggregator, viewer and inspector only. The code
agrees, and rules out a DaemonSet as currently written:
One-shot.Execute() (internal/inventory/agent/collect.go:104-136) collects,
writes SQLite, publishes over gRPC and returns. No loop, no interval flag. A
DaemonSet pod can only use restartPolicy: Always, so a cleanly-exiting binary
would be restarted forever.
Root required.collect.go:105-107 hard-fails when os.Geteuid() != 0.
Deep host access./dev/ipmi0, the 0400 product_serial and product_uuid
under /sys/class/dmi/id, /proc/cpuinfo, /proc/meminfo.
--db ./inventory.db is relative to the working directory; the design calls
it a node cache.
#633 accordingly ships it as a GoReleaser binary and does not build an image for
it.
Work
1. systemd service plus timer. The one-shot, root-only shape fits a timer
exactly. Precedent for unit assets the node agent installs: pkg/agent/phases/rootfs/assets/localdns.service, config-regeneration.service, pkg/agent/phases/nodestart/assets/kubelet.service.
2. Binary delivery. Either through the node agent's goalstate machinery
(pkg/agent/goalstates/agentupgrade.go shows the blue-green pattern for the
agent's own binary) or baked into images/host-ubuntu2404. Whichever is chosen has
to handle upgrade alongside the rest of the node's software.
3. Collector tool packages via pkg/agent/phases/host/apt.go. Each backs a
specific collector, and a missing binary means that collector silently reports
nothing:
Package
Collector
dmidecode
cpu.go, memory.go - DMI processor and DIMM tables
pciutils
gpu.go - lspci enumeration
ethtool
network.go - NIC driver and firmware
kmod
disk.go, gpu.go - modinfo driver versions
udev
disk.go - udevadm disk properties
lldpd
lldp.go - lldpctl switch neighbours
The NVIDIA tools (nvidia-smi, nvidia-imex-ctl) come from the host driver and
are version-locked to it; they cannot be installed here.
4. The collector endpoint.--collector defaults to inventory-collector:50051 (cmd/inventory/inventory-agent/main.go:28), a cluster
Service DNS name that cannot resolve from the host. It needs a NodePort,
LoadBalancer, hostNetwork Service or an explicitly configured address, and that
address has to reach each node as config. This couples to #636, where the Service
becomes inventory-collector-<site>.
5. --db path../inventory.db should become something durable such as /var/lib/unbounded/inventory.db rather than depending on the unit's working
directory.
6. Cadence and failure handling. At provisioning only, or periodically, and
with what jitter. What happens when the aggregator is unreachable: the local SQLite
cache already exists, so the question is whether the agent retries, or the next
timer firing simply re-publishes.
Open question: how to prove it works
This is where a true end-to-end test becomes possible, and it is also where it is
hardest. The collectors read DMI, IPMI, PCI, LLDP and NVLink, which return little
or nothing on a cloud VM, so a meaningful test needs real hardware - pointing at a
bare-metal site such as boulderlab, which is also the site that goes unreachable
and would need the tolerance treatment the DaemonSet deploy gate already has.
Cheaper options worth considering first: assert the unit is installed and enabled
after provisioning; assert one timer firing produces a non-empty local SQLite
database; assert a record for the node's serial appears in PostgreSQL within one
interval.
Acceptance criteria
After a node is provisioned, inventory-agent is installed, its unit is enabled,
and it has run at least once.
A record identified by the node's DMI serial reaches the aggregator.
The agent survives an unreachable aggregator without wedging the node or filling
the disk.
The collector address is configuration, not a hardcoded cluster Service name.
Activity