The row that keeps coming back

fleet-inventory reconciles the asset register against what the network actually answers, every night at 04:00, and opens a pull request if they disagree. It is a good job. It has caught four real problems since 2021, all of them decommissioned hardware someone forgot to unrack.

Since 2024-09-12 it has opened the same pull request every night.

The diff

--- a/inventory.csv
+++ b/inventory.csv
@@ -71,6 +71,7 @@
 corvid-a,192.0.2.71,4-2,U14,SM-2U-441902,2021-04-06,platform
 corvid-b,192.0.2.72,4-2,U15,SM-2U-441903,2021-04-06,platform
+corvid-c,192.0.2.73,,,,,
 gate-01,192.0.2.40,2-1,U03,SM-1U-338110,2016-02-19,network
 hesper-01,192.0.2.31,9-4,U21,SM-2U-402277,2019-08-30,platform

Seven columns: name, address, row, rack unit, serial, commissioned, owner. The proposed row has two of them.

The job fills a column when it can observe it. It reads the row and rack unit from the intelligent PDU, the serial from IPMI, and the commissioning date from the procurement export. For corvid-c it gets nothing from any of the three, and it will not invent a value, which is correct behaviour and the reason the job is trusted.

The pattern

The 123 pull requests opened since May are all open. Nobody has closed one, nobody has commented on one, and nobody has merged one.

The job checks for an existing identical pull request before opening a new one, so opening 123 of them means that on each of 123 nights it found none. The previous night's is on the same branch name.

What the register says now

$ git log --oneline -1 inventory.csv
9f2b0ac  2024-09-11  m.reyes  remove corvid-c (no matching asset)
$ grep -c '' inventory.csv
1182
$ grep -c corvid inventory.csv
3

The file has 1,182 lines: one header and 1,181 hosts. The estate has 1,181 hosts. There are three corvid rows.

The last commit that touched the file was made on 2024-09-11, and it removed the row.

See stories/quorum for the cluster, and stories/nightly-0300 for the other job that will not stop.