Hardware Census (Migrated)

See: https://uwaterloo.atlassian.net/wiki/x/dwINUAo

Statement of Work

Problem:

CSCF Inventory System information has (as of 2021-01-01) woefully nonexistent, incomplete, inaccurate, inconsistent information on IP enabled devices. If this information exists at all in inventory its generally entered freehand in a text field ("Comments"), not usable by any tooling and cannot be reconciled with reality.

Opportunity:

Computers, Linux in particular, will give you an exhaustive listing of all hardware on the system, quickly and accurately. Develop probes or https queryable service of at least the Infrastructure managed systems whose data can be used to validate or even update CSCF's inventory system.

Key Deliverables:

• Component database with names, descriptions and relevant attributes of:
  • CPU
  • Motherboard
  • Memory
  • Disks
  • Network Interfaces (with MAC Addresses) presently deployed in CSCF Infrastructure managed hardware (servers). All components should list a BIOS or Firmware version where possible.
• Software pipeline for incrementally populating component database and canonicalizing information (vendor names etc.) where necessary.
• Hardware database to map described components to individual pieces of deployed hardware by Inventory ID.
• Software pipeline for populating hardware database (periodic hardware census).
• API for querying databases to get a listing of the components in any particular piece of hardware
• API for querying databases to get a listing of hardware containing any particular component.
• API for querying databases to get a count of various components currently deployed.

Tasks:

1. Develop a method for exposing information about installed components on managed hardware (micro-service).
2. Create software pipeline for building "Components" database that accurately models the hardware components of the compute systems we manage that we care about. For each Component type:
   • Determine what information is valuable.
   • Augment database schema to store said information.
   • Develop tooling to extract that information from computers and load into database.
3. Create software pipeline for linking inventory asset IDs with components in Component database.
4. Create stored procedures to implement API functionality.
5. Create software to expose API, via command-line and REST API.

Challenges:

• Information as reported by tools might use inconsistent nomenclature, (for example Nvidia vs nvidia vs nVidia). We may need some hand written tooling to canonicalize information. Will likely require a 2 phase process, where the information is canonicalized and the Component database is created before a database linking those components to hardware under management is created.

Schedule:

• Deliverables due of April 19, 2021

Players:

• Staff:
  • Anthony Brennan
  • Ryan Kim
• Stakeholders:
  • Dave Gawley

Key Technologies:

• Postgres
• Python
• Bash

Compute Resources:

• Periodic use of:
  • 1 core(s)
  • 1 GB memory
  • 4MB scratch disk space per managed host
• Ongoing use of:
  • 1 core
  • 1 GB memory
  • 1 GB database space
  • 1 GB disk space for incremental database backup

Further work:

• Automated periodic hardware census, highlighting any changes

-- Anthony Brennan - 2021-02-01

Comments