An automated validation rig makes a strong promise: it will apply the same conditions, observe the same signals, and judge the same requirements repeatedly. The promise becomes dangerous when the rig itself can fail without being noticed.

A worn relay may introduce voltage drop. A serial adapter may reconnect under a different port. A camera can continue returning the last valid frame. A measurement channel can saturate. A robotic actuator may report that it reached position even though the tool slipped. All of those conditions can produce a red test result that looks like a product defect.

The opposite failure is worse. A broken observation path can produce a false pass because the automation never saw the behaviour it was supposed to judge.

Reliable validation therefore needs two systems under observation: the product and the rig.

Treat infrastructure as another engineered product

A test rig contains hardware, firmware, operating-system services, drivers, networks, fixtures, and software libraries. Each component has state, limits, calibration requirements, and failure modes.

Document the rig architecture as deliberately as the system under test:

  • power sources, relays, loads, and protection;
  • buses, adapters, termination, and addressing;
  • cameras, sensors, and measurement ranges;
  • actuator limits, homing, and collision boundaries;
  • host resources, services, and network dependencies;
  • software and configuration versions;
  • replaceable parts and expected maintenance.

This inventory becomes the basis for health checks. “The test computer is online” is not evidence that the complete observation and control chain is healthy.

Verify the path, not only the component

A device can report that it is healthy while the path around it is broken. A camera process may be running but producing frozen frames. A programmable supply may accept commands while its output is disconnected. A relay controller may respond even though the load contact has degraded.

Health checks should therefore verify the end-to-end effect where possible:

  • command a known output and measure it independently;
  • move an actuator to a reference target and confirm it through position feedback or vision;
  • inject a known bus message and confirm its appearance at the intended observation point;
  • display or place a reference target and verify that the camera sees a new frame;
  • switch a reference load and compare current with an expected range.

The reference does not need to reproduce the complete product. It needs to prove that the relevant rig path can produce and observe a known result.

Run a preflight before consuming a product

A preflight check should establish that the rig is fit to execute the requested test. It can include:

  1. identify every connected instrument and fixture;
  2. confirm approved firmware, driver, and configuration versions;
  3. verify calibration status and measurement range;
  4. check supply rails before connecting the product;
  5. validate communication quality and termination;
  6. home actuators and test safety interlocks;
  7. confirm that cameras and sensors are producing fresh data;
  8. execute a small reference sequence;
  9. record the resulting rig-health snapshot.

If the preflight fails, the product test should not start. Report the run as infrastructure unavailable rather than product failed.

That distinction protects defect statistics and prevents engineers from investigating the wrong system.

Preserve health during the run

Some failures appear only under load or after many cycles. Preflight alone is not enough.

Monitor useful rig signals continuously or at defined checkpoints:

  • supply voltage, current, and protection state;
  • communication error counters;
  • camera frame age and sequence number;
  • actuator position error and force;
  • host CPU, memory, storage, and temperature;
  • fixture identity and connection state;
  • emergency-stop and interlock status.

Attach those signals to the same run identifier as the product evidence. If a test fails at 14:03:12, the reviewer should be able to see whether the camera froze or the supply dipped at the same moment.

Do not turn every health signal into an immediate abort. Define severity. A critical interlock failure should stop the system. A rising communication-error count may mark the evidence as degraded and trigger a controlled retry or maintenance action.

Use independent observation for critical actions

The controller that performs an action should not always be the only source that proves it happened.

Examples include:

  • measure relay output rather than trusting the command acknowledgement;
  • observe actuator contact rather than trusting commanded position;
  • confirm product power consumption rather than assuming that “power on” succeeded;
  • use frame sequence and capture time to prove that an image is new;
  • read a separate reset or boot indicator where the primary interface may be unavailable.

Independent observation reduces common-mode failure. It is especially valuable when the action itself can damage the product or invalidate the rest of the test.

Classify failures before retrying

A blanket retry can hide a weak rig. Instead, classify the evidence:

  • Product failure: the rig is healthy and the product violates the acceptance rule.
  • Infrastructure failure: one or more required rig paths are unhealthy.
  • Inconclusive: the evidence is insufficient to assign either source safely.
  • Environmental violation: the requested condition was outside the test envelope.

The recovery action should follow the classification. Infrastructure failures can route to self-test, recalibration, reconnection, or maintenance. Product failures should preserve state and evidence before any reset. Inconclusive runs should not be reported as passes.

Repeated infrastructure failures are maintenance information, not test noise.

Track the rig as a versioned configuration

Results from two rigs are comparable only when their relevant differences are known. Record:

  • rig and fixture identifier;
  • wiring and hardware revision;
  • instrument serial numbers;
  • calibration records;
  • driver and library versions;
  • configuration and limit files;
  • recent maintenance or replaced parts.

Use a controlled change process. After changing a camera, cable, relay board, or driver, run a qualification suite before returning the rig to normal service.

Create a golden reference carefully

A known-good product can help detect changes in the rig, but it is not immortal. Its own hardware, battery, connectors, and software can drift.

Store it under controlled conditions, restrict its use, track its version, and periodically compare it with another reference or independent measurement. The reference is one diagnostic input, not unquestionable truth.

For fixtures that cannot use a complete reference product, build reference plugs, loopbacks, targets, loads, or signal sources around the paths that matter.

Make maintenance data visible

Rig-health history can reveal gradual degradation:

  • increasing relay resistance;
  • more frequent bus errors;
  • actuator position offset;
  • longer device-detection time;
  • camera exposure or focus changes;
  • growing recovery frequency.

Use trends to schedule inspection before the rig becomes unreliable. The goal is not predictive-maintenance theatre. It is to replace “random test failures” with observable infrastructure behaviour.

A trustworthy test system does not merely generate verdicts. It produces evidence that the product was tested by an infrastructure chain known to be functioning at the time. When the rig can prove its own health, a red result becomes a useful engineering signal rather than an accusation.

Source notes for review