Embedded teams often debate test environments as if one must win. Should a scenario run in software-in-the-loop, hardware-in-the-loop, on a laboratory bench, through vision-in-the-loop, with a robot, or only on the finished product?

That is the wrong competition. Each layer exposes a different set of failures at a different cost, speed, controllability, and level of realism. A strong strategy places each test at the earliest layer that can reveal the target risk, then keeps a smaller set of cross-layer tests to prove that the abstractions still match reality.

Begin with the failure you need to expose

The test environment is a consequence of the question.

If the risk is a calculation error in an algorithm, physical hardware may add noise without adding useful evidence. If the risk is a bus timing issue, a pure software model may hide it. If the risk is whether a user can wake a device by pressing a real control while wearing gloves, the physical interface is part of the requirement.

For every scenario, ask:

  • Which behaviour or failure mode is under investigation?
  • Which real interfaces are necessary to expose it?
  • Which parts can be simulated without invalidating the result?
  • Does the test need deterministic fault injection?
  • Does the verdict depend on timing, power, mechanics, optics, or human interaction?
  • How often must the test run, and how quickly must it return evidence?

Those answers place the scenario more reliably than a blanket rule such as “everything should run on HIL.”

Software and model layers

Software-in-the-loop and model-based environments are valuable when the target behaviour can be exercised without production electronics.

They are usually the best place for:

  • algorithmic behaviour and state transitions;
  • boundary-value and combinatorial exploration;
  • deterministic fault injection;
  • high-volume regression;
  • tests that need to run with every code change.

These layers are fast and controllable, but they can only reveal what the model represents. A missing electrical characteristic, inaccurate timing assumption, or simplified peripheral can create a clean result that the real product cannot reproduce.

Hardware-in-the-loop

HIL connects real controller hardware and its real interfaces to a simulated environment. It is especially useful when the controller, I/O, timing, communication, or hardware-software integration must be real while the surrounding plant remains simulated.

Typical HIL strengths include:

  • repeatable real-time stimulation;
  • safe injection of difficult or extreme conditions;
  • automated regression on actual controller hardware;
  • communication, I/O, and timing validation;
  • testing before the complete physical product is available.

HIL is not automatically realistic. The result is only as meaningful as the plant model, I/O behaviour, timing, wiring, calibration, and assumptions surrounding it.

Bench and device integration

A bench setup uses real devices, instruments, power control, communication interfaces, and selected physical components. It is often less elegant than a full HIL system but highly effective for product integration.

Bench validation is useful for:

  • startup, shutdown, reset, and power-state behaviour;
  • firmware flashing and recovery;
  • sensor, actuator, and peripheral integration;
  • diagnostic and communication behaviour;
  • long-running endurance scenarios;
  • measurements that require real electrical characteristics.

The bench is also where infrastructure quality becomes visible. Uncontrolled cables, shared equipment, undocumented manual setup, and hidden calibration assumptions can create more test instability than the product itself.

Vision-in-the-loop

For Lavaflows, vision-in-the-loop means using a camera or vision system as an observation channel inside the automated loop.

It is appropriate when:

  • a display or indicator has no reliable software interface;
  • visible state is part of the user-facing requirement;
  • the product must be observed without instrumenting its internal implementation;
  • geometry, appearance, alignment, or motion contributes to the verdict.

Vision adds real observation but also adds lighting, focus, pose, exposure, occlusion, and timing as test-system variables. A vision check must therefore include its own health and calibration evidence.

Robotic validation

Robotic validation brings repeatable physical interaction into the loop. It can press controls, move components, position devices, operate touchscreens, connect mechanisms, or reproduce user actions.

Use it when the physical interaction is inseparable from the risk:

  • force, travel, position, or mechanical tolerance matters;
  • the product state cannot be reached through an API;
  • interaction with the real enclosure, touchscreen, or control is required;
  • repeatable physical endurance or user-path validation is needed.

Robotic testing has the highest value when it complements earlier layers. It is a costly place to discover an error that a fast software test could have found minutes after the code changed.

Field and system validation

Some behaviour only becomes meaningful in the complete product and operating environment. Field testing provides the highest realism but usually the lowest controllability, slowest feedback, and most expensive diagnosis.

Reserve it for questions that genuinely require:

  • complete system interaction;
  • real environmental variation;
  • production mechanics, installation, and user behaviour;
  • validation of assumptions made by lower layers.

Field tests should confirm the strategy, not carry the entire regression burden.

Build a portfolio, not a pyramid slogan

A practical portfolio often looks like this:

  • many fast software and model tests;
  • focused HIL campaigns around controller and interface risks;
  • bench automation for real device integration;
  • vision checks for observable user-facing behaviour;
  • robotic tests for critical physical interactions;
  • a small, high-value set of field scenarios.

Reuse test intent across layers where possible, but allow layer-specific implementation and evidence. The same requirement may have a fast algorithm test, a HIL integration test, and one physical confirmation scenario.

Selection checklist

Choose the lowest layer that can answer “yes” to all required questions:

  • Are the necessary interfaces represented faithfully?
  • Can the target failure be induced and observed?
  • Is the timing model adequate?
  • Does the verdict remain meaningful without the omitted hardware or environment?
  • Can the result be reproduced and diagnosed?

Then add a higher-layer confirmation when model error, integration uncertainty, or physical behaviour could invalidate the lower-layer evidence.

The best validation architecture does not maximise realism everywhere. It spends realism where realism changes the answer.

Source notes for review