Robot Framework makes it easy to write a readable first test. That strength can become a weakness when every new requirement produces another high-level keyword, every resource file imports every other file, and device state leaks across suites.

A scalable project needs a clear answer to five questions:

  • Where is test intent expressed?
  • Where are reusable product workflows implemented?
  • Where does stateful device control live?
  • Where are protocol and tool details isolated?
  • Where is evidence created and correlated?

The answer should not be “wherever the last engineer found space.”

Layer 1: test cases express intent

A test case should read like a validation contract:

*** Test Cases ***
Device Recovers After Complete Power Loss
    Given Device Is Operational
    When Complete Power Cycle Is Performed
    Then Device Becomes Ready Within    ${BOOT_LIMIT}
    And No Unexpected Diagnostic Fault Is Present

The test names the behaviour and acceptance rule. It does not choose serial ports, manipulate relays directly, parse raw frames, or know where screenshots are stored.

Keep test cases small enough that a reviewer can understand the purpose, initial state, action, and expected outcome without following a chain of low-level implementation details.

Tags and metadata should support selection, ownership, requirement mapping, environment constraints, and reporting—not encode hidden execution logic.

Layer 2: domain resources define reusable workflows

Robot resource files are useful for product-language keywords:

  • Complete Power Cycle Is Performed
  • Device Becomes Ready Within
  • No Unexpected Diagnostic Fault Is Present

These keywords compose lower-level library operations into a workflow and apply product-specific rules.

Organise resources by domain or capability rather than creating one global keyword file. Examples:

  • power and lifecycle;
  • diagnostics;
  • communications;
  • user interaction;
  • update and recovery;
  • product configuration.

Avoid deep keyword chains that add no meaning. Each layer should either raise the abstraction or provide a clear reusable policy.

Resource files are not a substitute for stateful program logic. Complex polling, protocol parsing, concurrency, recovery, and typed data are usually easier to implement and test in Python.

Layer 3: Python libraries own device services

A custom Python library should expose bounded operations rather than raw implementation primitives.

For example:

def complete_power_cycle(self, off_timeout, boot_timeout):
    ...

Internally, it can:

  • verify the current state;
  • command the supply or relay;
  • measure discharge;
  • enforce off dwell;
  • apply power;
  • observe boot;
  • verify readiness;
  • capture evidence on failure;
  • return structured timing and state information.

The library should have explicit lifecycle and cleanup. Robot Framework supports test, suite, and global library scopes. Choose a scope based on ownership of state, then make the reset behaviour obvious. Shared mutable global state without disciplined cleanup creates order-dependent tests.

Keep library methods independently testable with simulated adapters.

Layer 4: adapters isolate protocols and tools

Below the device service, adapters handle concrete interfaces:

  • serial;
  • CAN or Ethernet;
  • programmable supplies;
  • relay controllers;
  • cameras;
  • measurement instruments;
  • robotic actuators.

An adapter translates between the external tool and a stable internal contract. It owns connection setup, framing, transport errors, and low-level timeouts.

Do not let a test case depend directly on a vendor command or raw byte sequence unless the protocol itself is the subject of the test.

Adapters also make simulation possible. A device service can be tested against a fake supply or fake transport before it is trusted on a real rig.

Layer 5: evidence is a shared service

Evidence should not be an afterthought inside individual test cases.

A shared run context can provide:

  • run and test identifiers;
  • product, rig, software, and configuration identity;
  • correlated timestamps;
  • structured events;
  • logs and measurements;
  • screenshot and trace references;
  • final verdict and failure classification.

Libraries and adapters should be able to attach evidence without deciding how the final report is rendered.

Robot Framework already produces logs, reports, and machine-readable output. Extend that foundation with listener interfaces or structured artifacts rather than replacing it with scattered print statements.

Keep configuration outside the tests

Environment-specific values belong in controlled configuration:

  • endpoints and ports;
  • device mappings;
  • instrument addresses;
  • timeout profiles;
  • measurement limits;
  • credentials or secret references;
  • artifact locations.

Separate product acceptance criteria from rig configuration. A requirement limit should not silently change because a different lab machine loaded a different local file.

Validate configuration at startup and show the resolved, non-secret values in the run evidence.

Make setup establish preconditions

Suite and test setup are useful when they prove that execution may begin:

  • required services are available;
  • rig health passes;
  • correct fixture is connected;
  • product identity is known;
  • device reaches the required initial state.

If setup cannot establish those conditions, classify the run as blocked or infrastructure failed. Do not let every test fail with a misleading product assertion.

Teardown should preserve evidence and return the system to a safe known state. It should still run after a failed test, but it should not overwrite the original failure.

Design keyword contracts

Each public keyword should document:

  • arguments and units;
  • preconditions;
  • postconditions;
  • returned data;
  • timeouts;
  • side effects;
  • failure categories;
  • evidence captured;
  • recovery behaviour.

Prefer typed objects or dictionaries internally, while returning readable values at the test layer.

Avoid keywords that combine unrelated outcomes or silently retry until a pass. A keyword should fail with enough structured context for the report to explain why.

Keep tests independent

The framework should support running:

  • one test;
  • a subset by tag;
  • suites in a different order;
  • compatible suites in parallel;
  • the same suite on another equivalent rig.

Independence requires explicit state, isolated test data, bounded shared resources, and cleanup.

When a resource truly must be shared, model ownership and locking. Accidental sharing is not optimisation.

Test the framework

Use unit and integration tests for Python libraries and adapters. Exercise error paths:

  • connection loss;
  • stale data;
  • invalid response;
  • timeout;
  • incomplete recovery;
  • evidence-write failure;
  • cancellation.

Run a smaller qualification suite against each rig and product simulation before the main validation set.

The goal is not a perfect directory tree. It is a framework in which a high-level test remains readable while each lower layer has one clear responsibility. That architecture lets the suite grow without turning the product language into a thin disguise over cables, ports, and sleeps.

Source notes for review