Pytest-f3ts Plugin For Hardware Testing

Introduction

This article covers how FixturFab leverages our open-source pytest-f3ts library to extend pytest features for hardware test software development. It builds on FixturFab’s Hardware Testing with Pytest Guide.

Pytest-f3ts is a pytest plugin developed to extend the standard capabilities of pytest for hardware testing applications. It enhances hardware test automation. This plugin is handy for functional testing of the Printed Circuit Board Assembly (PCBA) during manufacturing. It integrates smoothly with existing pytest workflows, adding functionality tailored for hardware test environments.

A test runner (also known as a test sequencer) is required to design software for any production turnkey test system. It’s an application that technicians on the factory floor can use. It will collect your measurement data and safely store it or process it for analysis.

A Pytest Plugin for Hardware Test Software: pytest-f3ts

Data Logging and Printing Results

The primary feature of the pytest-f3ts library is tools to help log and store measurement data so it is easier to access with other applications. This can help organize and track test cases through standardized data logging.

If you're taking a measurement, you can log that value using the record_property metadata feature. Any variables matching the pytest-f3ts format: “test_id”, “description”, “error_code” or “error_msg”, “meas”, “min_limit”, or “max_limit” can be automatically sent to other applications such hardware test sequencers for displaying results or test databases for storing results.

If we want to save the voltage measurement for our voltage rail tests, we can add them like so:

"""test_voltages.py"""

class TestMultipleVoltages:
    def test_3v3_voltage(hardware_interface_powered, test_config):
        """DUT 3.3V VoltageTest.

        Verify that the DUT is receiving 3.3V from the 5V supply.
        """

        # Measure
        voltage = hardware_interface_powered.get_voltage(test_point="3V3")

        # Send measurement to other applications
        record_property("meas", voltage)

        # Update error message for other applications
        error_msg = test_config.error_msg + f"{meas} outside of range!"
        record_property("error_msg", error_msg)

        # Verify measurement is withinlimits
        assert test_config.min_limit < meas < test_config.max_limit, error_msg

Or, if we want to collect all the data at once, we can use a pytest-f3ts utility called log_vars. This will record any variables that match the pytest-f3ts keyword variable names (ex/ “meas” and “error_msg”):

"""test_voltages.py"""

class TestMultipleVoltages:
    def test_3v3_voltage(hardware_interface_powered, test_config):
        """DUT 3.3V VoltageTest.

        Verify that the DUT is receiving 3.3V from the 5V supply.
        """

        # Measure and Update Error Message
        meas = hardware_interface_powered.get_voltage(test_point="3V3")
        error_msg = test_config.error_msg + f"{meas} outside of range!"

        # Auto record any variables that match a result key word such as
        # "meas", "error_msg", "min_limit", etc.
        log_vars(record_property)

        # Verify measurement is withinlimits
        assert test_config.min_limit < meas < test_config.max_limit, error_msg

Configuration and Limits

The pytest-f3ts plugin uses a simple config.yml file format for storing and configuring the test code in a way that is compatible with other applications. The plugin streamlines the testing process by managing test limits and configurations.

For example, our Test Runner can change the limits on the factory floor from the frontend application and keep track of what limits a board was tested with this structure. You can find more information on the config.yml file in the pytest-f3ts documentation.

If we wanted to write a config.yml file for the tests we developed in our Test Software Development guide here, it would look something like this:
‍

test_name: "example-test-plan"

test_cases:
  test_dut_en_5v:
    test_id: "1.1"
    error_code: 110
    description: "5VPower SupplyTest"
    error_msg: "5V Supply shorted to GND."

  test_3v3_voltage:
    test_id: "1.2"
    error_code: 120
    min_limit: 3.2
    max_limit: 3.4
    description: "3V3 RailTest"
    error_msg: "Voltage out of range."

  test_1v1_voltage:
    test_id: "1.3"
    error_code: 130
    min_limit: 1.0
    max_limit: 1.2
    description: "1V1 RailTest"
    error_msg: "Voltage out of range."

‍


You can rewrite your test plan to utilize these configurable test limits. The pytest-f3ts plugin provides a fixture named “test_config” that holds all the parameters set by your config.yml file (test_id, error_code, description, error_msg). All of these variables are set up such that they can also be overridden by the front end of the test runner during runtime. If we call this test_config fixture within a test case, pytest will be able to access these parameters and limits at runtime:

"""test_voltages.py"""

class TestMultipleVoltages:
    def test_3v3_voltage(hardware_interface_powered, test_config):
        """DUT 3.3V VoltageTest.

        Verify that the DUT is receiving 3.3V from the 5V supply.
        """

        # Measure
        voltage = hardware_interface_powered.get_voltage(test_point="3V3")

        # Verify measurement is withinlimits
        assert test_config.min_limit < voltage < test_config.max_limit, test_config.error_msg

‍


FixturFab Test Runner

What is a Test Runner?

A good test runner will allow you to manage users and track which operator ran which devices under which test limits. It will also allow you to collect and view measurement data meaningfully and give you easy access to analytics. Often, a custom turnkey test system software will need actions specific to the testing framework, such as opening a dialog window to ask the test technician for feedback.

FixturFab’s Test Runner application is a product we’ve developed as a tool for operators to use as an interface for any custom test system. It’s a user-friendly graphical user interface lets you run pytest code from the factory floor. It’s designed with tools to help manage users and user permissions, configure and debug test limits, and collect and store measurement data.

Developing with the FixturFab Test Runner

To develop test software compatible with the FixturFab Test Runner, you only need to write Python code that runs with our pytest-f3ts plugin. The pytest-f3ts plugin is a plugin that interfaces between your pytest software and the Test Runner. It automatically collects test run metadata and sends it to the frontend application through the Test Runner API.

The pytest-f3ts plugin includes tools to access limits and configuration data, automatically set the Test Runner, send messages to open dialog windows for the operator to interact with, or change status banners during a test run. We recommend looking through the getting started guide to learn more about how to install and use this plugin while developing your pytest code.

Conclusion

Pytest-f3ts extends pytest's capabilities specifically for hardware testing by providing:

• Standardized data logging for measurements and test results
• Configuration management through YAML files for test limits and parameters
• Automated metadata collection for production environments

The plugin simplifies integration with manufacturing tools by automatically handling:

• Test result collection and storage
• Limit configuration and validation
• Operator interactions through dialog windows
• Runtime parameter updates from test runner applications

This makes pytest-f3ts particularly valuable for teams developing production test systems who must bridge the gap between development and manufacturing environments.

If you’d like to read another example, this article covers using pytest-f3ts to integrate with a specific hardware data analytic platform.