English | 中文

---

Introduction

xbot is a lightweight, easy-to-use, and extensible test automation framework.

Installation

Install xbot via pip:

pip install xbot.framework

Type xbot --help to check:

$ xbot --help
usage: xbot [-h] [-d DIRECTORY] [-b TESTBED] [-s TESTSET] [-f {verbose,brief}] [-v] {init,run}

positional arguments:
{init,run}

optional arguments:
-h, --help            show this help message and exit
-d DIRECTORY, --directory DIRECTORY
                        directory to init (required by `init` command)
-b TESTBED, --testbed TESTBED
                        testbed filepath (required by `run` command)
-s TESTSET, --testset TESTSET
                        testset filepath (required by `run` command)
-f {verbose,brief}, --outfmt {verbose,brief}
                        output format (option for `run` command, options: verbose/brief, default: brief)
-v, --version         show program's version number and exit

Getting Started

Initialize a test project:

$ xbot init -d ./testproj
Initialized ./testproj

The test project directory structure:

./testproj
├── .gitignore
├── README.md
├── lib  # test libraries
│   ├── __init__.py
│   ├── testbed.py  # testbed base
│   └── testcase.py  # testcase base
├── requirements.txt
├── testbeds  # directory storing testbeds
│   └── testbed_example.yml
├── testcases  # directory storing testcases
│   ├── __init__.py
│   └── examples
│       ├── __init__.py
│       ├── block
│       │   ├── __init__.py
│       │   └── tc_eg_block_parent_setup_not_pass.py
│       ├── inst
│       │   ├── __init__.py
│       │   ├── tc_eg_install_the_software_to_be_tested_failed.py
│       │   └── tc_eg_install_the_software_to_be_tested_successful.py
│       ├── nonpass
│       │   ├── __init__.py
│       │   ├── tc_eg_nonpass_error_clsname.py
│       │   ├── tc_eg_nonpass_error_syntax.py
│       │   ├── tc_eg_nonpass_fail_setup_with_failfast_false.py
│       │   ├── tc_eg_nonpass_fail_setup_with_failfast_true.py
│       │   ├── tc_eg_nonpass_fail_step_with_failfast_false.py
│       │   ├── tc_eg_nonpass_fail_step_with_failfast_true.py
│       │   ├── tc_eg_nonpass_skip_excluded.py
│       │   ├── tc_eg_nonpass_skip_not_included.py
│       │   └── tc_eg_nonpass_timeout.py
│       └── pass
│           ├── __init__.py
│           ├── tc_eg_pass_create_dirs_and_files.py
│           └── tc_eg_pass_get_values_from_testbed.py
└── testsets  # directory storing testsets
    └── testset_example.yml

Testbed example(testbeds/testbed_example.yml):

# Testbed is used to store the information about the test environment.
# The information can be accessed by self.testbed.get() in the testcases.
example:
  key1: value1
  key2: 
    key2-1: value2-1
    key2-2: value2-2
  key3:
    - value3-1
    - value3-2
    - value3-3
  key4:
    - name: jack
      age: 20
    - name: tom
      age: 30

Testset example(testsets/testset_example.yml):

# Testset is used to organize testcases to be executed.

# Tags are used to filter testcases by matching them against the `TAGS`
# attribute of the testcases. Results of testcases that are not included
# or excluded will be marked as SKIP.
tags:
  # Include testcases with these tags.
  include:
    - tag1
  # Exclude testcases with these tags, higher priority than `include`.
  exclude:
    - tag2

# Relative paths of testcases. These can be file paths (ending in `.py`)
# or directory paths (not ending in `.py`). The execution order follows
# the order in which they are written, and directories will be recursively
# expanded into test case paths in alphabetical order.
testcases:
  # Testcases used to install the software to be tested. If any testcase
  # in this section fails, the test section will not be executed.
  # `tags` will not be used to filter testcases for this section.
  # This section can be empty.
  install:
    - testcases/examples/inst/tc_eg_install_the_software_to_be_tested_successful.py
  # Testcases used to test the installed software.
  test:
    - testcases/examples/pass/tc_eg_pass_get_values_from_testbed.py
    - testcases/examples/pass/tc_eg_pass_create_dirs_and_files.py
    # Recursively include all testcases in the directory,
    # only match files with the prefix `tc_` and suffix `.py`.
    - testcases/examples/nonpass/
    - testcases/examples/block/

Run the testcases(must execute under the test project directory):

$ xbot run -b testbeds/testbed_example.yml -s testsets/testset_example.yml
(^_^)    PASS     0:00:00  tc.setup
(^_^)    PASS     0:00:00  tc_eg.setup
(^_^)    PASS     0:00:00  tc_eg_inst.setup
(1/13)   PASS     0:00:00  tc_eg_install_the_software_to_be_tested_successful
(^_^)    PASS     0:00:00  tc_eg_inst.teardown
(^_^)    PASS     0:00:00  tc_eg_pass.setup
(2/13)   PASS     0:00:01  tc_eg_pass_get_values_from_testbed
(3/13)   PASS     0:00:01  tc_eg_pass_create_dirs_and_files
(^_^)    PASS     0:00:00  tc_eg_pass.teardown
(^_^)    PASS     0:00:00  tc_eg_nonpass.setup
(4/13)   ERROR    0:00:00  tc_eg_nonpass_error_clsname
(5/13)   ERROR    0:00:00  tc_eg_nonpass_error_syntax
(6/13)   FAIL     0:00:01  tc_eg_nonpass_fail_setup_with_failfast_false
(7/13)   FAIL     0:00:01  tc_eg_nonpass_fail_setup_with_failfast_true
(8/13)   FAIL     0:00:01  tc_eg_nonpass_fail_step_with_failfast_false
(9/13)   FAIL     0:00:01  tc_eg_nonpass_fail_step_with_failfast_true
(10/13)  SKIP     0:00:00  tc_eg_nonpass_skip_excluded
(11/13)  SKIP     0:00:00  tc_eg_nonpass_skip_not_included
(12/13)  TIMEOUT  0:00:03  tc_eg_nonpass_timeout
(^_^)    PASS     0:00:00  tc_eg_nonpass.teardown
(^_^)    FAIL     0:00:00  tc_eg_block.setup
(13/13)  BLOCK    0:00:00  tc_eg_block_parent_setup_not_pass
(^_^)    PASS     0:00:00  tc_eg_block.teardown
(^_^)    PASS     0:00:00  tc_eg.teardown
(^_^)    PASS     0:00:00  tc.teardown

report: /Users/zhaowcheng/Code/xbot/xbot.framework/testproj/logs/testbed_example/2026-09-13_14-27-20/report.html

Test report and logs will be generated in the logs subdirectory.

Example report:

Example log:

Testcase Development

Testcases are stored in the testcases subdirectory, below is a example(testcases/examples/pass/tc_eg_pass_create_dirs_and_files.py):

import os
import tempfile
import shutil

from xbot.framework.utils import assertx

from . import tc_eg_pass


class tc_eg_pass_create_dirs_and_files(tc_eg_pass):
    """
    Test creating directories and files.
    """
    TAGS = ['tag1']

    def setup(self):
        """
        Prepare.
        """
        self.workdir = tempfile.mkdtemp()
        self.info('Created workdir: %s', self.workdir)

    def step1(self):
        """
        Create a subdirectory 'dir' under the temporary working directory and check if it is created successfully.
        """
        self.dir1 = os.path.join(self.workdir, 'dir1')
        os.mkdir(self.dir1)
        assertx(os.path.exists(self.dir1), '==', True)

    def step2(self):
        """
        Create an empty file 'file1' under 'dir1' and check if it is created successfully.
        """
        self.file1 = os.path.join(self.dir1, 'file1')
        open(self.file1, 'w').close()
        assertx(os.path.exists(self.file1), '==', True)

    def step3(self):
        """
        Write 'hello world' to 'file1' and check if it is written successfully.
        """
        with open(self.file1, 'w') as f:
            f.write('hello world')
        with open(self.file1, 'r') as f:
            assertx(f.read(), '==', 'hello world')

    def teardown(self):
        """
        Cleanup.
        """
        shutil.rmtree(self.workdir)
        self.info('Removed workdir: %s', self.workdir)
        self.sleep(1)

• Testcase MUST directly inherit from the base class defined by its directory;
• Testcase MUST implement the preset steps in the setup method, write pass if there are no specific steps;
• Testcase MUST implement the cleanup steps in the teardown method, write pass if there are no specific steps;
• Test steps are named in the form of step1, step2, ..., the number at the end is the execution order;
• The TIMEOUT attribute defines the maximum execution time of the testcase(unit: seconds), the testcase will be forced to end and the result will be set to TIMEOUT if it exceeds the time limit;
• When FAILFAST attribute is True, the subsequent test steps will be skipped and the teardown will be executed immediately if a test step fails;
• The TAGS attribute defines the testcase tags, which can be used to filter testcases to be executed in the testset;

Test libraries development

Test libraries are stored in the lib subdirectory, write the test libraries according to the business requirements, import and use them in the testcases.

Plugins

Name	Description
xbot.plugins.ssh	SSH library for xbot.framework
xbot.plugins.pgsql	PostgreSQL library for xbot.framework
xbot.plugins.docker	Docker library for xbot.framework