Skip to main content
name: "My Test Suite"
login_url: "https://example.com/login"
context:
  - text: "Additional context for all test cases"
cases:
  - id: "test-001"
    name: "Login Test"
    description: "Test user login functionality"
    # ... additional test case fields

​
Root Level Fields​

​
name​

An optional string that provides a human-readable name for the test suite. If not specified, Testpilot will derive the name from the filename. 
name: "E-commerce Checkout Tests"

​
login_url​

An optional URL that specifies the login page for the application under test. When specified, Testpilot will perform authentication before running test cases. 
login_url: "https://myapp.com/auth/login"

​
context​

An optional array of context objects that provide additional information to all test cases in the plan. This context is inherited by all test cases and can be used to provide background information or setup details. 
context:
  - text: "This application requires users to be logged in"
  - text: "All tests assume a clean database state"

​
cases​

An array of test case objects that define the individual tests to be executed. Each test case represents a specific scenario or user journey to validate. 
cases:
  - id: "checkout-001"
    name: "Complete Purchase"
    # ... test case configuration

​
Test Case Fields​

Each test case in the cases array supports the following fields:

​
id (required)​

A unique identifier for the test case. This ID is used internally by Testpilot to track and reference test cases. 
id: "login-success-001"

​
name (required)​

A human-readable name for the test case that describes what the test validates. 
name: "Successful User Login"

​
description (required)​

A detailed description of what the test case does and what it validates. This helps with test documentation and debugging. 
description: "Verifies that a user can successfully log in with valid credentials"

​
steps (required)​

An array of strings that describe the steps to be performed during the test. Each step represents an action or validation that should be performed. 
steps:
  - "Navigate to the login page"
  - "Enter valid username and password"
  - "Click the login button"
  - "Verify successful login redirect"

​
url​

The URL where the test case should begin execution. This is the starting point for the test. 
url: "https://myapp.com/dashboard"

​
viewports​

An array of viewport configurations that define the screen sizes at which the test should be executed. This is useful for responsive testing across different device sizes. 
viewports:
  - name: "Desktop"
    size: [1920, 1080]
  - name: "Tablet"
    size: [768, 1024]
  - name: "Mobile"
    size: [375, 667]
Each viewport object contains:
  • name: A descriptive name for the viewport
  • size: An array of two integers representing width and height in pixels

​
context​

Test case-specific context that provides additional information for this particular test. This is merged with any root-level context. 
context:
  - text: "This test requires a user with admin privileges"
  - text: "Database should contain sample product data"

​
platform_config​

Configuration for testing on different platforms (web, mobile, etc.). This allows you to specify different entry points for different platforms. 
platform_config:
  url: "https://web.myapp.com"
  android_pkg: "com.mycompany.myapp"
  ios_bundle: "com.mycompany.MyApp"
The platform configuration includes:
  • url: The web URL for browser-based testing
  • android_pkg: The Android package name for mobile app testing
  • ios_bundle: The iOS bundle identifier for mobile app testing

​
Context Objects​

Context objects provide additional information that can be used by Testpilot during test execution. Currently, only text-based context is supported. 
context:
  - text: "User should have completed onboarding"
Each context object contains:
  • text: A string containing contextual information

​
File Formats​

Testpilot supports multiple file formats for pilot files: 
name: "My Tests"
cases:
  - id: "test-001"
    name: "Sample Test"
    description: "A sample test case"
    steps:
      - "Step 1"
      - "Step 2"

​
TOML Format​

name = "My Tests"

[[cases]]
id = "test-001"
name = "Sample Test"
description = "A sample test case"
steps = ["Step 1", "Step 2"]

​
JSON Format​

{
  "name": "My Tests",
  "cases": [
    {
      "id": "test-001",
      "name": "Sample Test",
      "description": "A sample test case",
      "steps": ["Step 1", "Step 2"]
    }
  ]
}

​
File Naming Conventions​

Testpilot recognizes pilot files with the following naming patterns:
  • *.pilot - Basic pilot file
  • *.pilot.yaml - YAML format pilot file
  • *.pilot.yml - YAML format pilot file (alternative extension)
  • *.pilot.toml - TOML format pilot file
  • *.pilot.json - JSON format pilot file

​
Remote Files​

Testpilot can load pilot files from remote URLs, including GitHub repositories. When using GitHub URLs, you can optionally set the GITHUB_TOKEN environment variable for authentication. 
# Load from a direct URL
testpilot run https://example.com/tests/pilot.yaml

# Load from GitHub
testpilot run https://github.com/user/repo/raw/main/tests/pilot.yaml

​
Example: E-commerce Testing Suite​

Here’s a complete example of a pilot file for testing an e-commerce application: 
name: "E-commerce Application Tests"
login_url: "https://shop.example.com/login"
context:
  - text: "Tests assume a clean database with sample products"
  - text: "Payment gateway is in test mode"
cases:
  - id: "product-search-001"
    name: "Product Search"
    description: "Verify users can search for products and view results"
    url: "https://shop.example.com"
    viewports:
      - name: "Desktop"
        size: [1920, 1080]
      - name: "Mobile"
        size: [375, 667]
    steps:
      - "Navigate to the homepage"
      - "Enter 'laptop' in the search box"
      - "Click the search button"
      - "Verify search results are displayed"
      - "Verify at least 3 products are shown"
    context:
      - text: "Search should return products from the electronics category"
  - id: "checkout-001"
    name: "Complete Checkout Process"
    description: "Test the full checkout flow from cart to payment"
    url: "https://shop.example.com/cart"
    steps:
      - "Add a product to cart"
      - "Navigate to checkout"
      - "Fill in shipping information"
      - "Select payment method"
      - "Complete purchase"
      - "Verify order confirmation"
    context:
      - text: "User must be logged in to complete checkout"
      - text: "Use test credit card: 4111111111111111"
  - id: "mobile-app-001"
    name: "Mobile App Login"
    description: "Test login functionality in mobile application"
    platform_config:
      url: "https://shop.example.com/mobile"
      android_pkg: "com.example.shop"
      ios_bundle: "com.example.Shop"
    steps:
      - "Launch the mobile application"
      - "Tap on login button"
      - "Enter valid credentials"
      - "Verify successful login"
    viewports:
      - name: "Mobile"
        size: [375, 667]
This example demonstrates:
  • Root-level configuration with name, login URL, and context
  • Multiple test cases with different purposes
  • Cross-platform testing configuration
  • Responsive testing with multiple viewports
  • Test-specific context and configuration
⌘I