Skip to main content

分片

介绍

Introduction

默认情况下,Playwright 在 parallel 中运行测试,并努力实现计算机上 CPU 核心的最佳利用率。为了实现更大的并行化,你可以通过在多台机器上同时运行测试来进一步扩展 Playwright 测试执行。我们称这种操作模式为 "sharding"。

By default, Playwright runs tests in parallel and strives for optimal utilization of CPU cores on your machine. In order to achieve even greater parallelisation, you can further scale Playwright test execution by running tests on multiple machines simultaneously. We call this mode of operation "sharding".

多台机器之间的分片测试

Sharding tests between multiple machines

要对测试套件进行分片,请将 --shard=x/y 传递到命令行。例如,要将套件分为四个分片,每个分片运行四分之一的测试:

To shard the test suite, pass --shard=x/y to the command line. For example, to split the suite into four shards, each running one fourth of the tests:

npx playwright test --shard=1/4
npx playwright test --shard=2/4
npx playwright test --shard=3/4
npx playwright test --shard=4/4

现在,如果你在不同的计算机上并行运行这些分片,你的测试套件的完成速度会快四倍。

Now, if you run these shards in parallel on different computers, your test suite completes four times faster.

合并来自多个分片的报告

Merging reports from multiple shards

在前面的示例中,每个测试分片都有自己的测试报告。如果你想要一个显示所有分片的所有测试结果的合并报告,你可以合并它们。

In the previous example, each test shard has its own test report. If you want to have a combined report showing all the test results from all the shards, you can merge them.

在 CI 上运行时,首先将 blob 报告器添加到配置中:

Start with adding blob reporter to the config when running on CI:

playwright.config.ts
export default defineConfig({
testDir: './tests',
reporter: process.env.CI ? 'blob' : 'html',
});

Blob 报告包含有关已运行的所有测试及其结果的信息以及所有测试附件(例如跟踪和屏幕截图差异)。Blob 报告可以合并并转换为任何其他 Playwright 报告。默认情况下,blob 报告将生成到 blob-report 目录中。

Blob report contains information about all the tests that were run and their results as well as all test attachments such as traces and screenshot diffs. Blob reports can be merged and converted to any other Playwright report. By default, blob report will be generated into blob-report directory.

要合并来自多个分片的报告,请将 blob 报告文件放入单个目录中,例如 all-blob-reports。Blob 报告名称包含分片编号,因此它们不会冲突。

To merge reports from multiple shards, put the blob report files into a single directory, for example all-blob-reports. Blob report names contain shard number, so they will not clash.

然后,运行 npx playwright merge-reports 命令:

Afterwards, run npx playwright merge-reports command:

npx playwright merge-reports --reporter html ./all-blob-reports

这将在 playwright-report 目录中生成标准 HTML 报告。

This will produce a standard HTML report into playwright-report directory.

GitHub Actions 示例

GitHub Actions example

GitHub Actions 支持使用 jobs.<job_id>.strategy.matrix 选项的 在多个作业之间分片测试matrix 选项将为所提供选项的每种可能的组合运行单独的作业。

GitHub Actions supports sharding tests between multiple jobs using the jobs.<job_id>.strategy.matrix option. The matrix option will run a separate job for every possible combination of the provided options.

以下示例向你展示如何配置作业以在四台计算机上并行运行测试,然后将报告合并为单个报告。不要忘记将 reporter: process.env.CI ? 'blob' : 'html', 添加到 playwright.config.ts 文件中,如上例所示。

The following example shows you how to configure a job to run your tests on four machines in parallel and then merge the reports into a single report. Don't forget to add reporter: process.env.CI ? 'blob' : 'html', to your playwright.config.ts file as in the example above.

  1. 首先,我们在作业配置中添加 matrix 选项,其中 shardTotal: [4] 选项包含我们要创建的分片总数,shardIndex: [1, 2, 3, 4] 选项包含分片编号数组。

    First we add a matrix option to our job configuration with the shardTotal: [4] option containing the total number of shards we want to create and shardIndex: [1, 2, 3, 4] with an array of the shard numbers.

  2. 然后我们使用 --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} 选项运行 Playwright 测试。这将为每个分片运行我们的测试命令。

    Then we run our Playwright tests with the --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} option. This will run our test command for each shard.

  3. 最后,我们将 blob 报告上传到 GitHub Actions Artifacts。这将使 Blob 报告可供工作流中的其他作业使用。

    Finally we upload our blob report to the GitHub Actions Artifacts. This will make the blob report available to other jobs in the workflow.

.github/workflows/playwright.yml
name: Playwright Tests
on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]
jobs:
playwright-tests:
timeout-minutes: 60
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
shardIndex: [1, 2, 3, 4]
shardTotal: [4]
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version: 18
- name: Install dependencies
run: npm ci
- name: Install Playwright browsers
run: npx playwright install --with-deps

- name: Run Playwright tests
run: npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}

- name: Upload blob report to GitHub Actions Artifacts
if: always()
uses: actions/upload-artifact@v4
with:
name: blob-report-${{ matrix.shardIndex }}
path: blob-report
retention-days: 1
  1. 所有分片完成后,你可以运行一个单独的作业来合并报告并生成合并的 HTML 报告。为了保证执行顺序,我们通过添加 needs: [playwright-tests] 使 merge-reports 作业在分片 playwright-tests 作业上成为 depend

    After all shards have completed, you can run a separate job that will merge the reports and produce a combined HTML report. To ensure the execution order, we make the merge-reports job depend on our sharded playwright-tests job by adding needs: [playwright-tests].

.github/workflows/playwright.yml
jobs:
...
merge-reports:
# Merge reports after playwright-tests, even if some shards have failed
if: always()
needs: [playwright-tests]

runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version: 18
- name: Install dependencies
run: npm ci

- name: Download blob reports from GitHub Actions Artifacts
uses: actions/download-artifact@v4
with:
path: all-blob-reports
pattern: blob-report-*
merge-multiple: true

- name: Merge into HTML Report
run: npx playwright merge-reports --reporter html ./all-blob-reports

- name: Upload HTML report
uses: actions/upload-artifact@v4
with:
name: html-report--attempt-${{ github.run_attempt }}
path: playwright-report
retention-days: 14

现在,你可以看到报告已合并,并且 GitHub Actions Artifacts 选项卡中提供了合并的 HTML 报告。

You can now see the reports have been merged and a combined HTML report is available in the GitHub Actions Artifacts tab.

image

合并报告 CLI

Merge-reports CLI

npx playwright merge-reports path/to/blob-reports-dir 从传递的目录中读取所有 blob 报告并将它们合并到单个报告中。

npx playwright merge-reports path/to/blob-reports-dir reads all blob reports from the passed directory and merges them into a single report.

当合并来自不同操作系统的报告时,你必须提供显式的合并配置来消除应将哪个目录用作测试根目录的歧义。

When merging reports from different OS'es you'll have to provide an explicit merge config to disambiguate which directory should be used as tests root.

支持的选项:

Supported options:

  • --reporter reporter-to-use

    要生成哪个报告。可以是多个报告器,用逗号分隔。

    Which report to produce. Can be multiple reporters separated by comma.

    示例:

    Example:

    npx playwright merge-reports --reporter=html,github ./blob-reports
  • --config path/to/config/file

    指定带有输出报告器的 Playwright 配置文件。使用此选项将附加配置传递给输出报告器。此配置文件可能与创建 blob 报告期间使用的配置文件不同。

    Specifies the Playwright configuration file with output reporters. Use this option to pass additional configuration to the output reporter. This configuration file can differ from the one used during the creation of blob reports.

    示例:

    Example:

    npx playwright merge-reports --config=merge.config.ts ./blob-reports
    merge.config.ts
    export default {
    testDir: 'e2e',
    reporter: [['html', { open: 'never' }]],
    };