Reporting

Allure Setup

Installation

macOS:

Linux:

Download from GitHub releases or use package manager.

Windows:

Download from GitHub releases and add to PATH.

Verify installation:

Install JUnit Logger

Add the JUnit XML logger to your test project:

This package generates JUnit XML format that Allure can consume.

Generating Reports

Step 1: Run Tests with JUnit Logger

Execute tests and generate JUnit XML output:

This creates:

• ./allure-results/junit.xml – Test execution results
• Additional metadata files in ./allure-results/

Step 2: Generate Allure Report

Convert JUnit XML to an Allure HTML report:

Options:

• --clean – Remove existing report before generating
• -o ./allure-report – Output directory

Step 3: Open Report

View the report in your browser:

This starts a local server and opens the report automatically.

Report Features

Overview Dashboard

The Allure dashboard provides:

• Test execution summary – Total, passed, failed, skipped
• Duration trends – Test execution time over time
• Status breakdown – Visual representation of test results
• Environment information – Browser, OS, framework versions

Test Details

Each test scenario includes:

• Step-by-step execution – Gherkin steps with timings
• Screenshots – Automatically captured on failure
• Video recordings – If enabled (see configuration)
• Console logs – Playwright and application logs
• Page source – HTML at time of failure
• Network requests – API calls made during test

History and Trends

Allure tracks:

• Historical data – Compare runs over time
• Flaky test detection – Identify unstable scenarios
• Performance trends – Execution time analysis
• Failure patterns – Common failure points

Enhanced Reporting

Attach Screenshots

Screenshots are automatically attached on failure. To attach custom screenshots:

[Then(@"I should see the dashboard")]
public async Task ThenIShouldSeeTheDashboard()
{
    await Page.ScreenshotAsync(new PageScreenshotOptions 
    { 
        Path = "screenshot.png" 
    });
    
    // Attach to Allure (if using Allure.SpecFlow)
    AllureLifecycle.Instance.AddAttachment("screenshot.png", "image/png", File.ReadAllBytes("screenshot.png"));
}

Attach Page Source

Capture page HTML on failure:

[AfterScenario]
public async Task CapturePageSourceOnFailure()
{
    if (ScenarioContext.ScenarioExecutionStatus == ScenarioExecutionStatus.TestError)
    {
        var pageSource = await Page.ContentAsync();
        File.WriteAllText($"artifacts/pagesource/{ScenarioContext.ScenarioInfo.Title}.html", pageSource);
    }
}

Attach Network Logs

Capture network activity:

[BeforeScenario]
public async Task SetupNetworkLogging()
{
    Page.Request += (sender, e) => 
    {
        Console.WriteLine($"Request: {e.Method} {e.Url}");
    };
    
    Page.Response += (sender, e) => 
    {
        Console.WriteLine($"Response: {e.Status} {e.Url}");
    };
}

CI/CD Integration

GitHub Actions

name: Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Setup .NET
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: '9.0.x'
      
      - name: Install Playwright
        run: |
          dotnet build ./Tests.E2E/Tests.E2E.csproj
          bash ./Tests.E2E/bin/Debug/net9.0/playwright.sh install --with-deps chromium
      
      - name: Run tests
        run: |
          dotnet test ./Tests.E2E/Tests.E2E.csproj \
            --logger:"junit;LogFileName=junit.xml" \
            --results-directory ./allure-results
      
      - name: Generate Allure report
        run: |
          npm install -g allure-commandline
          allure generate --clean ./allure-results -o ./allure-report
      
      - name: Publish Allure report
        uses: actions/upload-artifact@v3
        with:
          name: allure-report
          path: ./allure-report

Jenkins

pipeline {
    agent any
    
    stages {
        stage('Test') {
            steps {
                sh '''
                    dotnet test ./Tests.E2E/Tests.E2E.csproj \
                      --logger:"junit;LogFileName=junit.xml" \
                      --results-directory ./allure-results
                '''
            }
        }
        
        stage('Report') {
            steps {
                sh 'allure generate --clean ./allure-results -o ./allure-report'
            }
        }
    }
    
    post {
        always {
            allure([
                includeProperties: false,
                jdk: '',
                properties: [],
                reportBuildPolicy: 'ALWAYS',
                results: [[path: 'allure-results']]
            ])
        }
    }
}

Customising Reports

Allure Configuration

Create allure.config.json:

{
  "reportName": "Playwright C# Agentic Framework Test Report",
  "reportLanguage": "en",
  "logo": "path/to/logo.png"
}

Generate with config:

Environment Information

Add environment details to reports:

Create allure-results/environment.properties:

Browser=Chromium
Browser.Version=120.0
OS=macOS
OS.Version=14.0
Framework=Playwright
Framework.Version=1.40.0
.NET.Version=9.0.0

This file is automatically picked up during report generation.

Viewing Reports

Local Development

CI/CD Artifacts

• GitHub Actions: Download allure-report artifact
• Jenkins: Access via Allure plugin
• Azure DevOps: Publish as build artifact

Static Hosting

Upload allure-report to any static hosting:

Troubleshooting

No Report Generated

Ensure JUnit logger is installed:

Missing Screenshots

Verify screenshot capture is enabled in LifecycleHooks.cs and check artifacts/screenshots/ directory exists.

Allure Command Not Found

Install Allure CLI:

Empty Report

Check that allure-results/ contains junit.xml and other metadata files. Re-run tests with the JUnit logger if needed.