Skip to content

Before Class reported as Broken even it passed in retry #912

Description

@Apzal

Describe the bug
I have added retry plugin in my build.gradle, which retries when any test fails. The issue is when beforeclass annotated method fails in first run its reported as broken in Allure report but in retry if its get passed the status of before class is ignored and the report still shows as failed.

To Reproduce
Steps to reproduce the behavior:

  1. Add retry plugin in build.gradle file
plugins {
    id 'java'
    id 'io.qameta.allure'
    id "org.gradle.test-retry" version "1.5.2"
}

allure {
    def allureVersion = "2.15.0"
    version = allureVersion
}

test {
    useTestNG()
            {
        
                String groupName = System.getenv().getOrDefault('groups', 'REGRESSION')
                String testName = groupName
                includeGroups groupName
          
                setPreserveOrder(true)
                setParallel("classes")
                setThreadCount(10)
                setSuiteName("Test Suite")
                setTestName(testName)
                useDefaultListeners = true
                outputDirectory = file("$projectDir/TestNgReport")

            }
    retry {
        failOnPassedAfterRetry = false
        maxFailures = 200
        maxRetries = 2
    }
    reports.html.setDestination(file("$projectDir/GradleReport"))
    reports.junitXml.mergeReruns = true
}
  1. Run the TestNg test with before class which fails in first run and passes in second run. Build will be suceeded
  2. Generate Allure report
  3. Report shows before class as broken and Test as passed

Expected behavior
Before class should not be reported as a failed test instead report as passed.

Screenshots

image

Desktop (please complete the following information):

  • OS: Mac Monterey
  • Browser : Chrome
  • Version 112.0.5615.49

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions