Difference of values in duration in HTML report and xctest log · Issue #329 · XCTestHTMLReport/XCTestHTMLReport · GitHub
Skip to content

Difference of values in duration in HTML report and xctest log #329

Open

Description

@jbatra-wondermentapps

The HTML report and test-case logs have different values for duration

Is this a bug or some misunderstanding for interpretation?

Please route me to the right approach to read the logs.

Testing started
2023-05-26 12:24:58.207271+0530 PSSwiftExpressUITests-Runner[1191:259871] Running tests...
Test Suite 'Selected tests' started at 2023-05-26 12:24:58.269
Test Suite 'PSSwiftExpressUITests.xctest' started at 2023-05-26 12:24:58.270
Test Suite 'PerformanceTests' started at 2023-05-26 12:24:58.270
Test Case '-[PSSwiftExpressUITests.PerformanceTests testColdAppLaunchTime]' started.
    t =     0.00s Start Test at 2023-05-26 12:24:58.270
    t =     0.00s Set Up
executing: -[PerformanceTests testColdAppLaunchTime]
Battery before start:  0.96
RAM before start:  607993856
Free Disk space before start:  45.87 GB
    t =     0.01s Open com.adobe.PSMobile
    t =     0.01s     Launch com.adobe.PSMobile
    t =     0.01s         Terminate com.adobe.PSMobile:1168
    t =    21.39s         Setting up automation session
    t =    22.25s         Wait for com.adobe.PSMobile to idle
    t =    35.23s Open com.adobe.PSMobile
    t =    35.23s     Launch com.adobe.PSMobile
    t =    35.23s         Terminate com.adobe.PSMobile:1194
    t =    37.59s         Setting up automation session
    t =    37.70s         Wait for com.adobe.PSMobile to idle
    t =    49.34s Open com.adobe.PSMobile
    t =    49.34s     Launch com.adobe.PSMobile
    t =    49.34s         Terminate com.adobe.PSMobile:1198
    t =    51.61s         Setting up automation session
    t =    51.71s         Wait for com.adobe.PSMobile to idle
    t =    63.30s Open com.adobe.PSMobile
    t =    63.30s     Launch com.adobe.PSMobile
    t =    63.30s         Terminate com.adobe.PSMobile:1200
    t =    65.75s         Setting up automation session
    t =    65.85s         Wait for com.adobe.PSMobile to idle
    t =    77.44s Open com.adobe.PSMobile
    t =    77.44s     Launch com.adobe.PSMobile
    t =    77.44s         Terminate com.adobe.PSMobile:1203
    t =    79.95s         Setting up automation session
    t =    80.05s         Wait for com.adobe.PSMobile to idle
    t =    91.65s Open com.adobe.PSMobile
    t =    91.65s     Launch com.adobe.PSMobile
    t =    91.65s         Terminate com.adobe.PSMobile:1205
    t =    94.35s         Setting up automation session
    t =    94.37s         Wait for com.adobe.PSMobile to idle
/Users/usr/Documents/Adobe/Example/example-ios/PSSwiftExpressUITests/tests/PerformanceTests.swift:315: Test Case '-[PSSwiftExpressUITests.PerformanceTests testColdAppLaunchTime]' measured [Duration (AppLaunch), s] average: 0.436, relative standard deviation: 3.916%, values: [0.468968, 0.420271, 0.426677, 0.432267, 0.431298], performanceMetricID:com.apple.dt.XCTMetric_ApplicationLaunch-AppLaunch.duration, baselineName: "Local Baseline", baselineAverage: 2.786, polarity: prefers smaller, maxPercentRegression: 10.000%, maxPercentRelativeStandardDeviation: 10.000%, maxRegression: 0.000, maxStandardDeviation: 0.000
    t =   105.97s Tear Down
    t =   105.97s     -[PerformanceTests testColdAppLaunchTime]
    t =   107.15s     Terminate com.adobe.PSMobile:1206
Battery after end:  0.96
RAM after end:  590168064
Free Disk Space after end:  45.87 GB
Test Case '-[PSSwiftExpressUITests.PerformanceTests testColdAppLaunchTime]' passed (108.186 seconds).
Test Suite 'PerformanceTests' passed at 2023-05-26 12:26:46.458.
	 Executed 1 test, with 0 failures (0 unexpected) in 108.186 (108.188) seconds
Test Suite 'PSSwiftExpressUITests.xctest' passed at 2023-05-26 12:26:46.460.
	 Executed 1 test, with 0 failures (0 unexpected) in 108.186 (108.190) seconds
Test Suite 'Selected tests' passed at 2023-05-26 12:26:46.461.
	 Executed 1 test, with 0 failures (0 unexpected) in 108.186 (108.192) seconds
2023-05-26 12:26:46.379 xcodebuild[95428:2845548] [MT] IDETestOperationsObserverDebug: 113.146 elapsed -- Testing started completed.
2023-05-26 12:26:46.379 xcodebuild[95428:2845548] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
2023-05-26 12:26:46.379 xcodebuild[95428:2845548] [MT] IDETestOperationsObserverDebug: 113.146 sec, +113.146 sec -- end

Test session results, code coverage, and logs:
	/Users/usr/Documents/Adobe/Example/Example-ios/TestResults

** TEST SUCCEEDED **

usr@usr-MacBook-Pro Example-ios % xchtmlreport /Users/usr/Documents/Adobe/Example/Example-ios/TestResults.xcresult                                                     

Report successfully created at /Users/usr/Documents/Adobe/Example/Example-ios/index.html
usr@usr-MacBook-Pro Example-ios % xchtmlreport /Users/usr/Documents/Adobe/Example/Example-ios/TestResults.xcresult

HTML report

index 2.html.zip

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions