Exception in beforeTests/afterTests not reported in a meaningful way

Description

This comes out of finding a test suite was not reporting errors due to a Global Exception. This is a result of an exception occurring in the beforeTests function. Currently we use TestBox 2.4 using the ANTJUnit reporter and have our CI server running TeamCity 2019. The CI never picked up the global exception.

I replicated the same behaviour on latest TestBox v3.2.0 from CommandBox

BaseTest

For the antjunit
http://127.0.0.1:62348/testbox/tests/runner.cfm?reporter=antjunit

You can see there is no reporting of any test case.

I’m not exactly sure where or why the errors is set to 7 as it should be the number of unit tests not an arbitrary value.
https://github.com/Ortus-Solutions/TestBox/blob/development/system/runners/UnitRunner.cfc#L107:L108
I did see comment on about the 7 value. To me this is a strange way to report the error.

If you compare it to how setup/teardown handle exceptions it works as I would expect for all life cycle methods.

My expectation for an exception occurring in beforeTests/afterTests functions (a global exception) is that all unit tests of that TestCase would have that error. This would override the results of any unit tests that completed. This would bring their behaviour inline with setup/tearDown.

Assignee

Luis Majano

Reporter

Ross Phillips

Labels

None

Components

Fix versions

Affects versions

Priority

Major
Configure