component extends="testbox.system.BaseSpec" {
function run(labels="runLabel1,runLabel2"){
writeDump(var=arguments, label="run()'s args");
describe(
title = "with labels",
body = function(){
writeDump(var=arguments, label="describe()'s body args");
it(
title = "is fine",
body = function(){
writeDump(var=arguments, label="it()'s body args");
},
labels = "itLabel1,itLabel2",
skip = function(){
writeDump(var=arguments, label="it()'s skip args");
return false;
}
);
},
labels = "describeLabel1,describeLabel2",
skip = function(){
writeDump(var=arguments, label="describe()'s' skip() args");
return false;
}
);
}
}
I would expect all the callbacks here to receive the labels that they are running under (so it()'s body and skip callbacks would receive runLabel1,runLabel2,describeLabel1,describeLabel2,itLabel1,itLabel2 as those are all the labels applying to them. Similarly with the functions further up the sequence, receiving a subset of those. Otherwise the labels serve very little purpose beyond flags for the runner to use.
Specifically the skip() callback should receive them so one can use them in the decision-making process as to whether to skip the test or not.
I'm raising this as a bug because I don't see the point of the current work to implement labels unless this has also been done.
PS: I can't help but think that skip() should perhaps receive other test-run telemetry stuff as well. It seems odd these callbacks receive nothing.
Implementation Update
The skip() closure in the describe blocks now receives the following metadata arguments: title, body, labels, asyncAll and the actual suite metadata.
Actually, the labels for the suite are in the suite metadata argument. So you can get them from there as well
Adam Cameron September 5, 2014 at 5:17 PM
Good stuff @LMajano. Is there a way to get the upstream labels too, as per this thing I mentioned:
it()'s body and skip callbacks would receive runLabel1,runLabel2,describeLabel1,describeLabel2,itLabel1,itLabel2 as those are all the labels applying to them. Similarly with the functions further up the sequence, receiving a subset of those.
Currently it()'s callback is only getting itLabel1,itLabel2. I'm OK with that, but is there some sort of rc variable which I can use to seen ancestor ones too? Because I belief labels should accumulate as one moves down the tree (also consider nested describe() calls).
Very good first iteration of this though.
Luis Majano September 5, 2014 at 4:26 PM
Ok, added a few related issues to this to enhance the ticket completely so the spec can get much more information about its execution surroundings and telemetry.
See https://github.com/daccfml/scratch/blob/master/blogExamples/unittests/testbox/labels/TestLabels.cfc:
component extends="testbox.system.BaseSpec" { function run(labels="runLabel1,runLabel2"){ writeDump(var=arguments, label="run()'s args"); describe( title = "with labels", body = function(){ writeDump(var=arguments, label="describe()'s body args"); it( title = "is fine", body = function(){ writeDump(var=arguments, label="it()'s body args"); }, labels = "itLabel1,itLabel2", skip = function(){ writeDump(var=arguments, label="it()'s skip args"); return false; } ); }, labels = "describeLabel1,describeLabel2", skip = function(){ writeDump(var=arguments, label="describe()'s' skip() args"); return false; } ); } }I would expect all the callbacks here to receive the labels that they are running under (so
it()'sbodyandskipcallbacks would receiverunLabel1,runLabel2,describeLabel1,describeLabel2,itLabel1,itLabel2as those are all the labels applying to them. Similarly with the functions further up the sequence, receiving a subset of those. Otherwise the labels serve very little purpose beyond flags for the runner to use.Specifically the
skip()callback should receive them so one can use them in the decision-making process as to whether to skip the test or not.I'm raising this as a bug because I don't see the point of the current work to implement labels unless this has also been done.
PS: I can't help but think that
skip()should perhaps receive other test-run telemetry stuff as well. It seems odd these callbacks receive nothing.Implementation Update
The skip() closure in the describe blocks now receives the following metadata arguments: title, body, labels, asyncAll and the actual suite metadata.
suite.skip = arguments.skip( title=arguments.title, body=arguments.body, labels=arguments.labels, asyncAll=arguments.asyncAll, suite=suite );The skip() closure in the it blocks now receive the following metadata arguments: title, body, labels, spec metadata:
spec.skip = arguments.skip( title=arguments.title, body=arguments.body, labels=arguments.labels, spec=spec );