objective-c - 如何在 XCTest 中提取由 measureBlock 测量的性能指标

Question

我有一个简单的测试功能，它会点击一个按钮来衡量性能。我正在使用 XCTest。 measureBlock 返回后，我可以在控制台上看到一堆性能指标。我想在测试程序中得到这个，这样我就可以以编程方式在其他地方填充数据。事实证明，在测试控制台上查看测试数据很慢，因为我有很多测试用例。

- (void)testUseMeasureBlock {
    XCUIElement *launchTest1Button = [[XCUIApplication alloc] init].buttons[@"Launch Test 1"];
    void (^blockToMeasure)(void) = ^void(void) {
        [launchTest1Button tap];
    };
    // Run once to warm up any potential caching properties
    @autoreleasepool {
        blockToMeasure();
    }

    // Now measure the block
    [self measureBlock:blockToMeasure];
    /// Collect the measured metrics and send somewhere.

当我们运行测试时，它会打印:

measured [Time, seconds] average: 0.594, relative standard deviation: 0.517%, values: [0.602709, 0.593631, 0.593004, 0.592350, 0.596199, 0.593807, 0.591444, 0.593460, 0.592648, 0.592769], 

如果我能得到平均时间，那现在就足够了。

Answer 1

由于没有 API 来获取这些数据，您可以通过管道 stderr流式传输和解析测试日志以获取所需信息，例如平均时间。例如，您可以使用下一种方法:

@interface MeasureParser : NSObject

@property (nonatomic) NSPipe* pipe;
@property (nonatomic) NSRegularExpression* regex;
@property (nonatomic) NSMutableDictionary* results;

@end

@implementation MeasureParser

- (instancetype)init {
    self = [super self];
    if (self) {
        self.pipe = NSPipe.pipe;
        self.results = [NSMutableDictionary new];
        
        let pattern = [NSString stringWithFormat:@"[^']+'\\S+\\s([^\\]]+)\\]'\\smeasured\\s\\[Time,\\sseconds\\]\\saverage:\\s([^,]+)"];
        NSError* error = nil;
        self.regex = [NSRegularExpression regularExpressionWithPattern:pattern options:NSRegularExpressionCaseInsensitive error:&error];
        if (error) {
            return nil;
        }
    }
    return self;
}

- (void)capture:(void (^)(void))block {
    // Save original output
    int original = dup(STDERR_FILENO);

    setvbuf(stderr, nil, _IONBF, 0);
    dup2(self.pipe.fileHandleForWriting.fileDescriptor, STDERR_FILENO);
    
    __weak let wself = self;
    self.pipe.fileHandleForReading.readabilityHandler = ^(NSFileHandle *handle) {
        var *str = [[NSString alloc] initWithData:handle.availableData encoding:NSUTF8StringEncoding];
        
        let firstMatch = [wself.regex firstMatchInString:str options:NSMatchingReportCompletion range:NSMakeRange(0, str.length)];
        if (firstMatch) {
            let name = [str substringWithRange:[firstMatch rangeAtIndex:1]];
            let average = [str substringWithRange:[firstMatch rangeAtIndex:2]];
            wself.results[name] = average;
        }
        
        // Print to stdout because stderr is piped
        printf("%s", [str cStringUsingEncoding:NSUTF8StringEncoding]);
    };
    
    block();
    
    // Revert
    fflush(stderr);
    dup2(original, STDERR_FILENO);
    close(original);
}

@end

如何使用:

- (void)testPerformanceExample {
    
    let measureParser = [MeasureParser new];
    
    [measureParser capture:^{
        [self measureBlock:^{
            // Put the code you want to measure the time of here.
            sleep(1);
        }];
    }];
    
    NSLog(@"%@", measureParser.results);
}

// Outputs

{
    testPerformanceExample = "1.001";
}