I've been loving Benchmarking lately, but the Framework does this one quirky thing with the first result of a set. Specifically, the first return is always unusually high.

Lately, I've been spending time A/B benchmarking code against refactored versions assuming that both pieces of code pass their respective tests. One thing that always leaves me scratching my head is the way the first returned value of the Benchmark helper is always way higher than all other values. At first, I thought that this was due to some startup overhead in the framework, but I'm not so sure. Here's a test for you to try along with the result I got.

Here's the long-way-around code I played with for experimentation (each line is exactly the same):

php [ Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1] ];

Here's the result:

php [ 36.802083, 1.678583, 1.420791, 1.051042, 1.019792, 1.0015, 1.018708, 1.423625, 1.227666, 1.0715, 1.065666, 1.032791, 0.9855, 1.095708, 1.460583, 1.113666, ]

Weird right?

Here's another quirky example:

php $a = [ Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1] ]; $b = [ Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1], Benchmark::value(fn() => User::factory()->create())[1] ]; [$a,$b];

Result:

php [ [ 30.447542, 1.143333, 1.172375, 1.166459, 1.20575, 1.402584, 1.189042, ], [ 1.45775, 1.071958, 1.050291, 2.011417, 1.458583, 1.110292, 4.568708, ], ]

Methinks there be a ghost in the machine. submitted by /u/MuadDibMelange
[link] [comments]