Alexandre Linhares <linhares(a)clubofrome.org.br> writes:
Suppose the results from copy_if() are few, in relation to the
Wouldn't it be advantageous (performance-wise) to create a results-sized
buffer, copy the results there, and minimize buffer transfer?
In other words, am I introducing any bugs?
Source code here:
Thank you for your work. It is wonderful!
The interface to copy_if is as it is on purpose. As we start the scan to
do the main operation, we don't yet know how large the result will be,
so we have to allocate the full length.
From here, there are two possible ways forward, but which you choose is
- You can allocate a 'right-sized' output array and copy your result to
it. This is what you're suggesting, if I understand you right. The
advantage is that you don't waste memory for entries that are unused.
The downside is that the copy takes time. This is usually what you'd
do if the result is intended to be long-lived.
Btw, you can achieve this just via out[:count].copy().
- You can also just grab a smaller view of the results array by
out[:count], knowing that there's a larger array behind that being
kept alive 'behind' it. This is a good solution if the result is
If you're mainly concerned about copy costs, out[:count].get() should
be yet faster than your solution, because you don't incur the cost of
allocating the temporary and copying into it.
Since copy_if cannot make that trade-off for you (for example, it can't
know how long-lived the result is intended to be), the interface is the
way it is, allowing you to achieve a cost-optimal solution for your
situation with just a tiny bit of extra work. I'd love to take a patch
to the documentation that explains this, if you have time.
Hope that helps,