1

anonymous@undisclosed.example.com (Anonymous) — Tue, 15 Jun 2021 20:09:17 +0000

2012.04.06 - measurements

i've wrote about the need for measuring. quite a few times actually… it is so obvious, for an engineer, to perform an experiment to “see what happens”… or is it?

it appears that not that many engineers does that after all. that is why superstitions still lives on. “templates are bad”¹⁾, “exceptions have terrible impact on performance”, “if you use exceptions your program has an errors” and a lot, lot more bullshit bouncing here and there, on the internet and among programmers. some time ago i was “charged” of wasting time, while doing measurements!

if you're sure about your opinions, why not to check it with the proper experiment(s)? and if you have measured this 5 years ago, how come are you sure your results still holds? i highly recommend Bjarne Stroustrup's C++11 style lecture, presented during Going native 2012 conference²⁾. for the matter of this text, pay a special attention to the part from 44:50 to 52:48 of Bjarne's the lecture. since on that particular presentation important graph disappeared, along watching see the C++11 Style presentation, starting with page 43. now you have a time to watch it. i mean it. do it now – i'll wait! ok so if you've seen it – what was YOUR first reaction? what result did YOU expect? well – my expectation was wrong too… the funny thing is, as Bjarne noticed, it was so for everyone. times have changed and CPUs cache is now a crucial resource. if your expectations was that the list will be faster around 100-10k elements – it WAS true… about 10 years ago.

what just happened? knowledge outdated. this is normal. 10 years is quite a time, actually… :) and this is the clue of what i want to share – MEASURE and SAVE your test programs for future! even if you checked and confirmed your opinion it does not mean that a new compiler or a newer CPU won't make the results dramatically different – even counter intuitive.

in my opinion each project should have one more directory, along “src”, “tests”, “doc” and similar, namely “measurements”. this would be the location where a lot of (short) “checker” programs will be held, that check a number of means to solve each problem, team has decided to measure. this can be memory usage, run time or even as simple as output binary size. with rational description (simple documentation) on what and how is being measured, this can be used to periodically ensure that foundations of solutions being used still holds.

think about it as a way of extending an idea of automated testing, for the conceptual (design) decisions. tests are run at least daily, by CI tool. “measurements” tests can be run the same way, but in longer time spans – each time toolchain and/or hardware changed + every half a year (keep in mind, that if you're supposed to do it manually and rarely, you probably will forget about it ;)). cycle is simple: measure, save, re-measure and compare. results can be astonishing…

¹⁾

whatever does this mean…

²⁾

this particular lecture was also performed in Wrocław recently.

2

anonymous@undisclosed.example.com (Anonymous) — Tue, 15 Jun 2021 20:09:17 +0000

2012.04.06 - "undefined behavior" IS bad

among projects i worked on, there were cases, where undefined behavior was exploited, to achieve something useful. just to mention one case, when memory was allocated with new MyClass and freed… using free(void*). this was (still is?) done in the main messaging framework of the million-line project. for compiler version in use, it happened to work, as long as MyClass had POD-only members¹⁾.

i was always against such a hacks and promoted solutions that would not require it. in practice most of the times the answer was “it's working so don't touch it”. very ignorant approach at least…

when something is UB, you should see big red sign, that says: “if you cross this line ANYTHING can happen”. and so yesterday i came across a guy on the usenet, posting that his few-lines program does not work when compiled with optimizations on GCC 4.6. here it is:

#include<cstdlib>
#include<cstdio>
 
int main( int argc, char *argv[] )
{    
    printf("test");
    int t = 0;
    for( int i=0 ; i<30000 ; i++ )
        for( int j=0 ; j<10000 ; j++ )
            t += j;
    printf("%d",t);
    return 0;
}

when compiled with

g++ -Wall x.cpp

everything works fine – program executes and prints the result on the screen. adding optimizations, however causes program to loop forever. namely, if compiled with:

g++ -Wall x.cpp -O3

program never ends. what is the problem with this code? it uses UB of overflowing signed integer. adding -fno-strict-overflow (i.e. of-the-standard extension, making signed integer overflows defined) solves problem – compiler computes value in compile-time and puts a constant value in the code.

using the wrong side of force is asking for a trouble. if you do so, sooner or later you'll bounce from the wall, spend hours (or days) debugging just to find out you're not spec-compliant and having to fix code, that should have been written the right way from the very beginning. is it worth it?

btw: GCC 4.7 gives the same results as 4.6, while GCC 4.5 and Clang++ 3.0 gives “expected” results. unless you're an oracle, forget about using UBs!.

¹⁾

btw: MyClass was not a POD though.

BaSzErr - blog:2012:04:06

1

2012.04.06 - measurements

2

2012.04.06 - "undefined behavior" IS bad