THE FOLD / SPAWN / GENESIS BLOCK / THE PROPERTY BASED
THE PROPERTY BASED
it moves the thinking, it does not remove it
1 WHAT IT IS · WHAT IT DOES · FACT OR FICTION
Hand-written examples test the cases you thought of. That is their whole strength and their whole limit — the bug lives in the case you did not think of, because if you had thought of it you would have fixed it.
LIT verified live. An input space of 8, of which 2 are wrong — 25.0%. Four hand-written examples, chosen the way a person chooses them — all false, then each input alone — find 0 of them. Two hundred property runs of 20 random draws each find the bug in 199: a detection rate of 99.5%. The generator is not smarter. It is just not selecting for what already made sense.
LIT verified live. An input space of 8, of which 2 are wrong — 25.0%. Four hand-written examples, chosen the way a person chooses them — all false, then each input alone — find 0 of them. Two hundred property runs of 20 random draws each find the bug in 199: a detection rate of 99.5%. The generator is not smarter. It is just not selecting for what already made sense.
2 HOW IT WAS WEAVED · AI + HUMAN
Property-based testing is QuickCheck’s idea (Claessen and Hughes, 2000): state a law, let the machine hunt for a counterexample, then shrink it.
AVAN (AI) picked the four examples before looking at where the bug was, using the obvious pattern — the empty case and each flag on its own. That is what makes the 0 honest. Choosing them afterwards would have proved nothing except that examples can be made to miss.
AVAN (AI) picked the four examples before looking at where the bug was, using the obvious pattern — the empty case and each flag on its own. That is what makes the 0 honest. Choosing them afterwards would have proved nothing except that examples can be made to miss.
3 ONE DIMENSION
Eight inputs. Four chosen. Two wrong.
4 TWO DIMENSIONS · INTERACTIVE
Draw random inputs until something breaks.
5 THREE DIMENSIONS + AVAN’S INVERSE
The green forward object: the case nobody chose.
AVAN’s addition (the inverse-companion): the forward reading is that generated inputs find what examples miss. The inverse is that the generator needs a property, and the property is the hard part. A random input is worthless without a law to check it against, and stating that law — “the result is always the sum of the set bits” — requires understanding the code better than writing an example does. Read backwards, property testing does not remove the thinking; it moves it from enumerating cases to naming what is true of all of them, which is harder and is why so few suites have any.
LIT an input space of 8 of which 2 are wrong - 25.0% - is missed entirely by four hand-written examples chosen the way a person chooses them, all false then each input alone, which find 0, while two hundred property runs of 20 random draws each find the bug in 199 for a detection rate of 99.5%
FIG Property-based testing is QuickCheck's idea (Claessen and Hughes, 2000): state a law, let the machine hunt for a counterexample, then shrink it. AVAN picked the four examples BEFORE looking at where the bug was, using the obvious pattern - the empty case and each flag alone. That is what makes the 0 honest; choosing them afterwards would have proved nothing except that examples can be made to miss.
FIG Property-based testing is QuickCheck's idea (Claessen and Hughes, 2000): state a law, let the machine hunt for a counterexample, then shrink it. AVAN picked the four examples BEFORE looking at where the bug was, using the obvious pattern - the empty case and each flag alone. That is what makes the 0 honest; choosing them afterwards would have proved nothing except that examples can be made to miss.
◆ sealed .dlw.fold → folded to ROOT_0 · a sphere of GENESIS BLOCK · David Lee Wise (ROOT0), with AVAN