added examples for tests that compare the compiler output with a reference

This commit is contained in:
mrdudz
2022-11-03 17:40:13 +01:00
parent 0eba33ee74
commit 44b2e4f331
7 changed files with 140 additions and 5 deletions

View File

@@ -20,6 +20,28 @@ compiler is working as expected (when the tests behave as described):
library.
/ref - These tests produce output that must be compared with reference output.
Normally the reference output is produced by compiling the program on the
host (using gcc mostly) and then running them on the host. Tests should
be tweaked to produce the same output as on the host in the cases where
it would be different.
The Makefile also handles some special cases (add the tests to the
respective list in the makefile):
- Sometimes we want to check the warnings produced by the compiler. In
that case use the CUSTOMSOURCES list. Whatever output the compiler writes
to stderr will be compared against the matching .cref file. There is an
example in custom-reference.c/.cref
- Sometimes we want to check what kind of output the compiler produces
for a file that does not compile. In that case use the ERRORSOURCES list.
There is an example in custom-reference-error.c/.cref
Warning: please understand that comparing the compiler output against
a reference produces a moving target, ie the tests may break randomly
at any time when the compiler output changes for whatever reason. So
only ever use this as a last resort when something can not be tested by
other means.
/err - contains tests that MUST NOT compile