1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
|
Testing NASM
============
We use [Travis CI](https://travis-ci.org/) service to execute NASM tests,
which basically prepares the environment and runs our `nasm-t.py` script.
The script scans a testing directory for `*.json` test descriptor files
and runs test by descriptor content.
Test engine
-----------
`nasm-t.py` script is a simple test engine written by Python3 language
which allows either execute a single test or run them all in a sequence.
A typical test case processed by the following steps:
- a test descriptor get parsed to figure out which arguments
are to be provided into the NASM command line;
- invoke the NASM with arguments;
- compare generated files with precompiled templates.
`nasm-t.py` supports the following commands:
- `list`: to list all test cases
- `run`: to run test cases
- `update`: to update precompiled templates
Use `nasm-t.py -h` command to get the detailed description of every option.
Test descriptor file
--------------------
A descriptor file should provide enough information how to run the NASM
itself and which output files or streams to compare with predefined ones.
We use `JSON` format with the following fields:
- `description`: a short description of a test which is shown to
a user when tests are listed;
- `id`: descriptor internal name to use with `ref` field;
- `ref`: a reference to `id` from where settings should be
copied, it is convenient when say only `option` is different
while the rest of the fields are the same;
- `format`: NASM output format to use (`bin`,`elf` and etc);
- `source`: is a source file name to compile, this file must
be shipped together with descriptor file itself;
- `option`: an additional option passed to the command line;
- `update`: a trigger to skip updating targets when running
an update procedure;
- `target`: an array of targets which the test engine should
check once compilation finished:
- `stderr`: a file containing *stderr* stream output to check;
- `stdout`: a file containing *stdout* stream output to check;
- `output`: a file containing compiled result to check, in other
words it is a name passed as `-o` option to the compiler.
Examples
--------
A simple test where no additional options are used, simply compile
`absolute.asm` file with `bin` format for output, then compare
produced `absolute.bin` file with precompiled `absolute.bin.dest`.
```json
{
"description": "Check absolute addressing",
"format": "bin",
"source": "absolute.asm",
"target": [
{ "output": "absolute.bin" }
]
}
```
A slightly complex example: compile one source file with different optimization
options and all results must be the same. To not write three descriptors
we assign `id` to the first one and use `ref` term to copy settings.
Also, it is expected that `stderr` stream will not be empty but carry some
warnings to compare.
```json
[
{
"description": "Check 64-bit addressing (-Ox)",
"id": "addr64x",
"format": "bin",
"source": "addr64x.asm",
"option": "-Ox",
"target": [
{ "output": "addr64x.bin" },
{ "stderr": "addr64x.stderr" }
]
},
{
"description": "Check 64-bit addressing (-O1)",
"ref": "addr64x",
"option": "-O1",
"update": "false"
},
{
"description": "Check 64-bit addressing (-O0)",
"ref": "addr64x",
"option": "-O0",
"update": "false"
}
]
```
|
