source: trunk/symbol_table/README.txt @ 2113

Last change on this file since 2113 was 2113, checked in by ksherdy, 7 years ago

Updated Makefile. Updated comments.

File size: 4.5 KB
Line 
1Author: Ken Herdy
2Created on: May 17, 2012
3
4ABOUT
5
6    This project builds length sorted symbol table test driver(s)
7    to validate symbol table correctness and evaluate performance.
8
9    The test driver is structured to slurp and process a entire
10    'generated' CSV test file on execution.
11
12    Test files are generated using the python script
13    'test/gen_test_file.py' or 'test/st_test_file_generator.py'.
14
15    Test files:
16
17    (i) begin with and separate each symbol with a single comma,
18    (ii) contain symbols characters defined by the the character class [-a-zA-Z0-9_:.],
19    (iii) do not contain EOF characters. Avoid hand editing test files
20    within applications that append EOF characters,
21    such as gedit, or 'diff' tests report missing newline differences.
22
23HOW TO BUILD
24
25    This project currently supports symbol driver
26    builds for the each of the following length group strategies.
27
28    id                  G(L) = L
29    div2                G(L) = ceiling(L/2)
30    logbase2            G(L) = logbase2(L)
31    div2_logbase2       G(L) = div2(L) if L < k, k is even else logbase2(L)
32    bit_byte            G(L) = 7 if L < 8 else G(L) = 0
33
34    This project adopts a two stage build process in which,
35
36    (i)  the root directory Makefile compiles parallel bit stream source files to C++, and
37    (ii) the src directory Makefile compiles generated C++ code.
38
39    --- PERF BUILD ---
40
41    Builds symbol table driver for performance profiling.
42
43    cd symbol_table
44    make {id, div2, logbase2, div2_logbase2, bit_byte}
45    cd src
46    symbol_table/src/Makefile make {id, div2, logbase2, div2_logbase2, bit_byte}
47    ./id ../test/pin/\(L,0,U\)+_1_1.test
48
49    For example,
50
51    make id
52    cd src/
53    make id
54    ./id ../test/pin/\(1_1000_50\)_\(2_1000_50\)_\(3_1000_50\)_\(4_1000_50\)_1_1.test
55    BOM 16: 1 (avg time: 43735 cyc/kElem) Cumulative: 1 (avg: 43735 cyc/kElem)
56
57
58    --- GID TEST ---
59
60    A rudimentary test that evaluates that the correct number of unique GIDs
61    are allocated for each *.test file in the test/pin directory.
62
63    For example,
64
65    make id
66    cd src/
67    make id_gid_test
68
69    .
70    .
71    .
72    (9_1000_50)_(10_1000_50)_(11_1000_50)_(12_1000_50)_1_1.test
73    Expected: 200
74    Actual: 200
75    .
76    .
77    .
78    All pass.
79
80    NOTE:
81
82    File format: (Length, Occurences, Unique)...(Length, Occurences, Unique)_1_1.test
83
84    --- DIFF TEST ---
85
86    A round trip test that for each *.test file in the test/pin directory:
87    (i) resolves the unique symbol gid for each symbol,
88    (ii) writes the gid to the corresponding symbol starts position in the occurences array,
89    (iii) looks up each the gid for each symbol start position,
90    (iv) generates an output file to the test/pout directory, and
91    (v) diffs test/pin and test/pout.
92
93    make id
94    cd src/
95    make id_diff_test
96
97    diff -rq --exclude=".svn" ../test/pin ../test/pout || exit 0
98
99    NOTE: No differences reported indicates all diff tests pass.
100
101
102PROJECT STRUCTURE
103
104    symbol_table
105    │
106    ├── Compiler
107    ├── demo_strms.py
108    ├── bitutil.py
109    │
110    ├── bit_byte_group_strms.py
111    ├── div2_group_strms.py
112    ├── div2_logbase2_group_strms.py
113    ├── id_group_strms.py
114    ├── logbase2_group_strms.py
115    ├── group_strms_template.hpp
116    │
117    ├── hash_strms.py
118    ├── hash_strms_template.hpp
119    │
120    ├── main_template.cpp
121    │
122    ├── marker_strms.py
123    ├── marker_strms_template.hpp
124    │
125    ├── Makefile
126    ├── README.txt
127    │
128    ├── src
129    │   â”œâ”€â”€ basis_bits.hpp
130    │   â”œâ”€â”€ buffer.hpp
131    │   â”œâ”€â”€ byte_pool.hpp
132    │   â”œâ”€â”€ compare_strategy.hpp
133    │   â”œâ”€â”€ gid.hpp
134    │   â”œâ”€â”€ group_strms.hpp
135    │   â”œâ”€â”€ hash_strategy.hpp
136    │   â”œâ”€â”€ hash_strms.hpp
137    │   â”œâ”€â”€ hash_table.hpp
138    │   â”œâ”€â”€ Makefile
139    │   â”œâ”€â”€ marker_strms.hpp
140    │   â”œâ”€â”€ strategy_types.hpp
141    │   â”œâ”€â”€ symbol_table.hpp
142    │   â””── transpose.hpp
143    │
144    ├── test
145    │   â”œâ”€â”€ gen_test_file.py
146    │   â”œâ”€â”€ Makefile
147    │   â”œâ”€â”€ pin
148    │   â”‚   â””── (L_O_U)_1_1.test
149    │   â”œâ”€â”€ pout
150    │   â”‚   â””── (L_O_U)_1_1.test
151    │   â”œâ”€â”€ run_tests.py
152    │   â””── st_test_file_generator.py
153    │
154    ├── lib
155    ├── libtest
156    │   â”œâ”€â”€ bitscantest.cpp
157    │   â”œâ”€â”€ byte_compare_generator.cpp
158    │   â”œâ”€â”€ byte_compare_template.cpp
159    │   â”œâ”€â”€ hash_test.cpp
160    │   â”œâ”€â”€ Makefile
161    │   â””── pool_test.cpp
162    │
163    └── transpose.hpp
164
Note: See TracBrowser for help on using the repository browser.