source: docs/HPCA2012/04-methodology.tex @ 1644

Last change on this file since 1644 was 1643, checked in by ksherdy, 8 years ago

Minor edit.

File size: 5.0 KB
Line 
1\pagebreak
2\section{Evaluation Framework}
3\label{section:methodology}
4
5\paragraph{XML Parsers:}\label{parsers}
6We evaluate the Parabix XML parser described above
7against two widely available open-source parsers: Xerces-C \cite{xerces} and Expat \cite{expat}.
8Each of the parsers is evaluated on the task of implementing the
9parsing and well-formedness validation requirements of the full
10XML 1.0 specification\cite{TR:XML}.
11Xerces-C version 3.1.1 (SAX) is a validating XML
12parser written in C++ and is available as part of the the Apache project.
13Expat version 2.0.1 is a stream-oriented non-validating XML parser library written in C.
14To ensure a fair comparison, we restricted our analysis of Xerces-C to its WFXML scanner to
15eliminate the cost of non-well-formedness validation and used the SAX interface to avoid
16the memory cost of DOM tree construction.
17
18\paragraph{XML Workloads:}\label{workloads}
19XML is used for a variety of purposes ranging from databases to configuration
20files in mobile phones.
21A key predictor of the overall parsing performance of an XML file is \textit{Markup density} (i.e., the ratio of markup vs. the total XML document size.)
22This metric has substantial influence on the performance of
23traditional recursive descent XML parsers.  We use a
24mixture of document-oriented and data-oriented XML files
25to analyze performance over a spectrum of markup densities.
26
27Table \ref{XMLDocChars} shows the document characteristics of the XML
28input files selected for this performance study.  The jawiki.xml and
29dewiki.xml XML files represent document-oriented XML inputs and
30contain the three-byte and four-byte UTF-8 sequence required for the
31UTF-8 encoding of Japanese and German characters respectively.  The
32remaining data files are data-oriented XML documents and consist
33entirely of single byte  encoded ASCII characters.
34
35\begin{table*}[!h]
36\begin{center}
37{
38\footnotesize
39\begin{tabular}{|l||l|l|l|l|l|}
40\hline
41File Name               & dew.xml               & jaw.xml               & roads.gml     & po.xml        & soap.xml \\ \hline   
42File Type               & document              & document              & data          & data          & data   \\ \hline     
43File Size (kB)          & 66240                 & 7343                  & 11584         & 76450         & 2717 \\ \hline
44Markup Item Count       & 406792                & 74882                 & 280724        & 4634110       & 18004 \\ \hline
45Markup Density          & 0.07                  & 0.13                  & 0.57          & 0.76          & 0.87  \\ \hline
46\end{tabular}
47}
48\end{center}
49\caption{XML Document Characteristics} 
50\label{XMLDocChars} 
51\end{table*}
52
53
54\paragraph{Platform Hardware:}
55SSE extensions have been available on commodity Intel processors for
56over a decade since the Pentium III. They have steadily evolved with
57improvements in instruction latency, cache interface, register
58resources, and the addition of domain specific instructions. Here we
59investigate SIMD extensions across three different generations of
60intel processors (hardware details in Table \ref{hwinfo}). We compare
61the energy and performance profile of the Parabix under the platforms.
62We also analyze the implementation specifics of SIMD extensions under
63various microarchitectures and the newer AVX extensions supported by
64Sandybridge.
65
66
67We investigated the execution profiles of each XML parser
68using the performance counters found in the processor.
69We chose several key hardware events that provide insight into the profile of each
70application and indicate if the processor is doing useful work
71~\cite{bellosa2001, bertran2010}
72The set of events included in our study are: Branch instructions, Branch mispredictions,
73Integer instructions, SIMD instructions, and Cache misses. In
74addition, we characterize the SIMD operations and study the type and
75class of SIMD operations using the Intel Pin binary instrumentation
76framework.
77
78\begin{table*}[h]
79\begin{center}
80\footnotesize
81\begin{tabular}{|l||l|l|l|}
82\hline
83Processor & Core2 Duo (2.13GHz) & i3-530 (2.93GHz) & Sandybridge (2.80GHz) \\ \hline
84L1 D Cache & 32KB & 32KB & 32KB \\ \hline       
85L2 Cache & Shared 2MB & 256KB/core & 256KB/core \\ \hline 
86L3 Cache & --- & 4MB  & 6MB \\ \hline 
87Bus or QPI &  1066Mhz Bus & 1333Mhz QPI & 1333Mhz QPI \\ \hline
88Memory  & 2GB & 4GB & 6GB\\ \hline
89Max TDP & 65W & 73W &  95W \\ \hline
90\end{tabular}
91\caption{Platform Hardware Specs} 
92\label{hwinfo}
93\end{center}
94\vspace{-20pt}
95\end{table*}
96
97
98
99\paragraph{Energy Measurement:}
100A key benefit of the Parabix parser is its more efficient use of the
101processor pipeline which reflects in the overall energy usage.  We
102measure the energy consumption of the processor directly using a
103current clamp. We apply the Fluke i410 current clamp \cite{clamp} to the 12V wires
104that supply power to the processor sockets. The clamp detects the
105magnetic field created by the flowing current and converts it into
106voltage levels (1mV per 1A current). The voltage levels are then
107monitored by an Agilent 34410a digital multimeter at the granularity
108of 100 samples per second. This measurement captures the instantaneous
109power to the processor package, including cores, caches, northbridge
110memory controller, and the quick-path interconnects. We obtain samples
111throughout the entire execution of the program and then calculate overall
112total energy as  $12V*\sum^{N_{samples}}_{i=1} Sample_i$.
113
114
Note: See TracBrowser for help on using the repository browser.