Changeset 3382


Ignore:
Timestamp:
Jul 3, 2013, 3:40:47 PM (6 years ago)
Author:
linmengl
Message:

add pabloSupport, fix tiny problem

Location:
proto/CSV/csv2xml
Files:
5 edited

Legend:

Unmodified
Added
Removed
  • proto/CSV/csv2xml/Report on Perf.txt

    r2663 r3382  
    6666L2_2012-01   (103.2MB) 32
    6767L2_2012-02   (130.5MB) 39
     68
     69improved not much on files, but have improvement in File off, buffer on mode.
     70
     71Nov. 22nd
     72Put an end to this task, there are still something can be done:
     73
     741. Copy a faster version of memcpy using SSE2. There are such code online.
     752. Remove *writer. It seems prevent inline. But in reality memcpy is inlined.
     763. Multithread buffering. Cause you know how long is each block, how many "<col>" or "<row>" you need to insert in the block, so you can do data parallelism on blocks(maybe fixed as 10 blocks, you need the start position of each block's output, so parallel is possible)
     774. Two or more thread on input and output. Try to balance time spent on each I block and O block.
     78    I1 I2   I3   I4
     79        O1 O2  O3  O4
     805. [Rob] Reduce testing, by assuming every line contains the same number of columns. To reduce if clause by delicate design.
  • proto/CSV/csv2xml/pablo_template.cpp

    r2916 r3382  
    1616#include "../lib/s2p.hpp"
    1717#include "../lib/perflib/perfsec.h"
     18#include "../lib/pabloSupport.hpp"
    1819
    1920// Define the mappings for pablo.assert_0(strm, errkind) statements which
  • proto/CSV/csv2xml/src/Makefile

    r2663 r3382  
    55
    66CC= g++ $(CFLAGS)
    7 CFLAGS= $(SSE) -O3
     7CFLAGS= $(SSE) -O3 -std=gnu++0x
    88SSE=-msse2 #-msse4.1 -DUSE_PTEST #-mssse3 -msse4.1 -msse4.2
    99INCLUDES=-I../lib/ -I../util/
  • proto/CSV/csv2xml/src/csv.cpp

    r2916 r3382  
    1616#include "../lib/s2p.hpp"
    1717#include "../lib/perflib/perfsec.h"
     18#include "../lib/pabloSupport.hpp"
    1819
    1920// Define the mappings for pablo.assert_0(strm, errkind) statements which
  • proto/CSV/csv2xml/util/bitsegment_iterator.hpp

    r2663 r3382  
    33*/
    44
    5 #include "bitblock.hpp"
     5#include "../lib/bitblock.hpp"
    66#include "../lib/bitblock_iterator.hpp"
    77
Note: See TracChangeset for help on using the changeset viewer.