Ignore:
Timestamp:
Jul 31, 2011, 12:09:51 AM (8 years ago)
Author:
shermer
Message:

improvements to lexicalStream package.
The unused position-tracking code in CharClassDescriptor? was removed.
LexicalDescriptor? was not doing much, so its functionality was pulled down and removed.
other minor changes.

File:
1 edited

Legend:

Unmodified
Added
Removed
  • proto/pebble/trunk/src/lexicalStream/readme.txt

    r1242 r1249  
    3434                                placing streams in the correct order on a branch.  These
    3535                                structures get linked into a tree by MoveLexdefs...
    36        
    37         LexicalDescriptor
    38                                 An abstract base class for the other stream descriptors.
    3936       
    4037        CharClassDescriptor
     
    110107
    111108After the visitor-driven processing, the LexicalCollector may
    112 be asked to generate the lexical structs (for the C++ backend) which are
    113 required to store the lexical computation.  This should only be invoked for the
    114 top LexicalCollector in a tree.
     109be asked to generate the lexical struct classes (for the pablo/C++ backend)
     110which are required to store the lexical computation.  This should only be
     111invoked for the top LexicalCollector in a tree.
    115112
    116113
     
    127124stream.  It is be formed from the specifier by util.VariableNamer.
    128125
    129 The term "descriptor" is used for an instance of some subclass of
    130 LexicalDescriptor, and these instances (I hope) contain and
    131 coordinate all of the information about the lexical stream.  I note an
    132 exception in that whether a character class is complemented is not stored
    133 in the CharClassDescriptor.
     126The term "descriptor" is used for an instance of CharClassDescriptor or
     127SequenceDescriptor, and these instances sort-of contain and coordinate
     128all of the information about the lexical stream.  I note exceptions in
     129that (1) whether a character class is complemented is not stored in the
     130CharClassDescriptor, and (2) positions at which a character class are
     131required are stored in a LexicalCollector.CharClassPositionsPair, not in
     132the CharClass Descriptor.
    134133
    135134The term "canonical" means a unique description of a lexical stream; I
     
    164163descriptors are meant to be attached to canonized objects rather than the
    165164rough parsed objects. There should be one descriptor per lexical stream
    166 needed, even if it occurs in many places.  There should also be only one
    167 descriptor for the collection of any stream and its shifted versions.
    168 Anyhow, a clean distinction between the two types of classes, or a merging
    169 of them, would be helpful.
     165needed, even if it occurs in many places.  This is currently not the case,
     166in that (say) a character class needed in one sub-branch of the AST does not
     167have the same descriptor as the same character class needed in an unrelated
     168sub-branch of the AST.   
    170169
    171 This is an old comment: currently the collector knows the difference between
    172 the two types of LexicalDescriptors, with methods for each; thus it is
    173 coupled to that hierarchy.  It may be better to have a class that is
    174 parameterized by the LexicalDescriptor concrete type, in case more
    175 types are added.
     170Anyhow, a clean distinction between the two types of classes(TokenData and
     171Descriptors), or a merging of them, might be helpful.
    176172
    177173The collector now uses a (descriptor, positionList) pair for character classes.
    178174This adds complicating logic to the character-class half with no corresponding
    179 complication for the sequence half, so parameterization would be difficult.
    180 The collector is getting fairly complicated and probably should be split into
    181 smaller objects.
     175complication for the sequence half.  The collector is getting fairly complicated
     176and probably should be split into smaller objects.
    182177
    183178
Note: See TracChangeset for help on using the changeset viewer.