Menu

scScript Pattern Matching

scScript implements a comprehensive and powerful text matching system. The tables below summarize its functions, flags, directives, accessors, and templates. Detailed design and operation, along with multiple examples, are documented in sections that follows. There is a lot here, so buckle up!

Note: It is easier to just scan over these tables in the first reading; it will all make more sense later.

Pat Functions: (create a Pat first, then Match against data!)
Seq
P = Pat.Seq(F1, A1, A2, ...,
            F2, A10, A11, ...,
            ...
            Dir1, Arg1,
            Dir2, Arg2, ...);
 Returns pat to match concatenated sequence of args (conjunction).
F1, F2, ..., are Flags that affect subsequent args.
A1, A2, ..., are args. These can be Strs, pats, or PFunc (closures)

Dir1, Dir2, ..., are directives of where to stash, etc.
Arg1, Arg2, ...., are args for the directives, one per.
Alt
P = Pat.Alt(...same as Seq...);		 Returns pat to match one of the alternative args (disjunction).
Defaults to #SELF.
Span
P = Pat.Span(F1, SArg, F2, EArg,
             Dir1, Arg1,
             Dir2, Arg2, ...);
 Returns pat to span from SArg (start) all the way to to EArg (end),
skipping everything in between.

Only accepts flags and 2 string/pat args,
followed by stash directives.
Defaults to #MIN and #SELF.
Rep
P = Pat.Rep(Min, Max,
            ...same as Seq...);
 Returns pat to match the specified repetitions of the arg(s).

Min/Max can be dynamic values (from PDict) if given strings.
Min/Max can also be VFunc callbacks, return value at runtime.
Only #CONS/#SELF flags preceding first match arg are honored.
Defaults to #MIN and #CONS.
Iter
P = Pat.Iter(Cnt, 
             ...same as Seq...);
 Returns special Rep Pat, with MIN and MAX both set to Cnt!
Will match args exactly Cnt times.
Cnt can only be Int, use Rep for other options!
Char
P = Pat.Char(F1, A1, A2, ...,
             F2, A10, A11, ...,
             ...
             Dir1, Arg1,
             Dir2, Arg2, ...);
 Returns pat to match the given range of characters.
Args (A1, A2, etc.) are strings (perhaps pre-def) or other Char pats.
Strs list chars, can have ranges ("c-q") too. ('-' first if not range)
No PFunc and no #Sen/#InSen flags, succeeds if char is in the Pat.

Stores inclusion set only, #Neg Str/Pats must come after #Pos ones.
When matched, #S_Cnt is Unicode # for char, #S_Len1 is CharLen.
Default is #Smart (À matches if A is in), use #Exact for exclusion.
Any
P = Pat.Any(F1, F2, ...,
            Dir1, Arg1,
            Dir2, Arg2,
            ...);
 Returns pat to match any character.
More efficient version of Pat.Char(Pat.#All)!
Used to eat characters during a match.
Stashes everything Char does, except #S_Cnt will be UTF-8 CharLen.
Go
P = Pat.Go(Loc, ...Dir...);
 Returns pat to move match process (goto) to new data location.
Loc can be Int, Str (dynamic), or VFunc.
Loc defaults to #Rel and #H (chars), can also be #Abs and/or #V (lines).
At
P = Pat.At(Loc, ...Dir...);
 Returns pat that succeeds only if at specified location!
Loc can be Int, Str (dynamic), or VFunc.
#ABS Loc is from Buf beginning if >= 0, or Buf end if < 0.
#REL Loc is from FirstPos if >= 0, or LastPos if < 0.
#H deals with Pos, V deals with lines.
Does not stash any data, but can handle other directives.

Run Functions:
Make
P = Pat.Make(Type, a1, a2, ...);
 Return a pattern (like above), but takes Type as arg.
Match
Pat.Match(Bn, Pat/Str,
          Mode/MFunc,
          StartPos,
          FirstPos,
          LastPos,
          RFunc, GapRFunc)

         (PatDict,
          MStartPos, MEndPos);
 Match the given pattern against data.
BN (Required) BufferNumber from scEmacs.
Pat/Str (Required) is what is being matched.
Mode (Opt, Default #Once) determines Match operation mode.
StartPos (Opt, Default 0 is FirstPos) Pos to start matching.
FirstPos (Opt, Default 0) First permissible Pos in Buffer.
LastPos (Opt, Default 0 is Buf End) one past last permissible Pos.
RFunc (Opt callback) Called if match(s) succeeds (only for #Res Pats).
GapRFunc (Opt callback) Called for regions Match scans over.
PatDict (Opt) Set to internal PDict, or use PDict if starting with one.
MStartPos/MEndPos (Opt) Set to Start/End pos of successful match.

Flags: (Pat flags are sticky, once set, they apply to all subsequent args!)
Match Mode #Anchor, #Once, #Slide, or #Skip (also #Custom used by MFunc only)
Pat Types #Seq, #Alt, #Span. #Rep, #Char, #Any, #Go, or #At (Iter is just Rep)
Op Flags (Rep and Span) #Min (default) or #Max
Mode Flags
(Alt, Span, and Rep)		 #Self (default) or #Cons (default ONLY for Rep)
Result Flags #NoRes (default) or #Res (Store in RStack for RFunc)
Case Flags (Sticky!) #Insen (default) or #Sen
Char Flags (Sticky!) #Smart (default, root char can match accented version) or #Exact
Polarity Flags (Sticky!) #Pos (default) or #Neg (See Note ++++ in Format section)
Position Flags (Sticky!) #Rel (default) or #Abs
Direction Flag (Sticky!) #H (default) or #V
Sense Flag (Sticky!) #Str (default, string) or #Dyn (dynamic! Key for dict lookup)

Directives: (Do when Pat matches.)
#S_Pos (Key) Stash match position in PDict under Key (Str).
#S_Str (key) Stash match string in PDict under Key.
#S_Ord (Key) Stash match order in PDict under Key.
#S_Cnt (Key) Stash match count in PDict under Key.
#S_Len1 (Key) Stash match Len1 in PDict under Key.
#S_Len2 (Key) Stash match Len2 in PDict under key. (Only used by Span)
#S_Func (SFunc) Call SFunc (Closure) to store in PDict or affect client data.
Attachment Directives: (Do at Pat definition time.)
#S_Name (NameStr) Attach Name to Pat (Def time), for display and debugging!
#S_Data (Val) Attach Data to Pat (Def time), can be accessed by callbacks.

Accessors: (Called from callbacks to get data during match)
           (No Pat, Name, Data, or Type when called from GapRFunc!)
GetRange Returns current match range as (Int: (EndPos << 32) + BegPos).
GetStr Returns current match string (Str).
GetOrd Returns current match order (Int).
GetCnt Returns current match count (Int).
GetLen1 Returns current match Len1 (Int).
GetLen2 Returns current match Len2 (Int). (Only used by Span)
GetPat(Not GapRFunc) Returns Pat that is/was matched or 0 if called from GapRFunc.
GetName(Not GapRFunc) Returns #S_Name attached to Pat (Str, or 0 if none).
GetData(Not GapRFunc) Returns #S_Data attached to Pat (Val, or 0 if none).
GetType(Not GapRFunc) Returns Type of Pat that is/was matched or 0 if called from GapRFunc.
GetBuf Returns Buf Number for this match.
GetBufRange Returns permissible use range in buffer (Int: (EndPos << 32) + BegPos).

Callback Funcs: (Different function types, different args!)
MFunc
MFunc(PDict)(NextPos, SPos, EPos)
{...}		 Match Func (Called from Match after success)
Decides whether to do another match, and from where!
Returns: #Done to stop the match.
         #Custom, to continue match from NextPos!
         #Skip, to continue match after EPos.
         #Slide, to continue match from SPos+1 (will adjust for UTF-8).
SPos/EPos are start/end of current match.
VFunc
VFunc(PDict, VOrd)()
{...}		 Value Func (Provided in Pat definition to set required Op values)
Return Int value for VOrd Op arg.
VOrd is 1 for first Op arg, 2 for 2nd, etc.
VFunc is called at Match time, NOT when define/create Pat!
SFunc
SFunc(PDict, Data)()
{...}		 Stash Func (Provided in Pat definition, after #S_Func directive)
Returns Int 0 to abort whole match, everything else means proceed.
Data is from #S_Data, or Int 0.
Called after all other stash directives.
PFunc
PFunc(PDict)(Pos)
{...}		 Pat Func (Provided in Pat definition in place of sub-Pat or Str)
Returns Int 0 if match fails, Non-0 Int if success.
Otherwise returns Str or Pat that needs matching.
NOT for Char Pats!
RFunc (or GapRFunc)
RFunc(PDict, SPos, EPos, ROrd, Numb, PatP, PType)()
{...}		 Res Func (Called by Match to inform script of each succ. match)
Only works for Pats with #Res flag! ****
Return 0 to abort/terminate, non-0 to continue.
SPos and EPos are matched byte range.
ROrd is 1-based order in RStack.
Numb is which Elt (of Alt), which Rep, or what Span size matched.
PatP is the one matched and PType is its Pat type.
(ROrd, Numb, PatP, and PType are meaningless in GapRFunc.)

**** RFunc, like SFunc, are callbacks to convey match results to the controlling script. The process of matching a #Cons Pat normally involves searching, going down blind alleys and reversing course. This could be very confusing for the calling script, as the results communicated by SFunc callbacks could be subsequently invalidated. Instead, Match maintains an RStack (Result Stack) and can call the RFunc (instead of SFunc) after everything is done, using final results stored on the RStack.

Match invokes GapRFunc on text it scans over (gap) before matching (or matching again).

Char Range Defs:
#Alpha Alphanumeric A-Z, a-z, and 0-9
#Letter Letters A-Z, and a-z
#Upper Uppercase letters A-Z
#Lower Lowercase letters a-z
#Symbol Printing symbols in lower ASCII !-/, :-@, [-', and {-~" (Stops at Del)
#Digit Decimal digits 0-9
#Binary Binary digits 0 and 1
#Octal Octal digits 0-7
#Hex Hex digits A-F, a-f, and 0-9
#Quote Quote chars ' and "
#WSpace Whitespace chars, HTab, LF, CR, and Space
#Ctrl Control chars ASCII 0-31 and Del
#Print Printable ASCII !-~
#Punc Punctuation !,.:;? (excluding quotes)
#EPunc End-of-sentence punctuation !.?
#All All chars, ASCII + Unicode
#ASCII All ASCII chars \x01 - \xFF
#LoASCII Lower ASCII chars \x01 - \x7F
#HiASCII Higher ASCII chars \x80 - \xFF
#Unicode All Unicode chars \x80 - \xF4\x8F\xBF\xBF (Last Unicode is 0x10FFFF)