scScript Pat Package

The Pat package implements a comprehensive and powerful text matching system. The table below provides a cursory summary of its pattern definition and run functions. There is a great deal more to Pattern Matching, details along with many examples are given in the later Pattern Matching section.

Pat Definition:
Seq P = Pat.Seq(args...);	Returns pat to match concatenated sequence of args (conjunction). Args can be strings, other Pats, flags, and stash directives.
Alt P = Pat.Alt(args...);	Returns Pat to match one of the alternative args (disjunction). Defaults to selfish but can be made considerate.
Span P = Pat.Span(args...);	Returns pat to match span from StartArg all the way to to EndArg, ignoring everything in between. Defaults to minimizing and selfish.
Rep P = Pat.Rep(args...);	Returns pat to match the specified (n to m) repetitions of its arg(s). Defaults to minimizing but considerate.
Iter P = Pat.Iter(args...);	Returns special Rep Pat to match exactly n times, special form of Rep.
Char P = Pat.Char(args...);	Returns pat to match the given range of characters. Stores inclusion set of chars, many sets are predefined.
Any P = Pat.Any(args...);	Returns pat to match any character. More efficient version of Pat.Char(Pat.#All)!
Go P = Pat.Go(args...);	Returns pat to advance match process (goto) to new data location.
At P = Pat.At(args...);	Returns pat that succeeds only if at specified location!

Pat Match:
Match Pat.Match(args...)(args...);	Match the given pattern against data in scEmacs buffer. Takes many options and parameters, supports multiple callbacks. Provides rich feedback/debugging displays with compile flags.

Programmers generally view a text pattern search or transformation as a simple character-based linear (left to right) sequential process. Once a basic outline is designed, it is elaborated and flushed out, considering alternatives, exceptions, and other subtle points, much the same way all algorithms and code are conceived and created. But scripting languages typically require mapping from the simple sequential process to a complex representational grammar (RegEx, PEG, etc.) which the code then automatically interprets/executes at runtime.

Such grammars usually save keystrokes, but at the substantial cost of:

• Being generally write-only and very hard to read/maintain,
• Requiring knowledge of an arcane grammar and its cryptic symbols,
• Knowing how the program will interpret/execute the grammar,
• Awareness of non-obvious (hidden) tradeoffs in execution.

As a C programmer, I prefer to directly code a simple match, without resorting to magic runes and occult incantations. Ideally, the program understands and executes the pattern match instructions while taking care of low-level details. Scripting a pattern match should not be any harder than writing an arithmetic function.

To this end, scScript introduces a new, more procedural, Pattern (Pat) paradigm, drawing inspiration from SNOBOL as well as Parse Expression patterns. Simple Pat objects are first created, usually assigned to variables, then matched against text in editor buffers. This can be as simple as:

  PHello = Pat.Seq("Hello", " ", "world");
  Pat.Match(DataBuffN, PHello, Pat.#Once);
      

A minimal lexicon of Pat types, such as Seq, Alt, Rep, and Char, are predefined. But this simple operational model can easily accommodate more types as well as specialized and custom functionality to suit client applications. Importantly, text strings used to define Pat templates are just that, there are no additional in-band magic symbols and escape chars to shield them.

A Pat typically refers to strings, other Pats, and even callback script functions (closures) that should be called during a match. Once created, the Pat is simply stored in a variable, passed around, and used or re-used by many other Pats or matching functions.

Better yet, the matching process is re-entrant, allowing functional separation of different tasks. It is easy for a callback invoked from one Pat to grab data and match other Pats before returning results for the first Pat. For example, one can find paragraphs first, then check them for repeat words in a separate nested task, instead of coding one pattern to do everything at once!

Nevertheless, pattern matching is a complex process and requires a good deal of thought (and alas learning). The representational paradigm, no matter how elegant or simple, cannot eliminate this complexity. It is still all too easy to write a naive or over-simplified match and get surprised when it fails!