Scanning and Parsing…

Over the past 4 years I have been looking at interpreter and compiler creation specifically in the area of Scanning and Parsing.

One of the designs architectures that I designed some years ago was based on parallel processing of a stream of data (a file, and communications port/channel, etc.).

It is just this design that I aim to bring to life in the API’s I am creating over the next god knows how many months.

The concept is simple, but like many concepts there is a lot more to it that the text here implies; However, I do believe I have solved most of the architectural and performance based issues. This is due to both some rather large processing advances and superior development tools now on the market.

the concept itself is based on a pointer based parsing system, that uses various conditional logic to step forwards, backwards and indeed around various parts of a stream as it is being processed.

Traditionally this kind of Scanning/Parsing system is notoriously slow. but by looking at certain interpreter/compiler creation and performance optimization tricks, I think it may work at an acceptable performance level for real world systems… Guess I will find out in time.