DSL 101 – Domain Specific Languages

Today, I spent quite a while helping a friend to architect at high-level a DSL (Domain Specific Language).

Even though at the time he did not realize what he wanted to do was create a DSL. TO him all he needed was a way to do excel like formula expressions inside his code, but in a more English like syntax well as English as a chemical scientist can be I guess.

To this end, I thought it might be worthwhile explaining what a DSR is, since I will build a few of them for the project’s I mentioned that I am playing with for fun on this blog as I learn F#.

So just what is a DSL:

A domain-specific language is a programming/specification language customized to a particular problem domain.

This is quite an old concept as special purpose languages for modeling have always existed but recently have had a surge of emergence once again due to most newly created systems needing a more flexible way to extend logic used without the need of a programmer.

Creating a DSL and the tools required to support it can be worthwhile if the language allows a particular type of problem to be expressed more clearly than existing languages would allow. An example of this is the issue of applying simple logic used to route a document for approval in a document management system.

Another example of the new generation of DSL’s are products/tools like cucumber (from the Ruby world to do BDD), and in the F#/C# world we have tools that are DSL' based such as NaturalSpec (also a BDD tool this time for F#).

System Based Workflow Vs. Process Based Workflow…

There are two types of workflow that come to mind when people talk about workflow systems in software. Most of the time it seems that they get confused with each other since most people just use the term workflow to mean either of these types of workflow.

To clarify this, I decided to put down my views on what they are to me:

• System Based Workflow – A application level workflow system that executes processing logic in a sequence or based on a state in the system that has changed. A common Microsoft based workflow system of this type is Windows Workflow Foundation. (WFF)
• Process Based Workflow – A software level workflow or to be more accurate approval system. This type of workflow is used to approve changes in document management by routing update/changes/requests to other users in a system for approval or rejection or have the system automate the approval/rejection based on user defined logic. Document management and Content Management systems use this kind of workflow all of the time when it comes to publishing content/data. It is also the type of workflow used to implement 4-Eyes like maker/checker logic in software application.

Not sure if that helped, but there you have it.

About the Workflow system I am creating in this Blog…

Ok now it’s time to explain what I mean when I talk about creating a simple workflow system for this blog.

Without going into too much detail there are two main types of workflow system:

• Sequential Based Workflow – This is where Activities are executed one after another based on the results of a previous activities end state.
• State Base Workflow – This is where activities are executed based up a system state or event, they do not run in a logical sequence but more react to what happens inside a system.

The system I am creating for this blog will initially be a sequential based one, before i get to specifics it is better to detail the parts of a workflow system so here goes…

At a high level there are quite a few parts to a workflow these are listed below:

• Workflow system – The glue and process that enables all processing done in a workflow.
• Workflow Activity Map -  A collection of activities that has a natural start and end point with activities chained one after another in sequence. Workflow activity maps sometimes have embedded logic that helps the system select the next activity to process in a sequential workflow model. or just a collection of activities that have Zero or more dependencies in a state based workflow system).
• Workflow Activities – The actually activities that perform some process and have a return value or result. Activities also can point to the next logical activity.
• Workflow Workers – The heart of a workflow system is the worker. the worker is a token/object/item that contains context data related to what is being processed by the workflow activities in a workflow.
• Workflow States – only really useful in a state based workflow. This is a list of states that are available to be monitored by a state based workflow’s activities. (I will go into this more, when I create a simple state based workflow solution for this blog.)

A little more on Workflow Maps.

As stated above the workflow system I will create on this blog is a sequential based one that is a series of ACTIVITIES than are processed sequentially in a Workflow Map as shown below:

All sequential workflow systems have a start point where the workflow starts which basically just points to the first activity to process.

Once that activity has completed processing it will then move onto the next activity, Until it reaches the end of the workflow map.

Well I think that is enough for now, In a later post I wil explain how and where workflow workers come into play and then detail the types of activity that can be in a workflow system, and then detail which ones I plan to support in the system I am creating for this blog.

An issue of Context…

When creating a rule/logic/workflow engine there are various things that need to be considered in it’s architecture.

One of the most overlooked aspects of such systems is that of the CONTEXT in which a rule is run, and how it affects what a rule can do.

To elaborate a little more, rules normally know nothing of the environment that they run in, rules just execute logic based upon data that should be available when the rule is evaluated for its result. (a result is normally a logical true/false, but it can be other values and data types depending on its usage and context).

So what do I mean by Context, quite simple the context for an executing rule is the environment that it runs in.

A context normally defines what data is available to a rule for example:

• Global Values
• Results of nested logic that is used in the rule
• System data that is applicable to the rule being evaluated
• Functions or other rules that can be called by the rule

A rules engine when it is instantiated/run should go through the following processing steps:

• Setup context for the rule (normally defined by a flag or driven by a workflow system)
• Execute a rule or batch of rules in context
• Tear down context (sometimes a context is persisted depending on the type of rule logic and system it is in)

Later on, I will be creating a simple logic/rule engine that conforms to the above and deals with the issues in creating and synchronizing a context and its data.

I hope this has been at least a little helpful, soon the real fun and more technical stuff will be started.

Example Logic for a simple Rule Creation UI

Below is an example of a simple flow charts for  a couple of standard pieces of logic.(yes this rule structure will be in the API I am thinking of creating, and it is used in both a workflow API and also a simple rule engine/expert system.)

The format displayed below is common and is found in quite a few logic building systems, for example windows workflow foundation.

Copy from a single source field to a target field

Copy from two source fields and combine to a target field

 

I have not shown the exact logic/process of how this rule could be created/stored/implemented.

That can wait till I cover a simple architecture that can be used to create a basic rule engine and then produce a simple code based API that will execute simple rules like the one demonstrated.

One thing I will say, is the logic structure and all you see is meta-type and meta-data driven. the meta-type/data will then be converted into a format that can be later interpreted by the API to process the logic in question.

Open API structure requirements 101.

When creating an API for consumption by another system or systems, there are certain things I like API’s to do and also some assumptions that really should not be made. As I have done quite a few API of this nature I decided to document just a few things I try to do and also try to avoid doing.

As always, I will update this list with information based upon feedback should I get any ^^.

What an API should enable or be able to do:

• An API should be able to be plugged into any system and start to show it’s benefits
• An API should not conflict with an existing API or the system it is integrated in, if at all possible
• An API should document its REAL dependencies and requirements at a processing CPU/Memory level and also at database/network/disk space resource usage level.
• An API should also be able to produce Metrics on its operation and its environment
• An API should have Multi-threaded or Parallel processing where possible to enhance performance
• An API should be easily upgradable
• An API should be easily extended (usually via logic/rule engines, or plug-in architectures).
• An API should have logs/audits/performance-checks that can be disabled or enabled (when enabled performance will drop of course)
• An API should be documented with Unit tests and examples showing how it can be used.
• An API should where possible keep the same API signatures and objects backward compatible with previous versions of the API. (this is quite hard to do, but there are guidelines on depreciating old signatures with new ones, by introducing a new signature and flagging an old one as redundant and legacy, and then a release or two later removing the old signature)
• An API doe’s not need to reveal exactly how it does what it does in fine detail unless that was agreed in it’s design. (normally this is IP and NDA protected in most companies, although most of the time the technologies already exist in various forms already… very few new idea’s on data processing or system processing these days, just better tools that promote slightly different ways of doing things.)

What an API should never Assume:

1. It is responsible for exception management
   1. Error reporting, etc.
2. It will never have functional parts of its operation replaced
   1. Logging,
   2. error systems,
   3. UI aspects,
   4. Even processing logic (Ideally this should be done in such a way that it can be extended easily if at all possible, rule/logic engines allow this, and I will blog on rule/logic engine creation at a later date.)
3. It will never have its API syntax changed