Building a Flexible SMS Response System

Building SMS response system was my long pending dream! A few years back, I worked with SMS Service provider. Back then, the rates were high and the interfaces were not so easy.  But today, most SMS services offer cheap rates and very intuitive and easy to use, well documented APIs. Twillio is my choice for today. Let’s begin with a problem that’s being addressed here:

Problem:

My service needs to receive an SMS message from customer and process it and respond with an SMS message. The ask is very simple but if you can imagine, processing these SMS messages requires a flexible and configurable architecture, instead of a lots of if..else statements. I am not tying to do text analytics here, I expect my users to send specific keywords in SMS. For example, if they need help, they may send “help me”, “options”, or even “*”! If they need a list of nearby bars, we expect them to send “bars near me” or “nearby bars”….. you got the idea!

My Design Goals:

1. Ability to process incoming message and determine a response

2. The logic should be flexible and configurable. For instance, if I expect my users to send “help me” in order to receive a help message from us, it should be easy and configurable so that I can provided additional phrases that users can send without changing the code.

3. Break the problems into smaller chunks and make sure each piece of code has single responsibility

Possible Solutions

Given the problem and the design goals, I can think of two possible solutions.

Alternate Solution 1: Have separate classes created and each understands only a specific incoming message and processes it. Consider each class as a node which has ability to understand the request and provide response. If you like, consider it as an HTTP Module. Then build some sort of framework which passes the incoming message to each node in sequential fashion until it finds a node which can process the message.

Alternate Solution 2: Leverage a Service Bus architecture. A message can be put in a queue and then you can one subscriber which can process the message. And yes, the subscriber can pull the message from Queue and use the Alternate Solution 1 (above) to process the message. You may also have multiple subscribers and use the Topics mechanism of Azure Service Bus.

DISCLAIMER: If you are looking for a most reliable and scalable way to process the SMS messages, I recommend Alternate Solution 2 but for the low volume and fairly reliable solution, Solution 1 is not a bad choice either. While solution 1 is not the best, I’ll talk about that in this post, leaving solution 2 for another blog post. I also do not claim that these are the only two solutions, there could be many ways to solve such problems at hand. Sending and receiving SMS using Twilio is very simple. I’ll not talk about the mechanics of sending and receiving SMS messages because that depends from one provider to another.

So let’s take a look how does it (Alternate Solution 1) look conceptually:

And let’s zoon in a little bit into the SMS Processing Pipeline component:

The SMS Node processor in our case is a singleton class which initializes (and caches in its on state – it’s a Singleton, after all) the pipeline (series of processing nodes). It also has a method to receive a message and send response by routing the message through processing node.

A processing node implements a contract

   1:  interface ISmsProcessorNode

   2:      {

   3:          void ProcessMessage(SmsInfo message, SmsProcessingEventProperties eventProps);

   4:      }

The message represents everything that a typical incoming SMS message may include. For example, Twilio passes you following information along with some other information which we will not care:

   1:   public class SmsInfo

   2:      {

   3:          public string Message { get; set; }

   4:          public string ReceivedFrom { get; set; }

   5:          public string FromCity { get; set; }

   6:          public string FromCountry { get; set; }

   7:          public string FromState { get; set; }

   8:          public string FromZip { get; set; }

   9:      }

SmsProcessingEventProperties serves as a mechanism to receive configuration from the Node Processor and also as a mechanism to send response/status back to the node processor.

   1:  public enum SmsProcessingEventStatus

   2:      {

   3:          Processed = 0,

   4:          Skipped = 1, 

   5:          Warning =2,

   6:          Error = 3

   7:   

   8:      }

   9:      public class SmsProcessingEventProperties

  10:      {

  11:          public SmsProcessingEventStatus Status { get; set; }

  12:          public string ErrorMessage { get; set; }

  13:          public XElement Configuration { get; set; }

  14:          public string Response { get; set; }

  15:      }

Finally, here is the event processor looks like:

   1:   public class XmlNodePipelineProcessor

   2:      {

   3:          private static volatile XmlNodePipelineProcessor instance;

   4:          private static object syncRoot = new Object();

   5:   

   6:          List<SmsProcessingNode> _nodes = new List<SmsProcessingNode>();

   7:   

   8:          private XmlNodePipelineProcessor() { }

   9:   

  10:          public static XmlNodePipelineProcessor Instance

  11:          {

  12:              get

  13:              {

  14:                  

  15:                  if (instance == null)

  16:                  {

  17:                      lock (syncRoot)

  18:                      {

  19:                          var processorFile = HttpContext.Current.Server.MapPath("~/App_Data/SmsNodeProcessors.xml");

  20:                          if (instance == null)

  21:                          {

  22:                              instance = new XmlNodePipelineProcessor();

  23:                              instance.InitializeFromXml(processorFile);

  24:   

  25:                          }

  26:   

  27:                      }

  28:                  }

  29:   

  30:                  return instance;

  31:   

  32:              }

  33:                  

  34:              

  35:          }

  36:   

  37:          #region Public Members

  38:          public IEnumerable<SmsProcessingNode> GetProcessingNodes()

  39:          {

  40:              return _nodes;

  41:          }

  42:          public string GetResponseFor(SmsInfo incomingMessage)

  43:          {

  44:              string response = string.Empty;

  45:   

  46:              foreach (SmsProcessingNode node in _nodes)

  47:              {

  48:                  try

  49:                  {

  50:                      Assembly assembly = Assembly.Load(node.Assembly);

  51:                      Object classInstance = assembly.CreateInstance(node.ClassName);

  52:                      ISmsProcessorNode processingNode = classInstance as ISmsProcessorNode;

  53:                      if (processingNode == null)

  54:                      {

  55:                          Trace.WriteLine(string.Format("Assembly:{0}, Class{1} is not an implementation of {2}.  Node-{3} will be ignored.", assembly, node.ClassName, typeof(ISmsProcessorNode).FullName, node.Name));

  56:                          continue;

  57:                      }

  58:                      SmsProcessingEventProperties eventProps = new SmsProcessingEventProperties()

  59:                      {

  60:                          Configuration = node.Configuration,

  61:                          ErrorMessage = string.Empty,

  62:                          Status = SmsProcessingEventStatus.Skipped

  63:                      };

  64:                      processingNode.ProcessMessage(incomingMessage, eventProps);

  65:   

  66:                      // Check the modified eventProps and take actions

  67:                      if (eventProps.Status == SmsProcessingEventStatus.Skipped)

  68:                          continue;

  69:   

  70:                      if (eventProps.Status == SmsProcessingEventStatus.Error)

  71:                      {

  72:                          Trace.WriteLine(string.Format("Incoming Msg:{0}, Node:{1}, Error:{2}", incomingMessage.Message, node.Name, eventProps.ErrorMessage));

  73:                          continue;

  74:                      }

  75:                      if (eventProps.Status == SmsProcessingEventStatus.Processed && node.ContinueAfterProcessing)

  76:                      {

  77:                          continue;

  78:                      }

  79:                      if (eventProps.Status == SmsProcessingEventStatus.Processed && !string.IsNullOrEmpty(eventProps.Response))

  80:                      {

  81:                          return eventProps.Response;

  82:                      }

  83:                      

  84:                  }

  85:                  catch (Exception ex)

  86:                  {

  87:   

  88:                      Trace.WriteLine(string.Format( "Node:{0} failed to process message:'{1}', error message: {2}", node.Name, incomingMessage.Message, ex.Message));

  89:                  }

  90:                  

  91:              }

  92:   

  93:              return string.Format( "Sorry, we could not process your message: {0}", incomingMessage.Message);

  94:          }

  95:          #endregion

  96:   

  97:          private void InitializeFromXml(string fileName)

  98:          {

  99:   

 100:   

 101:              XmlDocument doc = new XmlDocument();

 102:              try

 103:              {

 104:                  doc.Load(fileName);

 105:              }

 106:              catch (XmlException ex)

 107:              {

 108:                  Trace.Write(ex);

 109:   

 110:              }

 111:   

 112:              if (doc.DocumentElement.LocalName != "NodeProcessors")

 113:              {

 114:                  Trace.WriteLine("File does not contain NodeProcessors");

 115:                  return;

 116:              }

 117:   

 118:   

 119:              XmlNodeReader nodeReader = new XmlNodeReader(doc.DocumentElement);

 120:              XDocument xDoc = XDocument.Load(nodeReader);

 121:   

 122:              Dictionary<string, string> assemblyRefs = new Dictionary<string, string>();

 123:              foreach (var item in xDoc.Descendants("Assembly"))

 124:              {

 125:                  //TODO: validate that item has Name and Value attributes

 126:                  assemblyRefs.Add(item.Attribute("Name").Value, item.Attribute("Value").Value);

 127:              }

 128:   

 129:              foreach (var item in xDoc.Descendants("Node"))

 130:              {

 131:   

 132:   

 133:                  if (!item.HasAttributes

 134:                      || item.Attribute("Name") == null

 135:                      || string.IsNullOrEmpty(item.Attribute("Name").Value)

 136:                      || item.Attribute("AssemblyRef") == null

 137:                      || string.IsNullOrEmpty(item.Attribute("AssemblyRef").Value)

 138:                      || !assemblyRefs.ContainsKey(item.Attribute("AssemblyRef").Value)

 139:                      || item.Attribute("Class") == null

 140:                      || string.IsNullOrEmpty(item.Attribute("Class").Value))

 141:                  {

 142:   

 143:   

 144:                      continue;

 145:                  }

 146:                  // All data seems to be okay, process it

 147:                  SmsProcessingNode newNode = new SmsProcessingNode()

 148:                  {

 149:                      Assembly = assemblyRefs[item.Attribute("AssemblyRef").Value.Trim()],

 150:                      ClassName = item.Attribute("Class").Value.Trim(),

 151:                      Name = item.Attribute("Name").Value,

 152:                      StopOnError = item.GetAttributeAsBool("StopOnError", false),

 153:                      ContinueAfterProcessing = item.GetAttributeAsBool("ContinueAfterProcessing", false),

 154:                      Configuration = item.Element("Configuration") 

 155:                  };

 156:   

 157:                  _nodes.Add(newNode);

 158:              }

 159:   

 160:   

 161:             

 162:          }

 163:      }

A node representation is here:

   1:  public class SmsProcessingNode

   2:      {

   3:          public string Name { get; set; }

   4:          public string Assembly { get; set; }

   5:          public string ClassName { get; set; }

   6:   

   7:          private bool _continueAfterProcessing = false;

   8:   

   9:          public bool ContinueAfterProcessing

  10:          {

  11:              get { return _continueAfterProcessing; }

  12:              set { _continueAfterProcessing = value; }

  13:          }

  14:   

  15:          private bool _stopOnError;

  16:   

  17:          public bool StopOnError

  18:          {

  19:              get { return _stopOnError; }

  20:              set { _stopOnError = value; }

  21:          }

  22:   

  23:          public XElement Configuration { get; set; }

  24:         

  25:      }

The XML based Node Configuration will look something like this:

<?xml version="1.0" encoding="utf-8" ?>

<NodeProcessors>

  <References>

    <Assemblies>

      <Assembly Name="PhoneService" Value="FULL NAME OF YOUR ASSEMBLY"></Assembly>

    </Assemblies>

  </References>

    <Nodes>

      <Node Name="PersistMessageProcessor" AssemblyRef="PhoneService" Class="FULL NAME OF CLASS TO SAVE MESSAGE IN DATABASE" ContinueAfterProcessing="true" StopOnError="false">

        <Configuration>

        </Configuration>

      </Node>

      <Node Name="HelpTextProcessor" AssemblyRef="PhoneService" Class="FULL NAME OF A CLASS TO SEND A RESPONSE WHEN  INCOMING MESSAGE IS TO SEEK HELP" ContinueAfterProcessing="false" StopOnError="false">

        <Configuration>

          <CatchPhrases>

            <Phrase>help</Phrase>

            <Phrase>help me</Phrase>

            <Phrase>help please</Phrase>

            <Phrase>what are my options?</Phrase>

            <Phrase>options</Phrase>

          </CatchPhrases>

          <Response>

            YOU NEED HELP! HERE ARE YOUR OPTIONS:...

          </Response>

        </Configuration>

      </Node>

    </Nodes>

</NodeProcessors>

As you can see, there are two examples that I have configured. The intention of the first node is to save the message in the database. The intention of 2nd node is to figure out if user is looking for help and if so, send a help response which is also configurable. Note that Configuration element in above XML has particular schema, the framework (node processor) will pass the entire configuration and it will be up to the node implementation to figure out how to use it.

So how does my HelpTextProcessor look like? well, i think that’s a good example which complete the loop in understanding how exactly it works. And here it is:

 public void ProcessMessage(SmsInfo message, Models.SmsProcessingEventProperties eventProps)

        {

            if (eventProps.Configuration == null)

            {

                eventProps.Status = Models.SmsProcessingEventStatus.Skipped;

            }

            try

            {

                foreach (var phrase in eventProps.Configuration.Descendants("Phrase"))

                {

                    if (phrase.Value.ToLower() == message.Message.ToLower().Trim())

                    {

                        eventProps.Status = Models.SmsProcessingEventStatus.Processed;

                        eventProps.Response = eventProps.Configuration.Descendants("Response").FirstOrDefault().Value.Trim();

                    }

                }

            }

            catch (Exception ex)

            {

                eventProps.Status = Models.SmsProcessingEventStatus.Error;

                eventProps.ErrorMessage = ex.Message;

            }