How will you design a web service that enables other developers to create online chat applications?

Me: Repeating the problem statement to make sure i understand what is being asked correctly, we want to design a web service that enables other developers to create online chat applications.

Interviewer: yes

Me: i would like to ask a few clarifying questions before dicing into solutions

Interviewer: Ok

Me: what type of scale are we talking ehre? is this an internal to company type of product? Or are we allowing users outside the org to utilize this feature? what is the user base like

Interviewer: It is internal to company. Essentially for other platforms to use this service that is already existing for chat purposes

Me: ok. What is the expected availability? Is this used globally or local to one place? 

Interviewer: this has a global usage

Me: What type of read queries per second are we looking at? 

Interviewer: 50,000 read queries per second

Me: How many queries per second does db receive?

Interviewer: around 500,000 queries per second

Me: For the sake of thisexercise, is it safe to assume that chat application is web based ?

interviewer: yes

Me: I would like to talk about the functionality that we want with this service

the most common chat application functionalities are:

- One to one conversation

- group conversations

- Send media like pictures or audio

- Receive delivered and read receipts

Is there any functionality you would like me to focus on?

Interviewer: No

Me: 

Since this is a web chat application, I would like to use REST APIs to perform the functional requests

One to one conversation and Group ocnversations : /api/chatSession

- GET conversation based on UserID, timestamp, chatID

- POST conversation based on UserID, timestamp, chatID

Send Media: /api/mediaContent

- GET media content based on UserID, SessionID, Timestamp, MediaID

POST media content based on UserID, SessionID, Timestamp, MediaID

Receipts like delivered and read: /api/messageStatus

GET message status based on UserID, Timestamp,SessionID, MessageID, Response

POST message to UserID,MessageID

Talking further, since this is light weight messaging application with read heavy model. And since the messages are not relational in nature, and can be delivered anytime (not necessary to maintain consistency), I would suggest going for a noSQL database. I would also suggest going ahead with Queue mechanisms like Kafka that can maintain publisher subscriber model to keep the status updated. In order to make sure huge media content is not being pushed through the webservice, I would suggest keeping it a separate call and to maintain a media heavy database that can store and retreive media.

How will you design a web service that enables other developers to create online chat applications?

Clarifying questions

1.        By Web Service I assume you mean a service that a site owner can insert onto their site that will enable two say chat between a web user and a rep at the company. Yes

2.       What kind of scale should the service be able to implement – I assume since this is MS and you mention developers – this is going to need to work at scale. 

3.       When you say developers – I assume that we will need levels of sophistication that allows this to work on an enterprise level.  – seems like the same question as above. 

4.       Chat applications – I assume two way chat, plus chatbot etc. – That would be nice

Goals

Develop a set of web services that a developer can use to create online chat.  Make sure the web services are scalable and ready for enterprise. 

Functionality outline

1.       Render chat on website

At CNET we rendered rich content onto partners pages.  We did this by installing a JS on their page.  They passed some parameters in their html.  The users browser would get the html and run our JS which would make a call to our servers.  We would return content.  The advantages of having a JS is that we got to handle a lot of the details from page layout, formatting etc.  I would imagine the same thing here if we give the developers a JS with some parameters then we can return the initial chat box to page and when we need to update functionality we aren’t reliant on the 3rd party developers to make those changes.  A few words of caution speed, SEO, vulnerability.  These are the 3 objections we got over time with asking partners to use a JS. 

·       Vulnerability was the 1st objection over time (last few years) this has diminished. 

·       SEO was a second big objection, we were rendering a full page content, this is an chat bot – not likely relevant

·       Speed – this was the objection I heard most recently – we will need to make sure that the chat bot returns to the users page very quickly.  At CNET we did this by having requests load balanced and keeping the content for sites up in edge cacheing.  I would imagine we would do something similar direct traffic to a geographically close system and employ cache to make sure that the item return quickly.  At CNET we had a unique customer id and version id as part of the url parameters in the JS.  In this system something similar

o   User denotes which client we are working with

o   Version denotes which config of the chat return.  You could imagine having different versions for different products, logged in users, anonymous users etc. 

o   Since this combination would be unique you could do very fast lookup of their code in cache. 

·       Finally it is worth nothing that some partners won’t want the JS.  At CNET our policy was lead with the JS it made everyone’s life easier but we will likely have to offer API calls that do much of the same functionality. 

o   One of the challenges when we stepped out of the JS mode is that reporting became more challenging.  With the JS we knew how many time we served the content, and then we knew interactions.  If they handle everything on chat setup on their side and only call the service in the next step then we may lose visibility at the very top of the funnel. 

2.       Initiate chat by user

a.       The user types info into the chat bot box this likely includes

                                                               i.      PII – we will need to treat this encryption and think about privacy laws in the US and rest of World

·       Name

·       Email,

·       Call back phone number

·       if they are logged in we may get session ID to help the site tie info back to their systems

·       customer number

                                                             ii.      They may select from a topic drop down

                                                           iii.      They will enter some text – the chat will likely prompt them to describe the problem

                                                            iv.      We will also get system info like IP, timestamp, and session (this happens automatically without user interjection).   

                                                             v.      Also we are storing enough info to link it back to the customer ID and version of the chat. 

b.       From a system need set this is essentially posting to a form the posted information is passed to our servers in an API maybe JSON format (but the engineers will decide what is best) 

c.       The developer may want to store this info in a 3rd party system (CRM).  I don’t think they will want to get these pings in real time (I could be wrong) but it’s more likely we will need a set of pull API latter on that allow the developer to batch up all the chats latter in the cycle. 

d.       The two most important steps are recording the data and passing the chat session to the queue. 

e.       From a system architecture – the posted data goes through the load balancer to the web server where the data is passed to storage and to the queue management. 

3.       user chat queue management

a.       each chat will pass to the queue.  The queue likely doesn’t want to store all the chat info just enough to identify the user and their placement in the queue. 

                                                               i.      Session ID

                                                             ii.      Queue order. 

b.       From a queue management standpoint, we would use classic queue management such as first in first out.  However, we may have enough info to help prioritize the queue a couple of approaches

                                                               i.      We could build a rules based engine that prioritized based on subject picked, who the user is and how long they have been in the queue. 

·       It is really important to note that in a prioritized queue time spent in queue will go up so we will need a function to look at those in queue and change their prioritization as time grows.  Quick example multiple people enter the queue – if there is no agent to take the 2nd user its possible the 3rd user will come in and bump that user.  If this happen over and over again then current.time – time(1) will grow and we need a function to weight that or a user could potentially never get serviced. 

Priority issue, priority customer, time 0

Non priority issue, non priority customer, time 1

Priority issue, priority customer, time 3

Priority issue, priority customer, time 4

                                                             ii.      We could also use machine learning to look a combination of factors as well as analyze words in the text to try to define out prioritization models. 

4.       Agent / user chats

a.       When an agent is available they will be ready for the next user in queue

b.       Developers will need a process to pop the next user from queue and the queue management will update the (stack, max heap) so that the next user is on top. 

c.       The agent and user will chat back and forth posting messages.  To the web server which will be deliver the chats back and forth to the users web browser. 

d.       At the end we should also ask for some satisfaction metrics from the user

e.       The agent may be making changes to the other systems and or taking notes – the developer will need a set of sessionid API’s from their own session to the chat session to tie everything together. 

f.        The developer will need a set of post API to send the message to the webserver and the clients will need a set of functionality that updates the chat messages in their browser / app From a system design standpoint it would be nice if the user and agent were load balanced onto proximate servers but there is a natural delay as user and agent type back and forth and this may not be necessary. 

g.       From a system design I think this is mostly passing though load balancing and the web server front end.  And that the chat is being stored – my guess is that a long from DB like NoSQL would be the choice but again as PM I help define functionality, they make the system work.  It will be important to store all chat message and the time of each chat.  Just like we stored user info we will want to know the agent and we will want this entire record linked to customer, chat version, chat session. 

5.       All chat sessions are stored

a.       I think I covered this as I went along

6.       Analytics

a.       I’m  going to generally break analytics into two main branches ability to pull out data to 3rd party system (CRM) and actual metrics

                                                               i.      The developer will want methods to pull all the chat data out of the MS system and then push that data into their CRMs

·       Microsoft may make integrations to common CRM’s and other integrated systems.  Here instead of the developer pulling the data they developer may define the fields that are used and call a process that writes to the 3rd party app, this may also be scheduled. 

b.       Last there is tressure trove of BI analytics which all need to be processed, mined and stored we can design all those systems latter but some of the data includes

                                                               i.      # of problems

                                                             ii.      # of problems by customer / user

                                                           iii.      # of problems by type of issue

                                                            iv.      Time and seasonality

                                                             v.      Queue times

                                                            vi.      # of problems by Agents

                                                          vii.      Time to resolve by agent

                                                        viii.      Satisfaction by the user

c.       The developer will want a way to pull all this data out to their own warehouse, crm, etc.  They will need s series of API pulls to get this data likely in some from of batch mode.  Again this is something that we can integrate into existing BI systems. 

7.       Last but not least we have talked a lot about API and processes and systems but for SMB clients we should wrap this all in a web portal and make it product not just a set of web services calls.  By doing the set of calls we make the system extensible for the biggest enterprise customers but by offering a portal that agents can log into.  Or a JS that can be installed on the agent back office and portal in the middle for management we likely have full blown product.