Open Source Dialog System Natural Language Processing Engine?

Question

Introduction and goal

I'm interested to develop (possibly in Ruby language) Conversational Commerce Chat Apps [1].

A Dialog System Engine ?

Specifically I want to implement a dialog system engine, a natural language server middleware to be used by business applications to achive specific deterministic business actions (online shopping: submit orders, booking: submit a reservation, supply any service) from bots running inside chat instant messaging platforms as http://www.Telegram.org (through Bot APIs, see my project [2][7]) or team-working chat as http://www.slack.com, or whatever IM system supply bot APis (maybe soon: Messenger/Whatsapp, Google Chat, etc. etc. ).

API.ai / WIT.ai approach

I'm studying now these two similar dialog system cloud / proprietary engines: http://api.ai and http://wit.ai (see interesting introductive articles by Patrick Cantazariti: [4][5][6] and my notes about excellent API.ai: [9][10]).

These platforms have a pretty similar approach (that someone call euristic/pragmatic systems, or dialog scripts systems), where a conversational application (called agent in api.ai terminology, or app in wit.ai case) is modeled as a set of intents acting on entities in certain domains, letting you to collect actionable data and so implementing a conversation that collect data to action what I call a deterministic goal (e.g. submitting an ecommerce order, reserve a room on a hotel, etc. etc.).

I feel that above approach to model a conversation (dialog modeled as domains/entities/intents/contexts/actionable data) is a good compromise between:

• a vast/complex "Artificial Intelligence" dialog engine that have to understand "any" natural language user sentence (e.g. the fact-totum user "assistant" approach a la Facebook "M", Apple Siri", etc.),

• and an hard-coded approach, as I done with my modest experimental project "dialogs" [3], where I modeled a conversation as a composition of dialogs, where each dialog is a sort of finite state machine.

Some notes/issues

1. Data privacy

   API.ai and WIT.ai have a great web interface to build-up easyly dialogs but i see a big problem: to model by example an ecommerce workflow, there is a need to define users entities where the personal data of the application and the users have to be "ingestioned"/stored on the proprietary cloud system (e.g. API.ai); that's a data privacy problem; in my opinion is bad that final business application and personal user data are maintained on a single (proprietary) cloud system.

2. Software Opening

   • A possible desiderata is to have a open-source software, to be instanstiated on private servers to maintains data private. All in all that "Dialog System" platform have to be as a database (DBMS): let's consider Postgresql/REDIS as great example of open software service platform paradigma: each developer install his own database server instance where store his private data.

   • Having something like API.ai as open-software could help also to SHARE dialog agent interchange format dialog scripts, maybe achieving a common standard (text files easy to download/upload).

The Main Question

1. There is something similar to API.ai but open-source ? If not:
   
2. There is any interest to build up now a community to create similar project ?

Any suggestion/comment/answer welcome.

giorgio

Links:

[1] my microblog
[2] BOTServer on github
[3] ((dialogs)) on github
[4] Five Simple Ways to Build Artificial Intelligence in 2016
[5] How to Build Your Own AI Assistant Using Api.ai
[6] Customizing Your Api.ai Assistant with Intent and Context
[7] Telegram Bot API
[8] Welcome to Slack API
[9] What are contexts and how are they used?
[10] API.ai vs Sirius ?

Answer 1

here are a couple resources I have found, that provide an open source "dialog system".

1. https://github.com/superscriptjs/superscript (Javascript) I just ran this yesterday locally, with the telnet interface for rules it looks interesting. There is an example using websockets; but I didn't get that working as of yet.

2. https://github.com/plison/opendial (Java) I just discovered this and have not tried it.

I am also interested in this. This area of develop "dialog systems" seems to be changing fast and getting a lot of resources. I do think the more intuitive setup and configuration like wi.ai offers (I have not yet used api.ai) is a needed piece that is lacking in open source. I'd like to work on something (assuming we all can find the time); I'd like to see maybe something built with NodeRed for an interface to then be able build the "dialog system"--but that is just an initial idea.

I am still researching this area--so there may be more and better options also.

Best regards!