Skip to main content

Complete On-Premise and Fully Customisable Chat Bot - Part 4 - Integrating the Natural Language Processor NLP

Welcome back Folks !!

In our part 3 we have seen how to communicate to the bot that we have built. Each and every people would talk the same thing in the different format and on his style. For each response if we start writing the code, Imagine how hard its gonna be.



What can we do for that ?

Natural Language Processor makes it easy to do. NLP is a component that identifies the user conversation and identifies the meaningful context from that.

What NLP is available ?

There are few NLP services like,
1. Dialog flow
2. Luie
3. IBM Voice etc.

But all these are the cloud services and we need to go internet for this. As we speaking about the on premise chat bot we will go for the RASA solution.

What is RASA ?

RASA is a Chatbot server and RASA NLU is the NLP server that comes handy and is built on the python tech stack. Its is a open source and backed by a strong community.

Lets get started with RASA ...

1. Installing the python environment : 

Since we know that the RASA is built on the python tech stack, we need to install the python. To install the python follow the below instructions,

Step 1: Open the respective page for the Operating system of your interest and follow the instruction and install the anaconda with python 3.x (contains most of the packages needed for the RASA)

  • Windows - https://conda.io/docs/user-guide/install/windows.html
  • MacOs - https://conda.io/docs/user-guide/install/macos.html
  • Linux - https://conda.io/docs/user-guide/install/linux.html
Step 2 : Test you Anaconda installation by using "conda list" in the command window.
Step 3 : Test your python using "python -V" command.
Step 4 : let's start installing the RASA NLU server. Open the command prompt and key in the following command 
                 
Step 5 :  RASA NLU used the SpaCy, MITE , SkLearn combinations to server the natural language processing. Most of the cases the combination of the SpaCy and Sklearn will work out. So to install the spacy.
                


Step 6 : Create a folder called "NLU" inside your bot project. < Revisit this post to see the directory structure >

Step 7: Our NLP processor is a model that learns the experiences from the past data and improves on the accuracy. So we need to provide the data that the user might have spoken for a particular intent of interest. So create the folder inside the NLU folder as data.

Step 8 : Following are the contents of the data folder, rasa_data.json file. The main components of the data file are as follows
Below one is the entire file created for a chinese restaurant use case
If you feel this process of editing the intents and entities are difficult. There is a handy tool at

  • online version : https://rasahq.github.io/rasa-nlu-trainer/
  • offline tool : https://github.com/RasaHQ/rasa-nlu-trainer
Just upload the file to the online / Offline version and add the intents and entities as shown in the picture below




Step 9: Now your are in the data directory of the RASA project. Come back a step (means to the root directory). Create the .json file and place the below contents.
Step 10:  Now it time to train the model with the data we created. Open the command window in the current directory and run this command "python -m rasa_nlu.train -c config_spacy.json"

Step 11: Once if the command completes successfully, you can see the projects folder getting created. At this point, invoke another command to run the HTTP server "python -m rasa_nlu.server -c config_spacy.json --server_model_dirs=<newly Created folder path>"

Tadaaaa !! the server is up and running. Using the postman hit the below url and see the magic.
http://localhost:3000/parse?q="Hello this is the user response".

 The Nlu server should understand this and give you back the JSON object that have the predicted Intent Name. Use this in the chatbot to respond back to the user.


Comments

  1. Anuj23 January 2019 at 02:52

    This is really awesome. An excellent step by step guide. I am also looking to use the BotKit for integrating it with Microsoft Teams. Do you have something (blog/github sample) that i can follow?

    ReplyDelete
    Replies
    1. Pranav Kumar14 February 2019 at 12:17

      Thanks Anuj ! Right now I don't have blog for integrating the bot with Microsoft Teams ! I can take this as a topic to help on others. Integrating with Microsoft Teams is a good idea.

      Delete
  2. Unknown11 February 2019 at 01:38

    Hi Pranav
    That's a very nice piece of article for beginners like us, thanks for that.

    I was able to setup both Bot and NLU server, but the article I think did not cover the integration path between them. Please help us few steps how to integrate

    Thanks

    ReplyDelete
    Replies
    1. Pranav Kumar14 February 2019 at 12:19

      Thank you ! Just noted the missing of linking between the two. I will update this with RASA and BOTKIT linking

      Delete
  3. WAQAR SHAIKH19 March 2019 at 01:12

    HELPFUL SITE.

    ReplyDelete
  4. WAKAR KHAN19 March 2019 at 01:14

    NICE

    ReplyDelete
  5. spaCy21 August 2019 at 02:47

    You have written that amazingly. One thing i want to know is how to embed code in blog posts like you have done here?

    ReplyDelete
    Replies
    1. Pranav Kumar5 October 2019 at 14:00

      Hi SpaCy with the help of git gist, we can embed the snippet in blogger

      Delete

Post a Comment

Popular posts from this blog

How to access the each view of item by position in Recycler View in android ?

How to access the each view of item by position in Recycler View in android ? There are some methods to access the view of each item view to change or update the view during the run time. To access the view of the item view , consider the Recycler view as below, RecyclerView mainRecyclerView = (RecyclerView)view.findViewById(R.id.main_recycler_view); RecyclerAdapter adapter = new RecyclerAdapter(mContext); mainRecyclerView.setAdapter(adapter); main.setLayoutManager(new LinearLayoutManager(mContext));  To access the itemView of each position the following functions can be used,  1. View itemView = mainRecyclerView.getChildAt(positionOfItem);  2. View itemView = mainRecyclerView.findViewHolderForAdapterPosition(Position).itemView;  3. View itemView = mainRecyclerView.findViewHolderForPosition(Position).itemView;  4. Long itemId = mainRecyclerView.getAdapter().getItemId(position);       View itemView = mainRecyclerView....
3 comments
Read more

How Machine Learning Works ? - Data Rules the Kingdom

W elcome Back Readers !, For those who have not read the previous post on How Machine Learning works ? Please do read previous one before continuing this so that it would be more easy to understand. You can find the previous post here How Machine Learning Works ? - Mathematics Every Where . For other who are ready to join with me in this journey to understand the Machine Learning, Thank you for being with this post. Till now we are discussing on the the title "How Machine Learning works? " and to understand that we have seen in the first post that there exist the patterns or trends in the data and we try to find that in terms of a mathematical equation called a model and try to fit to the new data that we have not came across for predicting the values. Wow may be we got the entire process complete with this. But there are few thinks that needs to be cared while training the model or creating the model or while processing the data for the model creation. ...
3 comments
Read more

A.P.I call or async await not working in Array forEach ?

H ello Readers, Welcome back. You would have wondered why does forEach function on the array does not wait for any asynchronous function even if we provide the async await in the node js. If you are the person, wondered about this ever, this post is right for you. Non working example : Lets consider the below snippet, this will not wait for the asynchronous process to wait. I am making a setTimeout to mock the API call async process. This will result the count as, Output : count = 0 OMG !! Why it doesn't work ? Answer probably might be lying inside the Array prototype from JavaScript. Lets take a look inside the "Array.prototype.forEach" function. From the snippet its clear that, the for loop in which they call the callback function does not wait for the asynchronous process to run in the callback. So this forEach is not build for asynchronous process itself. So can't I use forEach any more for running a asynchronous function ? Answer g...
Post a Comment
Read more