Natural Language Processing or NLP in short is the trending technology used by machines to understand the human’s natural language and process the text or request accordingly. It’s not that easy to train machines and make them to understand what is being requested and how to react to that communication.
In this article will start with a new series and learn a glimpse of Natural Language Processing, where to start with and how it can be implemented.
What is NLP?
Natural Language Processing, is a part of artificial intelligence that mainly focus on the interaction between computers and humans using the human understandable language. The objective of NLP is to read the unstructured data, analyze the data, extract insights, and summarize the content to make sense of the human languages in a manner that is valuable.
Most of the NLP techniques are achieved through on machine learning to get the meaningful output from extracted insights. In fact, a typical model using Natural Language Processing can be described as follows:
- Record the human speech as a audio file to process.
- Convert this captured audio to simple text.
- Processing of the text’s data by cleaning the data.
- Extracting the insights of the text given as a input to machine.
- The response from machine to the human for the provided input.
Applications of NLP:
Natural Language Processing is growing rapidly to help corporate world understand their business model, customer opinion about the product, improvement needed to make their business successful. It is a driving force behind many common applications that we use in our day-to-day life, few are listed below:
- Application such as Google translate, to translate text from one Language to another,
- Word processor such as MS-Word and Grammarly deploys the concept of NLP to check the grammatical errors in a text.
- Interactive Voice Response (IVR) applications implemented in call centers to automatically respond to requests made by users,
- Personal assistant such as "OK Google", "Siri", "Cortana", and "Alexa" uses NLP to act on the command or the request it receives.
- Product review and sentiment analysis on customer satisfaction on their service
How NLP works:
Unstructured human language is converted into a computer understandable vectorized format to extract the insights of text spoken by human. Sometimes it might be difficult for the machine to manipulate the meaning of sentence, which leads to the obscure output. NLP applies machine learning algorithms to identify the core topic and extract for the keywords. Entire process of NLP involves two main analytics with it. They are,
- Syntactical analytics
- Semantics analytics
Syntactical Processing:
Syntax processing is the way to understand the arrangement of word in a given sentence that makes a meaningful sense. With Syntactical processing, we in NLP, clean the input data for processing. This involves techniques such as,
- Tokenization
- Stop-words
- Stemming
- Lemmatization
- Understanding POS
Semantics Processing:
Semantics processing refers to the model or the machine algorithm built on top of the cleaned and trained data to understand the meaning of sentence. Commonly used technique in Semantic analytics are,
- Frequency Extraction
- Named Entity Recognition (NER)
- Word Sense
- Building Corpus
- Metrics Calculation
Chapters planning in upcoming post:
We are going to learn above techniques to process the text one by one in our upcoming post. I would recommend you to keep your machine installed with jupyter notebook and python to have a hands-on expenrience on upcoming chapters. To keep your setup ready, follow this link on "How to setup Spark in windows".
We use python libraries/packages such as "NLTK, Scapy, Sklearn, regex" to learn above techniques and finally build our model to classify the text, do sentiment analysis, etc.,
Have you worked on any NLP techniques in any of your projects? or you have any doubts or questions, share with me in the below comment box.
Happy learning!!!
We use python libraries/packages such as "NLTK, Scapy, Sklearn, regex" to learn above techniques and finally build our model to classify the text, do sentiment analysis, etc.,
Have you worked on any NLP techniques in any of your projects? or you have any doubts or questions, share with me in the below comment box.
Happy learning!!!
2 Comments
What are the cleaning and preprocessing steps require for NLP CHATBOT for user input?
ReplyDeleteMost common Data cleaning steps in any of the NLP application is,
Delete-> Tokenization - Depending on the type of data, choose word token or sentence token or even paragraph token. Mostly, splitting as individual words will be helpful
-> Case Normalization - Change all the text in your document into lower case, as machine treats Uppercase and Lowercase in a different manner.
-> Convert Non-alpha characters - Represent numbers in a full length words
-> Fix Basic Nuanced Errors - for example change, "what's" to "what is"; "email or e-mail" to "e mail"
-> Handling Stopwords and Punctuation - Remove punctuation and stopwords like the, a, is etc.,
Almost your data will be cleaned by the above process, you can also implement stemming/lemmetization for further processing.
We will look into these technique one by one in upcoming posts