Natural Language Processing: How it Works
Natural language processing is a part of the AI frameworks. So being a part of the AI frameworks, it’s a tool that we use that is part of AI.
Natural language processing is taking human spoken words and converting them into something a computer can use. NLP is the abbreviation for natural language processing.
When we’re working with NLP models or in general, we’re trying to take long, extensive lists of words. Words that a user may type into an email or say over the phone. And then we’re turning that into something that a computer can recognize.
In that process, what we’re doing is what we’d call a tokenization process. So we’ll take all the words that are part of an email, for example, and we’ll break it down into the important words. We’ll remove things like if, they’re, a, an, some of these extra words that don’t provide as much value to the machine.
We’ll then take that tokenized version of the NLP and we’ll run that through. Then consider what those tokenized words mean. After, it comes back with either classification or some other step after the NLP.
NLP and Tokenization
The NLP step is about extracting the important information out of an email. And then, normalizing the differences in words. This way we have something that the computer can then use, or the AI tools can then use. Then, going onto another step and take action on that NLP.
So natural language processing works by tokenizing a list of words or an email or voice texts that we speak. And it takes all those words and breaks them down into a list of the important words. But what it also does is normalize those words. For example, it’ll take out some of the language idioms that we may use. Or even the different types of words that we may use to describe the same thing.
So, for example, shipper, shipping, shipment, all these would kind of turn into the same type of a word. So, that when we do bring this tokenization, we’re not dealing with a million words. We’re dealing with a much smaller subset of words that we can work with.
Another example would be plurals. We’ll remove all the plurals, verbs and action. Then, normalize them into things that are a simple verb. This way we know that the computer knows you want to ship or move something. It might be so that we can bring these things together.
And for example, in the logistics space, we may use the word shipping and moving. We’ll combine those into the same word. There’s a lot of different ways to use NLP in the shipping industry. A lot of times we don’t even know when natural language processing is being used.
But, one of the common examples is when a voicemail comes in and that voicemail gets transcribed. Or when we talk to Google or Apple and it transcribes the words that we’re saying. That’s natural language processing in action.
How Teknowlogi Uses NLP
Now those are much bigger applications. But we use those throughout our day in the shipping industry as well. And then there are some very specific shipping industry versions of that too. Like for example, we can read documents.
So when we take a POD document at Teknowlogi, we’ll take that POD and we’ll extract information off of it. And that some of that is natural language processing that extracts that.
Another way that Teknowlogi uses NLP, is to extract information from an email. Then, process that information from an email. So for example, one of the products that we offer is an email assistant. And the email assistant can read an email from your customer as it is or how the customer sends it in.
So, we’ll take natural language processing tools and figure out the customer is asking for a quote. And if we find out they’re asking for a quote, we’ll extract all the information:
- ZIP code
- the density or the dimension
- the class of the shipment
All that information that’s required to get a quote. And we’ll run that through our quoting engine and send it back to your dispatchers. This is to help them shorten the time-frame that it takes them to run a quote.
So right now it takes dispatchers, customer service staff, or your salespeople. It takes them about 10 to 15 minutes to run a quote, not to mention the time it takes them before they read that email. But with our NLP tools at Teknowlogi and our email assistant, we’re able to:
- Immediately read your email from the customer
- Extract all that information
- Run a quote
And then allow your staff to approve the quote before it gets sent out. Allowing your staff to be 10 to 15 times more efficient at processing quotes and get them out faster. That way you’re more likely to win that business from your customers.