Extracting Numeric Entities using Duckling in Python

Wit.ai is a natural language processing (NLP) platform that allows developers to build conversational experiences for various applications. One of the key features of Wit.ai is its entity extraction system, which can recognize and extract entities from user input.

One of the key features provided by  Wit.ai is its entity extraction system called Duckling. Duckling is an open-source library that can extract entities such as time, date, duration, and numbers from text input. 

In this article, we will focus on the numeric entity tagging functionality provided by Duckling and how it can be implemented using Python.

Getting Started with Duckling :

Required Modules :

pip install wit
pip install duckling
pip install --force-reinstall JPype1==0.6.3 # To avoid a common dependency error

We can then create a Python file and import the necessary libraries:

# importing the necesary packages
import wit
import duckling

We can now use the wit library to connect to the Wit.ai API:

# initialize wit.ai instance using api-token
access_token = "your-access-token"
client = wit.Wit(access_token)

We can test our connection by sending a message to the Wit.ai API:

# capturing numeric entity using raw text(example0) using wit.ai client instance
example0 = "I want to read 3 geekforgeeks articles."
response = client.message(example0)
print(response)

The client. message() method sends a message to the Wit.ai API and returns a JSON response. The response should include the entities that Wit.ai was able to extract from the message. In this case, the response should look something like this:

{
 "text": "I want to read 3 geekforgeeks articles.",
 "intents": [],
 "entities": {
   "wit$amount_of_money:amount_of_money": [
     {
       "id": "12345678-1234-5678-1234-567812345678",
       "name": "wit$amount_of_money",
       "role": "amount_of_money",
       "start": 16,
       "end": 20,
       "body": "3",
       "confidence": 0.9975,
       "entities": [],
       "value": 3.0,
       "type": "value"
     }
   ]
 }
}

We can see that Wit.ai was able to extract the numeric entity "3" from the message and tag it as an amount of money. However, in this case, we want to extract the numeric entity without any specific tag. This is where Duckling comes in.

Using Duckling for Numeric Entity Tagging :

To use Duckling for numeric entity tagging, we first need to create a Duckling parser. 

# loading the opensource Duckling parser
parser = duckling.Duckling()

We can then use the parser to extract numeric entities from a message:

Example 1:

import json # to beautify json while printing

# using the opensource duckling instance 
# to capture a generalized numeric entities
example1 = "I want to read 3 geeksforgeeks articles."
response = parser.parse(example1)


print(json.dumps(response, indent=3))

The parser. parse() method sends the message to the Duckling parser and returns a list of entities that Duckling was able to extract. In this case, the response should look something as follows:

Output:

[
   {
      "dim": "number",
      "text": "3",
      "start": 15,
      "end": 16,
      "value": {
         "value": 3.0
      }
   },
   {
      "dim": "distance",
      "text": "3",
      "start": 15,
      "end": 16,
      "value": {
         "value": 3.0,
         "unit": null
      }
   },
   {
      "dim": "volume",
      "text": "3",
      "start": 15,
      "end": 16,
      "value": {
         "value": 3.0,
         "unit": null,
         "latent": true
      }
   },
   {
      "dim": "temperature",
      "text": "3",
      "start": 15,
      "end": 16,
      "value": {
         "value": 3.0,
         "unit": null
      }
   },
   {
      "dim": "time",
      "text": "3",
      "start": 15,
      "end": 16,
      "value": {
         "value": "2023-03-28T03:00:00.000+05:30",
         "grain": "hour",
         "others": [
            {
               "grain": "hour",
               "value": "2023-03-28T03:00:00.000+05:30"
            },
            {
               "grain": "hour",
               "value": "2023-03-28T15:00:00.000+05:30"
            },
            {
               "grain": "hour",
               "value": "2023-03-29T03:00:00.000+05:30"
            }
         ]
      }
   }
]

We can observe that Duckling was able to extract the numeric entity "3" from the message and tag it as a number. We get multiple different possible entities from the parsed text but we can extract this entity from the response using the following code:

# extracted the final response
entities = response[0]["value"]["value"]

print(entities)

Output:

3.0

Example 2:

# extracting the datetime entity from raw text
example2 = u"Let\'s meet at tomorrow at half past six to read a geekforgeeks article."

duck_parsed = parser.parse_time(example2)
print(json.dumps(duck_parsed[0], indent=3))

Output:

{
   "dim": "time",
   "text": "tomorrow at half past six",
   "start": 14,
   "end": 39,
   "value": {
      "value": "2023-03-28T06:30:00.000+05:30",
      "grain": "minute",
      "others": [
         {
            "grain": "minute",
            "value": "2023-03-28T06:30:00.000+05:30"
         },
         {
            "grain": "minute",
            "value": "2023-03-28T18:30:00.000+05:30"
         }
      ]
   }
}

As you can observe, duckling can recognize the relative date-time from the raw text and returns a date-time string as the value which is tomorrow's date 2023-03-28 (28th March 2023), and the specified time in the raw text(half past six) i.e. 6:30 AM. We can parse the date-time string which is present in the iso format to get a DateTime object as such:

import datetime

print(datetime.datetime.fromisoformat(duck_parsed[0]['value']['value']))

This outputs the DateTime object like so:

2023-03-28 06:30:00+05:30