Creating regex configurations

Begin by creating a module directory inside your notebook. This is a directory inside the notebook file system that is used temporarily to store the files created by the RBR training. This module directory can be the same directory that you created and used for dictionary-based entity extraction. Dictionaries and regular expressions can be used in the same training run.

To create the module directory in your notebook, enter the following in a code cell. Note that the module directory can't contain a dash (-).

import os
import watson_nlp
module_folder = "NLP_RBR_Module_2"
os.makedirs(module_folder, exist_ok=True)

A regex configuration is a Python dictionary, with the following attributes:

The regex configurations can be loaded using the following helper methods:

• To load a single regex configuration, use watson_nlp.toolkit.RegexConfig.load(<regex configuration>)
• To load multiple regex configurations, use watson_nlp.toolkit.RegexConfig.load_all([<regex configuration>)])

Code sample

This sample shows you how to load two different regex configurations. The first configuration detects person names. It uses the groups attribute to allow easy access to the full, first and last name at a later stage.

The second configuration detects acronyms as a sequence of all-uppercase characters. By using the token_boundary attribute, it prevents matches in words that contain both uppercase and lowercase characters.

from watson_nlp.toolkit.rule_utils import RegexConfig

# Load some regex configs, for instance to match First names or acronyms
regexes = RegexConfig.load_all([
    {
        'name': 'full names',
        'regexes': ['([A-Z][a-z]*) ([A-Z][a-z]*)'],
        'groups': ['full name', 'first name', 'last name']
    },
    {
        'name': 'acronyms',
        'regexes': ['([A-Z]+)'],
        'groups': ['acronym'],
        'token_boundary': {
            'min': 1,
            'max': 1
        }
    }
])

Training a model that contains regular expressions

After you have loaded the regex configurations, create an RBR model using the RBR.train() method. In the method, specify:

• The module directory
• The language of the text
• The regex configurations to use

This is the same method that is used to train RBR with dictionary-based extraction. You can pass the dictionary configuration in the same method call.

Code sample

# Train the RBR model
custom_regex_block = watson_nlp.resources.feature_extractor.RBR.train(module_path=module_folder, language='en', regexes=regexes)

Applying the model on new data

After you have trained the dictionaries, apply the model on new data using the run() method, as you would use on any of the existing pre-trained blocks.

Code sample

custom_regex_block.run('Bruce Wayne works for NASA')

Output of the code sample:

{(0, 11): ['regex::full names'], (0, 5): ['regex::full names'], (6, 11): ['regex::full names'], (22, 26): ['regex::acronyms']}

To show the matching subgroups or the matched text:

import json
# Get the raw response including matching groups
full_regex_result = custom_regex_block.executor.get_raw_response('Bruce Wayne works for NASA‘, language='en')
print(json.dumps(full_regex_result, indent=2))

Output of the code sample:

{
  "annotations": {
    "View_full names": [
      {
        "label": "regex::full names",
        "fullname": {
          "location": {
            "begin": 0,
            "end": 11
          },
          "text": "Bruce Wayne"
        },
        "firstname": {
          "location": {
            "begin": 0,
            "end": 5
          },
          "text": "Bruce"
        },
        "lastname": {
          "location": {
            "begin": 6,
            "end": 11
          },
          "text": "Wayne"
        }
      }
    ],
    "View_acronyms": [
      {
        "label": "regex::acronyms",
        "acronym": {
          "location": {
            "begin": 22,
            "end": 26
          },
          "text": "NASA"
        }
      }
    ]
  },
...
}

Parent topic: Creating your own models