Frequent Words

Last published at: 2024-04-06 10:10:33 UTC

The Frequent Words feature is available for Premium and PDM licenses.

The Frequent Words feature lets you define frequently used words that will not be taken into account when checking for duplicate records. Duplicate Check offers a standard list of common, frequent words, but you can also create a custom list with words that you choose.

For example, two accounts: Tokyo University and Leiden University, are by no means duplicates of each other, but might score medium high because they both contain "University". When University is added as a frequent word, only "Tokyo" and "Leiden" will be compared, scoring as 0% duplicates.

Create lists of frequent words for fields that you use in scenarios. Then indicate in your scenarios if and which frequent words list should be used when searching for duplicates.

The frequent words lists are applied per field, and you can only select one list per field. Therefore it makes sense to have lists of words that are of a similar type.
You can apply a frequent words list to multiple fields.

Frequent Words and Synonyms

Note that there is a difference between Frequent Words and Synonyms‍. 

  • Frequent Words ignore certain words when calculating duplicate scores, as they should not be considered duplicates of each other even though they are the same. The frequent word "University" in Tokyo University and Leiden University is not taken into account when calculating the duplicate score, only "Tokyo" and "Leiden" are compared. 
  • Synonyms match two different words as 100% duplicate of each other. "Bob" and "Robert" will score 100% if added as synonyms.

When both are in use in a scenario, the Frequent Words are applied first, and Synonyms second. Therefore there's no need to add frequent words in synonyms, and in fact it is recommended to leave frequent words out.
For example, "Street" and "St." only need to be added to the Frequent Words list, and don't need to be on the synonym list. You can add "Holland" and "Netherlands" to a country name synonym list, but "The Netherlands" should not be added if "the" is already used as a frequent word for the same field.

Configuring Frequent Words lists

Duplicate Check Frequent Words comes pre-filled with a couple of frequent words lists, containing an extensive collection of commonly-used frequent words. Before you create a new list, first review the existing lists and add or remove words if needed.

To configure your Frequent Words lists: 

  1. Go to DC Setup.
  2. At left, under General Setup, click Frequent Words.

The Frequent Words overview opens, with the focus on the first prefilled list, "Common".

  • At the top, switch between the "Common", "Company", and other tabs to review the different lists.
  • On a list, click Add to add new frequent words to that list. To enter multiple words in one go, separate them by commas.
    Then click Add to list .

  • At right, type one or more letters in the Filter Words box  to quickly find out if a word is already present in the list.
  • Click on a word's X icon to delete a word from a list.
    The default words provided by Duplicate Check cannot be removed.

Using Frequent Words in the search for duplicates

In your Scenarios, indicate which fields should ignore frequent words when deciding whether two values are duplicates. The next time you search for duplicates using those scenarios, words that are listed as frequent words will not be taken into account for the fields where the frequent words list is applied.

  1. Go to DC Setup and select an Object at left.
  2. At tab Scenarios, fold open the scenario where you want to apply a frequent words list.
  3. Find the field that can contain words from the frequent words list.
  4. Under Frequent Words next to the field, select the frequent words list you want to use for that field.