Skip to main content
Sign in

De-Duplication And Merge Against Master Data

  • Updated

Purpose

Check if records from a list are duplicates when compared to a master data list. If duplicates are found, use the data from the list to update the master data record. This task is ideal for checking for duplicates while loading a list into a system (Salesforce, Marketo, etc.) or database.

 

Category Location: All, Deduplication

 

Field Description

2.png

  1. Manual review of de-duplication result: Check this box to save the results to a manual data source. This allows for more review options.
  2. Master list Data Source to de-duplicate against: Select the data source to merge your input data source against.
  3. How to identify duplicates: Select the attributes to match exactly on the left and/or to match with the fuzzy option on the right. There are fuzzy level options in the advanced configuration section after an attribute has been selected.
  4. Map output attributes: Map the attributes from the input data source to the master list data source.
  5. How to merge records and review results: Check this box to set merging requirements. Options are: Fill only if empty from non-surviving records, use value from any record where, append values from all records, and do not merge.
  6. Advanced Configurations: This section will contain fuzzy options if you've matched an attribute in that section.  Additionally, select the Add Exception (arrow) option to add an exception to the how to merge records logic. (i.e. The selection "fill only if empty" has been selected in the how to merge records section, but you'd like the Description field to append values from all records.  You can add that exception for the Description field here)

Tips

  • This task has similar capabilities as the De-duplication and Merge task, with these differences:
    • The master data record is always the surviving record.
    • The output records are based on the master data record.
    • You must map the data attributes from the input list to the master data list.
  • Use the "Advanced Configuration" and "Add Exception" button to override the default merge logic on a per attribute level.  Each exception can use different merge logic to achieve even the most complicated merge requirements.
  • You can use the "Manual review of de-duplication result" section to specify a data source to store the de-duplication results and then review it using the "Data - Review De-dupe" option.
  • The de-duplication task supports multiple actions to perform sequential de-duplication. The results of each action is recorded as well as the final results as well. The results of each action is stored in attributes "Duplicates1", "Duplicates2", etc… and "Merged Attributes1", "Merged Attributes 2", etc., where the number is the corresponding action in top to bottom order. The final results are stored in the attributes "Duplicates" and "Merged Attributes".

 

Example

Before loading a trade show list into Marketo, check if any record from the list already exists in Marketo.

 

 

Related articles