Skip to main content
Sign in

Manage Data Sources

  • Updated

 

This article explains how to use various features to manage existing data sources. Manage an existing data source by clicking on the data source tile. 

DSM1.png

Configuration, View Data

DSM1.png

  • Configuration – select to change the configuration of the data. This is useful if you want to change the attributes you are pulling in from your Salesforce, Marketo, etc. system. When changing the data source configuration, make sure you select the Parse option (or tab) before proceeding to the Map option. During loading of the Parse page, Openprise goes to your source system to get the latest data configuration (commonly called a schema), so if you skip the Parse step, you will not be able to view the most up-to-date data configuration.
  • View Data – use this option to view data belonging to the data source.

Import Options

DSM3.png

  • Import Now – Select this option to load any new data into the Data Source. It is important to note that creating a Data Source doesn't load the data automatically. You must import the data either by selecting this option, or creating a bot with a step that imports data into the data source.  With each Data Source import, Openprise gets any changed or new records from the source system. If there is no method for Openprise to determine if a record is new or has been changed since the last import, all data in the source system will be imported. Note: There is a limit for how many direct connections can be made at the same time to the same target system. Due to this limitation, it may take some time to import data as each import and export must be queued to accommodate the system's limitations.
  • Re-sync - Full Re-sync - This option is currently available for Salesforce and Marketo. This option will remove deleted records, add new records, add or remove attributes and update all data to time of re-sync. Jobs and/or Bots will continue to run when using this option.
  • Re-sync - Full re-sync with filter - This option is currently available for Salesforce and Marketo. Use this option to apply a filter when re-syncing the data source. The filter you set will be shown in a pop-up. Select Import if it is correct or No if you'd like to make changes.
  • Re-sync - Deleted record detection only - This option is currently available for Salesforce and Marketo. This option checks every record already imported in Openprise to see if it was deleted in the users CRM system. If a record has been deleted from the CRM system, it will be purged from the data source inside Openprise. Jobs and/or Bots will continue to run when using this option.

Download

DSM5.png

Download – select this option to download your data. After naming the data download file, the data is available under the Data Downloads tab. Large data sources will take some time to be ready for download.

Purge Options

DSM4.png

  • Purge – Purge all data – Use this option to delete all data in your data source. This option, along with a subsequent data import, is useful for systems that delete records directly (as opposed to "marking them deleted"). For these systems (Marketo, Salesforce, etc.), Openprise cannot tell when a record has been deleted in the target system directly, so a periodic purge and re-import is recommended to sync the data between Openprise and the target system. When purging and re-importing, all jobs and/or bots will need to be paused to avoid errors based on data not being available.
  • Purge – Filter then purge – This option works in the same manner as "Purge all data", except you can first filter the records before purging to delete a portion of the data in your Openprise Data Source.

Delete

DSM6.png

Delete – use this option to delete the data source AND the data in the data source. If the data source belongs to a bot, an error message will be displayed and the data source will not be deleted. However, if the data source is used by a job and is deleted, the job will be marked with errors (as seen in red text on the job screen).

Advanced Topics:

Filtering records on import

In some environments, there may be reasons to limit the records imported into Openprise. For example, if there are sensitive records in the source system that cannot be exported or if there are extraneous records that do not need to be in Openprise. In these cases, an import filter can be defined in the data source configuration as seen in the filter below.

DSM7.png

Samples of the import filter are as follows:

For string fields
 LastName = 'foo'
  LastName = null
  Lastname LIKE 'appl%'
  City != 'San Francisco'
  BillingState IN ('California', 'New York')
  BillingState NOT IN ('California', 'New York') 

For multi-select picklist fields
 MSP1__c = 'AAA;BBB'
  MSP1__c includes ('AAA;BBB', 'CCC')

The first filter returns records where MSP1__c has both AAA and BBB selected.
The second filter returns records where MSP1__c either have AAA and BBB selected or just CCC.

For date fields
 CreatedDate  2005-10-08T01:02:03Z
  CreatedDate <1999-01-01 createddate="CreatedDate"> LAST_WEEK</1999-01-01>

For boolean fields
 BooleanField = TRUE
  BooleanField = FALSE

Using logical operations with multiple fields
 LastName = 'foo' OR FirstName != 'bar'
  Company IN ('openprise', 'opt') AND Country = 'USA'
  CreatedDate  2021-01-01T00:00:00Z OR (Company IN ('openprise', 'opt') AND Country = 'USA')

Using object relationship in filters
 Account.Name = 'foo'

NOTE: this feature is currently available for Salesforce related data sources only. It will be expanded to other source systems in future releases.

 

Reference:

  1. Learn what data sources are and how to create them by clicking HERE. 

 

Related articles