Skip to main content
Data Templates
Updated over a week ago

Overview

As a Salesforce App administrator, you can selectively restore records or metadata from a data set from one environment to another. You can also use data templates to restore archived records. We also understand how critical it is to safeguard the functional production data when copying the data to a non-production instance.

To meet your requirements, we bring to you the following via the use of Data Templates:

  • The granular level of restoration of your Salesforce records and metadata using record selection, and query-based selection of data.

  • The exclusion of objects and associated fields. A good use-case would be ensuring certain personal information of a customer is skipped from a data copy (Sandbox Seeding) process. This is per regulations set by the General Data Protection Regulations (GDPR).

  • The application of data masking and translation to obscure production data into fully functional data. This is to avoid the usage of actual production data in a non-production instance after a data copy or sandbox seeding process.

In a data template, you can selectively configure the data set for Data Copy (Sandbox Seeding) or Data Restore and apply any data transformations, validations, exclusions, and masking.

Step 1: Creating a Data Template

Create a data template for data copy, data restore, or archival restore. For more information, see Creating a Data Template.

Step 2: Configuring a Data Template

Configure the created data template for data selection methods and additional data template settings.

The following data selection methods are available.

Selection Method

Description

Record Selection enables you to select individual records and related records from an object to restore or copy.

Use object queries to select records dynamically. You can add a single object query or multiple object queries.

Manage the data template settings, such as:

  • Excluding objects and fields

  • Applying validations, data masks, and data translations

  • Specifying the batch size of records for copying data to a destination organization

  • Discover the child objects of a parent object when working with junction objects

For more information, see Configuring a Data Template.

Creating a Data Template


โ— Important

Ensure that the required profile permissions are set to begin with data template creation. The profile permissions for managing data templates are Manage Data Templates and Perform Data Copy. For more information, see User Profiles.


If you are invited to a project, add the source and destination environments from the Environment Hub page. These environment settings are used in data templates.

For more information, see Adding a Salesforce Environment.

Procedure

  1. On the Salesforce App console, click Data Template.

  2. Click New. The Create Data Template window appears.

Configuring_Data_Templates_Create_a_data_template

3. Assign values to the following fields:

Fields

Descriptions

Projects

Projects are the organization tools to gather all of the data templates and users associated with one project in one place. Projects can be created as part of the organization settings from the Settings page.

Name

Specify a unique template name within the project.

Source Org

Specify the source organization to which you want to apply the data copy or restore template.


๐Ÿ“ Note

Source Org is only applicable if the data template type is Data Copy.


Destination Org

Specify the destination organization to which you want to apply the data copy or restore template.

4. Select the data template type, which can be either Data Copy, Data Restore, or Archival Restore.

5. If you have selected Data Restore, the following fields appear.

Fields

Descriptions

Backup Definition

Specify a backup definition from the dropdown list to link this Data Restore template to a backup definition.

Backup Job

Specify a backup job from the dropdown list to link this Data Restore template to a backup job.

6. If you have selected Archival Restore, the following fields appear.

Fields

Descriptions

Archival Set

Select the archival set that you want to restore objects from.

7. Click Save.

The Data Template fetches the records from the source organization. You can go a step further to configure your data template to filter records using data selection methods. This helps to achieve a granular data copy (Sandbox Seeding) or data restore process.

Configuring a Data Template

The Data Template page enables users to select their data set for data copy or restore and apply any data transformations.

The Data Template page is divided into three different views:

  1. Environment Details: The top part of the Data Template or Data Archival Restore Template page displays the source and destination environments for this specific data copy. You can change the source or destination by selecting from the dropdown list. For more information about configuring the environment, see Adding a Salesforce Environment.

data_template.png

๐Ÿ“ Note

For the Archival Restore template, you will additionally see an option to Start Archival Restore. The rest of the workflow remains the same.


2. File and Data Statistics: The center part of the Data Template page displays two progress bars, one for data and the other for files.

The different color indicators of the progress bars indicate the following:

  • Blue: Existing data and file size within the destination environment.

  • Red: Data and file size selected for data copy or restore.

  • Green: Free space within the destination environment.

Salesforce_App_Data_Template_Data_File_Statistics.png

3. Selection Methods and Template Settings: The bottom part of the Data Template page contains the data selection methods and data template settings.

The following two ways of selecting data are available:

data_selection_method.png

Record Selection

Record Selection enables you to select individual records and related records from an object to restore or copy.

record_selection.png

Step

Description

Step 1

Enter the object name in the search box.

The table below populates the associated records.

Step 2

Select the checkbox next to the record or records you would like to copy.

Step 3

You can type in a SOQL statement in the search box below the Add/Remove Records button.


๐Ÿ“ Note

You can skip the WHERE keyword in the SOQL statement.


Step 4

Select the level of record relationship to copy from the dropdown list, which can be any of the following:

  • All Child Objects

  • None

  • 1 Level

  • 2 Levels

  • 3 Levels

  • Custom

The related parent records are excluded by default during custom child object selection. Select Include Related Parent Records to include parent records. For more information, see Understanding Object Relationships.

Step 5

Click Add/Remove Records to discover the records from the source environment.

The Tasks tab on the bottom-right corner displays the status of the record inclusion task. Click on the task entry to learn more about the status of this task. This will redirect you to the Tasks page to display additional information about the associated task.

Step 6

The gear icon to the left of the WHERE clause provides control over the table of records. Select among the following options:

Column Selection

This enables you to select the columns that you want to view in the records table. After selecting this option, the Column Selection window appears.

  • Use the directional arrows to select a field from the list on the left and move it to the list on the right.

  • Use the directional arrows to determine the order of the columns.


๐Ÿ“ Note

The table can display a maximum of 20 fields.


Show Selected Rows

This option displays the records for the designated object only, which are selected for migration.

Query-based selection

Create an object query to select records dynamically. If records are altered in your source environment, the data copy activity continues to run using this method. You can add a single object query or multiple object queries.

Adding a Single Object Query

Perform the following steps to query records for a single object.

1. Click Add Query. The Add Object Query window appears.

Salesforce_App_Data_Template_Query Selection.png

2. Enter information for the following fields:

Field

Description

Environment Object

Select the object to query.

Include All Records

Discover all records that meet the search criteria. Uncheck this box to specify the total number of records to include in the Total Numbers to Include text box.

Total Records to Include

Discover a specified number of records only that meet the search criteria.

Total Records to Include Per Parent Record

Discover a specified number of child records only per parent record that meet the search criteria.

For example, if the parent record contains 10 child records, you can limit it to only include 5.

Where Clause - e.g., Id IN ('1232') AND Name = 'Test'

Specify the syntax of the condition expression for a WHERE clause of a SOQL query. For more information, see Condition Expression Syntax.


๐Ÿ“ Note

Skip the WHERE keyword when specifying the condition expression.


Include Related Child Records

Select the level of child records to include from the dropdown list.


๐Ÿ“ Note

Ensure caution when selecting All Child Objects as the record may be a part of a nested relationship chain.


For more information, see Understanding Object Relationships.

3. Click Save.

The created queries appear in a table in the Query-based Selection tab.

Adding Multiple Queries

Perform the following steps to discover records across multiple objects.

1. Click Add Multiple Queries. The Add Multiple Queries window appears.

2. Enter information for the following fields:

Field

Description

Environment Object

Select all objects to query by selecting the objects in the Available Objects list and using the arrows to move them into the Selected Object list.

Include All Records

Discover all records that meet the search criteria. Uncheck this box to specify the total number of records to include in the Total Numbers to Include text box.

Total Records to Include

Discover a specified number of records only that meet the search criteria.

Where Clause - e.g., Id IN ('1232') AND Name = 'Test'

Specify the syntax of the condition expression for a WHERE clause of a SOQL query. For more information, see Condition Expression Syntax.


๐Ÿ“ Note

Skip the WHERE keyword when specifying the condition expression.


Include Related Child Records

Select the level of child records to include from the dropdown list.


๐Ÿ“ Note

Ensure caution when selecting All Child Objects as the record may be a part of a nested relationship chain.


For more information, see Understanding Object Relationships.

3. Click Save.

The created queries appear in a table in the Query-based Selection tab.

Managing Queries

You can manage queries and add them to a data template.

query_based_selection.png

The following table describes the additional actions that you can perform for created queries.

Field

Description

Add to Data Template

Click Add to Data Templates to discover records from the source environment.

Clear All Queries

Click Clear All Queries to delete all queries listed from the bottom table.

Query Row Action

Every query in the bottom table includes a dropdown arrow on the right. This gives users additional options, which are as follows.

  • Add Sub-Query - A subquery enables you to query child records related to the parent object. For example, you can query cases related to specific accounts. Perform the steps from Adding a Single Object Query to add a subquery.

  • Edit Query - The Add Object Query window appears enabling you to edit the existing query.

  • Delete Query - Deletes the selected query.


๐Ÿ“ Note

A second subquery created for the same object under a query will not be reflected during discovery.


Data Template Settings

You can manage the settings of a data template from the Data Template Settings tab.

Salesforce_App_Data_Template_Settings.png

Overview

Get an overview of the configured data template settings from the Overview tab. You can see the following information for all objects:


๐Ÿ“ Note

When creating a Data Template, some of the data template settings are applied by default. For example, the AccountContactRelation, ApexClass, and ApexPage objects are excluded. You can change these settings from the Exclude Objects and Exclude Fields tabs. For a detailed list of exclusions, see Understanding Backup Exclusions.


External ID

Salesforce uses External IDs to create the parent-child relationships between the source and the target object.

When a field is marked with an External ID, its values can be used to match and associate related records. The Salesforce App uses this to determine if the record already exists on the destination org. As a result, the Salesforce App can prevent duplicate inserts by performing an update operation in place of an insert operation.

When to create External IDs?

If you have manually created records on the destination org or used another application to load data into the destination org, you must mark fields with external IDs.

For example, consider a scenario where you are importing data from a .csv file with records that already exist in the destination environment without a Salesforce record ID. To prevent creating a duplicate record, you can mark a field with an external ID to map the source and destination fields for that record.

If the data is populated through a sandbox refresh (Partial or Full Sandbox), or by the Salesforce App, you do not have to define External IDs. In this case, the application automatically tracks the ID mapping between the source and destination.


๐Ÿ“Note

External IDs support composite keys.


Creating an External ID

  1. Click New External ID from the External Id tab. An External Id Form appears.

  2. Enter the following fields:

Field

Description

Environment Object

Search the environment object from the search box to apply the external ID.

Source Field

Select a field from the source environment.

Destination Field

Select a field from the destination environment.

Source RegEx

You can apply a regular expression to the source field for its value to match with the destination field.

Source Substitution

Specify the value which should replace the regular expression matched value.

3. Click Save.

Perform the following actions to manage external IDs for an object.

  • Click on the object name to see a list of the added external IDs.

  • Click the plus icon to add additional external IDs for the same object.

  • Click the tick icon to test the objectโ€™s RegEx.

  • Click the red bin icon to remove all object external IDs.

  • Click the disk icon to update an external ID.

Exclude Objects

Exclude Objects option is used to exclude any objects from data copy into the destination org to consider data storage limitations or to filter sensitive data.

Perform the following steps to exclude an object from a data copy:

  1. Enter the object name or the search criteria to filter from the listed object names.


๐Ÿ“ Note

If an object is not listed, the following object and field-level permissions may not be set to true:

  • Queryable

  • Creatable

  • Updateable

Set queryable to true and set either creatable or updatable to true for the Salesforce App to perform upsert and update operations.


2. Select the required field and click Exclude or Include.


๐Ÿ“ Note

You cannot exclude objects containing a positive record count. To exclude that object, perform an exclude record operation from the Record Selection tab.


Exclude Fields

Select fields of an object to exclude or include from a data copy. By default, the Salesforce App excludes the following types of fields:

  • Encrypted fields

  • Fields without a Blob type

  • Fields without a calculation

Change the field settings in your source environment to populate the fields in the Salesforce App.

Perform the following steps to exclude a field.

  1. Enter an object name in the search box. The following search options are available.

Option

Description

Search by object

Search a field by a specific object name.

Search across objects

Search a field across objects.

Show all excluded fields

Displays a list of all excluded fields.

2. Select the desired field and click Exclude or Include.

Data Translations

Data translations use regular expressions to transform field values for an object from the source environment and replace them with data to match the requirements of the destination environment.

Perform the following steps to translate data in the source environment.

  1. Enter an object name in the search box using the search by object option. Alternatively, select search across objects to search using field name.

  2. Create regular expressions and substitutions to translate data using either of the following options.

Option 1 - Adding manually

  1. Select the desired field names and click Edit. The Add Data Translation window appears with the selected object field.

  2. Enter the regular expression and its substitution pair in the text boxes and click Add. To delete a pair, click Remove.

If multiple object fields are selected, this window displays the common data translations for the selected fields.

3. Click Save.

Option 2 - Uploading Files

  1. Select the desired field names and click Upload Files.

The file format must be .csv with the following formatting:

Sample File Name

Required Formatting

myFirstImport.csv

OR

myFirstImport.csv

  • \w,test

  • \s,fefe

  • myFirstImpor

2. Click Edit to add a regular expression and a substitution pair for the data translation.

To remove a data translation, select the field name(s) and click Delete on the upper-right corner. For more information, see Using Regular Expressions.


๐Ÿ“ Note

To match a NULL field value, you can use any one of the following regular expression values:

  • (.*)

  • (^$)

  • (^)

  • ($)


Data Mask

You can manage and create data masks for projects, in contrast to creating global data masks from the Settings page. These data masks are applicable only to the current data template.

Data mask definitions created at the organizational or project level take precedence over those created within the current data template.

Consider the following example:

  • An organization's administrator creates a data mask definition to mask Social Security Number fields using a regular expression pattern or substitution values.

  • A developer also adds a data mask definition to mask Social Security Number fields with a different regular expression pattern or substitution values.

  • In this scenario, the data mask definition created by the administrator is applied as it is created at an organizational level.

For more information about data masking, see Data Masking.

For a step-by-step guide and further information on creating data masks, see Creating a Data Mask.

Validations

Validations in Salesforce evaluate the data to meet defined field guidelines before saving the record.

Salesforce App disables the following Salesforce validation types by default during a data copy:

  • Validation Rules

  • Required Fields

  • Lookup Filters

  • Allow Re-parenting Fields

  • Unique Fields

  • Hidden Fields

  • Read-Only Fields

  • Inaccessible Record Types

  • Triggers

  • Flows

  • Duplicate Rules

  • Workflows

Perform the following steps to disable or enable a validation.

  1. On the Validations tab, select a Validation Type from the dropdown list. A metadata list for the selected validation type appears.

  2. Enter the search criteria in the Search By Name search box to filter from the metadata list.

  3. Click Save Selection.

Some points to consider when enabling or disabling validations:

  • The application does not support disabling certain metadata types for metadata defined in managed packages.

  • The application supports enabling or disabling validations associated with standard fields.

For example, if a managed package validation rule exists on a Standard Object, the application disables it. Conversely, if a managed package trigger exists, the application cannot disable this trigger as only the managed package vendor can modify it. For more information about managed packages, see Manage Packages.

  • Triggers are deployed using the Metadata API's Deploy() method. All other validation types are enabled/disabled using CRUD operations.

  • If the application is inactivating the trigger in Salesforce, the trigger must not have any active Apex Scheduled Jobs (Future methods) running.

  • The application uses the latest API version (i.e. Generally Available (GA)) when working with metadata.

  • If an object does not have a default record type defined, the application automatically sets the first record type as the default.

  • You can enable individual validations/validation types during the data copy process. During a data copy or sandbox seeding process, the application disables only the validations related to objects that are part of the data copy process (except Process Builder and Visual Flows objects).

Validation Workflow

The following diagram illustrates the workflow when performing data copy with validations.

Data_Copy_with_Validations.png
  1. Initiate the Data Copy. A Data Copy Confirmation window appears. For more information about the Data Copy workflow, see Data Copy.

  2. Select Disable destination metadata during data copy to disable the validations during the data copy process.


๐Ÿ“ Note

This option is disabled if the validations on the destination org are already disabled.


3. If the validations are already disabled for the org, the application does not disable/re-enable the validations.

4. If validations are not already disabled, the application retrieves the validations associated with the objects, which are applicable for data copy.

5. The Salesforce App saves a backup of the validations, which will be disabled.

6. It will disable the validations on the destination org.

7. The Data Copy operation begins.

8. The Salesforce App retrieves the validations associated with the objects, which are applicable for data copy.

9. It enables the validations, which were previously disabled in step 6.

10. It also deletes the backup of the validations taken in step 5.

Manage Validations for an Organization

Perform the following steps to disable, enable, or refresh all validations for an org.

  1. On the Salesforce App console, click Environment Hub.

  2. Click the dropdown icon on the right for an org.

  3. Select among the following options as per your requirements.

Option

Description

Disable Validations

Disables all validations for an org.

Enable Validations

Enables all validations for an org.

Refresh Validations

Identifies all validations for an org.

Batch Size

You can specify the batch size of records for copying data to a destination organization. By default, the Salesforce App loads a maximum of 200 records.

Perform the following steps to specify the batch size:

  1. On the Batch Size tab, click New. A Batch Size window appears.

  2. Select the object from the Environment Objects dropdown list.

  3. Enter the batch size in the Batch Size text box.

  4. Switch the Process Serial toggle to enable serial processing.

  5. Click Save.

Object Child Discovery

You can discover the child objects of a parent object when working with junction objects from the Object Child Discovery tab. Junction objects are used to create many-to-many relationships between two objects in Salesforce.

For more information about junction objects in Salesforce, see Junction Object.

Perform the following steps to identify all child objects for a parent object:

  1. On the Object Child Discovery tab, enter the search criteria in the search box. Alternatively, select an alphabetic character to filter the object.

  2. Select the desired object and click Save.

Cloning a Data Template

You can clone the data template by making a copy of the original template. Cloning enables you to create a data template quickly with auto-populated configurations.

Procedure

  1. On the Salesforce App console, click Data Template. The Data Template page appears with a list of data templates.

    Percentage_deprecation_final.gif
  2. Select the template you want to clone and click Show Details. The Data Template window appears.

  3. Click Clone Data Template in the top-right corner. The Clone Data Template window appears with the auto-populated fields for Project, Name, Source Org, and Destination Org. For more information about assigning values to these fields, see Configuring a Data Template.

    clone_template.png
  4. Select the required items in the Items to Clone list:

Field

Description

Data Record

Data records of the template.

Query Selection

Select records dynamically by creating an object query.

External Id

External IDs to create the parent-child relationships between the source and the target object.

Excluded Object

Excluded objects from data copy into the destination org to consider data storage limitations or to filter sensitive data.

Excluded Field

Excluded fields from a data copy. By default, the Salesforce App excludes the following types of fields:

  • Encrypted fields

  • Fields without a Blob type

  • Fields without a calculation

Data Translation

Transform field values for an object from the source environment and replace them with data to match the requirements of the destination environment.

Batch Size

Batch size of records for copying data to a destination organization.

Validations

Validations to evaluate the data to meet defined field guidelines before saving the record.

Data Mask

Data masking replaces sensitive information while copying data. The data mask configurations that were created in the original template will be added as a part of the cloned template.

5. Click OK. The data template is cloned and appears in the data templates list.

Schedule Sandbox Seeding

You can now automate seeding data into your sandboxes using the data template scheduler. Using the data template scheduler, you can:

  • Copy data to your sandboxes on a daily, weekly, or monthly basis

  • Ensure your sandboxes have adequate test data at all times

  • Accelerate your Salesforce development tasks as operational data is available regularly to your sandboxes.

Letโ€™s see how you can schedule a data copy activity.

Procedure

  1. On the Salesforce App console, click Data Template.

  2. Click Scheduler. The Data Template Scheduling window appears.

Scheduler_Data_Copy_Window.png

3. Click New Schedule.

4. Enter the following details in the Data Template Selection area.

Field

Description

Project

Select the project, which includes the data copy template.

Data Template

Select the data copy template using which you want to schedule a data copy job.

Source Org

Select the source org from which you want to copy the data.

Destination Org

Select the destination org to which you want to copy the data.

5. Click Add to include the data template in the schedule.

6. Enter the following details in the Scheduling area.

Field

Description

Repeat every

Specify how often your data copy schedule should execute.

Start Date

Select the start date for the data copy schedule.

Start Time

Select the start time for the data copy schedule.

End Date

Select the end date for the data copy schedule.

7. Specify the following details in the Record Selection area.

Field

Description

Run Query-based Selection

This setting is disabled by default. When enabled, any queries defined in the Query-based Selection section within the data template are executed prior to starting the data copy operation.

For more information, see Query-based selection.

Clear existing record selection

This setting is disabled by default. When enabled, any records which are already added to the data template will be cleared.

You can disable this setting if you have already selected specific records, which you want to copy. You can also optionally use the Run Query-based Selection option to append additional records to the data template.

8. Configure the following details in the Data Copy Settings area as per your requirements.

  • Disable destination metadata during the data copy

  • Replace references to inactive user IDs with your user ID.

  • Replace inactive and blank record type IDs with the object's default record type ID.

  • Specify the email address to send notifications after the data copy completion.

9. Specify the maximum number of attempts for the data copy activity. This configuration will retry the data copy activity for the specified maximum number of times in case of any failures.

10. Click Save Schedule.

The data template scheduler appears in the list of created schedulers on the left pane and executes according to the configured frequency. You can click on a scheduler to edit or delete it from the right pane.

Scheduler_Data_Copy_Details.png

You can navigate to the Tasks tab from the Salesforce App console to view the following task details:

Task

Description

Data Template Scheduler Post

Provides details of the data template scheduler.

Data Copy

Provides details of the data copy task, such as status and reasons for failure, if any.

For more information, see Managing Tasks.

Using Regular Expressions

The following features in the Salesforce App enable you to use regular expressions to match patterns and substitute values.

  • External IDs

  • Data Mask

  • Data Translations

Regular expressions are patterns applied to strings to match character combinations. This is used to apply actions to data.

For example, you may want to change the suffix of all the user email addresses in your data set from .source to .destination.

You can apply the following RegEx pattern to the data set and use โ€˜destinationโ€™ as the substitution.


๐Ÿ’ก Tip

To help you construct your regular expression, see Regular Expressions for a detailed guide on how to create regular expressions. Additionally, RegEx Tester is a useful tool to test the created RegEx.


Where to go from here

Did this answer your question?