Overview
As a Salesforce App administrator, you can selectively restore records or metadata from a data set from one environment to another. You can also use templates to restore archived records. We also understand how critical it is to safeguard the functional production data when copying the data to a non-production instance.
To meet your requirements, we bring to you the following via the use of Templates:
The granular level of restoration of your Salesforce records and metadata using record selection and query-based selection of data.
The exclusion of objects and associated fields. A good use-case would be ensuring certain personal information of a customer is skipped from a Data Seeding (Sandbox Seeding) process. This is per regulations set by the General Data Protection Regulations (GDPR).
The application of data masking and translation to obscure production data into fully functional data. This is to avoid the usage of actual production data in a non-production instance after a Data Seeding or sandbox seeding process.
In a template, you can selectively configure the data set for Data Seeding (Sandbox Seeding) or Data Restore and apply any data transformations, validations, exclusions, and masking.
Step 1: Creating a Template
Create a template for data seeding, data restore, or archival restore. For more information, see Creating a Template.
Step 2: Configuring a Template
Configure the created template for data selection methods and additional data template settings.
The following data selection methods are available.
Selection Method | Description |
Record Selection enables you to select individual records and related records from an object to restore or copy. | |
Use object queries to select records dynamically. You can add a single object query or multiple object queries. | |
Manage the template settings, such as:
|
For more information, see Configuring a Template.
Creating a Template
❗ Important
Ensure that the required profile permissions are set to begin with template creation. The profile permissions for managing templates are Manage Templates and Perform Data Seeding. For more information, see User Profiles.
If you are invited to a project, add the source and destination environments from the Environment Hub page. These environment settings are used in templates.
For more information, see Adding a Salesforce Environment.
Procedure
On the Salesforce App console, go to Restore & Seeding.
Click the New Template button and select the desired template from the dropdown. The Create Template window appears.
3. Assign values to the following fields:
Fields | Descriptions |
Projects | Projects are the organization tools to gather all of the templates and users associated with one project in one place. Projects can be created as part of the organization settings from the Settings page. |
Name | Specify a unique template name within the project. |
Source Org | Specify the source organization to which you want to apply the Data Seeding or restore template. 📝 Note Source Org is only applicable if the template type is Data Seeding. |
Destination Org | Specify the destination organization to which you want to apply the Data Seeding or restore template. |
4. Select the template type, which can be either Data Seeding, Data Restore, or Archival Restore.
5. If you have selected Data Restore, the following fields appear.
Fields | Descriptions |
Backup Definition | Specify a backup definition from the dropdown list to link this Data Restore template to a backup definition. |
Backup Job | Specify a backup job from the dropdown list to link this Data Restore template to a backup job. |
6. If you have selected Archival Restore, the following fields appear.
Fields | Descriptions |
Archival Set | Select the archival set that you want to restore objects from. |
7. Click Save.
The Template fetches the records from the source organization. You can go a step further to configure your template to filter records using data selection methods. This helps to achieve a granular Data Seeding (Sandbox Seeding) or data restore process.
Configuring a Template
The Template page enables users to select their data set for Data Seeding or restore and apply any data transformations.
The Template page is divided into three different views:
Environment Details: The top part of the Template or Data Archival Restore Template page displays the source and destination environments for this specific data seeding. You can change the source or destination by selecting from the dropdown list. For more information about configuring the environment, see Adding a Salesforce Environment.
📝 Note
For the Archival Restore template, you will additionally see an option to Start Archival Restore. The rest of the workflow remains the same.
2. File and Data Statistics: The center part of the Template page displays two progress bars, one for data and the other for files.
The different color indicators of the progress bars indicate the following:
Blue: Existing data and file size within the destination environment.
Red: Data and file size selected for Data Seeding or restore.
Green: Free space within the destination environment.
3. Selection Methods and Template Settings: The bottom part of the Template page contains the data selection methods and template settings.
The following two ways of selecting data are available:
Record Selection
Record Selection enables you to select individual records and related records from an object to restore or copy.
Step | Description |
Step 1 | Enter the object name in the search box. The table below populates the associated records. |
Step 2 | Select the checkbox next to the record or records you would like to copy. |
Step 3 | You can type in a SOQL statement in the search box below the Add/Remove Records button. 📝 Note You can skip the WHERE keyword in the SOQL statement. |
Step 4 | Select the level of record relationship to copy from the dropdown list, which can be any of the following:
The related parent records are excluded by default during custom child object selection. Select Include Related Parent Records to include parent records. For more information, see Understanding Object Relationships. |
Step 5 | Click Add/Remove Records to discover the records from the source environment. The Tasks tab on the bottom-right corner displays the status of the record inclusion task. Click on the task entry to learn more about the status of this task. This will redirect you to the Tasks page to display additional information about the associated task. |
Step 6 | The gear icon to the left of the WHERE clause provides control over the table of records. Select among the following options: Column Selection This enables you to select the columns that you want to view in the records table. After selecting this option, the Column Selection window appears.
📝 Note The table can display a maximum of 20 fields. Show Selected Rows This option displays the records for the designated object only, which are selected for migration. |
Query-based selection
Create an object query to select records dynamically. If records are altered in your source environment, the Data Seeding activity continues to run using this method. You can add a single object query or multiple object queries.
Adding a Single Object Query
Perform the following steps to query records for a single object.
1. Click Add Query. The Add Object Query window appears.
2. Enter information for the following fields:
Field | Description |
Environment Object | Select the object to query. |
Include All Records | Discover all records that meet the search criteria. Uncheck this box to specify the total number of records to include in the Total Numbers to Include text box. |
Total Records to Include | Discover a specified number of records only that meet the search criteria. |
Total Records to Include Per Parent Record | Discover a specified number of child records only per parent record that meet the search criteria. For example, if the parent record contains 10 child records, you can limit it to only include 5. |
Where Clause - e.g., Id IN ('1232') AND Name = 'Test' | Specify the syntax of the condition expression for a WHERE clause of a SOQL query. For more information, see Condition Expression Syntax - 📝 Note Skip the WHERE keyword when specifying the condition expression. |
Include Related Child Records | Select the level of child records to include from the dropdown list. 📝 Note Ensure caution when selecting All Child Objects, as the record may be a part of a nested relationship chain. For more information, see Understanding Object Relationships. |
3. Click Save.
The created queries appear in a table in the Query-based Selection tab.
Adding Multiple Queries
Perform the following steps to discover records across multiple objects.
1. Click Add Multiple Queries. The Add Multiple Queries window appears.
2. Enter information for the following fields:
Field | Description |
Environment Object | Select all objects to query by selecting the objects in the Available Objects list and using the arrows to move them into the Selected Object list. |
Include All Records | Discover all records that meet the search criteria. Uncheck this box to specify the total number of records to include in the Total Numbers to Include text box. |
Total Records to Include | Discover a specified number of records only that meet the search criteria. |
Where Clause - e.g., Id IN ('1232') AND Name = 'Test' | Specify the syntax of the condition expression for a WHERE clause of a SOQL query. For more information, see Condition Expression Syntax - 📝 Note Skip the WHERE keyword when specifying the condition expression. |
Include Related Child Records | Select the level of child records to include from the dropdown list.
📝 Note Ensure caution when selecting All Child Objects as the record may be a part of a nested relationship chain. For more information, see Understanding Object Relationships. |
3. Click Save.
The created queries appear in a table in the Query-based Selection tab.
Managing Queries
You can manage queries and add them to a template.
The following table describes the additional actions that you can perform for created queries.
Field | Description |
Add to Template | Click Add to Data Templates to discover records from the source environment. |
Clear All Queries | Click Clear All Queries to delete all queries listed from the bottom table. |
Query Row Action | Every query in the bottom table includes a dropdown arrow on the right. This gives users additional options, which are as follows.
📝 Note A second subquery created for the same object under a query will not be reflected during discovery. |
Data Seeding Settings
You can manage the settings of a data template from the Data Seeding Settings tab.
Overview
Get an overview of the configured data Seeding settings from the Overview tab. You can see the following information for all objects:
📝 Note
When creating a Template, some of the template settings are applied by default. For example, the AccountContactRelation, ApexClass, and ApexPage objects are excluded. You can change these settings from the Exclude Objects and Exclude Fields tabs. For a detailed list of exclusions, see Understanding Backup Exclusions.
External ID
Salesforce uses External IDs to create the parent-child relationships between the source and the target object.
When a field is marked with an External ID, its values can be used to match and associate related records. The Salesforce App uses this to determine if the record already exists on the destination org. As a result, the Salesforce App can prevent duplicate inserts by performing an update operation in place of an insert operation.
When to create External IDs?
If you have manually created records on the destination org or used another application to load data into the destination org, you must mark fields with external IDs.
For example, consider a scenario where you are importing data from a .csv file with records that already exist in the destination environment without a Salesforce record ID. To prevent creating a duplicate record, you can mark a field with an external ID to map the source and destination fields for that record.
If the data is populated through a sandbox refresh (Partial or Full Sandbox), or by the Salesforce App, you do not have to define External IDs. In this case, the application automatically tracks the ID mapping between the source and destination.
📝Note
External IDs support composite keys.
Creating an External ID
Click New External ID from the External Id tab. An External Id Form appears.
Enter the following fields:
Field | Description |
Environment Object | Search the environment object from the search box to apply the external ID. |
Source Field | Select a field from the source environment. |
Destination Field | Select a field from the destination environment. |
Source RegEx | You can apply a regular expression to the source field for its value to match with the destination field. |
Source Substitution | Specify the value which should replace the regular expression matched value. |
3. Click Save.
Perform the following actions to manage external IDs for an object.
Click on the object name to see a list of the added external IDs.
Click the plus icon to add additional external IDs for the same object.
Click the tick icon to test the object’s RegEx.
Click the red bin icon to remove all object external IDs.
Click the disk icon to update an external ID.
Exclude Objects
The Exclude Objects option excludes any objects from the data seeding into the destination org to consider data storage limitations or to filter sensitive data.
Perform the following steps to exclude an object from a Data Seeding:
Enter the object name or the search criteria to filter from the listed object names.
📝 Note
If an object is not listed, the following object and field-level permissions may not be set to true:
Queryable
Creatable
Updateable
Set queryable to true and set either creatable or updatable to true for the Salesforce App to perform upsert and update operations.
2. Select the required field and click Exclude or Include.
📝 Note
You cannot exclude objects containing a positive record count. To exclude that object, perform an exclude record operation from the Record Selection tab.
Exclude Fields
Select fields of an object to exclude or include from a Data Seeding. By default, the Salesforce App excludes the following types of fields:
Encrypted fields
Fields without a Blob type
Fields without a calculation
Change the field settings in your source environment to populate the fields in the Salesforce App.
Perform the following steps to exclude a field.
Enter an object name in the search box. The following search options are available.
Option | Description |
Search by object | Search a field by a specific object name. |
Search across objects | Search a field across objects. |
Show all excluded fields | Displays a list of all excluded fields. |
2. Select the desired field and click Exclude or Include.
Data Translations
Data translations use regular expressions to transform field values for an object from the source environment and replace them with data to match the requirements of the destination environment.
Perform the following steps to translate data in the source environment.
Enter an object name in the search box using the search by object option. Alternatively, select search across objects to search using field name.
Create regular expressions and substitutions to translate data using either of the following options.
Option 1 - Adding manually
Select the desired field names and click Edit. The Add Data Translation window appears with the selected object field.
Enter the regular expression and its substitution pair in the text boxes and click Add. To delete a pair, click Remove.
If multiple object fields are selected, this window displays the common data translations for the selected fields.
3. Click Save.
Option 2 - Uploading Files
Select the desired field names and click Upload Files.
The file format must be .csv with the following formatting:
Sample File Name | Required Formatting |
myFirstImport.csv
OR
myFirstImport.csv |
|
2. Click Edit to add a regular expression and a substitution pair for the data translation.
To remove a data translation, select the field name(s) and click Delete on the upper-right corner. For more information, see Using Regular Expressions.
📝 Note
To match a NULL field value, you can use any one of the following regular expression values:
(.*)
(^$)
(^)
($)
Data Mask
You can manage and create data masks for projects, in contrast to creating global data masks from the Settings page. These data masks are applicable only to the current template.
Data mask definitions created at the organizational or project level take precedence over those created within the current template.
Consider the following example:
An organization's administrator creates a data mask definition to mask Social Security Number fields using a regular expression pattern or substitution values.
A developer also adds a data mask definition to mask Social Security Number fields with a different regular expression pattern or substitution values.
In this scenario, the data mask definition created by the administrator is applied as it is created at an organizational level.
For more information about data masking, see Data Masking.
For a step-by-step guide and further information on creating data masks, see Creating a Data Mask.
Validations
Validations in Salesforce evaluate the data to meet defined field guidelines before saving the record.
Salesforce App disables the following Salesforce validation types by default during a Data Seeding:
Validation Rules
Required Fields
Lookup Filters
Allow Re-parenting Fields
Unique Fields
Hidden Fields
Read-Only Fields
Inaccessible Record Types
Triggers
Flows
Duplicate Rules
Workflows
Perform the following steps to disable or enable a validation.
On the Validations tab, select a Validation Type from the dropdown list. A metadata list for the selected validation type appears.
Enter the search criteria in the Search By Name search box to filter from the metadata list.
Click Save Selection.
Some points to consider when enabling or disabling validations:
The application does not support disabling certain metadata types for metadata defined in managed packages.
The application supports enabling or disabling validations associated with standard fields.
For example, if a managed package validation rule exists on a Standard Object, the application disables it. Conversely, if a managed package trigger exists, the application cannot disable this trigger as only the managed package vendor can modify it. For more information about managed packages, see Manage Packages (https://help.salesforce.com/articleView?id=sf.managing_packages.htm&type=5).
Triggers are deployed using the Metadata API's Deploy() method. All other validation types are enabled/disabled using CRUD operations.
If the application is inactivating the trigger in Salesforce, the trigger must not have any active Apex Scheduled Jobs (Future methods) running.
The application uses the latest API version (i.e. Generally Available (GA)) when working with metadata.
If an object does not have a default record type defined, the application automatically sets the first record type as the default.
You can enable individual validations/validation types during the Data Seeding process. During a Data Seeding or sandbox seeding process, the application disables only the validations related to objects that are part of the Data Seeding process (except Process Builder and Visual Flows objects).
Validation Workflow
The following diagram illustrates the workflow when performing Data Seeding with validations.
Initiate the Data Seeding. A Data Seeding Confirmation window appears. For more information about the Data Seeding workflow, see Data Seeding.
Select Disable destination metadata during Data Seeding to disable the validations during the data Seeding process.
📝 Note
This option is disabled if the validations on the destination org are already disabled.
3. If the validations are already disabled for the org, the application does not disable/re-enable the validations.
4. If validations are not already disabled, the application retrieves the validations associated with the objects, which are applicable for Data Seeding.
5. The Salesforce App saves a backup of the validations, which will be disabled.
6. It will disable the validations on the destination org.
7. The Data Seeding operation begins.
8. The Salesforce App retrieves the validations associated with the objects, which are applicable for Data Seeding.
9. It enables the validations, which were previously disabled in step 6.
10. It also deletes the backup of the validations taken in step 5.
Manage Validations for an Organization
Perform the following steps to disable, enable, or refresh all validations for an org.
On the Salesforce App console, click Environment Hub.
Click the dropdown icon on the right for an org.
Select among the following options as per your requirements.
Option | Description |
Disable Validations | Disables all validations for an org. |
Enable Validations | Enables all validations for an org. |
Refresh Validations | Identifies all validations for an org. |
Batch Size
You can specify the batch size of records for copying data to a destination organization. By default, the Salesforce App loads a maximum of 200 records.
Perform the following steps to specify the batch size:
On the Batch Size tab, click New. A Batch Size window appears.
Select the object from the Environment Objects dropdown list.
Enter the batch size in the Batch Size text box.
Switch the Process Serial toggle to enable serial processing.
Click Save.
Object Child Discovery
You can discover the child objects of a parent object when working with junction objects from the Object Child Discovery tab. Junction objects are used to create many-to-many relationships between two objects in Salesforce.
For more information about junction objects in Salesforce, see Junction Object (https://help.salesforce.com/articleView?id=sf.relationships_manytomany.htm&type=5).
Perform the following steps to identify all child objects for a parent object:
On the Object Child Discovery tab, enter the search criteria in the search box. Alternatively, select an alphabetic character to filter the object.
Select the desired object and click Save.
Cloning a Template
You can clone the template by making a copy of the original template. Cloning enables you to create a template quickly with auto-populated configurations.
Procedure
On the Salesforce App console, click Restore & Seeding. The list of Templates appears.
Select the template you want to clone and click Show Details. The Template window appears.
Click Clone Template in the top-right corner. The Clone Template window appears with the auto-populated fields for Project, Name, Source Org, and Destination Org. For more information about assigning values to these fields, see Configuring a Data Template.
Select the required items in the Items to Clone list:
Field | Description |
Data Record | Data records of the template. |
Query Selection | Select records dynamically by creating an object query. |
External Id | External IDs to create the parent-child relationships between the source and the target object. |
Excluded Object | Excluded objects from Data Seeding into the destination org to consider data storage limitations or to filter sensitive data. |
Excluded Field | Excluded fields from a Data Seeding. By default, the Salesforce App excludes the following types of fields:
|
Data Translation | Transform field values for an object from the source environment and replace them with data to match the requirements of the destination environment. |
Batch Size | Batch size of records for copying data to a destination organization. |
Validations | Validations to evaluate the data to meet defined field guidelines before saving the record. |
Data Mask | Data masking replaces sensitive information while copying data. The data mask configurations that were created in the original template will be added as a part of the cloned template. |
5. Click OK. The template is cloned and appears in the templates list.
Schedule Sandbox Seeding
You can now automate seeding data into your sandboxes using the template scheduler. Using the template scheduler, you can:
Copy data to your sandboxes on a daily, weekly, or monthly basis
Ensure your sandboxes have adequate test data at all times
Accelerate your Salesforce development tasks as operational data is available regularly to your sandboxes.
Let’s see how you can schedule a Data Seeding activity.
Procedure
On the Salesforce App console, click Restore & Seeding.
Click Scheduler. The Data Template Scheduling window appears.
3. Click New Schedule.
4. Enter the following details in the Data Template Selection area.
Field | Description |
Project | Select the project, which includes the data Seeding template. |
Data Template | Select the Data Seeding template using which you want to schedule a Data Seeding job. |
Source Org | Select the source org from which you want to copy the data. |
Destination Org | Select the destination org to which you want to copy the data. |
5. Click Add to include the data template in the schedule.
6. Enter the following details in the Scheduling area.
Field | Description |
Repeat every | Specify how often your Data Seeding schedule should execute. |
Start Date | Select the start date for the Data Seeding schedule. |
Start Time | Select the start time for the Data Seeding schedule. |
End Date | Select the end date for the Data Seeding schedule. |
7. Specify the following details in the Record Selection area.
Field | Description |
Run Query-based Selection | This setting is disabled by default. When enabled, any queries defined in the Query-based Selection section within the data template are executed prior to starting the Data Seeding operation. For more information, see Query-based selection. |
Clear existing record selection | This setting is disabled by default. When enabled, any records which are already added to the data template will be cleared.
You can disable this setting if you have already selected specific records, which you want to copy. You can also optionally use the Run Query-based Selection option to append additional records to the data template. |
8. Configure the following details in the Data Seeding Settings area as per your requirements.
Disable destination metadata during the Data Seeding
Replace references to inactive user IDs with your user ID.
Replace inactive and blank record type IDs with the object's default record type ID.
Specify the email address to send notifications after the Data Seeding completion.
9. Specify the maximum number of attempts for the Data Seeding activity. This configuration will retry the Data Seeding activity for the specified maximum number of times in case of any failures.
10. Click Save Schedule.
The data template scheduler appears in the list of created schedulers on the left pane and executes according to the configured frequency. You can click on a scheduler to edit or delete it from the right pane.
You can navigate to the Tasks tab from the Salesforce App console to view the following task details:
Task | Description |
Data Template Scheduler Post | Provides details of the data template scheduler. |
Data Seeding | Provides details of the Data Seeding task, such as status and reasons for failure, if any. |
For more information, see Managing Tasks.
Using Regular Expressions
The following features in the Salesforce App enable you to use regular expressions to match patterns and substitute values.
External IDs
Data Mask
Data Translations
Regular expressions are patterns applied to strings to match character combinations. This is used to apply actions to data.
For example, you may want to change the suffix of all the user email addresses in your data set from .source to .destination.
You can apply the following RegEx pattern to the data set and use ‘destination’ as the substitution.
Original Data Set | Applied RegEx | Resulting Data Set |
(.)@(.) or (\d{3})[^0-9]*(\d{2})[^0-9]*(\d{4}) |
💡 Tip
To help you construct your regular expression, see Regular Expressions (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions) for a detailed guide on how to create regular expressions. Additionally, RegEx Tester (https://www.regextester.com/) is a useful tool to test the created RegEx.