Disable "Pre-process incoming data" in Configurable SynchronisationsDisable "Pre-process incoming data" in Configurable Synchronisations

What

In the Configurable Synchronizations there is a setting called “Pre-process incoming data for data changes to reduce the amount of logging and speed up the synchronisation". This is designed to ensure best possible performance on the sync jobs. It marks the content with a hash value and uses this value to calculate whether the content needs to be updated on the job run. 

• The setting is enabled as default and was introduced in 5.12.X.
• It is not supported by all data providers, only Database (views) and XML upload.
• If enabled, the synchronisation will scan the incoming data for changes before trying to synchronise the data.
• You will see a hash value added to the content, which is generated from the Person synchronisation source data.
• When a new synchronisation runs, Pure generates the hash again with the latest source data. If there are any changes made to the source data, then the hash will be different, and Pure will know that a change has been made.
• Only content with changes in the source will be updated.
• If needed, the setting can be disabled in the job configuration.
  • Go to Administrator > Jobs > Cron job scheduling and find the job.
  • Click on Edit configuration
  • Click on Synchronisation configurations
  • Toggle off: “Pre-process incoming data”
    

If this setting is off, hashes will be disabled, meaning Pure will run through each field in the source data and cross-reference it with each field on that Person. This means the synchronisation jobs will take longer. However, sometimes it can be useful to disable it.

• Because the hash value is used to skip content, you might find that some content does not get updated.
  • If a field is not being synchronised even though the configurations of the job should allow the synchronisation, and there hasn't been any manual changes to the field, then testing the synchronisation with the setting toggled off might help.
  • If the raw data is not changed, but the referenced data is changing, then it might be preferable to run the job with this setting toggled off - for example where the URL to a profile picture stays the same, but the picture itself can be changed locally.

The configuration is available on these jobs:

• Configurable Activity Synchronisation
• Configurable Course Synchronisation
• Configurable Dataset Synchronisation
• Configurable Equipment Synchronisation
• Configurable Ethical Review Synchronisation
• Configurable Event Synchronisation
• Configurable External Organisation Synchronisation
• Configurable External Person Synchronisation
• Configurable Impact Synchronisation
• Configurable Journal Synchronisation
• Configurable Organisation Synchronisation
• Configurable Person Synchronisation
• Configurable Prize Synchronisation
• Configurable Publisher Synchronisation
• Configurable Student Thesis Synchronisation
• Configurable Supervisor Synchronisation

Configuration changes and Pure upgrades will always trigger a full run of the synchronisation. If this setting is on when when Pure is updated from one version to another, all hashes will be regenerated, which means all Persons will be synced again. In this instance there will be records for which only the hash value has changed.

More information

General Configurable Synchronisation Errors