Dataflow: How to store data from integrations

Intent

Templates and text properties can be completed manually or through automations like the “Import reconciliation data” functionality. Both approaches assume some level of user control over the data. Fully automated processes can also populate text properties through syncs and integrations.

Because sync and integration data originates outside of Silverfin (and may update frequently, e.g. daily), users typically have less control over its validity and the downstream consequences it can trigger.

Therefore, as a template author, it is important to be deliberate about where you store data that comes in through syncs and integrations.

There is no single “right” approach: different solutions fit different use cases. This pattern describes the available options, their pros and cons, and examples of where each could be used best.

Problem

Having data from outside of the platform flow in can lead to several UX issues, for example:

  • Users don’t realise data is being overwritten.
  • Updated data triggers consequences further downstream in the data flow.
  • Missing overview of where the data is stored and how it is used.

Solution

In this pattern, we discuss different ways to store data from syncs and integrations, list benefits and risks for each solution, and provide an example use case.

Data stored in people drop

Sync/integration data can be stored in predefined methods or customs under the people drop.

Usage

  • Data from admin syncs

Benefits

  • Easily accessible
  • Data is centrally stored
  • Data can be accessed directly from database across templates (i.e. no results needed)
  • Banner showing (TBC - can be that it’s always the case)
  • Period specific

Risks/downsides

  • Rollforward/copy data
  • Make non-period-specific data period specific
  • Insights?

Data stored in company drop

Sync/integration data can be stored in predefined methods or customs under the company drop.

Usage

  • Data from public government syncs
  • Admin syncs

Benefits

  • Not period specific
  • Easily accessible
  • Data is centrally stored
  • Data can be accessed directly from database across templates (i.e. no results needed)
  • Banner showing (TBC - can be that it’s always the case)

Risks/downsides

  • Can only be added to templates as fixed data or default
  • Not period specific
  • Insights?

Data stored in text property under period drop

Sync/integration data can be stored as text property under the period drop

Usage

  • Data needed across templates
  • Admin data

{{ period.custom.some.thing }}

Benefits

  • Data is centrally stored
  • Data can be accessed directly from database across templates (i.e. no results needed)
  • Less dependencies
  • Banner showing (TBC - can be that it’s always the case)
  • Period specific

Risks/downsides

  • It is not clear for a user that data will not copied when displayed in a template when using the copy data functionality
  • Delete data

Data stored in reconciliation template text property

Sync/integration data can be stored as text property in a predefined template

Usage

  • Data needed in a specific template

Benefits

  • Stores data exactly where it’s needed
  • Rollforward, copy data, delete data
  • Banner showing (TBC - can be that it’s always the case)
  • Period specific

Risks/downsides

  • If for some reason data is needed elsewhere it can become a bit cumbersome (since we need to fetch results or access text properties from other templates which created dependencies)

Data stored in account text property

Sync/integration data can be stored as text property in a predefined account

Usage

  • Transaction data related to an account that is not supported in the platform

Benefits

  • Stores data exactly where it’s needed
  • Banner showing in case admin sync
  • Period specific

Risks/downsides

  • Data is stored at ‘lowest’ level - meaning if we need it anywhere else it can become quite cumbersome as well (since we need to fetch results or access text properties from other templates which created dependencies)

Examples / Applicability

Once decided where to store the data, you should decide on how to display the data. This can be done in three ways:

  • As default to a text property
  • Display as an editable text property
  • Fixed value

All of these solutions bring in benefits and downsides on their own. Depending on the use-case one approach can be preferred over another.

  • As default to a text property
    • (+) Users cannot overwrite synced data
      • Users cannot overwrite synced data
    • (-) Once overwritten, it’s unclear if source data has been updated
      • Once overwritten, it’s unclear if source data has been updated
    • (-) Banner not showing
      • Banner not showing
  • Display as an editable text property
    • (+)Immediately clear for users in case source data has updated
      • Immediately clear for users when source data has updated
    • (-)User can overwrite synced data - which can be reverted when the sync/intergration runs again
      • Users can overwrite synced data, which can be reverted when the sync/integration runs again
  • Fixed value
    • (+) Users cannot overwrite synced data
      • Users cannot overwrite synced data
    • (+) Immediately clear for users in case source data has updated
      • Immediately clear for users when source data has updated
    • (-) Users cannot correct or change in case synced data contains mistakes or typos - should be corrected in the source where data is coming from
      • Users cannot correct mistakes/typos in Silverfin — issues must be fixed at the source
    • (-) No comment box in review
      • No comment box in review