New Features

Ignore lookup issues

When resolving lookup issues during a data load it is now possible to ignore/hide lookup issues. Ignored issues are hidden from the list in subsequent uploads for this origin. Issues can be un-ignored at any time.

The feature can be used with some lookup commands (MartLookup, HierarchyLookup and SysIDLookups) but is particularly useful when subnational place names are being resolved with the HierarchyLookup command. In this case, it may be impossible to resolve some subnational place names until the underlying geo-reference data is updated to include new places, while other issues are resolvable, like misspellings of places which already exist in the underlying geo-reference data. When the geo-reference data is updated, issues can be un-ignored.

For more information, please see the how-to article on ignoring lookup issues.

Enhancements

Data Change Notifications

Support row-level security (RLS) in data change notifications

Data change notifications now apply any row-level security settings defined on a table. Instead of being notified when any change happens in a table by anyone, you will only be notified if there are changes to rows to which you have access.

Pipelines

Pipeline editor auto-save feature

The pipeline editor is now equipped wth an auto-save feature. If an unexpected shutdown, browser closure, end of session, etc. happens, you will be prompted to restore unsaved changes from the previous session.

pipeline auto-save

FindReplace support for multiple columns

The FindReplace transformation command now accepts a comma-separated list of values in the Column property. This way, FindReplace can be applied to a set of columns in one statement rather than having a long list of FindReplace.

PostRun > RunPipeline: specify batch comment for triggered pipeline

When triggering a subsequent pipeline in the PostRun section of a pipeline, it is now possible to define the batch comment.

<RunPipeline OriginCode="XXX" BatchComment="my comment" />

Option to generate warnings in RenameColumnsByLookup

It is possible now to produce warnings from the RenameColumnsByLookup command by configuring a new RegisterMissingAsIssues property. If a source columnn is not found in the lookup table, a warning or error can now optionally be displayed to the data loader.

Other changes

4.25.1 - 4.26.0

  • #5089: Data Change Notification emails not being sent
  • #5111: GetMart - default DATE formatting is confusing
  • #5096: Make explicit LoadTable optional when using Load Auto
  • #5091: Private OData API returning http 500 rather than 403? -> timeout exception
  • #5030: api/catalog/view_info errors
  • #4794: Refactor IsFileBased flag at Origin level instead of pipeline_draft / published
  • #4943: Prevent model update deadlocks
  • #5064: BPK uniqueness not being respected (duplicates after PK change)
  • #5086: AddColumn - weird behavior and ability to override existing values
  • #5051: AddColumn - validate expression during pipeline Validation
  • #4759: Upgrade to dotnet 7 then 8