New Features
#3496: Improved batch email notifications
It is now possible to configure batch email notifications in a more precise manner. For example, you can set up notifications only when certain origins or pipelines run, or when certain storage tables have been updated. You can tweak it further based on whether a batch results in data changes or not, the issue level (warnings and/or errors), whether the batch was invalid (problem with pipeline or source data) or resulted in a system error.
The email notification itself has been improved and includes more information about the batch.
A list of your email notifications and a sentence-based summary of each is available. Note that it is not possible to unsubscribe from Approval Requests, since not being aware of approvals could be a blocker to data loading.
The how-to article on email notifications provides further details.
#2771: Custom Data Entry Forms
Since version 4.15 it has been possible to edit a single record of a table in form layout. This is because every table automatically has an unmodifiable "standard data entry form" which exactly matches the field structure and sequence of its model. See the how-to article on standard forms .
Version 4.18 introduces “custom data entry forms”.
Like standard forms, custom forms target a single storage table, but several aspects of the form can be customized (which fields, sequence, field labels, display read-only fields, set default values, conditionally hide fields, filter items in foreign key dropdowns, add validation expressions and group fields).
Please refer to the how-to document on how to create, use and customize data entry forms.
Essentially, standard data entry forms are for internal data managers making direct changes to storage tables, while custom data entry forms are for external data providers submitting data to your mart. A summary of differences between standard and custom data entry forms is described in the xMart data entry forms article.
Enhancements
#3640: OData API link reflects UI table filters
It is now easier to generate a filtered and ordered OData API hyperlink. On the data view page, adjust filters and sort order in the UI as normal. Clicking the API button will generate a hyperlink that includes the filter, sort settings, page size and page number of the gridview.
For example, to produce an OData URL of WPRO countries:
- browse to the REF_COUNTRY table in REFMART
- filter by ‘WPR’ and adjust page size to 1000
- click the API button
Generated URL of WPRO countries: https://frontdoor-r5quteqglawbs.azurefd.net/REFMART/REF_COUNTRY?$top=1000&$filter=contains(GRP_WHO_REGION,%20%27WPR%27)
#3564: Filter issues during batch loading
When viewing issues during loading on the batch preview page, it is now possible to filter the issues by clicking on a relevant tag. Each tag corresponds to a particular pipeline command such as a validation command (ie TestRow. TestValueInRange) or a lookup command (ie MartLookup, HierarchyLookup). The text shown on the tag can be customized in the pipeline by editing the tag attribute of the command:
#3743: Can filter batch issues list
When there are many hundreds or thousands of issues, it helps to filter them. This also helps when issues are delegated to different groups as with some Polio datasets managed in AFRO: data is loaded at the IST level but resolution of subnational geographic data is delegated to countries. The filter helps each country focus on their own issues.
#3769: Excel export: change exported date format to be understood by Excel
Exported dates are now better understood in Excel.
#3216: New <LookupInterval> element on the HierarchyLookup to manage effective dating (to filter on end-dated geoplaces)
Some lookup tables use effective dating; they have a start date and an end date to indicate when a record is effective. GEOGRAPHY TABLES When doing a lookup on such tables, we only want to do a lookup on records which are effective for the source/business data record that is being loaded. For example, when loading case-based data containing a date of onset field, we want to RETRIEVE THE RIGHT RECORD.
A new <LookupInterval> element has been added to the HierarchyLookup command which uses a date column in the “source” data being loaded to filter
#3448: HierarchyLookup supports remove rows option when lookup fails
In a pipeline, when the HierarchyLookup command fails to resolve, there is now an option to remove the source row from the data load rather than ignoring it or raising a warning.
HierarchyLookup.RegisterMissingAsIssues was previously a Boolean and is now a list of values. Note that this change has been automatically applied to existing pipelines so no action to take in this case. However, exported pipelines saved externally would need to be updated.
|
Old value |
New Value |
|
False |
None |
|
True |
Warning |
|
n/a |
ErrorRemoveRows |
#3631 Support mart pipeline variables in package export / deploy
Mart-level pipeline variables are now supported during deployment (ie mart export-import). Only the names of the variables are exported, not the values. This allows different environments to have different values.
#3728Batch preview issue resolution - showing issue resolved despite not being resolved
Sometimes it is not possible to resolve a foreign key lookup issue simply by creating a synonym or adding a new record to the lookup table. The system now shows a message to this effect rather than indicating that the issue “is resolved”.
Other Changes
#3673: Data entry for failed lookups: pre-fill Synonym filter
#3712: Active roles selector: apply changes when panel is closed
#3725: DbMigrate: output the SQL upgrade script, and ability to run it
#3755: Mart Import fails to import package for newer version of xmart with new permissions
#3719: upgrade dotnet core version following security alert
#3618: Top level Data menu will now be hidden if user has no DATA_BROWSE permission in the mart
#3660: Load page keeps displaying the Data tab
#3667: Filter popup for type NUMBER UI glitch
#3656: Backup files were not being cleaned up properly
#3628: Public OData endpoint not refreshing
#3455: Admin cleanup page too slow: load in chunk
#3586: CATALOG_CATEGORIES: remove Sys_Deleted* columns from model
#3565: Upgrade angular v14
#3789: Need more specific message or catch during form definition validation
#3783: Custom forms URL param initialization failures for some field types (BOOLEAN / DATE)
#3761: Form input: hide read-only label in "View" mode (the whole form is disabled, not each field)
#3775: Fix date picker opening position
#3693: Data Edit Hide "beware" message if you are the editor
#3640: In Data View, generated API link should reflect table filters
#3788: Batch preview Filename changing when approvals happen
#3786: Data view: date filter end dates is not inclusive
#3780: Expand width of launch origin dropdown on pipeline page
#3772: Tags Not Assigning to Pipeline if pre-selected
#3771: Notifications: autocomplete does not seem to work as expected for Code / Title
#3794: Catalog : register new custom view does not save Discoverability settings properly
xMart Service Team