New Features
Row-level security (RLS)
Row-level security (RLS) permits user access to some rows in a table or view while denying access to the other rows. “User access” in this case could mean viewing data, loading data using custom pipelines, editing data (in UI or via export-import), form-based data entry and/or consuming the OData API.
RLS was designed to allow multiple countries to store data in the same table, while not being able to see or change data of other countries. But it is generic in nature and can be used for any similar scenario (security by major office, security by org unit or team, etc.).
The row-level security how-to article provides full details on how to set it up and use it.
Enhancements
Data catalog: display objects the user can access
To improve the usefulness of xMart’s data catalogs, by default, users now only see views and tables which the user has access to. Protected (private) tables and views are hidden unless the “Show protected views” checkbox is checked. Checking this box reveals the metadata (titles, descriptions and tags) of all objects for discoverability purposes, but does not mean that the user can access the content of those tables.
Export staged data
It is now possible to download staged data during batch preview. Staged data continues to be downloadable until it is removed from the server (~ 3 days after batch commit).
Exported staged data includes all data columns in the format of the target table, plus a number of system columns that indicate the result of the analysis performed by the system for each record.
Please consult the export staged data how-to article for more information and details on the system columns included in the export.
Process multiple files in a zip file using GetZip
Not really new behavior, but now the pipeline script guide explains how to process multiple files in a single zip file using GetZip. See Example 3.
As usual, use the file-specific Getter inside GetZip, ie GetText, GetExcel, etc.
Extract data from older EpiInfo Rec files: GetRecfile command
Older EpiInfo Rec files can now be processed using the GetRecFile command.
RecFiles contain both the data as well as the structure of the data and uses a unique fixed-width format.
GetRecFile can be used inside the GetZip command.
If you are curious, a peek at the internal structure of a .REC file:
Send email upon approval failure
When a batch is rejected during the approval process, an email notification is sent to the person who created the batch.
Very large files can be exported/downloaded in user interface
Previously, exporting very large tables/views (> 500 Mb) might fail (“stream too long” error). This has been improved so that even 2 Gb or larger files can be exported. Larger files will now automatically be downloaded as a zip file while smaller files continue to be downloaded uncompressed so they can immediately be opened by desktop applications without (ie Excel) without the need to unzip first.
Faster REPLACE mechanism when whole table is being replaced
When using the REPLACE load strategy in a pipeline to replace the entire table (a common scenario), the performance has been improved.
Other changes
- #3995: Hide Edit Model button user has no permission
- #3988: Data edit: user cannot clear a DATE field
- #3906: Prevent duplicate row counting
- #3879: Intra-batch FK fails with implicit sysidlookup and auto column mappings
- #3986: Batch Preview optim Sort by BPK
- #3990: Hotfix/uat refmart optim: FillStrictTable by chunk
- #3809: Improve Data Catalog page load time
- #3911: Date field is always required even though populated (manually)
- Disable rollback if table is referenced by others table
- #3907: Counting rows error even though rows are counted
- #3890: Catch this special invalid xml error in pipeline
- #3895: Sys Admin Import error message isn’t very useful.
- #3896: Mart pipeline import broken because of MART_VARIABLES missing Type (SQL and pipeline already hotfixed)
- #3882: Update filtering by TABLE_CODE + UI glitch fix
- #3934: GetWebService can’t handle gzip compression