Pipelines can be tested from the editing page. There are 2 options at the top, Test which will run the pipeline in test mode and Debug which will add the debugging to it.

TestDebug_Start

For an unpublished pipeline, this will Validate, Save and then Run the pipeline.

If the pipeline has been published then the Debug will appear next to the Load button.

Test_Debug_Load_Debug

If there are many origins, you can select the origin from the Test button by clicking on the down arrow.

TestDebug_Test_Select_Origin

If the pupeline has been published then you can do the same from the Load button.

TestDebug_Load_Select_Origin

To use Debug mode, you can either click it before clicking Test or you can press Test and select the Debug from the Batch Start pop-up

Test_Debug_Set_Debug_After_Test

You can do this when you start or re-run any pipeline.

If you select Debug mode

TestDebug_Debug_Mode

When the pipeline runs, it will be in Debug Mode and so it will look like this.

Note that for an Unpublished pipeline, like this one, the Commit option is greyed out.

TestDebug_Unpublished_NoCommit

If you click on Preview the pipeline will run in Debug Mode.

Using Debug Mode

The debug mode screen looks like this:

Debug_Mode_Entry

If you press on the down arrow,

Debug_DownArrow

It will show the the step you are looking at. This is done in 2 ways, firstly, the row is highlighted and secondly, the Debug step appears in the Selected Debug at the top. This number corresponds to the DebugID field.

The data is the state of the data at that step.

Debug_Simple_Data

To move through the steps, you can use the up and down arrow keys at the top.

Debug_UpDown_Arrow

Column Selector on data panel

On the data panel, there is a column selector so enable you to select certain columns. This can be useful if you are doing a Transform. In this case, an AddColumn statement has been added to the pipeline. If you go to the line in debug mode, it will only show the result column.

Debug_Select_Columns_Orig

In order to see the source columns, use the column selector to pick the source columns.

Debug_Select_Columns

And then it makes the transform easier to understand.

Debug_Show_Columns

Leaving Debug Mode

THere are 4 buttons in the top left which are mainly about leaving Debug Mode

Debug_ther_Buttons

Batch Preview

Debug_Batch_Preview     goes to the Batch Preview Page

Load Again

Debug_Load_Again     Load the batch again using a new file

Replay

Debug_Replay     Creates a new batch using the same file as this batch

Pipeline Editor

Debug_Pipeline_Editor     Goes to the pipeline editor for the current pipeline.

Getting back to Debug Mode

The symbol for getting back into debug mode is this one

Test_Debug_View_Debug_Tables

It appears on the batch preview

Debug_Icon_Batch_Preview

And on the batch list

Debug_Icon_Batch_List

How to Publish an Unpublished Pipeline and Commit the Data

Once a pipeline has been run in Test mode, you have the option to publish it and commit the data

Test_Pipeline_Publish_Commit

How to Tell if a Batch was run in Test Mode

There are 2 ways to tell if a batch has been run in Test mode.

Firstly, on the Batch Preview, there is a Test Mode icon

Test_Icon_Batch_Preview

It is also shown next to the batch status in the Load list

Test_Debug_View_Debug_Table_Publish_Commit