Quantcast
Channel: SCN : All Content - Data Services and Data Quality
Viewing all articles
Browse latest Browse all 4013

order preservation in data flow

$
0
0

Hi Experts,

                I would like to know whether the order of data flowing through a dataflow is preserved or not.

 

consider the scenario as follows.

I have a source table

I fetch it to a query transform to set an error flag based on some column values(say if column1 or column2 or column3 is null then error flag is E else null)

Then I have a case statement to split the flow into two based on the error flag value( If Error flag is E then Error else success )

on both the flows i am having one table comparison each before loading it to target tables.

 

Since i am using sorted input option in table comparison I need the data to be sorted. Should i do it separately using one query transform each before the Table comparisons or can i use an order by in the first Query transform itself. Also If i am using the order by in the first query transform, Will this operation be pushed down to the DB?

 

Thanks & Regards

Alex Oommen


Viewing all articles
Browse latest Browse all 4013

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>