Hi,
I am working on the Job where I am loading the data from the flat file(.csv) to Sybase IQ table.
Flat file having 94 column with 4+millions record.
Design:
Its having two dataflow first with bulk load option its taking just some minutes to load to Sybase IQ table.
Second flow is having order by and gen_row_num_by_group() in one query transform to find the duplicate and validation transform to remove the duplicates and then target table of Sybase IQ.This flow is taking hours to load the data without bulk load option.
Is there anything which I can do to improve the performance?.
Thanks