Hi there,
I have a request to group an in-bound dataflow by two fields combination, no aggregation is really needed (so you can also call it a sort by). The challenge is to output the dataflow into multiple files with all records with all possible combinations on the group by. Simple Example:
1 A a 1 A a
2 A a 2 A a
3 B b ->>>> 5 A a 1,2,5 in one file
4 B a 3 B b 3 in one file
5 A a 4 B a 4 in one file
Data profiling tells me that the real case has over 80 possible combinations for the group by fields. So if to hard code it with where clause in Query I need to build 80+ queries as well as 80+ flat files as output. If the combination grows in the future not easy to maintain as well.
So is there a way to auto split the data flow based on possible value combinations on certain fields? The Case transform can save some time but you still need to key in the 80+ conditions on the field values...
Thanks in advance!