Quantcast
Channel: SCN : All Content - Data Services and Data Quality
Viewing all articles
Browse latest Browse all 4013

Nested fixed format file

$
0
0

Hi Team,

 

I got a requirement to produce a fixed format file, which will have a nested structure. Data source is XML file with following structure.

 

Root

     +ECOS (Record type 100 to 104)

          +PARTS (Record type 200 to 204)

               +FUNCTIONS (Record type 300 to 304)

 

So, there are three levels. Data can have multiple ECO which can have multiple PARTS which can further have multiple function within it..

Each level got 5 records with different length and mapping.. So a sample OP data for an ECO with a Part which has a Function would look like..

 

100 ECO attributes..............

101 ECO Description

102 ECO Change log information

103 ECO owner, activity

104 ECO release information

200 Parts attributes..............

201 PartsDescription

202 PartsChange log information

203 Parts owner, activity

204 Parts release information

300 Parts Function attributes..............

301 Parts FunctionDescription

302 Parts Function Change log information

303 Parts Function owner, activity

304 Parts Function release information

 

To achieve this, I used three nested while loops. Process is like ..

1. Count total ECO and get first ECO as a current

2. Write current ECO data to file

     3. Count Parts for current ECO, get first part as a current

     4. Write Current Part data to file

          5. Count Function for current Part, get first function as a current

          6. Write Current function data to file


As all these 15 records type got different length, I created separate mapping, added ranking and then concatenated in a 2064 char field before writing to file in each data-flow.

 

It's working fine, but taking much time. for e.g. for 1 ECO with 46 parts and where each part has 1 function means 46 function it's taking 9 minutes..

I have to generate this file each 15 minutes and data I used is smaller. I may get 1000 Parts in a sinigle ECO.

 

Please suggest any better way..


Viewing all articles
Browse latest Browse all 4013

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>