Generally source files are one of the main data source for ETL.
We can load this files as full load and delta load.
This document will give clear understanding for beginners.
Main Points in text file loading:
1) Choosing right delimiter
2) Data Type of the columns
3) Error Handling
4) Delta Loading
Go to Format Tab of the Local Object Library
Select Flat Files
Right Click and Click on New
Fill the options as shown in below screen
I have taken EMP table data in text file for our easy understanding.
Browse the Text files source folder for Root Directory
Select the File Name
Select the delimiter Tab.
We can use custom delimiter also.Here we can see default delimiters comma, semi colon, Tab and space.
I have chosen Tab.
Give the appropriate data types
For First load select the choose the components as shown below components.
Once first load is completed, we can use delta load as shown below:
We can see EMPNO as the Primary Key/Unique record.
So,In table comparison follow the below things:
Table Comparison Tab: Point to EMP table for destination table
Comparison method: Row-by_row select ( Based on your requirement, for more info refer Designer guide)
Input primary key columns : EMPNO
Compare columns : Bring the below columns
ENAME
JOB
MGR
HIREDATE
SAL
COMM
DEPTNO
Use two Map_Operations one for Insert and another for Update the records.
We should use Key_Generation transform to load the Key Generation for newly inserted records.
Thank you.
Best Regards,
Arjun