When reading from a flat file source or loading into flat files, delimiters' specification including text, column and row delimiters are essential to ensure that the data is not corrupted due to lack of delimiters. One such example is the text delimiter. The absence of text delimiter can cause the data read to be interrupted or result in error on cases where the data in a given field entry contains multiple lines thus having row delimiter in a given cell. The screenshots below provide an illustration of using the text delimiter in flat file.
The illustration in Fig 1 below shows that the absence of the text delimiter in the file format causes the reading of the flat file resulted in error.
Fig 1
The illustration in Fig 2 below shows that after introducing a text delimiter in the file format
Fig 2
There are several scenarios other than the one illustrated above where the delimiter setting in the flat file format can help in ETL involving flat file data from legacy systems or flat file targets.