Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

Overview

This document describes the best practice in parsing csv files

General Tips

  • Parsing csv with header extraction from the file as of release 6.2.x does not work when used on large files and multiple smaller files

  • Recommendation: parse as csv with skipping header in Wrangler. This would entail the following steps 

    • Add a filter condition to skip header

      • filter-row-if-true offset == 0

    • Drop offset

      • drop offset

    • Parse as csv skipping header

      • parse-as-csv :COLUMN 'SEPRATOR' false

    • Rename the parsed entities in wrangler (using set-headers)

      • set-headers COL1,COL2,…,COLN


  • No labels