We use microservice written in Crystal to parse large CSV files (about 1.5Gb). Some rows in these files may contain no closed " characters:
,Y,FEDERAL NATIONAL MORTGAGE ASSOCIATION "F,,
With Crystal default CSV parse settings this row and everything after it won't be parsed correctly because
DEFAULT_QUOTE_CHAR constant is equal to
". Of couse you can override
quote_char param in
CSV contstructor with something that cannot be found in your document.
From my point of view the best is to use zero byte which is
'\u0000' in Crystal.
csv = CSV.new(file, headers: true, strip: true, quote_char: '\u0000') while csv.next # ... end