Skip to content

Commit 8f385ef

Browse files
committed
docs: add note about BOMs
1 parent 407d48e commit 8f385ef

1 file changed

Lines changed: 26 additions & 0 deletions

File tree

README.md

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -296,6 +296,32 @@ source stream can be done neatly with a modules such as:
296296
Or native [`iconv`](http://man7.org/linux/man-pages/man1/iconv.1.html) if part
297297
of a pipeline.
298298

299+
## Byte Order Marks
300+
301+
Some CSV files may be generated with, or contain a leading [Byte Order Mark](https://en.wikipedia.org/wiki/Byte_order_mark#UTF-8). This may cause issues parsing headers and/or data from your file. From Wikipedia:
302+
303+
>The Unicode Standard permits the BOM in UTF-8, but does not require nor recommend its use. Byte order has no meaning in UTF-8.
304+
305+
To use this module with a file containing a BOM, please use a module like [strip-bom-stream](https://github.com/sindresorhus/strip-bom-stream) in your pipeline:
306+
307+
```js
308+
const fs = require('fs');
309+
310+
const csv = require('csv-parser');
311+
const stripBom = require('strip-bom-stream');
312+
313+
fs.createReadStream('data.csv')
314+
.pipe(stripBomStream())
315+
.pipe(csv())
316+
...
317+
```
318+
319+
When using the CLI, the BOM can be removed by first running:
320+
321+
```console
322+
$ sed $'s/\xEF\xBB\xBF//g' data.csv
323+
```
324+
299325
## Meta
300326

301327
[CONTRIBUTING](./.github/CONTRIBUTING)

0 commit comments

Comments
 (0)