Data Corruption LZ4

[klauspost]

Using https://files.klauspost.com/compress/nyc-taxi-data-10M.csv.zst (uncompresed)

Execute:

lz4c compress nyc-taxi-data-10M.csv&&mv nyc-taxi-data-10M.csv nyc-taxi-data-10M.csv.org&&lz4c uncompress nyc-taxi-data-10M.csv.lz4
nyc-taxi-data-10M.csv.lz4 32.47%
nyc-taxi-data-10M.csv  99% |███████████████████████████████████████ |  [1s:0s]lz4: invalid frame checksum: got a106b1a0; expected 4608b3de

Blocks appear to be out of order or something else is off. Reproduces on master, Go 1.16.2, Windows 10, 64 bits.

[pierrec]

Interesting. Will work on it when time permits.

[rvrangel]

this seems to be happening when concurrency is used (if disabled via flags, this error doesn't happen). I put a PR with a possible fix: #127