Why comma as a column separator

[nfcg]

Hi,
In the ipverse/asn-info repo the CSV currently uses comma (,) as a column separator.
This creates a problem in some lines in the description column, as it contains commas.
Why not use a semicolon as a column separator?

[trick77]

Hi, thanks for pointing this out.
The use of a comma as the separator is intentional and mainly historical. This file follows the same format as other established ASN/IP lists, and keeping it identical allows it to be used as a drop-in replacement without requiring changes on the consumer side.
Descriptions containing commas are therefore not expected and should be treated as data issues. I’m happy to fix those, but I’ll need concrete examples.
Could you please point me to the specific entries or line numbers where this causes problems?
Thanks for the feedback.

[nfcg]

Importing the CSV file to SQLite will throws the error/warning "expected 3 columns but found 4 - extras ignored" and report the lines.
e.g.: sqlite3 as.db -cmd ".import --csv --skip 1 as.csv asn_info" '';
lines.txt: https://gist.github.com/nfcg/703a99659833bd22d974465a9567e0a6 (--skip 1)

I am currently using this command:
sed -E 's/^([^,]+),([^,]+),(.*)$/\1,\2,"\3"/' as.csv > as_fixed.csv
which adds (") to the data in the description column and solving the issue this way....

[trick77]

I can't help you with that import but regarding the comma, a quick csvtool col 3 as.csv | grep ',' doesn't show any.

[trick77]

That actually was a dumb test, let me have a look at it again.

[trick77]

While the bug was fixed in August, existing database entries were not migrated. This has now been corrected. Thanks for the heads-up.

[trick77]

Btw. the list gets exported to JSON too now if you don't like (faulty) CSV's.