Setting escapcahr and quotechar to double quote acts unexpected

[zelima]

@anuveyatsu what is the reason using double quotes as escapechar, besides csv-parser using it as default? 398235d

Cause in python by default it's None https://docs.python.org/2/library/csv.html#csv.Dialect.escapechar

Cause it acts really weird if both, escapechar and quotechar are set to ". Eg take a look at this validation report: https://pkgstore.datahub.io/81429cbbddcfb180f54c142fac32f83b/schema/validation_report/data/874d49bd554630f5b536216ce390d4d9/validation_report.json
It thinks that everything after "{\"one\"... is one column. Same will happen even if it's quoted in a simple way, like "one, two"

I've just deleted that line from data.js and pushed, it's processed successfully https://datahub.io/zelima/schema/v/20

Do you think we can remove it?

Ans from @anuveyatsu

@zelima the reason IMO is that it is very common, e.g., go to google spreadsheet create a table with ” in values. Then export it as CSV and you’d see that ” was used as the escape char. I haven’t tried the same operation with Excel, but read in the web that it is the same.

Update

• I removed the line from above and quotechars are working fine
• Also created spreadsheet and exported into CSV(that automatically decides escapecahr to be ") - this worked fine as well
  • https://docs.google.com/spreadsheets/d/e/2PACX-1vTzTPL9ktfZ2dz38LeE5nu7sMiky13cm50XIGc_4yXqAK7hSuySlNl5Mpt8j9ReH7m3kZDZKRY76ief/pub?gid=0&single=true&output=csv

So think we don't really need to define it (and especially hardcode it to be ").

[zelima]

FIXED. We allow guessing the escapechar to the processing libraries (tabulator) in python for now. Issue for guessing it here #33