I have a few lists of 2 million rows or so consisting of:
Name, Address, Phone Number
but these have errors like multiple commas in a row.
When I import these into postgres as .txt and comma delimited it shows errors "Extra data after column Phone". I then have to go find the line manually, edit out the extra comma, then reupload/repeat. It's been 7 days and I'm still working on the same list and I'm not even past 250k records. This is really obnoxious and there has to be a way to fix this.
Thoughts and suggestions? Don't mind paying for good software.
Maybe there is a program similar to Excel for this kind of work? Something that can handle millions of lines with similar functions/tools as Excel that will allow me to edit this large data sets easily? Excel is good for my datasets that are less than 1 million lines, but can't open larger ones.
I tried the delimit and emeditor editors, but they didn't work for me.