Hot answers tagged

3

Have a look at q q is a command line tool that allows direct execution of SQL-like queries on CSVs/TSVs (and any other tabular text files). https://github.com/harelba/q


2

Although not free, there's a cheap (~5$) option - Data Transformer (disclaimer - I'm its developer). It converts between CSV, JSON, XML, and YML locally. It offers a number of conversion settings (with sensible defaults) so you can match the data for your purposes. You can check it out on the Mac App Store or the Microsoft Store.


2

One possibility is a streaming XSLT 3.0 processor, which given your constraints means in practice Saxon/C Enterprise Edition (this has a Python language binding). There is actually a CSV-to-XML stylesheet published as a worked example in the XSLT 3.0 specification, but sadly no counterpart to do the reverse. However, you can see the principle in some of the ...


2

Pandas can do a lot of the work for you and is well worth looking at. Reading .csv files, (also .xlsx & others) Format conversions for date & value columns Filtering Are all available.


2

I've been using EmEditor for years. Can open practically any size file (up to 248GB realistically) and can split CSV and other delimited text into columns. With tons of other features. There's a free version, but if you need it for a company, then I highly recommend getting the lifetime license. https://www.emeditor.com/


1

if you need to process these in your intranet- get a server or PC with lots of memory. Command line tools such as grep, sort and uniq -c are a good first start to do simple analyses, assuming that the data is reasonably clean. Alternatively process datafiles in the cloud. New customers get a free tier there. Upload files to, say, Google BigQuery and process ...


1

The XML Utilities library is worth a try, assuming valid & flat XML structure - it even comes with a command line xml2csv utility. It specifically states: xmlutils.py is a set of Python utilities for processing xml files serially for converting them to various formats (SQL, CSV, JSON). The scripts use ElementTree.iterparse() to iterate through ...


1

Use the R function read.csv().


1

I'd suggest you look at Data Transformer (disclaimer - I'm its developer). It can convert CSV/JSON/XML to SQL. The generated SQL contains "insert" statements for each line and a "create table" statement. The app works offline, and your data never leaves your computer. You can get it from the Mac App Store or the Microsoft Store.


1

I would suggest keeping an eye out for pandas official documentation on IO. One's option keeps changing based on the development cycle and new formats get added all time. They also publish the benchmark.


Only top voted, non community-wiki answers of a minimum length are eligible