2

I have a CSV file of more than 5 billion entries in it. I want to create a line or area chart for this data (data row number as x, value as y). Which software I can use to do so?

The file has only one column and all values are floating values with 3 decimal points. The file size is 34GB.

Sample:

23287.323
50982.287
71124.691
[...]
  • 2
    We'll need much more information to provide a solid recommendation. Please, take some time to read through our question quality guidelines and edit your question to match them. What requirements do you have? What would be nice to have, but not required? What operating system should it run on? After you edit your question to contain more information, we can reopen it. Thanks! – Undo Oct 24 '15 at 18:01
  • 3
    Again, as @Undo said: Which OS? Price? Do you have a budget? Are there any additional requirements? – ᔕᖺᘎᕊ Oct 24 '15 at 22:15
  • I am on MAC OS X 10.11 – Hemen Ashodia Oct 26 '15 at 7:13
  • So you just want a (any) program that will do that on your mac? What do you want to do with the chart afterwards? – user416 Oct 26 '15 at 8:46
1

The fastest way to generate this graph is probably with gnuplot.

Let's say your data is in a file called data.csv. Then put the following script in a file called graph.gnuplot:

set yrange [0:100000]
set grid
set term png
set output 'graph.png'
plot \
  0.01 with lines lt 1, \
  'data.csv' u 0:1:2 with line lt -1 lw 1

Then run this command:

gnuplot graph.gnuplot

And the graph will be generated as graph.png. You will probably need to adjust yrange (and maybe other variables) so give it a try with just a few values first.

| improve this answer | |
0

Conceivably reading the file into Python's Pandas DataFrame object is better than using just gnuplot:

http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html#pandas.read_csv

The advantage is that you can slice, dice and display data further using Pandas facilities. QtConsole is very useful interactive shell for this kind of work.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.