68

At the office certain server log files grow way too big and trying to go through the logs can be a real pain, when for instance you have a several GB size text file and all the normal text editors load the entire file in memory, becoming really slow or crashing.

Thus, looking for a lightweight text editor that doesn't try to load the entire file at once, since we're generally only interested in a small subset of the data, going through logs that happened somewhere in the vicinity of a certain timestamp or then hopping through all the lines logged for a certain user over time.

Sure, this would be solvable with command line tools from say Cygwin or if we actually had SSH access to production servers, but looking for a GUI solution that also people who're less CLI savvy could use when a huge logfile gets dumped on their lap.

14 Answers 14

18

I had the same issue with over 10 gigabytes large MySQL log and found 010Editor with the help of this question on Stack Overflow. It opened up the file fine and fast. The search results are presented very well at the bottom of the editor and searching huge files is surprisingly fast.

There is a free 30-day trial, and the pricing for extended use can be seen here.

  • 1
    Could you elaborate a bit more about this software? What are your experiences? Where is this tool great? Have a look at Joshs Answer below which looks splendid now :) – Angelo Fuchs Feb 19 '14 at 18:41
  • 1
    @Angelo There's nothing more I can say about it. I used it for the trial period a while ago and my experiences with it are already mentioned in the answer, as are the few features I used (huge files, great search) – 3ventic Feb 19 '14 at 18:51
  • 1
    what about memory consumption? Can you filter the lines? Follow changes and have it make "beep" when something changes? Or if it can do nothing like this, that would be noteworthy too. – Angelo Fuchs Feb 19 '14 at 19:12
19

You wrote that a read-only app would work. Then I highly recommend you to try Lister.

It's a part of Total Commander but has standalone version. It's a small app that can view really huge files very fast. Features included:

  • Print file
  • Find in file (supports RegEx search)
  • Different view modes and charsets (you can see in screenshot)
  • Supports plugins for different filetypes and new features

Lister window

All that in ~500 kb file. For screenshot I opened VirtualBox hdd image which size is 10 Gb. Navigating in file is instant. Searching for string took less than 30 seconds untill first hit in the middle of file (I suppose that it depends on your HDD speed). And RAM consumption is 5.7 Mb for this file.

  • I confirm this is very good for big files and i am still looking for an open source alternative. – eadmaster Jul 5 '16 at 16:28
16

I use Microsoft's Configuration Manager Trace Log Viewer for exactly this sort of task. It's part of the System Center 2012 R2 Configuration Manager Toolkit (free download), but once you've extracted that, the cmtrace.exe in there is just 678KB and completely portable; you can drop it onto any machine and run it with no install. I keep a copy on my network home drive and most USB sticks that I own.

Though it comes as part of the SCCM toolkit, it happily opens any text file. The first time you run Cmtrace on a machine it will ask if you want to associate it with *.log files (or not). Then just has a big empty window ready to open a log file.

It "tails" log files, so you can see the live log data scrolling past (but also has a pause button, so you can actually read fast-moving logs). It automatically colour highlights lines with keywords like "error", "failure" or "warning" in, and you can set it to highlight other keywords.

While I normally use it to open log files of around 1MB, I've have often used it to open 500MB to 800MB JBoss log files in the past without any problem.

  • 1
    This looked so promising! It works great on small log files. It even highlights warning and error lines. But I tried it on a 242MB log file, and it failed to ever respond. I repeated the test several times, but it would not work. – RockPaperLizard Jul 16 '15 at 6:32
  • 2
    Maybe it works for logfiles, but it says "not responding" for me on a simple 33 MB text file. – Thomas Weller Jan 28 '16 at 8:41
  • Also known as Trace32. Unfortunately it is a lot slower than others like BareTail when loading huge log files. – KERR Sep 22 '17 at 14:52
11

I like to use LogFusion for this purpose.

Its targeted at being used with log files, which tends to be the majority of the giant text files I've encountered. But I've found it useful for all sorts of files, like SQL database script files, giant CSV files, server logs, etc.

It opens the file at the end of the file contents, and loads a small subset of the file into memory. The speed is with which it opens the file is independent of the file size. Scanning to specific points in the file using the scroll-bar is simple, and instant. Filtering lines out with keywords is nearly instant.

It really shines for log reading though. It has features which help with log reading, like a customizable highlighting filter, which lets you jump between highlighted entries, or show or hide only highlighted entries. The highlight filter would let you to tag a certain user, and only show log lines for them. Then you could scan through their timeline!

It also lets you have a live log in the viewing window so that you can watch the log entries roll in as they happen. It has an option to keep you at the bottom of the file so you always see the newest entries.

Another thing you were looking for is "light weight". I currently have a 4 GB log file loaded, and LogFusion.exe is taking a total of 112mb of memory. Opening the application with nothing loaded, it starts at 65mb. The LogFusion.exe file is a scant 1.6mb.

They have both Free and PRO licensing.
http://www.logfusion.ca/Compare/

11

Try EmEditor ($40 USD initally; Updates are $20/year after first year).

They highlight Large File Support (up to 248GB) as one of their biggest selling points.

I just discovered this editor recently when I was looking for an editor that can open open large files. Once every month or two, I seem to have a need to look at a large text file and I decided I would hunt down my definitive tool(s) for doing as such. At the time, I had a 52GB SQL database dump that I wanted to, at least, peek into. That's when I discovered EmEditor, and it is quite impressive in it's handling of large files. I was able to open that 52GB file quite easily. I was even able to use EmEditor to cut out sections of the file and "Save As" the cut down text to a new file (~20GB).

EmEditor with it's large file controller

  • 1
    +1. I tried it and it's really good at files of any size. – Martin Ba Mar 20 '14 at 6:13
  • I've been using it for years for all text editing. Huge fan. – Jon Sep 29 '17 at 2:29
9

You can use gVIM for Win32:

https://sourceforge.net/projects/portablegvim/

It uses virtual rendering and can easily handle 50+ GB files.
Works on our logfile dump (if you experience any trouble, switch off syntax highlighting)

enter image description here enter image description here enter image description here

  • 5
    Your log consists of TPL, Java and Python code? Cool! – Thomas Weller Apr 18 '16 at 17:27
6

Open source, but reader-only: glogg. Unlike 010 Editor, it doesn't have many features (I'm missing tabs), but it's free and never crashed on my computer. Also, glogg is available for Linux and soon Mac OS.

enter image description here

  • 1
    +1, it's a promising piece. (But note that it will (v0.99) for files with too many lines to fit in it's index. (usually multi 100M files with very short lines.) – Martin Ba Mar 20 '14 at 6:11
4

BareTail (free edition) is good enough for my log monitoring activities:

  • Real-time file viewing
    • Optimised real-time viewing engine
    • View files of any size (> 2GB)
    • Scroll to any point in the whole file instantly
    • View files over a network
  • Follow tail mode
  • Tail multiple files
  • Configurable highlighting
  • International character sets
  • Many file formats
  • Single small executable, no installer

and many other nice to have features.

2

If you really need an editor (and not only viewer), then you can go with PilotEdit Lite. I successfully used it to modify and save 1.6 GB log file.

2

Notepad++ 7 is also available as a 64-bit build, and I have tested it on 3+ GB file: considering the size of the file, it is doing a great job.

Even if it's a file editor and a bit slower than a read-only tool, not all free-for-commercial-use tools are supporting searching with regular expressions.

It also supports log monitoring (tail -f).

https://notepad-plus-plus.org/download/

1

I've been looking for text editors able to view and modify large files, and my opinion is:

The two best commercial editors are Editpad and Textpad, they use very little memory and can view, edit and perform many advanced tasks, such as comparing files, syntax highlight...

EditPad pro
https://www.editpadpro.com/
enter image description here

TextPad Pro
https://www.textpad.com/products/textpad/screenshots/index.html
enter image description here

Now, two programs that can work as regular editor but specially designed to work with csv files. The program can show the file as a spreadsheet. They don't have as many options as regular editor though.

EmEditor
https://www.emeditor.com/

I've had problems with it. Every few seconds it tries to reload or reshape the file and it doesn't let you work properly.

enter image description here

Delimit
http://delimitware.com/
enter image description here

All the aforementioned programs can deal with files much larger than memory using very few resources.

There are many other programs but most of them use a large quantity of memory.

Among the free editors I would choose gvim, it's interface may be not as comfortable for most Windows users but it works.

gvim x64
enter image description here You can find the 64bit version of this editor originally created for Windows here:
https://tuxproject.de/projects/vim/

1

If you are looking for a reader only:

HTMLPen.com is a free online tool that can open very large files(TB+) instantly, can do regex searches, and counts, export the matches. You do not need to download anything. It works on any OS with a modern browser. It also has syntax coloring if you open a data or code file.

0

You can use Large Text File Viewer (aka. LTFViewr5u).

As of July 2015, their site is down, so here is a link to an archive of the site.

  • view-only
  • free
  • Windows
  • portable and lightweight (700 KB)
  • designed for viewing large (>1GB) text files
  • uses little memory and is able to open a gigabyte file instantly (from the website's description and I can confirm: it uses 4 MB for a 500 MB file, which loaded in less than 5 sec on my SSD Crucial M5)

enter image description here

0

With ultraedit you can open and edit big files. It has a mode where you can tell it not to create a temporary backup file.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.