Text editor or reader for working with huge files in Windows

Question

At the office certain server log files grow way too big and trying to go through the logs can be a real pain, when for instance you have a several GB size text file and all the normal text editors load the entire file in memory, becoming really slow or crashing.

Thus, looking for a lightweight text editor that doesn't try to load the entire file at once, since we're generally only interested in a small subset of the data, going through logs that happened somewhere in the vicinity of a certain timestamp or then hopping through all the lines logged for a certain user over time.

Sure, this would be solvable with command line tools from say Cygwin or if we actually had SSH access to production servers, but looking for a GUI solution that also people who're less CLI savvy could use when a huge logfile gets dumped on their lap.

Answer 1

I had the same issue with over 10 gigabytes large MySQL log and found 010Editor with the help of this question on Stack Overflow. It opened up the file fine and fast. The search results are presented very well at the bottom of the editor and searching huge files is surprisingly fast.

There is a free 30-day trial, and the pricing for extended use can be seen here.

Answer 2

You wrote that a read-only app would work. Then I highly recommend you to try Lister.

It's a part of Total Commander but has standalone version. It's a small app that can view really huge files very fast. Features included:

• Print file
• Find in file (supports RegEx search)
• Different view modes and charsets (you can see in screenshot)
• Supports plugins for different filetypes and new features

All that in ~500 kb file. For screenshot I opened VirtualBox hdd image which size is 10 Gb. Navigating in file is instant. Searching for string took less than 30 seconds untill first hit in the middle of file (I suppose that it depends on your HDD speed). And RAM consumption is 5.7 Mb for this file.

Answer 3

I use Microsoft's Configuration Manager Trace Log Viewer for exactly this sort of task. It's part of the System Center 2012 R2 Configuration Manager Toolkit (free download), but once you've extracted that, the cmtrace.exe in there is just 678KB and completely portable; you can drop it onto any machine and run it with no install. I keep a copy on my network home drive and most USB sticks that I own.

Though it comes as part of the SCCM toolkit, it happily opens any text file. The first time you run Cmtrace on a machine it will ask if you want to associate it with *.log files (or not). Then just has a big empty window ready to open a log file.

It "tails" log files, so you can see the live log data scrolling past (but also has a pause button, so you can actually read fast-moving logs). It automatically colour highlights lines with keywords like "error", "failure" or "warning" in, and you can set it to highlight other keywords.

While I normally use it to open log files of around 1MB, I've have often used it to open 500MB to 800MB JBoss log files in the past without any problem.

Answer 4

I like to use LogFusion for this purpose.

Its targeted at being used with log files, which tends to be the majority of the giant text files I've encountered. But I've found it useful for all sorts of files, like SQL database script files, giant CSV files, server logs, etc.

It opens the file at the end of the file contents, and loads a small subset of the file into memory. The speed is with which it opens the file is independent of the file size. Scanning to specific points in the file using the scroll-bar is simple, and instant. Filtering lines out with keywords is nearly instant.

It really shines for log reading though. It has features which help with log reading, like a customizable highlighting filter, which lets you jump between highlighted entries, or show or hide only highlighted entries. The highlight filter would let you to tag a certain user, and only show log lines for them. Then you could scan through their timeline!

It also lets you have a live log in the viewing window so that you can watch the log entries roll in as they happen. It has an option to keep you at the bottom of the file so you always see the newest entries.

Another thing you were looking for is "light weight". I currently have a 4 GB log file loaded, and LogFusion.exe is taking a total of 112mb of memory. Opening the application with nothing loaded, it starts at 65mb. The LogFusion.exe file is a scant 1.6mb.

They have both Free and PRO licensing.
http://www.logfusion.ca/Compare/

Answer 5

Try EmEditor ($40 USD initally; Updates are $20/year after first year).

They highlight Large File Support (up to 248GB) as one of their biggest selling points.

I just discovered this editor recently when I was looking for an editor that can open open large files. Once every month or two, I seem to have a need to look at a large text file and I decided I would hunt down my definitive tool(s) for doing as such. At the time, I had a 52GB SQL database dump that I wanted to, at least, peek into. That's when I discovered EmEditor, and it is quite impressive in it's handling of large files. I was able to open that 52GB file quite easily. I was even able to use EmEditor to cut out sections of the file and "Save As" the cut down text to a new file (~20GB).

Answer 6

Open source, but reader-only: glogg. Unlike 010 Editor, it doesn't have many features (I'm missing tabs), but it's free and never crashed on my computer. Also, glogg is available for Linux and soon Mac OS.

Answer 7

You can use gVIM for Win32:

https://sourceforge.net/projects/portablegvim/

It uses virtual rendering and can easily handle 50+ GB files.
Works on our logfile dump (if you experience any trouble, switch off syntax highlighting)

Answer 8

BareTail (free edition) is good enough for my log monitoring activities:

• Real-time file viewing
  • Optimised real-time viewing engine
  • View files of any size (> 2GB)
  • Scroll to any point in the whole file instantly
  • View files over a network
• Follow tail mode
• Tail multiple files
• Configurable highlighting
• International character sets
• Many file formats
• Single small executable, no installer

and many other nice to have features.

Answer 9

I've been looking for text editors able to view and modify large files, and my opinion is:

The two best commercial editors are Editpad and Textpad, they use very little memory and can view, edit and perform many advanced tasks, such as comparing files, syntax highlight...

EditPad pro
https://www.editpadpro.com/

TextPad Pro
https://www.textpad.com/products/textpad/screenshots/index.html

Now, two programs that can work as regular editor but specially designed to work with csv files. The program can show the file as a spreadsheet. They don't have as many options as regular editor though.

EmEditor
https://www.emeditor.com/

I've had problems with it. Every few seconds it tries to reload or reshape the file and it doesn't let you work properly.

Delimit
http://delimitware.com/

All the aforementioned programs can deal with files much larger than memory using very few resources.

There are many other programs but most of them use a large quantity of memory.

Among the free editors I would choose gvim, it's interface may be not as comfortable for most Windows users but it works.

gvim x64
You can find the 64bit version of this editor originally created for Windows here:
https://tuxproject.de/projects/vim/

Kainet
http://www.kainet.ch/Editor-for-very-large-files
I've just found this new free editor. It's very basic: search and replace, codification and not much more. but it's really fast. It can open very large files almost instantly. It only has Windows version.

Answer 10

If you really need an editor (and not only viewer), then you can go with PilotEdit Lite. I successfully used it to modify and save 1.6 GB log file.

Answer 11

You can use Large Text File Viewer (aka. LTFViewr5u).

As of July 2015, their site is down, so here is a link to an archive of the site.

• view-only
• free
• Windows
• portable and lightweight (700 KB)
• designed for viewing large (>1GB) text files
• uses little memory and is able to open a gigabyte file instantly (from the website's description and I can confirm: it uses 4 MB for a 500 MB file, which loaded in less than 5 sec on my SSD Crucial M5)

Answer 12

Notepad++ 7 is also available as a 64-bit build, and I have tested it on 3+ GB file: considering the size of the file, it is doing a great job.

Even if it's a file editor and a bit slower than a read-only tool, not all free-for-commercial-use tools are supporting searching with regular expressions.

It also supports log monitoring (tail -f).

https://notepad-plus-plus.org/download/

Answer 13

If you are looking for a reader only:

HTMLPen.com is a free online tool that can open very large files(TB+) instantly, can do regex searches, and counts, export the matches. You do not need to download anything. It works on any OS with a modern browser. It also has syntax coloring if you open a data or code file.

Answer 14

I was using the free baretail for a very long time but was missing always some features.

So I wrote another logviewer: loxx. It supports real time viewing and filtering, huge and multiple views, regular expressions in filters and searches, split views and much more.

So have a try on it -- any feedback and ideas are welcome!

_{(source: mommos-software.com)}

Answer 15

With ultraedit you can open and edit big files. It has a mode where you can tell it not to create a temporary backup file.

Answer 16

wxHexEditor is (nearly) perfect for this.

• Opens 10 GB+ files without any problem

• Very fast text search: ~1 minute to search a pattern in a 10 GB file whereas it takes 21 hours with another tool I tried (LTFViewer)

• Only drawback: I haven't found an option to display the text with newlines like a normal text. Only a hex view and a string preview right panel:

  

  Here it would be interesting to have it displayed as:

  hello
  world
  

  in another view than the "Hex" view.

---

For me Notepad++ doesn't work: even with the 64-bit version, I get "File is too big to be opened by Notepad++" for a 7 GB file.