1

I'm looking for a way that I can convert several thousand .xml files into .csv. I've had no problems doing so in small quantities, but the data that I am dealing with is within several thousand folders. More specifically, I am looking at MLB game data over the course of a single season. Each game has its own separate folder within other folders for day and month. Within each game folder are several files, however I am only interested in one .xml file called inning_all.xml for each game. I need to convert these ~2,500 files into .csv files in order to be interpreted by Stata (stata has a built in xml import function but is incompatible with these specific files; they are able to imported after converting to .csv).

  • And you are looking for software to do so (otherwise your question is off-topic here)? What OS, what price? We also need to see an example (part of) such a XML file, and your required CSV output. A hierarchical XML structure and a flat CSV structure do not match 'naturally' . – user416 Oct 27 '16 at 13:05
  • The fact that they are all in different folders is not very relevant to the question because many tools let you collect them in one folder. Unless you need to have the CSV in the same folders, or they have duplicate names and cannot be renamed. Please say so. – user416 Oct 27 '16 at 13:06
  • Please explain this "I've had no problems doing so in small quantities". For me this sounds like you already have an application or script which solves your needs. How did you do this task for small quantities. What is the problem with your current app/script? – guettli Oct 27 '16 at 15:31
1

You haven't said anything about your technology constraints, but

(a) converting XML to CSV is easily done with any XSLT processor

(b) XSLT 2.0 processors such as Saxon typically have the ability to process multiple files in directory structures using the collection() or uri-collection() functions.

The number of files isn't a problem - I've processed much larger quantities than this in a minute or two, and Saxon's implementation of the collection() function is multi-threaded so it scales well.

0

I would suggest using python with the built in os.walk, csv and xml parse libraries such as expat.

  • Free, gratis & Open Source
  • Cross Platform including Windows
  • Quick and easy to learn
  • os.walk will "walk" a directory tree to find all of the files of interest.
  • there are multiple xml parsers but you might even be better off using the super powerful regular expression processing of the re library.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.