4

I'm conducting data science on medium-size datasets (~5000 entries with ~100 fields) that include input by humans. I am using the pandas python module developed by Wes McKinney.

I spend a lot of time bugfixing due to errors in the datasets. For example, someone has input ">23" in a field that should contain integers, or "xxx" in a field that should contain something more useful. Using Ipython notebook and a bit of data wrangling, these entries can be found quite quickly. However when the scripts get too large I migrate them into modules, upon which the dataframe is a local variable in a function. A debugger is then necessary to view or manipulate the data.

My questions:

Is there an IDE that correctly displays pandas dataframes?

  • equivalent to the "view array" function of PyCharm (this works only for NumPy arrays)

Is there a debugger that allows easy manipulation of pandas dataframes?

2

Is there an IDE that correctly displays pandas dataframes?

PyCharm released version 2016.2 (a week after I posted this question!), which includes a pandas dataframe viewer.

example right-click

example df view

I still want to know if software exists that allows easy, ipython-like manipulation of such "local-variable" dataframes.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.