I'm conducting data science on medium-size datasets (~5000 entries with ~100 fields) that include input by humans. I am using the pandas python module developed by Wes McKinney.
I spend a lot of time bugfixing due to errors in the datasets. For example, someone has input ">23" in a field that should contain integers, or "xxx" in a field that should contain something more useful. Using Ipython notebook and a bit of data wrangling, these entries can be found quite quickly. However when the scripts get too large I migrate them into modules, upon which the dataframe is a local variable in a function. A debugger is then necessary to view or manipulate the data.
Is there an IDE that correctly displays pandas dataframes?
- equivalent to the "view array" function of PyCharm (this works only for NumPy arrays)
Is there a debugger that allows easy manipulation of pandas dataframes?