6

I'm conducting data science on medium-size datasets (~5000 entries with ~100 fields) that include input by humans. I am using the pandas python module developed by Wes McKinney.

I spend a lot of time bugfixing due to errors in the datasets. For example, someone has input ">23" in a field that should contain integers, or "xxx" in a field that should contain something more useful. Using Ipython notebook and a bit of data wrangling, these entries can be found quite quickly. However when the scripts get too large I migrate them into modules, upon which the dataframe is a local variable in a function. A debugger is then necessary to view or manipulate the data.

My questions:

Is there an IDE that correctly displays pandas dataframes?

  • equivalent to the "view array" function of PyCharm (this works only for NumPy arrays)

Is there a debugger that allows easy manipulation of pandas dataframes?

Improve this question

1 Answer 1

Reset to default
3

Is there an IDE that correctly displays pandas dataframes?

PyCharm released version 2016.2 (a week after I posted this question!), which includes a pandas dataframe viewer.

example right-click

example df view

I still want to know if software exists that allows easy, ipython-like manipulation of such "local-variable" dataframes.

Improve this answer
1
  • Mark, please accept your own answer. Doing so will help others who read this question in future
    – Mawg
    Commented Jul 2, 2021 at 6:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.