Share code with Spyder variable editor

[s-celles]

Hello,

Spyder IDE https://github.com/spyder-ide/spyder provide a convenient user interface named "variable editor" (MIT license)

it will be great to use a part of their code to have a standalone version of this variable editor.

Kind regards

See spyder-ide/spyder#2553

[wavexx]

It's a nice idea, but I'd like to see more options. Are you aware of other spreadsheet-looking things done in python? (as long as it's not tk - I'm fine with qt, wx or gtk in order of preference).

The thing is, the table controller in QT is extremely slow. As you will see, it can barely handle 1m cells even on a decent i5. There are means to implement the data/view controller itself, but then there's a lot of work to do. You also cannot reasonably have more than a single header row/column (which is a pity, since DF would look so nice) without also re-implementing a lot of stuff.

In that sense, I'd like to take as much as what's already available.

[s-celles]

Sorry I don't know other spreadsheet looking Python project.

I try this in Spyder (IPython).

import numpy as np
import pandas as pd
df = pd.DataFrame(np.random.random((5000000,3)))

and go to Spyder variables editor

Scroll down... you will notice that table is only feed with (around) 500 rows

Going to the bottom of DataFrame is really difficult.

[wavexx]

Meanwhile I discovered pandas had something like this already:

https://github.com/pydata/pandas/blob/1d8717446d8666207b63ae324e56be60a0b01b07/pandas/sandbox/qtpandas.py

It has been split into a separate module:

https://github.com/datalyze-solutions/pandas-qt

Ironically, it handles multi-level indexes worse than what gtabview is already doing, and seems to be equally slow :(

[wavexx]

Or maybe not. I now implemented some simple models, I can display millions of rows without problems.

[s-celles]

That's very efficient! It can display millions of rows
but also millions of columns!
You did a great work
Thanks

[wavexx]

On 21/07/15 15:47, scls19fr wrote:

That's very efficient! It can display millions of rows
but also millions of columns!
You did a great work

Now it just needs a few keybindings to make it similar to navigate to
tabview.

[s-celles]

Yes with keybindings it will be more convenient.

Keeping window size will be probably an other interesting improvement.
Several size policies could exists:

• calling view(...) always displays a window which have a defined default size
• calling view(...) with same object (might use id(obj)) displays a window which have last size

[wavexx]

On 21/07/15 15:58, scls19fr wrote:

Yes with keybindings it will be more convenient.

Keeping window size will be probably an other interesting improvement.
Several size policies could exists:

• calling |view(...)| always displays a window which have a defined
  default size
• calling |view(...)| with same object (might use |id(obj)|) displays
  a window which have last size

This is already done.
You have two more parameters in gtabview.view: modal and recycle.

By default, modal=True and it will block the python process as
tabview.view does. However, this being a gui, with modal=False the
window will be asyncronous (you can keep working). calling
view(modal=False) will reuse the same window (recycle=True by default).

[s-celles]

Thanks for these tips.

I wasn't aware of this non-modal mode.

But you should notice that:

• if you change data in dataframe from ipython (df.loc[0, 0] = 1.23), window shows these changes (and that's a good thing)
• if you add new column to dataframe from ipython (df["new_col"] = 1), window won't update (unfortunately) you need to call view(df, modal=False)

Calling (again) view(df, modal=False) should also put the window on top

But I'm sorry I think I wasn't clear enouth
if you close window (modal or not) you don't keep window size.

[wavexx]

Try now. I changed the defaults/names to wait=False, recycle=True.

[s-celles]

I think you changed something else.

With Python 3

In [18]: from gtabview import view
  File "//anaconda/lib/python3.4/site-packages/gtabview/__init__.py", line 132
    app.postEvent(self._view, QtCore.QEvent(QtCore.QEvent.None))
                                                             ^
SyntaxError: invalid syntax

[s-celles]

With Python 2

pc:~ scls$ source activate py2
discarding //anaconda/bin from PATH
prepending //anaconda/envs/py2/bin to PATH
(py2)pc:~ scls$ pip uninstall gtabview
Cannot uninstall requirement gtabview, not installed
(py2)pc:~ scls$ pip install git+git://github.com/wavexx/gtabview
Collecting git+git://github.com/wavexx/gtabview
  Cloning git://github.com/wavexx/gtabview to /var/folders/j_/v8b1bst93_94t724ptsswfsr0000gn/T/pip-J7fZM7-build
Requirement already satisfied (use --upgrade to upgrade): setuptools in /anaconda/envs/py2/lib/python2.7/site-packages/setuptools-18.0.1-py2.7.egg (from gtabview==0.1)
Installing collected packages: gtabview
  Running setup.py install for gtabview
Successfully installed gtabview-0.1
(py2)pc:~ scls$ ipython
Python 2.7.10 |Anaconda 2.3.0 (x86_64)| (default, May 28 2015, 17:04:42)
Type "copyright", "credits" or "license" for more information.

IPython 3.2.1 -- An enhanced Interactive Python.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: from gtabview import view

In [2]:

In [2]: import pandas as pd

In [3]: df = pd.DataFrame([[1, 2, 3], [4, 5, 6]],
   ...:                   columns=['a', 'b', 'c'], index=['x', 'y'])

In [4]: view(df)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
TypeError: 'DataFrame' object is not callable
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
TypeError: 'DataFrame' object is not callable
---------------------------------------------------------------------------

(...)

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)
TypeError: 'DataFrame' object is not callable
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
TypeError: 'DataFrame' object is not callable
Out[4]: <gtabview.ViewController at 0x102264790>

In [5]:

[wavexx]

Yes, I changed a lot of stuff in order to keep the GUI off the main thread, depending if you use matplotlib or not (which also uses QT). In essence, if you use matplotlib, you should import it first in order to gtabview initialize correctly. If matplotlib is not present, gtabview assumes control of QT in order to have a completely detached GUI.

I'm not sure it's the best approach. Maybe detach=False should always be a default.
I'll write some notes in the README about this.

I fixed the issue with python 3 (didn't know that 'None' is now not usable in contextes like this)

[wavexx]

I pushed some support for MultiIndex for both rows and columns. gtabview -H[n] now works correctly.
But so does this:

import numpy as np
import pandas as pd
import gtabview

arrays = [['bar', 'bar', 'baz', 'baz', 'foo', 'foo', 'qux', 'qux'],
          ['one', 'two', 'one', 'two', 'one', 'two', 'one', 'two']]
tuples = list(zip(*arrays))
index = pd.MultiIndex.from_tuples(tuples, names=['first', 'second'])

arrays2 = [['bar', 'bar', 'bar', 'bar', 'baz', 'baz', 'baz', 'baz'],
           ['foo', 'foo', 'qux', 'qux', 'foo', 'foo', 'qux', 'qux'], 
           ['one', 'two', 'one', 'two', 'one', 'two', 'one', 'two']]
tuples2 = list(zip(*arrays2))
index2 = pd.MultiIndex.from_tuples(tuples2, names=['first', 'second', 'third'])

df = pd.DataFrame(np.random.randn(8, 8), index=index2, columns=index)
gtabview.view(df)

There is some drudgery to make it work, so I'd like a few comments on how it works/looks on mac.