Skip to content
Merged

maint #301

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
132 changes: 71 additions & 61 deletions doc/source/changes/version_0_35.rst.inc
Original file line number Diff line number Diff line change
Expand Up @@ -3,100 +3,110 @@
New features
^^^^^^^^^^^^

* allow displaying *many* more different kinds of objects, and not only arrays
from larray. One specific goal when developing this new feature was speed.
* allow displaying **many** more different kinds of objects, and not only arrays
from LArray. One specific goal when developing this new feature was speed.
Most of these viewers should be fast (when at all possible), even on (very)
large datasets. We only support displaying (not editing) all the new types.

The following types are supported so far (but adding more is relatively easy):

* Python builtin objects:
- tuple (including named tuple), list (sequences), dict (mappings),
dict views, memoryview and array
* Python **builtin** objects:

- ``tuple`` (including named tuple), ``list`` (sequences), ``dict``
(mappings), dict views, ``memoryview`` and ``array``
- text and binary files
* Python stdlib objects:
- pathlib.Path

* Python **standard library** objects:

- ``pathlib.Path``

* if the path points to a directory, it will display the content of the
directory
* if the path points to a file, it will try to display it, if we
implemented support for that file type (see below for the list
of supported types).
- sqlite3.Connection (and their tables)
- pstats.Stats (results of Python's profiler)
- zipfile.ZipFile and zipfile.Path
* new objects from LArray: Axis, Excel Workbook (what you get from
larray.open_excel()), Sheets and Range
* IODE "collections" objects: Comments, Equations, Identities, Lists, Tables,
Scalars and Variables, as well as Table objects
* Pandas: DataFrame, Series and DataFrameGroupBy
* Polars: DataFrame and LazyFrame
* Numpy: ndarray
* PyArrow: Array, Table, RecordBatchFileReader (reader object for feather
files) and ParquetFile
* Narwhals: DataFrame and LazyFrame
* PyTables: File, Group (with special support for Pandas DataFrames written
in HDF files), Array and Table
* IBIS: Table
* DuckDB: DuckDBPyConnection and DuckDBPyRelation (what you receive from any
query)

File types (extensions) currently supported:
- Iode files: .ac, .ae, .ai, .al, .as, .at, .av, .cmt, .eqs, .idt, .lst,
.scl, .tbl, .var
- Text files: .bat, .c, .cfg, .cpp, .h, .htm, .html, .ini, .log, .md,
.py, .pyx, .pxd, .rep, .rst, .sh, .sql, .toml, .txt, .wsgi,
.yaml, .yml
- HDF5 files: .h5, .hdf
- Parquet files: .parquet
- Stata files: .dta
- Feather files: .feather
- SAS files: .sas7bdat
It is limited to the first few thousand rows (the exact number depends on
the number of columns), because reading later rows get increasingly slow,
to the point of being unusable.
- CSV files: .csv
- Gzipped CSV files: .csv.gz
- Excel files: .xls, .xlsx
- Zip files: .zip
- DuckDB files: .ddb, .duckdb

* the editor now features a new "File Explorer" (accessible from the "File"
- ``sqlite3.Connection`` (and their tables)
- ``pstats.Stats`` (results of Python's profiler)
- ``zipfile.ZipFile`` and ``zipfile.Path``

* new objects from **LArray**: ``Axis``, Excel ``Workbook`` (what you get from
larray.open_excel()), ``Sheet`` and ``Range``
* **IODE** "collections" objects: ``Comments``, ``Equations``, ``Identities``,
``Lists``, ``Tables``, ``Scalars`` and ``Variables``, as well as ``Table``
objects
* **Pandas** objects: ``DataFrame``, ``Series`` and ``DataFrameGroupBy``
* **Polars** objects: ``DataFrame`` and ``LazyFrame``
* **Numpy** objects: ``ndarray``
* **PyArrow** objects: ``Array``, ``Table``, ``RecordBatchFileReader``
(reader object for feather files) and ``ParquetFile``
* **Narwhals** objects: ``DataFrame`` and ``LazyFrame``
* **PyTables** objects: ``File``, ``Group`` (with special support for Pandas
DataFrames written in HDF files), ``Array`` and ``Table``
* **IBIS** objects: ``Table``
* **DuckDB** objects: ``DuckDBPyConnection`` and ``DuckDBPyRelation`` (what
you receive from any query)

File types (extensions) currently supported:

- **IODE** files: .ac, .ae, .ai, .al, .as, .at, .av, .cmt, .eqs, .idt, .lst,
.scl, .tbl, .var
- **Text** files: .bat, .c, .cfg, .cpp, .h, .htm, .html, .ini, .log, .md,
.py, .pyx, .pxd, .rep, .rst, .sh, .sql, .toml, .txt, .wsgi, .yaml, .yml
- **HDF5** files: .h5, .hdf
- **Parquet** files: .parquet
- **Stata** files: .dta
- **Feather** files: .feather
- **SAS** files: .sas7bdat
It is limited to the first few thousand rows (the exact number depends on
the number of columns), because reading later rows get increasingly slow,
to the point of being unusable.
- **CSV** files: .csv
- Gzipped CSV files: .csv.gz
- **Excel** files: .xls, .xlsx
- **Zip** files: .zip
- **DuckDB** files: .ddb, .duckdb

* the editor now features a new "**File Explorer**" (accessible from the "File"
menu) so that one can more easily make use of all the above file viewers.

* added a new SQL Console (next to the iPython console) for querying Polars
* added a new **SQL Console** (next to the iPython console) for querying Polars
structures (DataFrame, LazyFrame and Series) as SQL tables. The console
features auto-completion for SQL keywords, table names and column names
and stores the last 1000 queries (even across sessions). Recalling a query
from history is done with the up and down arrows and like in the iPython
console, it searches through history with the current command as prefix.
This console will only be present if the polars module is installed.

* allow sorting some objects by column by pressing on an horizontal label.
* allow **sorting** some objects by column by pressing on an horizontal label.
This is currently implemented for the following objects:

- python built-in sequences (e.g. tuples and lists)
- python pathlib.Path objects representing directories
- LArray (only for 2D arrays)
- Pandas DataFrame
- Polars DataFrame and LazyFrame
- Narwhals LazyFrame
- Pandas ``DataFrame``
- Polars ``DataFrame and ``LazyFrame``
- Narwhals ``LazyFrame``
- SQLite tables
- DuckDB relations

* allow filtering some objects by pressing on an horizontal label.
* allow **filtering** some objects by pressing on an horizontal label.
This is currently implemented for the following objects:
- Pandas DataFrame
- Polars DataFrame and LazyFrame

- Pandas ``DataFrame``
- Polars ``DataFrame`` and ``LazyFrame``
- DuckDB relations

* allow comparing arrays/sessions with different axes in :py:obj:`compare()`.
The function gained ``align`` and ``fill_value`` arguments and the interface
has a new combobox to change the alignment method during the comparison:
- outer: will use a label if it is in any array (ordered like the first array).
This is the default as it results in no information loss.
- inner: will use a label if it is in all arrays (ordered like the first array).
- left: will use the first array axis labels.
- right: will use the last array axis labels.
- exact: raise an error when axes are not equal.

- *outer*: will use a label if it is in any array (ordered like the first array).
This is the default as it results in no information loss.
- *inner*: will use a label if it is in all arrays (ordered like the first array).
- *left*: will use the first array axis labels.
- *right*: will use the last array axis labels.
- *exact*: raise an error when axes are not equal.

Closes :editor_issue:`214` and :editor_issue:`251`.

* double-clicking on a name in the variable list will open it in a new window
Expand Down
32 changes: 7 additions & 25 deletions doc/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,16 +44,17 @@
'sphinx.ext.autodoc',
'sphinx.ext.autosummary',
'sphinx.ext.doctest',
'sphinx.ext.extlinks',
'sphinx.ext.intersphinx',
'sphinx.ext.mathjax',
'sphinx.ext.ifconfig',
'sphinx.ext.viewcode',
'sphinx.ext.extlinks',
'numpydoc',
]

extlinks = {
'issue': ('https://github.com/larray-project/larray-editor/issues/%s', 'issue '),
# needed even in larray-editor config to allow pointing to larray issues (this was done at least once)
'issue': ('https://github.com/larray-project/larray/issues/%s', 'issue %s'),
'editor_issue': ('https://github.com/larray-project/larray-editor/issues/%s', 'issue %s')
}

# scan all found documents for autosummary directives, and to generate stub pages for each.
Expand Down Expand Up @@ -83,7 +84,7 @@
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None
# language = None

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
Expand All @@ -110,7 +111,7 @@
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# html_static_path = ['_static']

# Custom sidebar templates, must be a dictionary that maps document names
# to template names.
Expand Down Expand Up @@ -179,28 +180,9 @@
'Miscellaneous'),
]


# -- Options for Epub output -------------------------------------------------

# Bibliographic Dublin Core info.
epub_title = project

# The unique identifier of the text. This can be a ISBN number
# or the project homepage.
#
# epub_identifier = ''

# A unique identification for the text.
#
# epub_uid = ''

# A list of files that should not be packed into the epub file.
epub_exclude_files = ['search.html']


# -- Extension configuration -------------------------------------------------

# -- Options for intersphinx extension ---------------------------------------

# Example configuration for intersphinx: refer to the Python standard library.
intersphinx_mapping = {'https://docs.python.org/3': None}
intersphinx_mapping = {'python': ('https://docs.python.org/3', None)}
18 changes: 4 additions & 14 deletions doc/source/index.rst
Original file line number Diff line number Diff line change
@@ -1,21 +1,11 @@
.. LArray-Editor documentation master file, created by
sphinx-quickstart on Fri Apr 12 14:25:10 2019.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
.. this file is only used to locally test rendering the larray-editor
changelog before it is included in the main larray changelog.


Welcome to LArray-Editor's documentation!
=========================================

.. toctree::
:maxdepth: 2
:caption: Contents:
:maxdepth: 1



Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
changes
7 changes: 4 additions & 3 deletions make_release.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
PACKAGE_NAME = "larray-editor"
SRC_CODE = "larray_editor"
SRC_DOC = join('doc', 'source')
CONDA_BUILD_ARGS = {'--user': 'larray-project'}
ANACONDA_UPLOAD_ARGS = {'--user': 'larray-project'}

GITHUB_REP = "https://github.com/larray-project/larray-editor"
UPSTREAM_CONDAFORGE_FEEDSTOCK_REP = "https://github.com/conda-forge/larray-editor-feedstock.git"
Expand Down Expand Up @@ -69,5 +69,6 @@ def update_version_in_json_used_by_menuinst(build_dir, release_name, package_nam
SRC_CODE, *argv[2:], tmp_dir=TMP_PATH_CONDA)
else:
local_repository = abspath(dirname(__file__))
make_release(local_repository, PACKAGE_NAME, SRC_CODE, *argv[1:], src_documentation=SRC_DOC, tmp_dir=TMP_PATH,
conda_build_args=CONDA_BUILD_ARGS)
make_release(local_repository, PACKAGE_NAME, SRC_CODE, *argv[1:],
src_documentation=SRC_DOC, tmp_dir=TMP_PATH,
anaconda_upload_args=ANACONDA_UPLOAD_ARGS)
84 changes: 84 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,87 @@
[build-system]
build-backend = "setuptools.build_meta"
requires = [
"setuptools >= 77.0.3",
]

[project]
name = "larray-editor"
version = "0.35"
description = "Graphical User Interface for LArray library"
readme = { file = "README.rst", content-type = "text/x-rst" }

authors = [
{name = "Gaetan de Menten", email = "gdementen@gmail.com"},
{name = "Alix Damman", email = "ald@plan.be"},
{name = "Geert Bryon"},
{name = "Yannick Van den Abbeel"},
]
classifiers = [
"Development Status :: 4 - Beta",
"Operating System :: OS Independent",
"Intended Audience :: Science/Research",
"Intended Audience :: Developers",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
"Topic :: Scientific/Engineering",
"Topic :: Software Development :: Libraries",
]
license = "GPL-3.0-only"
license-files = ["LICENSE"]

requires-python = ">=3.9"
dependencies = [
# jedi >=0.18 to workaround incompatibility between jedi <0.18 and
# parso >=0.8 (see #220)
"jedi >=0.18",
# Technically, we should require larray >=0.35 because we need align_arrays
# for compare(), but to make larray-editor releasable, we cannot depend on
# larray X.Y when releasing larray-editor X.Y (see utils.py for more
# details)
# TODO: require 0.35 for next larray-editor version and drop shim in
# utils.pyk
"larray >=0.32",
"matplotlib",
"numpy",
# Pandas is required directly for a silly reason (to support converting
# pandas dataframes to arrays before comparing them). We could make it an
# optional dependency by lazily importing it but since it is also
# indirectly required via larray, it does not really matter.
"pandas",
# we do not actually require PyQt6 but rather either PyQt5, PyQt6 or
# PySide6 but I do not know how to specify this
"PyQt6",
"qtpy"
]

[project.optional-dependencies]
test = ["pytest"]
# for the "Copy to Excel" context-menu action
excel = ["xlwings"]
# (=PyTables) to load the example datasets from larray
hdf5 = ["tables"]

[project.gui-scripts]
larray-editor = "larray_editor.start:main"

[project.urls]
homepage = "https://github.com/larray-project/larray-editor"
repository = "https://github.com/larray-project/larray-editor"
issues = "https://github.com/larray-project/larray-editor/issues"

[tool.setuptools.package-data]
larray_editor = ["images/*"]

[tool.setuptools.packages.find]
where = ["."]
namespaces = false

[tool.pytest.ini_options]
minversion = "6.0"
testpaths = [
Expand Down
Loading