Enable layered architectures to provide useful context-dependent error information

[thorwhalen]

What

We want to have tools to be able to define rules of error handling, and apply them to our code with minimal tampering with the code itself.
This error handling is focused on being able to provide useful error information in a layered architecture.
One criteria of usefulness is that the error message (and possibly type) is relevant to the layer/context raising it.
This means that there should be a mechanism to catch errors from lower layers, translate them to the current context/layer, and raise that translated error so that the next higher level can do the same.

We want this functionality to be SoC, so want to have

• decorators that can apply error handling logic externally (without having to modify the code of the handled object)
• a means centralize the error handling rules for a whole layer

Motivation

The layered approach that the i2i tools are built for has a major problem: The errors a user will see at a top layer are very obscure and unhelpful. Each layer is focused on a separate concern and tries to use a language that is focused on that concern, purposely avoiding to use the language that might be more appropriate for its usage.

One will find similar problems when following the uncle bob dogma of "many tiny functions" as soon as these functions are reused in more than one (language-consistent) context.

It's not a bad thing.

It's a good thing.

But it does come with a cost: A large part from the error message obscurity mentioned above.

So we want to have each layer provide error messages that are relevant to that layer, in the language of that layer. When errors happen at deeper layers we want higher layers to catch these, and raise their own context-specific errors.

But also, we don't want to litter our code with try-catch clauses.
That would significantly increase the indentation-base-complexity of the program,
hiding the main logic.

No, error handling is a separate concern, so should be done in a different... layer.

Also, we'd like to make it easy to enhance, and possibly change, the error handling aspect without tampering with the business-logic code itself. Centralizing the error handling rules will help doing that.

Note

• Similar arguments as above could be said about argument validation: This is a typical use of decorators in fact.
• We want to make sure that we don't make things unpicklable when decorating, so should use i2.wrappers or careful decorator design (using functools.partial) for our work here

Example code

from functools import partial, wraps
from typing import Callable, Any


def __call_func_handling_error(
        error_handler: Callable, func: Callable, *args, **kwargs
):
    try:
        return func(*args, **kwargs)
    except BaseException as error_obj:
        return error_handler(error_obj, func, args, kwargs)


def _call_func_handling_error(error_handler: Callable, func: Callable):
    return wraps(func)(partial(__call_func_handling_error, error_handler, func))


def _handle_error(error_handler: Callable):
    return partial(_call_func_handling_error, error_handler)


HandlerSpecs = Any
ErrorHandlerFactory = Callable[[HandlerSpecs], Callable]

def handle_error(
        handler_specs: HandlerSpecs,
        error_handler_factory: ErrorHandlerFactory
):
    error_handler = error_handler_factory(handler_specs)
    return _handle_error(error_handler)

The intent is then to provide the user with some tools to easily make error_handler_factory for their context.
These factories could be checking only the type of the error object (see below),
but observe that the error_handler above is called on
(error_obj, func, args, kwargs), which means that it has access to the error object
itself (thus the message) as well as the function and arguments that it was called on.

This means that the user can compose error handlers of any precision.

These ideas point towards a generalization of the code found currently in know.base:

# TODO: Could consider (topologically) ordering the exceptions to reduce the matching
#  possibilities (see _handle_exception)
def _get_handle_exceptions(
        handle_exceptions: HandledExceptionsMapSpec,
) -> HandledExceptionsMap:
    if isinstance(handle_exceptions, BaseException):
        # Only one? Ensure there's a tuple of exceptions:
        handle_exceptions = (handle_exceptions,)
    if not isinstance(handle_exceptions, Mapping):
        handle_exceptions = {exc_type: do_nothing for exc_type in handle_exceptions}
    return handle_exceptions


def _handle_exception(
        instance, exc_val: BaseException, handle_exceptions: HandledExceptionsMap
) -> ExceptionHandlerOutput:
    """Looks for an exception type matching exc_val and calls the corresponding
    handler with
    """
    inputs = dict(exc_val=exc_val, instance=instance)
    if type(exc_val) in handle_exceptions:  # try precise matching first
        exception_handler = handle_exceptions[type(exc_val)]
        return _call_from_dict(inputs, exception_handler, Sig(exception_handler))

    else:  # if not, find the first matching parent
        for exc_type, exception_handler in handle_exceptions.items():
            if isinstance(exc_val, exc_type):
                return _call_from_dict(
                    inputs, exception_handler, Sig(exception_handler)
                )
    # You never should get this far, but if you do, there's a problem, let's scream it:
    raise ExceptionalException(
        f"I couldn't find that exception in my handlers: {exc_val}"
    )