diff --git a/README.md b/README.md index d1460354..7c03ca8b 100644 --- a/README.md +++ b/README.md @@ -1,57 +1,59 @@ -hydrus [![Build Status](https://travis-ci.com/HTTP-APIs/hydrus.svg?branch=master)](https://travis-ci.com/HTTP-APIs/hydrus) +Hydrus [![Build Status](https://travis-ci.com/HTTP-APIs/hydrus.svg?branch=master)](https://travis-ci.com/HTTP-APIs/hydrus) =================== hydrus is a set of **Python** based tools for easier and efficient creation of Hypermedia driven REST-APIs. hydrus utilises the power of [Linked Data](https://en.wikipedia.org/wiki/Linked_data) to create a powerful REST APIs to serve data. -hydrus uses the [Hydra(W3C)](http://www.hydra-cg.com/) standard for creation and documentation of it's APIs. +hydrus uses the [hydra(W3C)](http://www.hydra-cg.com/) standard for creation and documentation of it's APIs. -Start-up the demo ------------------ -* with *Docker* and *docker-compose* installed, run `docker-compose up --build` -* open the browser at `http://localhost:8080/api/vocab` +## Start-up the demo +- With [*Docker*](https://www.docker.com/) and [*docker-compose*](https://docs.docker.com/compose/) installed, run +``` bash +docker-compose up --build +``` +- Open the browser at `http://localhost:8080/api/vocab`. You should be displaying the example API as served by the server. -Add your own Hydra documentation file -------------------------------------- -To serve your own Hydra-RDF documentation file: -* create a `doc.py` file as the ones in `examples/` directory containing your own *ApiDoc* -* set the `APIDOC_REL_PATH` variable in `docker-compose.yml`. This should the relative path from the project root -* start-up the demo as above. +## Add your own hydra documentation file +To serve your own hydra-RDF documentation file: + +1. Create a `doc.py` file as the ones in `examples/` directory containing your own *ApiDoc*. +2. Set the `APIDOC_REL_PATH` variable in `docker-compose.yml`. This should the relative path from the project root. +3. Start-up the demo as above. You should be displaying your API as served by the server. -Table of contents -------------- +## Table of contents * [Features](#features) * [Requirements](#req) * [Demo](#demo) * [Usage](#usage) -Features -------------- +## Features hydrus supports the following features: -- A client that can understand Hydra vocabulary and interacts with a Hydra supporting server to basic [CRUD](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete) operations on data. +- A client that can understand hydra vocabulary and interacts with a hydra supporting server to basic [CRUD](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete) operations on data. - A generic server that can serve required data and metadata(in the form of API documentation) to a client over HTTP. - A middleware that allows users to use the client to interact with the server using Natural Language which is processed machine consumable language. **(under development)** -Requirements -------------- +## Requirements The system is built over the following standards and tools: +- [Python](https://www.python.org/downloads/) 3.6 and above - [Flask](http://flask.pocoo.org/) a Python based micro-framework for handling server requests and responses. - [JSON-LD](http://json-ld.org/spec/latest/json-ld/) as the preferred data format. -- [Hydra](http://www.hydra-cg.com/) as the API standard. +- [hydra](http://www.hydra-cg.com/) as the API standard. - [SQLAlchemy](http://www.sqlalchemy.org/) as the backend database connector for storage and related operations. Apart from this, there are also various Python packages that hydrus uses. Using `python setup.py install` installs all the required dependencies. -**NOTE:** You'll need to use `python3` not `python2`. Hydrus does not support python < 3.6 +**NOTE:** You'll need to use `python3` not `python2`. hydrus does not support python < 3.6 +To check your python version run +```bash +python --version +``` -Demo -------------- +## Demo To run a demo for hydrus using the sample API, just do the following: - 1. Clone hydrus: ```bash git clone https://github.com/HTTP-APIs/hydrus @@ -78,7 +80,8 @@ NOTE: there is an alternative way to install dependencies with `poetry`: pip3 install poetry poetry install ``` -This is mostly used to check dependencies conflicts among packages and to release to `PyPi`. + +This is mostly used to check dependencies conflicts among packages and to release to [`PyPi`](https://pypi.org/). After installation is successful, to *run the server*: ```bash @@ -88,14 +91,11 @@ hydrus serve The demo should be up and running on `http://localhost:8080/serverapi/`. -Usage -------------- +## Usage For more info, head to the [Usage](http://www.hydraecosystem.org/01-Usage.html) section of the [wiki](http://www.hydraecosystem.org/). -Development -------------- - +## Development From the `hydrus` directory: * To run formatter: `pip install black && black *.py` * To test for formatting: `flake8 *.py` diff --git a/docs/STARTING.md b/docs/STARTING.md index 0fffa123..8928be1d 100644 --- a/docs/STARTING.md +++ b/docs/STARTING.md @@ -3,14 +3,16 @@ This document is just a collection of annotations at the moment. ### Objectives This project in basically aimed to solve these issues: -* provide an attractive demo API to show HYDRA specs, we decided to leverage space sciences as they provide a subject of great interest; -* define a multi-layered (and multi-peer?) approach to make the API usable and adaptable to different filtering/querying needs; -* make useful experiments about the right way to approach aggregation/filtering procedures among data served by multiple HYDRA-enhanced endpoints (starting from well-know industry standards for database management and data warehousing, and some glimpses to GraphQL). +* Provide an attractive demo API to show hydra specs, we decided to leverage space sciences as they provide a subject of great interest; +* Define a multi-layered (and multi-peer?) approach to make the API usable and adaptable to different filtering/querying needs; +* Make useful experiments about the right way to approach aggregation/filtering procedures among data served by multiple hydra-enhanced endpoints (starting from well-know industry standards for database management and data warehousing, and some glimpses to [GraphQL](https://graphql.org/)). ### Starting assumptions -We suppose the server has to be build with at least two layers of abstraction: a low-level strict-HYDRA-RDF API that can map as directly as possible the data to resources/collections ("Server"), and a middleware API ("Client") to perform some sort of opinionated intermediation between the lower level and the "final" client ("UI") above. The middleware has to implement data querying on the lower endpoints, leveraging HYDRA's metadata framework; to do this it has in some way to implement precise choices about the aggregation mechanism that is going to fetch and "map-reduce" (aggregate) the data from the lower level (this part is really open for discussions, as a starting example you can check Mongo's pipelines and its matching/grouping system); because of the different patterns that makes possible this kind of results, some choices about tools and techniques have to be discussed and options defined. +#### We suppose the server has to be build with at least **two layers of abstraction**: + +A **low-level strict-HYDRA-RDF API** that can map as directly as possible the data to resources/collections **("Server")**, and a middleware API **("Client")** to perform some sort of opinionated intermediation between the lower level and the **"final"** **client ("UI")** above. The middleware has to implement data querying on the lower endpoints, leveraging HYDRA's metadata framework. To do this it has in some way to implement precise choices about the aggregation mechanism that is going to fetch and "map-reduce" (aggregate) the data from the lower level (this part is really open for discussions, as a starting example you can check Mongo's pipelines and its matching/grouping system), because of the different patterns that makes possible this kind of results, some choices about tools and techniques have to be discussed and options defined. ### About the option of having endpoints that are type- (class-) based As we discussed, the fun idea would be to have somenthing that can "mix" REST and RPC in some way, not technically but at least conceptually. This approach comes from the observation that the statefulness of a REST API can be in some sort (at a level we are trying to understand for the purpose of our implementation) related to the concept of invariability of the output of a pure-function. This would be practically accomplished by dedicating some endpoints to resources obviously but also to "operations" (functions that accepts some kind of paramethers and run some kind of reasoning on a data structure, we temporarly define these endpoints `hydra:Operation`s). @@ -19,14 +21,14 @@ As we discussed, the fun idea would be to have somenthing that can "mix" REST an What we mean with "aggregation" and "filtering" with HYDRA? Let's see a simple example. As HYDRA is meant to let clients to interoperate automatically, we try here to subset the problem posing it on this shape: starting from an initial input from a human/user, how can different layers of HYDRA-featured APIs respond consistantly by querying the provided endpoints? -* "UI" layer: a user (or a machine from another network) is wishful to know "what is most distant from the Sun, Earth or Mars?" -* "Client" layer: the client knows that some endpoints are available at a lower level, and we suppose that it knows it has to look for some kind of length value. It looks for the endpoints that can help, we suppose it can understand the fact that it needs the `/api/planet/calculate_average_au` (look for the right operation to perform on the "planet" class, that is basically a documentation problem); so it pass the parameters (Earth and Mars) to it. This layer is commonly referred to as *middleware*; we temporarly define `/api/planet/calculate_average_au` as a `hydra:Operation` (an endpoint that performs come kind of aggregation on other endpoints, something more similar to an RPC call more than a REST call probably), all the `hydra:Operation`s referred to a `hydra:Class` type has to be listed in the class' definition; -* "Server" layer: the server endpoints are queried and they serve the required data to calculate and respond: "Mars!". This server is the one that works like a traditional HYDRA-enhanced API, with all the descriptive features provided by the spec. +* **"UI" layer:** a user (or a machine from another network) is wishful to know "what is most distant from the Sun, Earth or Mars?" +* **"Client" layer:** the client knows that some endpoints are available at a lower level, and we suppose that it knows it has to look for some kind of length value. It looks for the endpoints that can help, we suppose it can understand the fact that it needs the `/api/planet/calculate_average_au` (look for the right operation to perform on the "planet" class, that is basically a documentation problem); so it pass the parameters (Earth and Mars) to it. This layer is commonly referred to as *middleware*; we temporarly define `/api/planet/calculate_average_au` as a `hydra:Operation` (an endpoint that performs come kind of aggregation on other endpoints, something more similar to an RPC call more than a REST call probably), all the `hydra:Operation`s referred to a `hydra:Class` type has to be listed in the class' definition; +* **"Server" layer:** the server endpoints are queried and they serve the required data to calculate and respond: "Mars!". This server is the one that works like a traditional HYDRA-enhanced API, with all the descriptive features provided by the spec. In this scenario, allowed operations are defined by the possible interaction of different (or same-kind) classes (this idea is inspired by the Scala programming language's typing system). For example, two instances of a `Planet` (of the Solar System) class can have in common the defined "operation" of `calculate_average_au` (compute the average distance of two planets using the Sun-Earth distance as unit(AU)). This way in the API we could do: `/api/planet/calculate_average_au` and pass to it the related parameters. This kind and body of the request: -``` +```json POST /api/planet/calculate_average_au { @type: "hydra:Collection", @@ -37,7 +39,7 @@ POST /api/planet/calculate_average_au } ``` This endpoint could respond with: -``` +```json { @type: "vocab:Result", # or the given type in the HYDRA framework @returns: "umbel:Float", @@ -46,7 +48,7 @@ This endpoint could respond with: } ``` This RPC-like endpoint could be meta-described as: -``` +```json { "@context": { "hydra": "http://www.w3.org/ns/hydra/context.jsonld" @@ -69,9 +71,9 @@ This RPC-like endpoint could be meta-described as: ### Stack -* initial version: local Flask ("Server") and in-memory low-footprint actors ("Client") reading from local vocabularies, use ZeroMQ; -* development version: use a cache (Mongo or Redis) for documents and try to represent a graph; -* stable version: add a some kind of graph database under the cache layer; +* **Initial version:** local Flask ("Server") and in-memory low-footprint actors ("Client") reading from local vocabularies, use ZeroMQ; +* **Development version:** use a cache (Mongo or Redis) for documents and try to represent a graph; +* **Stable version:** add a some kind of graph database under the cache layer; * ... ### Implementation diff --git a/docs/TASKS.md b/docs/TASKS.md index 55efc685..dd0f2dc8 100644 --- a/docs/TASKS.md +++ b/docs/TASKS.md @@ -1,7 +1,7 @@ -Establish a protocol for building routes in a way that can fit the HYDRA spec: +# Establish a protocol for building routes in a way that can fit the HYDRA spec: -* Every route (see `server.routes.py`) has to be documented with a `hydra:ApiDocumentation` (automatically): it can be used the Werkzeug `Rule.endpoints` property to name each route (within the `add_url_rule` method) and map to an apidoc response; +* Every route (see `server.routes.py`) has to be documented with a `hydra:ApiDocumentation` (automatically): it can be used the Werkzeug `Rule.endpoints` property to name each route (within the `add_url_rule` method) and map to an apidoc response;. * Rejoin all the different vocabularies in just one big vocabulary and make the server filter by argument if necessary (example vocabulary are "astronomy" and "solarsystem"); * Routes, outside the basic collections, has to be opinionated considering the concern of filtering and data creation required by the instantiation of the objects based on the provided classes (*or do it in a separate layer above*); in this case we would have a "smart middleware" that relies on a lower server layer. -* Start with the lower layer, that should be meant to provide resource one-by-one or by collections \ No newline at end of file +* Start with the lower layer, that should be meant to provide resource one-by-one or by collections. diff --git a/docs/TUTORIALS.md b/docs/TUTORIALS.md index f0451f07..e6e2f847 100644 --- a/docs/TUTORIALS.md +++ b/docs/TUTORIALS.md @@ -1,35 +1,35 @@ -# hydrus Workflow +# Hydra Workflow -This document can be used to understand how hydrus works.
+This document can be used to understand how hydra works.
To understand the concepts related to hydrus:
https://www.hydraecosystem.org/Starting-Material ### How is an ```ApiDoc``` defined: - There are two ways of defining an ApiDoc: - 1. Writing a Hydra-compliant JSON-LD like [here]( https://github.com/HTTP-APIs/hydrus/blob/master/hydrus/samples/doc_writer_sample_output.py) - 1. Defining the ApiDoc programmatically using the code like in [here](https://github.com/HTTP-APIs/hydrus/blob/master/hydrus/samples/doc_writer_sample.py) + 1. Writing a Hydra-compliant JSON-LD like [here]( https://github.com/HTTP-APIs/hydrus/blob/master/hydrus/samples/doc_writer_sample_output.py). + 2. Defining the ApiDoc programmatically using the code like in [here](https://github.com/HTTP-APIs/hydrus/blob/master/hydrus/samples/doc_writer_sample.py). - Any valid ApiDoc can be passed to the server so that it can parse the doc, generate a database and start the server. - The document containing the context is generated by the doc_maker file in the hydra_python_core repo. hydrus provides the link in its response for the context. ### Points : #### Description of some files: -1. hydrus/app.py - 1. Used for logging. (Line 22) - 1. Starts the SQLAlchemy engine and creates a session (Line 25-26) - 1. Creates an Api Documentation using the doc_maker() of hydra_python_core - 1. Gets the parsed classes from hydrus/data/doc_parse.py - 1. Creates database tables and its metadata Line 34-36 - 1. Checks if Authentication is True, adds user to the session.Check: https://www.hydraecosystem.org/Auth - 1. Calls app_factory class' constructor - 1.Sets authentication,api_name,session etc. which can be later retrieved using the corresponding get functions. - 1. Runs the API on the Port mentioned. -1. hydrus/app_factory.py - 1. Used to add resources to the API from resources.py. +1. [*hydrus/app.py*](https://github.com/HTTP-APIs/hydrus/blob/develop/hydrus/app.py). + 1. Used for logging. [Line 22](https://github.com/HTTP-APIs/hydrus/blob/c6a8587c7904afe64c31f74da1f9d4edc7139ed2/hydrus/app.py#L22). + 2. Starts the SQLAlchemy engine and creates a session [Line 30](https://github.com/HTTP-APIs/hydrus/blob/c6a8587c7904afe64c31f74da1f9d4edc7139ed2/hydrus/app.py#L30). + 3. Creates an Api Documentation using the [doc_maker() of hydra_python_core](https://hydra-python-core.readthedocs.io/_/downloads/en/develop/pdf/). + 4. Gets the parsed classes from [hydrus/data/doc_parse.py](https://github.com/HTTP-APIs/hydrus/blob/develop/hydrus/data/doc_parse.py). + 5. Creates database tables and its metadata [Line 34-36](https://github.com/HTTP-APIs/hydrus/blob/c6a8587c7904afe64c31f74da1f9d4edc7139ed2/hydrus/data/doc_parse.py#L34). + 6. Checks if Authentication is True, adds user to the session.[Check](https://www.hydraecosystem.org/Auth). + 7. Calls app_factory class' constructor. + 8. Sets authentication,api_name,session etc. which can be later retrieved using the corresponding get functions. + 9. Runs the API on the Port mentioned. +1. [*hydrus/app_factory.py*](https://github.com/HTTP-APIs/hydrus/blob/develop/hydrus/app_factory.py) + 1. Used to add resources to the API from `resources.py`. 1. Configures properties of API. - 1. Enables CORS -1. hydrus/resources.py + 1. Enables CORS. +1. [*hydrus/resources.py*](https://github.com/HTTP-APIs/hydrus/blob/develop/hydrus/resources.py) 1. Contains classes which are added as resources to the API. -1. hydrus/samples/hydra_doc_sample.py - 1. This file contains the sample ApiDoc that is displayed when the demo for hydrus is run. It contains the JSON-LD document that is displayed. +1. [*hydrus/samples/hydra_doc_sample.py*](https://github.com/HTTP-APIs/hydrus/blob/develop/hydrus/samples/hydra_doc_sample.py). + 1. This file contains the sample ApiDoc that is displayed when the demo for hydrus is run. It contains the [JSON-LD](https://json-ld.org/) document that is displayed. diff --git a/docs/hydrus_database.md b/docs/hydrus_database.md index 1653e900..8be8e11f 100644 --- a/docs/hydrus_database.md +++ b/docs/hydrus_database.md @@ -1,4 +1,4 @@ -## hydrus +## Hydrus `hydrus` is a core part of the HydraEcosystem. It is the server which powers Hydra-based API Docs. @@ -21,4 +21,4 @@ The overview of the process of making a database from `apidoc` object is: ### Example ER Diagram It might be easier to understand what we discussed above if we could see a ER diagram for a Hydra API Doc. So, let us take the sample [Drone API Doc](https://github.com/HTTP-APIs/hydrus/blob/develop/hydrus/samples/hydra_doc_sample.py) as our example. -The ER diagram for the database generated for this Hydra doc can be found [here](er_diagram.pdf). \ No newline at end of file +The [ER diagram](https://en.wikipedia.org/wiki/Entity%E2%80%93relationship_model) for the database generated for this Hydra doc can be found [here](er_diagram.pdf). diff --git a/docs/wiki/images/db_schema.png b/docs/wiki/images/db_schema.png index ca239748..7d3832d3 100644 Binary files a/docs/wiki/images/db_schema.png and b/docs/wiki/images/db_schema.png differ diff --git a/docs/wiki/images/hydra_dataflow.png b/docs/wiki/images/hydra_dataflow.png index 9f6f6155..73afc9ae 100644 Binary files a/docs/wiki/images/hydra_dataflow.png and b/docs/wiki/images/hydra_dataflow.png differ diff --git a/docs/wiki/images/rdf_dataflow.png b/docs/wiki/images/rdf_dataflow.png index 0d256cb9..640d534a 100644 Binary files a/docs/wiki/images/rdf_dataflow.png and b/docs/wiki/images/rdf_dataflow.png differ diff --git a/docs/wiki/images/use_case1.png b/docs/wiki/images/use_case1.png index 6e772359..b0ac2da1 100644 Binary files a/docs/wiki/images/use_case1.png and b/docs/wiki/images/use_case1.png differ diff --git a/hydrus/app_factory.py b/hydrus/app_factory.py index 3184166d..11d0ffaf 100644 --- a/hydrus/app_factory.py +++ b/hydrus/app_factory.py @@ -36,14 +36,8 @@ def root_url(): api.add_resource(Index, f"/{api_name}/", endpoint="api") api.add_resource(Vocab, f"/{api_name}/{vocab_route}", endpoint="vocab") - api.add_resource( - Contexts, f"/{api_name}/contexts/.jsonld", endpoint="contexts" - ) - api.add_resource( - Entrypoint, - f"/{api_name}/contexts/EntryPoint.jsonld", - endpoint="main_entrypoint", - ) + api.add_resource(Contexts, f"/{api_name}/contexts/.jsonld", endpoint="contexts") + api.add_resource(Entrypoint,f"/{api_name}/contexts/EntryPoint.jsonld",endpoint="main_entrypoint",) api.add_resource( ItemCollection, f"/{api_name}/", endpoint="item_collection" ) diff --git a/hydrus/resources.py b/hydrus/resources.py index b0e0408b..6697e06c 100644 --- a/hydrus/resources.py +++ b/hydrus/resources.py @@ -1,22 +1,22 @@ -"""Imports : - flask.json.jsonify : Turns the JSON output into a Response object with the - application/json mimetype Ref- http://flask.pocoo.org/docs/0.12/api - flask.request : The request object used by default in Flask. - Remembers the matched endpoint and view arguments. - Ref - http://flask.pocoo.org/docs/0.12/api - flask.abort : Raises an HTTPException for the given status code or WSGI - application: Ref - http://flask.pocoo.org/docs/0.12/api - flask_restful.Resource : Represents an abstract RESTful resource. - Ref - http://flask-restful.readthedocs.io/en/latest/api.html - hydrus.data.crud : Function/Class to perform basic CRUD operations for the server - hydrus.auth.authenticate : Decorator for checking authentication of each request - hydrus.utils.get_session : Gets the database session for the server - hydrus.utils.get_doc : Function which gets the server API documentation - hydrus.utils.get_api_name : Function which gets the server API name - hydrus.utils.get_hydrus_server_url : Function the gets the server URL - hydrus.utils.get_authentication : Function that checks whether API needs to be - authenticated or not -""" # nopep8 +# Imports : +# flask.json.jsonify : Turns the JSON output into a Response object with the +# application/json mimetype Ref- http://flask.pocoo.org/docs/0.12/api +# flask.request : The request object used by default in Flask. +# Remembers the matched endpoint and view arguments. +# Ref - http://flask.pocoo.org/docs/0.12/api +# flask.abort : Raises an HTTPException for the given status code or WSGI +# application: Ref - http://flask.pocoo.org/docs/0.12/api +# flask_restful.Resource : Represents an abstract RESTful resource. +# Ref - http://flask-restful.readthedocs.io/en/latest/api.html +# hydrus.data.crud : Function/Class to perform basic CRUD operations for the server +# hydrus.auth.authenticate : Decorator for checking authentication of each request +# hydrus.utils.get_session : Gets the database session for the server +# hydrus.utils.get_doc : Function which gets the server API documentation +# hydrus.utils.get_api_name : Function which gets the server API name +# hydrus.utils.get_hydrus_server_url : Function the gets the server URL +# hydrus.utils.get_authentication : Function that checks whether API needs to be +# authenticated or not + import json @@ -54,18 +54,18 @@ class Index(Resource): - """Class for the EntryPoint.""" + # Class for the EntryPoint. def get(self) -> Response: - """Return main entrypoint for the api.""" + # Return main entrypoint for the api. return set_response_headers(jsonify(get_doc().entrypoint.get())) class Vocab(Resource): - """Vocabulary for Hydra.""" + # Vocabulary for Hydra. def get(self) -> Response: - """Return the main hydra vocab or a fragment of the main hydra vocab.""" + # Return the main hydra vocab or a fragment of the main hydra vocab. try: resource = request.args.getlist("resource")[0] return set_response_headers(jsonify(get_fragments(resource))) @@ -74,26 +74,26 @@ def get(self) -> Response: class Entrypoint(Resource): - """Hydra EntryPoint.""" + # Hydra EntryPoint. def get(self) -> Response: - """Return application main Entrypoint.""" + # Return application main Entrypoint. response = {"@context": get_doc().entrypoint.context.generate()} return set_response_headers(jsonify(response)) class Item(Resource): - """Handles all operations(GET, POST, PATCH, DELETE) on Items - (item can be anything depending upon the vocabulary).""" + # Handles all operations(GET, POST, PATCH, DELETE) on Items + # (item can be anything depending upon the vocabulary). @authenticate def get(self, id_: str, path: str) -> Response: - """ - GET object with id = id_ from the database. - :param id_ : Item ID - :param path : Path for Item ( Specified in APIDoc @id) - :return : object with id=id_ - """ + + # GET object with id = id_ from the database. + # :param id_ : Item ID + # :param path : Path for Item ( Specified in APIDoc @id) + # :return : object with id=id_ + id_ = str(id_) collections, parsed_classes = get_collections_and_parsed_classes() is_collection = False @@ -121,11 +121,11 @@ def get(self, id_: str, path: str) -> Response: @authenticate def post(self, id_: str, path: str) -> Response: - """ - Update object of type at ID with new object_ using HTTP POST. - :param id_ - ID of Item to be updated - :param path - Path for Item type( Specified in APIDoc @id) - """ + + # Update object of type at ID with new object_ using HTTP POST. + # :param id_ - ID of Item to be updated + # :param path - Path for Item type( Specified in APIDoc @id) + id_ = str(id_) collections, parsed_classes = get_collections_and_parsed_classes() is_collection = False @@ -146,11 +146,11 @@ def post(self, id_: str, path: str) -> Response: @authenticate def put(self, id_: str, path: str) -> Response: - """ - Add new object_ optional parameter using HTTP PUT. - :param id_ - ID of Item to be updated - :param path - Path for Item type( Specified in APIDoc @id) to be updated - """ + + # Add new object_ optional parameter using HTTP PUT. + # :param id_ - ID of Item to be updated + # :param path - Path for Item type( Specified in APIDoc @id) to be updated + id_ = str(id_) collections, parsed_classes = get_collections_and_parsed_classes() is_collection = False @@ -166,11 +166,11 @@ def put(self, id_: str, path: str) -> Response: @authenticate def delete(self, id_: str, path: str) -> Response: - """ - Delete object with id=id_ from database. - :param id_ - ID of Item to be deleted - :param path - Path for Item type( Specified in APIDoc @id) to be deleted - """ + + # Delete object with id=id_ from database. + # :param id_ - ID of Item to be deleted + # :param path - Path for Item type( Specified in APIDoc @id) to be deleted + id_ = str(id_) params = request.args.to_dict() if params.get("instances"): @@ -193,15 +193,15 @@ def delete(self, id_: str, path: str) -> Response: class ItemCollection(Resource): - """Handle operation related to ItemCollection (a collection of items).""" + # Handle operation related to ItemCollection (a collection of items). @authenticate def get(self, path: str) -> Response: - """ - Retrieve a collection of items from the database. - :param path : Path of the Collection - :return : collection of items - """ + + # Retrieve a collection of items from the database. + # :param path : Path of the Collection + # :return : collection of items + endpoint_ = checkEndpoint("GET", path) if not endpoint_["method"]: # If endpoint and Get method not supported in the API @@ -210,11 +210,11 @@ def get(self, path: str) -> Response: @authenticate def put(self, path: str) -> Response: - """ - Method executed for PUT requests. - Used to add an item to a collection - :param path - Path for Item type ( Specified in APIDoc @id) - """ + + # Method executed for PUT requests. + # Used to add an item to a collection + # :param path - Path for Item type ( Specified in APIDoc @id) + endpoint_ = checkEndpoint("PUT", path) if not endpoint_["method"]: # If endpoint and PUT method is not supported in the API @@ -229,11 +229,11 @@ def put(self, path: str) -> Response: @authenticate def delete(self, path): - """ - To delete multiple objects - :param path: endpoints - :return: - """ + + # To delete multiple objects + # :param path: endpoints + # :return: + params = request.args.to_dict() if params.get("instances"): int_list = params.get("instances") @@ -242,8 +242,8 @@ def delete(self, path): class Contexts(Resource): - """Dynamically generated contexts.""" + # Dynamically generated contexts. def get(self, category: str) -> Response: - """Return the context for the specified class.""" + # Return the context for the specified class. return get_context(category) diff --git a/hydrus/utils.py b/hydrus/utils.py index dcf5f69d..06f2636f 100644 --- a/hydrus/utils.py +++ b/hydrus/utils.py @@ -1,355 +1,319 @@ -""" -Pluggable utilities for hydrus. -=============================== -This module is for Web server-related utilities. For data-related - utilities use `data.helpers` - -Imports : -contextlib.contextmanager : This function is a decorator that can be used to -define a factory function for with statement context managers, without needing -to create a class or separate __enter__() and __exit__() methods. -Ref- https://docs.python.org/2/library/contextlib.html#contextlib.contextmanager -flask.appcontext_pushed : Signal is sent when an application context is pushed. -The sender is the application. -Ref- http://flask.pocoo.org/docs/0.12/api/#flask.appcontext_pushed -Ref- https://speakerdeck.com/mitsuhiko/advanced-flask-patterns-1 -flask.g : Used to attach values to global variables -doc_writer_sample : Sample script used to create Hydra APIDocumentation -hydra_python_core.engine : An SQLalchemy DB engine -sqlalchemy.orm.sessionmaker : Used to create a SQLalchemy Session -sqlalchemy.orm.session.Session : SQLalchemy Session class -Ref- http://docs.sqlalchemy.org/en/latest/orm/session_basics.html -hydra_python_core.doc_writer.HydraDoc : Class for Hydra Documentation -""" # nopep8 +# Pluggable utilities for hydrus. +# =============================== +# This module is for Web server-related utilities. For data-related +# utilities use `data.helpers` +# Imports : + +# contextlib module used + +# contextlib.contextmanager : This function is a decorator that can be used to +# define a factory function for with statement context +# managers, without needing to create a class or separate +# __enter__() and __exit__() methods. +# Ref- https://docs.python.org/2/library/contextlib.html#contextlib.contextmanager from contextlib import contextmanager -from typing import Dict, List -from flask import appcontext_pushed -from flask import g, Response, jsonify -from hydrus.samples import doc_writer_sample -from hydrus.data.db_models import engine -from hydra_python_core.doc_writer import HydraError + +# sqlalchemy module used + +# sqlalchemy.orm.sessionmaker : Used to create a SQLalchemy Session +# sqlalchemy.orm.scoped_session : Used to create a SQLalchemy Scoped_session +# sqlalchemy.orm.session.Session : SQLalchemy Session class +# Ref- http://docs.sqlalchemy.org/en/latest/orm/session_basics.html from sqlalchemy.orm import sessionmaker, scoped_session from sqlalchemy.orm.session import Session -from hydra_python_core.doc_writer import HydraDoc -from flask.app import Flask -from typing import Any, Iterator +# flask module used -@contextmanager -def set_authentication(application: Flask, authentication: bool) -> Iterator: - """ - Set the whether API needs to be authenticated or not (before it is run in main.py). +# flask.g : Used to attach values to global variables +# flask.Response : +# flask.jsonify : +# flask.appcontext_pushed : +# flask.app.Flask : +# flask.appcontext_pushed : Signal is sent when an application context is pushed. +# The sender is the application. +# Ref- http://flask.pocoo.org/docs/0.12/api/#flask.appcontext_pushed +# Ref- https://speakerdeck.com/mitsuhiko/advanced-flask-patterns-1 +from flask import g, Response, jsonify , appcontext_pushed , Flask - :param application: Flask app object - - :param authentication : Bool. API Auth needed or not - +# typing +# +from typing import Any, Iterator , Dict, List - Raises: - TypeError: If `authentication` is not a boolean value. +# hrdyus modules - """ - if not isinstance(authentication, bool): - raise TypeError("Authentication flag must be of type ") +# hydrus.samples.doc_writer_sample : Sample script used to create Hydra APIDocumentation +# hydrus.data.db_models.engine : An SQLalchemy DB engine +# hydra_python_core.doc_writer.HydraDoc : Class for Hydra Documentation +# hydra_python_core.doc_writer.HydraError : Class for Hydra Error +from hydrus.samples import doc_writer_sample +from hydrus.data.db_models import engine +from hydra_python_core.doc_writer import HydraDoc , HydraError - def handler(sender: Flask, **kwargs: Any) -> None: - g.authentication_ = authentication +def error(value,data_type,error_msg): + if not isinstance(value,data_type): + raise TypeError(error_msg) + +def error_2(value_1 , data_type_1 , value_2 , data_type_2 , error_msg): + if not isinstance(value_1, data_type_1) and not isinstance(value_1, data_type_2): + raise TypeError(error_msg) +def try_except_block(value_1,value_2,value_3,value_4): + try: + value_1 = getattr(g, value_2) + except AttributeError: + value_1 = value_3 + value_4 = value_1 + return value_1 + +def appcontext_pushed_function(): with appcontext_pushed.connected_to(handler, application): yield +@contextmanager +def set_authentication(application: Flask, authentication: bool) -> Iterator: + # set_authentication : + # Set the whether API needs to be authenticated or not (before it is run in main.py). -def get_authentication() -> bool: - """ - Check whether API needs to be authenticated or not. - Return and sets False if not found. - :return authentication : Bool. API Auth needed or not - - """ - try: - authentication = getattr(g, "authentication_") - except AttributeError: - authentication = False + # :param application: Flask app object + # + # :param authentication : Bool. API Auth needed or not + # + + # Raises: + # TypeError: If `authentication` is not a boolean value. + + error(authentication, bool,"Authentication flag must be of type ") + + def handler(sender: Flask, **kwargs: Any) -> None: g.authentication_ = authentication - return authentication + + appcontext_pushed_function() + + +def get_authentication() -> bool: + + # Check whether API needs to be authenticated or not. + # Return and sets False if not found. + # :return authentication : Bool. API Auth needed or not + # + + return try_except_block(authentication,"authentication_",False,g.authentication_) @contextmanager def set_api_name(application: Flask, api_name: str) -> Iterator: - """ - Set the server name or EntryPoint for the app (before it is run in main.py). - :param application: Flask app object - - :param api_name : API/Server name or EntryPoint - - Raises: - TypeError: If `api_name` is not a string. + # Set the server name or EntryPoint for the app (before it is run in main.py). + # :param application: Flask app object + # + # :param api_name : API/Server name or EntryPoint + # + + # Raises: + # TypeError: If `api_name` is not a string. + - """ - if not isinstance(api_name, str): - raise TypeError("The api_name is not of type ") + error(api_name, str,"The api_name is not of type ") def handler(sender: Flask, **kwargs: Any) -> None: g.api_name = api_name - with appcontext_pushed.connected_to(handler, application): - yield + appcontext_pushed_function() def get_api_name() -> str: - """ - Get the server API name. - Returns an sets "api" as api_name if not found. - :return api_name : API/Server name or EntryPoint - - """ - try: - api_name = getattr(g, "api_name") - except AttributeError: - api_name = "api" - g.api_name = api_name - return api_name + + # Get the server API name. + # Returns an sets "api" as api_name if not found. + # :return api_name : API/Server name or EntryPoint + # + + return try_except_block(api_name,"api_name","api",g.api_name) @contextmanager def set_page_size(application: Flask, page_size: int) -> Iterator: - """ - Set the page_size of a page view. - :param application: Flask app object - - :param page_size : Number of maximum elements a page can contain - - Raises: - TypeError: If `page_size` is not an int. + # Set the page_size of a page view. + # :param application: Flask app object + # + # :param page_size : Number of maximum elements a page can contain + # + + # Raises: + # TypeError: If `page_size` is not an int. + - """ - if not isinstance(page_size, int): - raise TypeError("The page_size is not of type ") + error(page_size, int , "The page_size is not of type ") def handler(sender: Flask, **kwargs: Any) -> None: g.page_size = page_size - with appcontext_pushed.connected_to(handler, application): - yield + appcontext_pushed_function() def get_page_size() -> int: - """ - Get the page_size of a page-view. - :return page_size : Number of maximum elements a page view can contain. - - """ - try: - page_size = getattr(g, "page_size") - except AttributeError: - page_size = 10 - g.page_size = page_size - return page_size + + # Get the page_size of a page-view. + # :return page_size : Number of maximum elements a page view can contain. + # + + return try_except_block(page_size,"page_size",10,g.page_size) @contextmanager def set_pagination(application: Flask, paginate: bool) -> Iterator: - """ - Enable or disable pagination. - :param application: Flask app object - - :param paginate : Pagination enabled or not - - Raises: - TypeError: If `paginate` is not a bool. + # Enable or disable pagination. + # :param application: Flask app object + # + # :param paginate : Pagination enabled or not + # + + # Raises: + # TypeError: If `paginate` is not a bool. - """ - if not isinstance(paginate, bool): - raise TypeError("The CLI argument 'pagination' is not of type ") + error(paginate, bool,"The CLI argument 'pagination' is not of type ") + def handler(sender: Flask, **kwargs: Any) -> None: g.paginate = paginate - with appcontext_pushed.connected_to(handler, application): - yield + appcontext_pushed_function() def get_pagination() -> bool: - """ - Get the pagination status(Enable/Disable). - :return paginate : Pagination enabled or not - - """ - try: - paginate = getattr(g, "paginate") - except AttributeError: - paginate = True - g.paginate = paginate - return paginate + + # Get the pagination status(Enable/Disable). + # :return paginate : Pagination enabled or not + # + + return try_except_block(paginate,"paginate",True,g.paginate) @contextmanager def set_doc(application: Flask, APIDOC: HydraDoc) -> Iterator: - """ - Set the API Documentation for the app (before it is run in main.py). - :param application: Flask app object - - :param APIDOC : Hydra Documentation object - - - Raises: - TypeError: If `APIDOC` is not an instance of `HydraDoc`. - - """ - if not isinstance(APIDOC, HydraDoc): - raise TypeError( - "The API Doc is not of type " - ) + + # Set the API Documentation for the app (before it is run in main.py). + # :param application: Flask app object + # + # :param APIDOC : Hydra Documentation object + # + + # Raises: + # TypeError: If `APIDOC` is not an instance of `HydraDoc`. + + + error(APIDOC, HydraDoc,"The API Doc is not of type ") def handler(sender: Flask, **kwargs: Any) -> None: g.doc = APIDOC - with appcontext_pushed.connected_to(handler, application): - yield + appcontext_pushed_function() @contextmanager def set_token(application: Flask, token: bool) -> Iterator: - """Set whether API needs to implement token based authentication. + # Set whether API needs to implement token based authentication. - Raises: - TypeError: If `token` is not a boolean value. + # Raises: + # TypeError: If `token` is not a boolean value. - """ - if not isinstance(token, bool): - raise TypeError("Token flag must be of type ") + error(token, bool,"Token flag must be of type ") def handler(sender: Flask, **kwargs: Any) -> None: g.token_ = token - with appcontext_pushed.connected_to(handler, application): - yield + appcontext_pushed_function() def get_doc() -> HydraDoc: - """ - Get the server API Documentation. - Returns and sets doc_writer_sample.api_doc if not found. - :return apidoc : Hydra Documentation object - - """ - try: - apidoc = getattr(g, "doc") - except AttributeError: - g.doc = apidoc = doc_writer_sample.api_doc - return apidoc + + # Get the server API Documentation. + # Returns and sets doc_writer_sample.api_doc if not found. + # :return apidoc : Hydra Documentation object + # + + return try_except_block(apidoc,"doc",g.doc,doc_writer_sample.api_doc) def get_token() -> bool: - """Check wether API needs to be authenticated or not.""" - try: - token = getattr(g, "token_") - except AttributeError: - token = False - g.token_ = token - return token + # Check wether API needs to be authenticated or not. + return try_except_block(token,"token_",False,g.token_) @contextmanager def set_hydrus_server_url(application: Flask, server_url: str) -> Iterator: - """ - Set the server URL for the app (before it is run in main.py). - :param application: Flask app object - - :param server_url : Server URL - - Raises: - TypeError: If the `server_url` is not a string. + # Set the server URL for the app (before it is run in main.py). + # :param application: Flask app object + # + # :param server_url : Server URL + # + + # Raises: + # TypeError: If the `server_url` is not a string. - """ - if not isinstance(server_url, str): - raise TypeError("The server_url is not of type ") + error(server_url, str,"The server_url is not of type ") def handler(sender: Flask, **kwargs: Any) -> None: g.hydrus_server_url = server_url - with appcontext_pushed.connected_to(handler, application): - yield + appcontext_pushed_function() def get_hydrus_server_url() -> str: - """ - Get the server URL. - Returns and sets "http://localhost/" if not found. - :return hydrus_server_url : Server URL - - """ - try: - hydrus_server_url = getattr(g, "hydrus_server_url") - except AttributeError: - hydrus_server_url = "http://localhost/" - g.hydrus_server_url = hydrus_server_url - return hydrus_server_url + + # Get the server URL. + # Returns and sets "http://localhost/" if not found. + # :return hydrus_server_url : Server URL + # + + return try_except_block(hydrus_server_url,"hydrus_server_url","http://localhost/",g.hydrus_server_url) @contextmanager def set_session(application: Flask, DB_SESSION: scoped_session) -> Iterator: - """ - Set the Database Session for the app before it is run in main.py. - :param application: Flask app object - - :param DB_SESSION: SQLalchemy Session object - - - Raises: - TypeError: If `DB_SESSION` is not an instance of `scoped_session` or `Session`. - - """ - if not isinstance(DB_SESSION, scoped_session) and not isinstance( - DB_SESSION, Session - ): - raise TypeError( - "The API Doc is not of type or" - " " - ) + + # Set the Database Session for the app before it is run in main.py. + # :param application: Flask app object + # + # :param DB_SESSION: SQLalchemy Session object + # + + # Raises: + # TypeError: If `DB_SESSION` is not an instance of `scoped_session` or `Session`. + + + error_2(DB_SESSION, scoped_session,DB_SESSION, Session,"The API Doc is not of type or\n") def handler(sender: Flask, **kwargs: Any) -> None: g.dbsession = DB_SESSION - with appcontext_pushed.connected_to(handler, application): - yield + appcontext_pushed_function() def get_session() -> scoped_session: - """ - Get the Database Session from g. - Returns and sets a default Session if not found - :return session : SQLalchemy Session object - - """ - try: - session = getattr(g, "dbsession") - except AttributeError: - session = scoped_session(sessionmaker(bind=engine)) - g.dbsession = session - return session - - -def set_response_headers( - resp: Response, - ct: str = "application/ld+json", - headers: List[Dict[str, Any]]=[], - status_code: int = 200, -) -> Response: - """ - Set the response headers. - :param resp: Response. - :param ct: Content-type default "application/ld+json". - :param headers: List of objects. - :param status_code: status code default 200. - :return: Response with headers. - """ + + # Get the Database Session from g. + # Returns and sets a default Session if not found + # :return session : SQLalchemy Session object + # + + return try_except_block(session,"dbsession",scoped_session(sessionmaker(bind=engine)),g.dbsession) + + +def set_response_headers(resp: Response,ct: str = "application/ld+json",headers: List[Dict[str, Any]]=[],status_code: int = 200,) -> Response: + + # Set the response headers. + # :param resp: Response. + # :param ct: Content-type default "application/ld+json". + # :param headers: List of objects. + # :param status_code: status code default 200. + # :return: Response with headers. + resp.status_code = status_code for header in headers: resp.headers[list(header.keys())[0]] = header[list(header.keys())[0]] @@ -364,12 +328,12 @@ def set_response_headers( def error_response(error: HydraError) -> Response: - """ - Generate the response if there is an error while performing any operation - - :param error: HydraError object which will help in generating response - :type error: HydraError - :return: Error response with appropriate status code - :rtype: Response - """ + + # Generate the response if there is an error while performing any operation + + # :param error: HydraError object which will help in generating response + # :type error: HydraError + # :return: Error response with appropriate status code + # :rtype: Response + return set_response_headers(jsonify(error.generate()), status_code=error.code) diff --git a/setup.py b/setup.py index d46f92df..a789defe 100644 --- a/setup.py +++ b/setup.py @@ -9,7 +9,10 @@ from pip._internal.req import parse_requirements install_requires = parse_requirements("requirements.txt", session=PipSession()) - dependencies = [str(package.requirement) for package in install_requires] + try: + dependencies = [str(package.requirement) for package in install_requires] + except: + dependencies = [str(package.req) for package in install_requires] except ImportError: msg = "Your pip version is out of date, please run `pip install --upgrade pip setuptools`" raise ImportError(msg)