Async Client

[Jayclifford345]

Check whether it is currently possible to create an Aysnc version of the client based on the underlining Arrow library.

1. Write API currently supports Aysnc through parent library inheritance
   The support was removed at 56 remove v2 library dependency #62 (note by @bednar)
2. Query API needs to be investigated

[thopewell]

@Jayclifford345 has any progress been made in the investigation into async on the query api for Influx3.0?

Similar to this discussion/issue in the v2 python client: influxdata/influxdb-python#452
I'm adding an endpoint to an api, the endpoint will query influx 3.0 and return some data, but the framework I'm plugging into uses async.

I'm not sure how best to do this:

• using something like aiohttp and querying the v1 api might work. Its a bit confusing that you have to set up a "dbrp mapping" via a v2 endpoint before query works (https://docs.influxdata.com/influxdb/cloud/query-data/influxql/dbrp/?t=InfluxDB+API#create-dbrp-mappings)
• the python v2 client does support async query, but expects flux
• I'm trying to figure out whether I can use async with the pyarrow.flight given: https://docs.influxdata.com/influxdb/cloud-serverless/reference/client-libraries/flight/python-flight/

[Jayclifford345]

Hi @thopewell,
I am currently struggling to create a clean solution for this since from what I can see underlining pyarrow flight library does not currently have an inherently Async solution. It might be something we need to raise with the greater pyarrow community

[LostInDarkMath]

Yes, an async client would be great :)

[Jayclifford345]

Hi there, I belive we need to wait for the upstream library Pyarrow to implement the necessary functionality before we are able to move forward: apache/arrow#3151

[LostInDarkMath]

Thanks for your fast reply!

The issue you mentioned is closed and there is a linked gist. So I guess the gist is not enough to provide an async influxDB client?

[pvardanis]

Is it possible to run concurrent requests on an Arrow Flight RPC server without using an async FlightClient (since it doesn't exist yet)?

[jules-ch]

I have been delegating queries to threads using the same client. I haven't seen any problems so far. I don't know if the Pyarrow implementation is thread safe.
You can also use aiogrpc since Flight is using GRPC under the hood (tried it, it works but you need to deal with the IPC messages and header bytes). The pyArrow implementation is using the C++ client from the Python bindings with Cython which releases the GIL.
Don't know about performance, just use pure sync and threads I'd say until pyarrow release an asynchronous version.

[karel-rehor]

I've spent a couple of days looking at how this might be implemented, especially considering that work on pyarrow async seems to have stalled.

I would suggest wrapping the synchronous calls to FlightClient.doGet in an executor provided or coordinated by the Python asyncio event loop. For example...

    loop = asyncio.get_running_loop()

    result = await loop.run_in_executor(None, client.query, query, "sql", "")

A summary of my investigations into using asyncio with Influxdb3-python are in the attached markdown file.

Python3Async.md

[bednar]

The query part is done by: #123, the write will be in next iteration.