Tracking Issue for Async I/O migration

[jebrosen]

Status

Rocket's master branch is currently fully asynchronous and supports async/await!

Pending

• (async) Remove or replace typed headers #1067 - Remove or replace typed headers (PR: Fix #1067 by importing typed headers from hyperx #1535)
• Fully update documentation for 0.5 #1329 - Documentation fixes, additions, improvements.
• Provide a lower-level connection interface #1070 - Expose APIs for customizing the underlying I/O, instead of being restricted to TCP with optional TLS.
• Detect blocking I/O in async: it seems the best we can do for now is to document the issues.
  A "blocking-in-async" lint in clippy or rustc would be much nicer, if feasible, but would occur upstream...someday.
• Miscellaneous small TODOs (tagged as TODO.async) throughout code.
• Response bodies are passed back to hyper using a channel.

Done

• Async I/O everything!
• Harden and Enforce Data Read Limits #1325 - Data reading and connection limits.
• Keep-alive and idle client timeouts
• Handling of blocking database connections.
• Gracefully handle panics with 500. (PR Catch and gracefully handle panics in routes and catchers. #1491)
• (async) Add a Write-like interface for responses #1066 - Add a Write-like interface for responses, and consideration of AsyncRead vs Stream<Item=Result<Chunk, Error>>
• Clean shutdown? #180, Graceful shutdown timeout #1374 - fixing graceful shutdown when using "infinite" responders
• Support for Server Sent Events #33 - Server-Sent Events
• Tracking Issue for Asynchronous Database Pool Support #1117 - Asynchronous database pools, or at the very least forward compatibility with them.

Design

The async APIs are based on rust's async/await feature and std::future::Future, running on the tokio (0.2) runtime.

• High-level APIs (routes and catchers):
  • Routes can now be defined as #[get|post|route(...)] async fn(params...) -> R where R is any Responder type. Routes that do not await anything can remain unchanged. #[catch]ers can also be defined as an async fn.
  • Methods on Data now return Futures that must be .awaited. As a consequence, any route that uses Data directly must be async. Routes that use data guards such as Json are not affected by this change.
  • Routes that make their own network requests or other I/O should be async, as it will allow other requests to continue while waiting for the operation.
• Mid and low-level APIs (FromTransformedData and FromData, Fairing, Responder)
  • Several traits are now defined using async-trait, re-exported as #[rocket::async_trait].
  • Data is read asynchronously: DataStream implements tokio::io::AsyncRead instead of Read. Thus, FromTransformedData::transform and From(Transformed)Data::from_data are async fn or return Futures.
  • Response bodies are streamed to the client asynchronously: set_body and related methods expect tokio::io::AsyncRead instead of Read.
    • Consequently, Fairing::on_response and Responder::respond_to are now async fn. This allows Responders and Fairings to access and/or replace the body data.
• The testing APIs have been moved to rocket::local::asynchronous and rocket::local::blocking. The blocking test API is simpler to use, and recommended when possible.
• #[rocket::main], #[rocket::launch], and #[rocket::async_test] have been added to make async fn easily accessible for main() and test().
• Many long-standing bugs with keep-alive, graceful shutdown (Clean shutdown? #180), performance, and more are fixed or will be easier to fix by using actively-supported versions of libraries.
• Some new functionality will be possible to implement, such as SSE and WebSockets. These will likely not make it into Rocket 0.5. At this point, Rocket should not be missing any APIs necessary to implement SSE "by hand". WebSockets is a bit more complicated, and tracked in Native WebSocket support #90 .

[jhpratt]

The async_await feature is now required in addition to proc_macro_hygine and decl_macro

Wasn't decl_macro removed on the master branch a while back?

Currently, routes must either be fn -> R where R: Responder (non-async) or async fn -> R where R: Responder (async). Should we try to support (non-async) fn -> impl Future<Output=R> where R: Responder?

I don't see why not. Though it would be relatively trivial to create it async and await the returned value(s).

I'll take a look at uuid tomorrow to see if any changes are necessary. I suspect the answer is no.

@jebrosen Roughly how much of the documentation can be switched over to async? Are we going to primarily document in async, or will we essentially double everything up?

[jebrosen]

Wasn't decl_macro removed on the master branch a while back?

Right! I'll fix that.

Currently, routes must either be fn -> R where R: Responder (non-async) or async fn -> R where R: Responder (async). Should we try to support (non-async) fn -> impl Future<Output=R> where R: Responder?

I don't see why not. Though it would be relatively trivial to create it async and await the returned value(s).

The issue is the return type of routes. Currently the codegen assumes that a non-async function returns some type R that implements Responder and an async function returns a Future whose output type is some R implementing Responder. Allowing functions to return either a Responder or a Future is hairy because both are traits and pretty soon we would end up wrestling with specialization. But I might have missed something in my analysis, and I would really like to support it if we can!

@jebrosen Roughly how much of the documentation can be switched over to async? Are we going to primarily document in async, or will we essentially double everything up?

The only place where there is a choice is routes; currently everything else is an async-only API and documentation should reflect that. I was going to wait a bit before pushing hard for docs, as there are likely a few design issues to be resolved with which APIs return which futures with what lifetime bounds.

My admittedly hand-wavy assumption was that we would explain traits like FromData and Responder in terms of async and futures, but use mostly non-async routes. We should definitely have a few examples of async routes, perhaps for HTTP requests or File I/O. For a lot of routes I do expect it to be as easy as changing fn to async fn and using .await in the right places.

Another really important point here is that manual FromData and Responder implementations should be relatively rare and are much more in the domain of API documentation than in the guide. I am personally more worried about the accessibility and understandability of the guide.

[jhpratt]

The issue is the return type of routes. Currently the codegen assumes that a non-async function returns some type R that implements Responder and an async function returns a Future whose output type is some R implementing Responder. Allowing functions to return either a Responder or a Future is hairy because both are traits and pretty soon we would end up wrestling with specialization. But I might have missed something in my analysis, and I would really like to support it if we can!

That certainly makes sense. Adding support for -> impl Future would be backwards compatible, as it doesn't currently work. So that's a point that can certainly be postponed to see if there's even demand after async lands.

The only place where there is a choice is routes; currently everything else is an async-only API and documentation should reflect that. I was going to wait a bit before pushing hard for docs, as there are likely a few design issues to be resolved with which APIs return which futures with what lifetime bounds.

Probably best to hold off on docs for the time being, then.

[Nemo157]

compression - Not for the faint of heart; there might be some important APIs that are not yet available. HELP WELCOME

Quick advertisement for https://github.com/rustasync/async-compression, this should allow just drop-in wrappers to transparently compress/decompress AsyncRead/AsyncWrite instances. Not all encodings are supported for the read/write interfaces yet as the first framework it was used in used the impl Stream<Item = io::Result<Bytes>> adaptors instead, but it should be straightforward to expand them (at least once I merge the initial AsyncWrite support).

[theduke]

@jebrosen this is awesome!

Just as a side note, both tokio and hyper master are working with std::future::Future now. It's still a WIP but in my testing the essentials are all working, so this could already be based on the master branches instead of dealing with the compat layer.

[jhpratt]

contrib/lib/src/uuid.rs looks good to me as-is. It's just a thin wrapper around the uuid crate, so there's not really any room for making things async, unless we were to go all in and parse it asynchronously.

[alexsherman]

databases - We are not necessarily pushing for asynchronous database engines, but we should at least ensure they will be usable. postgres is a good candidate for exploring this.

Hi - what would you be looking for in a pull request to explore this? I've been developing some projects using Rocket and have been using both tokio-postgres and the synchronous rust-postgres, as well as their connection pools, and I would like to help if I think I know how.

[jebrosen]

Hi - what would you be looking for in a pull request to explore this?

I have not tried too hard to test what "ensure they will be usable" entails. There are a few problems I can see ahead of time:

• Futures-aware database engines probably need to be run within a specific runtime (i.e. tokio 0.1 or tokio 0.2). So once Rocket uses tokio 0.2 ((async) Use the futures 0.3 - compatible versions of hyper and tokio #1071), you wouldn't be able to use the current version of tokio-postgres. This is a non-obvious compatibility problem that isn't expressed in any type signature.
• "Connect" might be an asynchronous operation. This seems to be the case for tokio-postgres, for example. So we should evaluate if making FromRequest return a future would be worth it for making databases (among other things) easier to work with.

At a minimum, I think we should be able to:

• Create a connection pool with tokio-postgres and/or bb8 and put it in managed state. This should tell us if on_attach and on_launch need to be async-capable.
• Create a request guard type wrapping a connection (or a future that will resolve to a connection). This and the previous bullet point together are more or less what the guide demonstrated for databases in Rocket 0.3 before we had rocket_contrib::databases.
• Use said connection via the request guard within a route to query/alter the database. This should work fine, because routes can now be async fn.
• Test, figure out, and document cross-runtime compatibility (or lack thereof). That is, a definitive statement in the form of "Rocket uses tokio 0.2, so your database futures have to as well" if that's the case. Alternatively, maybe we could find a proof of concept of running two runtimes - tokio 0.2 (for rocket) alongside tokio 0.1 (for tokio-postgres) with some kind of message passing between them. I leave this intentionally vague because it's an annoying solution to an annoying problem and someone other than I will probably think of a better idea.

[jebrosen]

I am mostly waiting on #1071 (after a new tokio alpha and hyperium/hyper#1897) so that we don't need to significantly rewrite too much code that directly deals with the low-level bits -- which are pretty different between tokio 0.1 and 0.2. For example, I suspect #1070 will take almost as much work again as it would to implement it now.

At present all public APIs use async/await and futures 0.3, but everything runs on a tokio 0.1 runtime. This means that I/O types from tokio 0.2 cannot be used directly in routes yet, for example.

---

For anyone interested in kicking the tires pretty early on, I have written a short and possibly incomplete guide to trying projects against the async branch. At this stage the most helpful feedback would be examples of "things I can't do anymore" or that now require expensive allocations, especially in implementations of FromData or Fairing.

• Depend on rocket = { git = "https://github.com/SergioBenitez/Rocket", branch = "async" } and rocket_contrib = { git = "https://github.com/SergioBenitez/Rocket", branch = "async" }
• Add #![feature(async_await)] to your crate. You can remove #[feature(decl_macro)] thanks to Remove use of decl_macro by using a different hack. #964.
• Typed headers are removed and a replacement has not been decided on ((async) Remove or replace typed headers #1067), so you will need to build headers manually and/or use raw headers at least for now
• Implementations of FromData, Fairing, AdHoc::on_response, and Responder must be converted to use futures. For some cases, wrapping the function body in Box::pin(async move { }) will be enough. For some traits, just like with FromData for rocket_contrib::json::Json, you might need to copy some data out of the request before creating the future.
• Some dependencies such as ring and handlebars have been updated, which may consequently break your application. You should get obvious build failures if this affects you.
• You can now declare routes as async, and await futures inside of them.

[jhpratt]

@jebrosen have you taken a look at async-trait? I know I've mentioned it before, and it reduces some of the overhead associated with async trait fns, like lifetimes and boxing. As a bonus, all we'd have to do after GATs are (some day) implemented is remove the attribute.

[jebrosen]

I did try async-trait today and ran into some issues that might be able to be worked around. I would prefer to use async-trait on either all affected traits or none of them, and I'm hopeful that it will be usable.