From 89c63fc18742470a863456356f45767f58a72887 Mon Sep 17 00:00:00 2001 From: Joel Dice Date: Mon, 16 Dec 2024 12:04:41 -0700 Subject: [PATCH 1/9] Expand and clarify consitency/durability docs in store.wit The existing docs are somewhat vague about how the "read your writes" consistency model works in practice, so I've tried to make them more explicit. Also, they don't mention durability at all, so I've added a section dedicated to that. Note that I've generally erred on the side of maximum portability across host implementations at the expense of strong guarantees for the guest. Based on previous conversations, my understanding is that we _do_ want to support implementations backed by eventually consistent distributed systems, and that means portable guest code cannot assume a stronger consistency model than what such systems can deliver. Concretely, we must consider the scenario where a host has a pool of connections to multiple replicas in such a system such that a single component instance which opens the same bucket multiple times might get a different replica (each with its own view of the state) each time. If we feel the guarantees described in these docs are too weak, we can certainly strengthen them at the expense of host implementation flexibility. Alternatively, we could add new APIs for querying and/or controlling the durability and consistency models provided by the implementation -- or even allow the guest to statically declare that it requires some specific consistency model by importing a specific interface corresponding to that model, analogous to what we did with the `atomics` interface. Regardless of what set of (non-)guarantees and features we settle on, my main priority is to be as clear as possible about them so that application developers are not caught by surprise. Signed-off-by: Joel Dice --- wit/store.wit | 65 ++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 54 insertions(+), 11 deletions(-) diff --git a/wit/store.wit b/wit/store.wit index a35c2d4..049261b 100644 --- a/wit/store.wit +++ b/wit/store.wit @@ -7,22 +7,65 @@ /// ensuring compatibility between different key-value stores. Note: the clients will be expecting /// serialization/deserialization overhead to be handled by the key-value store. The value could be /// a serialized object from JSON, HTML or vendor-specific data types like AWS S3 objects. +/// +/// ## Consistency and Cache Coherency /// /// Data consistency in a key value store refers to the guarantee that once a write operation /// completes, all subsequent read operations will return the value that was written. /// /// Any implementation of this interface must have enough consistency to guarantee "reading your -/// writes." In particular, this means that the client should never get a value that is older than -/// the one it wrote, but it MAY get a newer value if one was written around the same time. These -/// guarantees only apply to the same client (which will likely be provided by the host or an -/// external capability of some kind). In this context a "client" is referring to the caller or -/// guest that is consuming this interface. Once a write request is committed by a specific client, -/// all subsequent read requests by the same client will reflect that write or any subsequent -/// writes. Another client running in a different context may or may not immediately see the result -/// due to the replication lag. As an example of all of this, if a value at a given key is A, and -/// the client writes B, then immediately reads, it should get B. If something else writes C in -/// quick succession, then the client may get C. However, a client running in a separate context may -/// still see A or B +/// writes." In particular, this means that a `get` call for a given key on a given `bucket` +/// resource should never return a value that is older than the the last value written to that key +/// on the same resource, but it MAY get a newer value if one was written around the same +/// time. These guarantees only apply to reads and writes on the same resource; they do not hold +/// across multiple resources -- even when those resources were opened using the same string +/// identifier by the same component instance. +/// +/// The following pseudocode example illustrates this behavior. Note that we assume there is +/// initially no value set for any key and that no other writes are happening beyond what is shown +/// in the example. +/// +/// bucketA = open("foo") +/// bucketB = open("foo") +/// bucketA.set("bar", "a") +/// // The following are guaranteed to succeed: +/// assert bucketA.get("bar").equals("a") +/// assert bucketB.get("bar").equals("a") or bucketB.get("bar") is None +/// // ...whereas this is NOT guaranteed to succeed immediately (but should eventually): +/// // assert bucketB.get("bar").equals("a") +/// +/// Once a value is `set` for a given key on a given `bucket`, all subsequent `get` requests on that +/// same bucket will reflect that write or any subsequent writes. `get` requests using a different +/// bucket may or may not immediately see the new value due to e.g. cache effects and/or replication +/// lag. +/// +/// Continuing the above example: +/// +/// bucketB.set("bar", "b") +/// bucketC = open("foo") +/// value = bucketC.get("bar") +/// assert value.equals("a") or value.equals("b") or value is None +/// +/// In other words, the `bucketC` resource may reflect either the most recent write to the `bucketA` +/// resource, or the one to the `bucketB` resource, or neither, depending on how quickly either of +/// those writes reached the replica from which the `bucketC` resource is reading. However, +/// assuming there are no unrecoverable errors -- such that the state of a replica is irretrievably +/// lost before it can be propagated -- one of the values ("a" or "b") will eventually be considered +/// the "latest" and replicated across the system, at which point all three resources will return +/// that same value. +/// +/// ## Durability +/// +/// This interface does not currently make any hard guarantees about the durability of values +/// stored. A valid implementation might rely on an in-memory hash table, the contents of which are +/// lost when the process exits. Alternatively, another implementation might synchronously persist +/// all writes to disk -- or even to a quorum of disk-backed nodes at multiple locations -- before +/// returning a result for a `set` call. Finally, a third implementation might persist values +/// asynchronously on a best-effort basis without blocking `set` calls, in which case an I/O error +/// could occur after the component instance which originally made the call has exited. +/// +/// Future versions of the `wasi-keyvalue` package may provide ways to query and control the +/// durability and consistency provided by the backing implementation. interface store { /// The set of errors which may be raised by functions in this package variant error { From aaeec5425aeb94514574fd66008c16ef44d3196e Mon Sep 17 00:00:00 2001 From: Joel Dice Date: Mon, 16 Dec 2024 15:40:58 -0700 Subject: [PATCH 2/9] rename consistency heading --- wit/store.wit | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/wit/store.wit b/wit/store.wit index 049261b..9879610 100644 --- a/wit/store.wit +++ b/wit/store.wit @@ -8,7 +8,7 @@ /// serialization/deserialization overhead to be handled by the key-value store. The value could be /// a serialized object from JSON, HTML or vendor-specific data types like AWS S3 objects. /// -/// ## Consistency and Cache Coherency +/// ## Consistency /// /// Data consistency in a key value store refers to the guarantee that once a write operation /// completes, all subsequent read operations will return the value that was written. From ee77cef53af4fe11a1d92653881358a854468617 Mon Sep 17 00:00:00 2001 From: Joel Dice Date: Tue, 17 Dec 2024 07:47:52 -0700 Subject: [PATCH 3/9] remove first paragraph in `Consistency` section It was somewhat redundant (and potentially misleading) given that the following paragraph says the same thing less ambiguously and defines exactly in which circumstances the "read your writes" guarantee applies. Signed-off-by: Joel Dice --- wit/store.wit | 3 --- 1 file changed, 3 deletions(-) diff --git a/wit/store.wit b/wit/store.wit index 9879610..f83443e 100644 --- a/wit/store.wit +++ b/wit/store.wit @@ -10,9 +10,6 @@ /// /// ## Consistency /// -/// Data consistency in a key value store refers to the guarantee that once a write operation -/// completes, all subsequent read operations will return the value that was written. -/// /// Any implementation of this interface must have enough consistency to guarantee "reading your /// writes." In particular, this means that a `get` call for a given key on a given `bucket` /// resource should never return a value that is older than the the last value written to that key From b4f3fffb9e5690155c2bb844e2d6903b9d3e898b Mon Sep 17 00:00:00 2001 From: Joel Dice Date: Fri, 17 Jan 2025 10:31:17 -0700 Subject: [PATCH 4/9] expand and clarify consistency wording based on review feedback Signed-off-by: Joel Dice --- wit/store.wit | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/wit/store.wit b/wit/store.wit index f83443e..f749497 100644 --- a/wit/store.wit +++ b/wit/store.wit @@ -11,7 +11,12 @@ /// ## Consistency /// /// Any implementation of this interface must have enough consistency to guarantee "reading your -/// writes." In particular, this means that a `get` call for a given key on a given `bucket` +/// writes" for read operations on the same `bucket` resource instance. Reads from `bucket` +/// resources other than the one used to write are _not_ guaranteed to return the written value +/// given that the other resources may be connected to other replicas in a distributed system, even +/// when opened using the same name bucket identifier. +/// +/// In particular, this means that a `get` call for a given key on a given `bucket` /// resource should never return a value that is older than the the last value written to that key /// on the same resource, but it MAY get a newer value if one was written around the same /// time. These guarantees only apply to reads and writes on the same resource; they do not hold @@ -31,10 +36,10 @@ /// // ...whereas this is NOT guaranteed to succeed immediately (but should eventually): /// // assert bucketB.get("bar").equals("a") /// -/// Once a value is `set` for a given key on a given `bucket`, all subsequent `get` requests on that -/// same bucket will reflect that write or any subsequent writes. `get` requests using a different -/// bucket may or may not immediately see the new value due to e.g. cache effects and/or replication -/// lag. +/// Once a value is `set` for a given key on a given `bucket` resource, all subsequent `get` +/// requests on that same resource will reflect that write or any subsequent writes. `get` requests +/// using a different bucket may or may not immediately see the new value due to e.g. cache effects +/// and/or replication lag. /// /// Continuing the above example: /// @@ -107,7 +112,14 @@ interface store { /// 6. Memcached calls a collection of key-value pairs a slab /// 7. Azure Cosmos DB calls a collection of key-value pairs a container /// - /// In this interface, we use the term `bucket` to refer to a collection of key-value pairs + /// In this interface, we use the term `bucket` to refer to a connection to a collection of + /// key-value pairs. + /// + /// Note that opening two `bucket` resources using the same identifier may result in connections + /// to two separate replicas in a distributed database, and that writes to one of those + /// resources are not guaranteed to be readable from the other resource promptly (or ever, in + /// the case of a replica failure). See the `Consistency` section of the `store` interface + /// documentation for details. resource bucket { /// Get the value associated with the specified `key` /// From 5b3c65babe2e5d2855a4eb6ae4af198dcfe9964b Mon Sep 17 00:00:00 2001 From: Joel Dice Date: Tue, 21 Jan 2025 09:39:18 -0700 Subject: [PATCH 5/9] update consistency docs to use RFC-2119-style terminology Signed-off-by: Joel Dice --- wit/store.wit | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/wit/store.wit b/wit/store.wit index f749497..8463fa9 100644 --- a/wit/store.wit +++ b/wit/store.wit @@ -10,14 +10,14 @@ /// /// ## Consistency /// -/// Any implementation of this interface must have enough consistency to guarantee "reading your +/// Any implementation of this interface MUST have enough consistency to guarantee "reading your /// writes" for read operations on the same `bucket` resource instance. Reads from `bucket` /// resources other than the one used to write are _not_ guaranteed to return the written value /// given that the other resources may be connected to other replicas in a distributed system, even -/// when opened using the same name bucket identifier. +/// when opened using the same bucket identifier. /// /// In particular, this means that a `get` call for a given key on a given `bucket` -/// resource should never return a value that is older than the the last value written to that key +/// resource MUST never return a value that is older than the the last value written to that key /// on the same resource, but it MAY get a newer value if one was written around the same /// time. These guarantees only apply to reads and writes on the same resource; they do not hold /// across multiple resources -- even when those resources were opened using the same string @@ -33,7 +33,7 @@ /// // The following are guaranteed to succeed: /// assert bucketA.get("bar").equals("a") /// assert bucketB.get("bar").equals("a") or bucketB.get("bar") is None -/// // ...whereas this is NOT guaranteed to succeed immediately (but should eventually): +/// // ...whereas this is NOT guaranteed to succeed immediately (but SHOULD eventually): /// // assert bucketB.get("bar").equals("a") /// /// Once a value is `set` for a given key on a given `bucket` resource, all subsequent `get` @@ -48,13 +48,13 @@ /// value = bucketC.get("bar") /// assert value.equals("a") or value.equals("b") or value is None /// -/// In other words, the `bucketC` resource may reflect either the most recent write to the `bucketA` +/// In other words, the `bucketC` resource MAY reflect either the most recent write to the `bucketA` /// resource, or the one to the `bucketB` resource, or neither, depending on how quickly either of /// those writes reached the replica from which the `bucketC` resource is reading. However, /// assuming there are no unrecoverable errors -- such that the state of a replica is irretrievably -/// lost before it can be propagated -- one of the values ("a" or "b") will eventually be considered -/// the "latest" and replicated across the system, at which point all three resources will return -/// that same value. +/// lost before it can be propagated -- one of the values ("a" or "b") SHOULD eventually be +/// considered the "latest" and replicated across the system, at which point all three resources +/// will return that same value. /// /// ## Durability /// @@ -115,7 +115,7 @@ interface store { /// In this interface, we use the term `bucket` to refer to a connection to a collection of /// key-value pairs. /// - /// Note that opening two `bucket` resources using the same identifier may result in connections + /// Note that opening two `bucket` resources using the same identifier MAY result in connections /// to two separate replicas in a distributed database, and that writes to one of those /// resources are not guaranteed to be readable from the other resource promptly (or ever, in /// the case of a replica failure). See the `Consistency` section of the `store` interface From 94183c2e44eee51b60958c05f276f5670faf5677 Mon Sep 17 00:00:00 2001 From: Joel Dice Date: Tue, 21 Jan 2025 13:43:18 -0700 Subject: [PATCH 6/9] update markdown files Signed-off-by: Joel Dice --- imports.md | 71 ++++++++++++++++++++++++++++++++++++++---------- watch-service.md | 71 ++++++++++++++++++++++++++++++++++++++---------- 2 files changed, 112 insertions(+), 30 deletions(-) diff --git a/imports.md b/imports.md index ca31a78..756e64a 100644 --- a/imports.md +++ b/imports.md @@ -23,20 +23,55 @@ the common denominator for all data types defined by different key-value stores ensuring compatibility between different key-value stores. Note: the clients will be expecting serialization/deserialization overhead to be handled by the key-value store. The value could be a serialized object from JSON, HTML or vendor-specific data types like AWS S3 objects.

-

Data consistency in a key value store refers to the guarantee that once a write operation -completes, all subsequent read operations will return the value that was written.

-

Any implementation of this interface must have enough consistency to guarantee "reading your -writes." In particular, this means that the client should never get a value that is older than -the one it wrote, but it MAY get a newer value if one was written around the same time. These -guarantees only apply to the same client (which will likely be provided by the host or an -external capability of some kind). In this context a "client" is referring to the caller or -guest that is consuming this interface. Once a write request is committed by a specific client, -all subsequent read requests by the same client will reflect that write or any subsequent -writes. Another client running in a different context may or may not immediately see the result -due to the replication lag. As an example of all of this, if a value at a given key is A, and -the client writes B, then immediately reads, it should get B. If something else writes C in -quick succession, then the client may get C. However, a client running in a separate context may -still see A or B

+

Consistency

+

Any implementation of this interface MUST have enough consistency to guarantee "reading your +writes" for read operations on the same bucket resource instance. Reads from bucket +resources other than the one used to write are not guaranteed to return the written value +given that the other resources may be connected to other replicas in a distributed system, even +when opened using the same bucket identifier.

+

In particular, this means that a get call for a given key on a given bucket +resource MUST never return a value that is older than the the last value written to that key +on the same resource, but it MAY get a newer value if one was written around the same +time. These guarantees only apply to reads and writes on the same resource; they do not hold +across multiple resources -- even when those resources were opened using the same string +identifier by the same component instance.

+

The following pseudocode example illustrates this behavior. Note that we assume there is +initially no value set for any key and that no other writes are happening beyond what is shown +in the example.

+

bucketA = open("foo") +bucketB = open("foo") +bucketA.set("bar", "a") +// The following are guaranteed to succeed: +assert bucketA.get("bar").equals("a") +assert bucketB.get("bar").equals("a") or bucketB.get("bar") is None +// ...whereas this is NOT guaranteed to succeed immediately (but SHOULD eventually): +// assert bucketB.get("bar").equals("a")

+

Once a value is set for a given key on a given bucket resource, all subsequent get +requests on that same resource will reflect that write or any subsequent writes. get requests +using a different bucket may or may not immediately see the new value due to e.g. cache effects +and/or replication lag.

+

Continuing the above example:

+

bucketB.set("bar", "b") +bucketC = open("foo") +value = bucketC.get("bar") +assert value.equals("a") or value.equals("b") or value is None

+

In other words, the bucketC resource MAY reflect either the most recent write to the bucketA +resource, or the one to the bucketB resource, or neither, depending on how quickly either of +those writes reached the replica from which the bucketC resource is reading. However, +assuming there are no unrecoverable errors -- such that the state of a replica is irretrievably +lost before it can be propagated -- one of the values ("a" or "b") SHOULD eventually be +considered the "latest" and replicated across the system, at which point all three resources +will return that same value.

+

Durability

+

This interface does not currently make any hard guarantees about the durability of values +stored. A valid implementation might rely on an in-memory hash table, the contents of which are +lost when the process exits. Alternatively, another implementation might synchronously persist +all writes to disk -- or even to a quorum of disk-backed nodes at multiple locations -- before +returning a result for a set call. Finally, a third implementation might persist values +asynchronously on a best-effort basis without blocking set calls, in which case an I/O error +could occur after the component instance which originally made the call has exited.

+

Future versions of the wasi-keyvalue package may provide ways to query and control the +durability and consistency provided by the backing implementation.

Types

variant error

@@ -85,7 +120,13 @@ depending on the specific implementation. For example:

  • Memcached calls a collection of key-value pairs a slab
  • Azure Cosmos DB calls a collection of key-value pairs a container
    • -

    In this interface, we use the term bucket to refer to a collection of key-value pairs

    +

    In this interface, we use the term bucket to refer to a connection to a collection of +key-value pairs.

    +

    Note that opening two bucket resources using the same identifier MAY result in connections +to two separate replicas in a distributed database, and that writes to one of those +resources are not guaranteed to be readable from the other resource promptly (or ever, in +the case of a replica failure). See the Consistency section of the store interface +documentation for details.

    Functions

    open: func

    Get the bucket with the specified identifier.

    diff --git a/watch-service.md b/watch-service.md index a65115a..2e17b04 100644 --- a/watch-service.md +++ b/watch-service.md @@ -21,20 +21,55 @@ the common denominator for all data types defined by different key-value stores ensuring compatibility between different key-value stores. Note: the clients will be expecting serialization/deserialization overhead to be handled by the key-value store. The value could be a serialized object from JSON, HTML or vendor-specific data types like AWS S3 objects.

    -

    Data consistency in a key value store refers to the guarantee that once a write operation -completes, all subsequent read operations will return the value that was written.

    -

    Any implementation of this interface must have enough consistency to guarantee "reading your -writes." In particular, this means that the client should never get a value that is older than -the one it wrote, but it MAY get a newer value if one was written around the same time. These -guarantees only apply to the same client (which will likely be provided by the host or an -external capability of some kind). In this context a "client" is referring to the caller or -guest that is consuming this interface. Once a write request is committed by a specific client, -all subsequent read requests by the same client will reflect that write or any subsequent -writes. Another client running in a different context may or may not immediately see the result -due to the replication lag. As an example of all of this, if a value at a given key is A, and -the client writes B, then immediately reads, it should get B. If something else writes C in -quick succession, then the client may get C. However, a client running in a separate context may -still see A or B

    +

    Consistency

    +

    Any implementation of this interface MUST have enough consistency to guarantee "reading your +writes" for read operations on the same bucket resource instance. Reads from bucket +resources other than the one used to write are not guaranteed to return the written value +given that the other resources may be connected to other replicas in a distributed system, even +when opened using the same bucket identifier.

    +

    In particular, this means that a get call for a given key on a given bucket +resource MUST never return a value that is older than the the last value written to that key +on the same resource, but it MAY get a newer value if one was written around the same +time. These guarantees only apply to reads and writes on the same resource; they do not hold +across multiple resources -- even when those resources were opened using the same string +identifier by the same component instance.

    +

    The following pseudocode example illustrates this behavior. Note that we assume there is +initially no value set for any key and that no other writes are happening beyond what is shown +in the example.

    +

    bucketA = open("foo") +bucketB = open("foo") +bucketA.set("bar", "a") +// The following are guaranteed to succeed: +assert bucketA.get("bar").equals("a") +assert bucketB.get("bar").equals("a") or bucketB.get("bar") is None +// ...whereas this is NOT guaranteed to succeed immediately (but SHOULD eventually): +// assert bucketB.get("bar").equals("a")

    +

    Once a value is set for a given key on a given bucket resource, all subsequent get +requests on that same resource will reflect that write or any subsequent writes. get requests +using a different bucket may or may not immediately see the new value due to e.g. cache effects +and/or replication lag.

    +

    Continuing the above example:

    +

    bucketB.set("bar", "b") +bucketC = open("foo") +value = bucketC.get("bar") +assert value.equals("a") or value.equals("b") or value is None

    +

    In other words, the bucketC resource MAY reflect either the most recent write to the bucketA +resource, or the one to the bucketB resource, or neither, depending on how quickly either of +those writes reached the replica from which the bucketC resource is reading. However, +assuming there are no unrecoverable errors -- such that the state of a replica is irretrievably +lost before it can be propagated -- one of the values ("a" or "b") SHOULD eventually be +considered the "latest" and replicated across the system, at which point all three resources +will return that same value.

    +

    Durability

    +

    This interface does not currently make any hard guarantees about the durability of values +stored. A valid implementation might rely on an in-memory hash table, the contents of which are +lost when the process exits. Alternatively, another implementation might synchronously persist +all writes to disk -- or even to a quorum of disk-backed nodes at multiple locations -- before +returning a result for a set call. Finally, a third implementation might persist values +asynchronously on a best-effort basis without blocking set calls, in which case an I/O error +could occur after the component instance which originally made the call has exited.

    +

    Future versions of the wasi-keyvalue package may provide ways to query and control the +durability and consistency provided by the backing implementation.

    Types

    variant error

    @@ -83,7 +118,13 @@ depending on the specific implementation. For example:

  • Memcached calls a collection of key-value pairs a slab
  • Azure Cosmos DB calls a collection of key-value pairs a container
    • -

    In this interface, we use the term bucket to refer to a collection of key-value pairs

    +

    In this interface, we use the term bucket to refer to a connection to a collection of +key-value pairs.

    +

    Note that opening two bucket resources using the same identifier MAY result in connections +to two separate replicas in a distributed database, and that writes to one of those +resources are not guaranteed to be readable from the other resource promptly (or ever, in +the case of a replica failure). See the Consistency section of the store interface +documentation for details.

    Functions

    open: func

    Get the bucket with the specified identifier.

    From cf03f8a169093b6cbc903846230059d7aeb6a97f Mon Sep 17 00:00:00 2001 From: Joel Dice Date: Mon, 3 Mar 2025 11:47:42 -0700 Subject: [PATCH 7/9] remove read-your-writes guarantee This removes the "read-your-writes" guarantee since several of the backing stores we wish to support either do not support it or do not support it by default. Signed-off-by: Joel Dice --- imports.md | 69 +++++++++++++++++++++++++++--------------------- watch-service.md | 69 +++++++++++++++++++++++++++--------------------- wit/store.wit | 61 +++++++++++++++++++++++------------------- 3 files changed, 112 insertions(+), 87 deletions(-) diff --git a/imports.md b/imports.md index 756e64a..0c11a41 100644 --- a/imports.md +++ b/imports.md @@ -24,44 +24,53 @@ ensuring compatibility between different key-value stores. Note: the clients wil serialization/deserialization overhead to be handled by the key-value store. The value could be a serialized object from JSON, HTML or vendor-specific data types like AWS S3 objects.

    Consistency

    -

    Any implementation of this interface MUST have enough consistency to guarantee "reading your -writes" for read operations on the same bucket resource instance. Reads from bucket -resources other than the one used to write are not guaranteed to return the written value -given that the other resources may be connected to other replicas in a distributed system, even -when opened using the same bucket identifier.

    -

    In particular, this means that a get call for a given key on a given bucket -resource MUST never return a value that is older than the the last value written to that key -on the same resource, but it MAY get a newer value if one was written around the same -time. These guarantees only apply to reads and writes on the same resource; they do not hold -across multiple resources -- even when those resources were opened using the same string -identifier by the same component instance.

    -

    The following pseudocode example illustrates this behavior. Note that we assume there is -initially no value set for any key and that no other writes are happening beyond what is shown -in the example.

    -

    bucketA = open("foo") +

    An implementation of this interface MUST be eventually consistent, meaning that, after some time +with no further updates, all replicas in the (potentially distributed) system will eventually +converge on a consistent state for all values. This allows replicas to temporarily diverge to +ensure low latency and high availability. Implementations based on a centralized or local +backing store may provide a stronger consistency model, but guest components which are intended +to be portable to any wasi-keyvalue implementation should not rely on anything stronger than +eventual consistency.

    +

    Given that each bucket resource may represent a connection to a different replica in a +distributed system, values read for a given key from two different buckets may differ, even if +those bucket resources were opened using the same string identifier. In addition, consecutive +operations on a single bucket resource may produce temporarily inconsistent results if +e.g. the implementation is forced to reconnect to a different replica due to a connection +failure. For example, a write followed by a read may not return the value just written, even if +no other recent or subsequent writes have occurred.

    +

    Consider the following pseudocode example (and assume we start with an empty store and no other +concurrent activity):

    +
    bucketA = open("foo")
 bucketB = open("foo")
 bucketA.set("bar", "a")
-// The following are guaranteed to succeed:
-assert bucketA.get("bar").equals("a")
+
+// These are guaranteed to succeed:
+assert bucketA.get("bar").equals("a") or bucketA.get("bar") is None
 assert bucketB.get("bar").equals("a") or bucketB.get("bar") is None
-// ...whereas this is NOT guaranteed to succeed immediately (but SHOULD eventually):
-// assert bucketB.get("bar").equals("a")
    
-
    Once a value is set for a given key on a given bucket resource, all subsequent get
-requests on that same resource will reflect that write or any subsequent writes. get requests
-using a different bucket may or may not immediately see the new value due to e.g. cache effects
-and/or replication lag.
    
-
    Continuing the above example:
    
-
    bucketB.set("bar", "b")
+
+// This is likely to succeed, but not guaranteed; e.g. `bucketA` might need to reconnect to a
+// different replica which hasn't received the above write yet.  It will _eventually_
+// succeed, provided there are no irrecoverable errors which prevent the propagation of the
+// write.
+assert bucketA.get("bar").equals("a")
+
+// Likewise, this will _eventually_ succeed in the absence of irrecoverable errors:
+assert bucketB.get("bar").equals("a")
+
+bucketB.set("bar", "b")
 bucketC = open("foo")
 value = bucketC.get("bar")
-assert value.equals("a") or value.equals("b") or value is None
    
+
+// This is guaranteed to succeed:
+assert value.equals("a") or value.equals("b") or value is None
+

    In other words, the bucketC resource MAY reflect either the most recent write to the bucketA resource, or the one to the bucketB resource, or neither, depending on how quickly either of those writes reached the replica from which the bucketC resource is reading. However, -assuming there are no unrecoverable errors -- such that the state of a replica is irretrievably -lost before it can be propagated -- one of the values ("a" or "b") SHOULD eventually be -considered the "latest" and replicated across the system, at which point all three resources -will return that same value.

    +assuming there are no irrecoverable errors -- such that the state of a replica is irretrievably +lost before it can be propagated -- one of the values ("a" or "b") MUST eventually be considered +the "latest" and replicated across the system, at which point all three resources will return +that same value.

    Durability

    This interface does not currently make any hard guarantees about the durability of values stored. A valid implementation might rely on an in-memory hash table, the contents of which are diff --git a/watch-service.md b/watch-service.md index 2e17b04..fe83324 100644 --- a/watch-service.md +++ b/watch-service.md @@ -22,44 +22,53 @@ ensuring compatibility between different key-value stores. Note: the clients wil serialization/deserialization overhead to be handled by the key-value store. The value could be a serialized object from JSON, HTML or vendor-specific data types like AWS S3 objects.

    Consistency

    -

    Any implementation of this interface MUST have enough consistency to guarantee "reading your -writes" for read operations on the same bucket resource instance. Reads from bucket -resources other than the one used to write are not guaranteed to return the written value -given that the other resources may be connected to other replicas in a distributed system, even -when opened using the same bucket identifier.

    -

    In particular, this means that a get call for a given key on a given bucket -resource MUST never return a value that is older than the the last value written to that key -on the same resource, but it MAY get a newer value if one was written around the same -time. These guarantees only apply to reads and writes on the same resource; they do not hold -across multiple resources -- even when those resources were opened using the same string -identifier by the same component instance.

    -

    The following pseudocode example illustrates this behavior. Note that we assume there is -initially no value set for any key and that no other writes are happening beyond what is shown -in the example.

    -

    bucketA = open("foo") +

    An implementation of this interface MUST be eventually consistent, meaning that, after some time +with no further updates, all replicas in the (potentially distributed) system will eventually +converge on a consistent state for all values. This allows replicas to temporarily diverge to +ensure low latency and high availability. Implementations based on a centralized or local +backing store may provide a stronger consistency model, but guest components which are intended +to be portable to any wasi-keyvalue implementation should not rely on anything stronger than +eventual consistency.

    +

    Given that each bucket resource may represent a connection to a different replica in a +distributed system, values read for a given key from two different buckets may differ, even if +those bucket resources were opened using the same string identifier. In addition, consecutive +operations on a single bucket resource may produce temporarily inconsistent results if +e.g. the implementation is forced to reconnect to a different replica due to a connection +failure. For example, a write followed by a read may not return the value just written, even if +no other recent or subsequent writes have occurred.

    +

    Consider the following pseudocode example (and assume we start with an empty store and no other +concurrent activity):

    +
    bucketA = open("foo")
 bucketB = open("foo")
 bucketA.set("bar", "a")
-// The following are guaranteed to succeed:
-assert bucketA.get("bar").equals("a")
+
+// These are guaranteed to succeed:
+assert bucketA.get("bar").equals("a") or bucketA.get("bar") is None
 assert bucketB.get("bar").equals("a") or bucketB.get("bar") is None
-// ...whereas this is NOT guaranteed to succeed immediately (but SHOULD eventually):
-// assert bucketB.get("bar").equals("a")
    
-
    Once a value is set for a given key on a given bucket resource, all subsequent get
-requests on that same resource will reflect that write or any subsequent writes. get requests
-using a different bucket may or may not immediately see the new value due to e.g. cache effects
-and/or replication lag.
    
-
    Continuing the above example:
    
-
    bucketB.set("bar", "b")
+
+// This is likely to succeed, but not guaranteed; e.g. `bucketA` might need to reconnect to a
+// different replica which hasn't received the above write yet.  It will _eventually_
+// succeed, provided there are no irrecoverable errors which prevent the propagation of the
+// write.
+assert bucketA.get("bar").equals("a")
+
+// Likewise, this will _eventually_ succeed in the absence of irrecoverable errors:
+assert bucketB.get("bar").equals("a")
+
+bucketB.set("bar", "b")
 bucketC = open("foo")
 value = bucketC.get("bar")
-assert value.equals("a") or value.equals("b") or value is None
    
+
+// This is guaranteed to succeed:
+assert value.equals("a") or value.equals("b") or value is None
+

    In other words, the bucketC resource MAY reflect either the most recent write to the bucketA resource, or the one to the bucketB resource, or neither, depending on how quickly either of those writes reached the replica from which the bucketC resource is reading. However, -assuming there are no unrecoverable errors -- such that the state of a replica is irretrievably -lost before it can be propagated -- one of the values ("a" or "b") SHOULD eventually be -considered the "latest" and replicated across the system, at which point all three resources -will return that same value.

    +assuming there are no irrecoverable errors -- such that the state of a replica is irretrievably +lost before it can be propagated -- one of the values ("a" or "b") MUST eventually be considered +the "latest" and replicated across the system, at which point all three resources will return +that same value.

    Durability

    This interface does not currently make any hard guarantees about the durability of values stored. A valid implementation might rely on an in-memory hash table, the contents of which are diff --git a/wit/store.wit b/wit/store.wit index 8463fa9..93e751a 100644 --- a/wit/store.wit +++ b/wit/store.wit @@ -10,51 +10,58 @@ /// /// ## Consistency /// -/// Any implementation of this interface MUST have enough consistency to guarantee "reading your -/// writes" for read operations on the same `bucket` resource instance. Reads from `bucket` -/// resources other than the one used to write are _not_ guaranteed to return the written value -/// given that the other resources may be connected to other replicas in a distributed system, even -/// when opened using the same bucket identifier. +/// An implementation of this interface MUST be eventually consistent, meaning that, after some time +/// with no further updates, all replicas in the (potentially distributed) system will eventually +/// converge on a consistent state for all values. This allows replicas to temporarily diverge to +/// ensure low latency and high availability. Implementations based on a centralized or local +/// backing store may provide a stronger consistency model, but guest components which are intended +/// to be portable to any `wasi-keyvalue` implementation should not rely on anything stronger than +/// eventual consistency. /// -/// In particular, this means that a `get` call for a given key on a given `bucket` -/// resource MUST never return a value that is older than the the last value written to that key -/// on the same resource, but it MAY get a newer value if one was written around the same -/// time. These guarantees only apply to reads and writes on the same resource; they do not hold -/// across multiple resources -- even when those resources were opened using the same string -/// identifier by the same component instance. +/// Given that each `bucket` resource may represent a connection to a different replica in a +/// distributed system, values read for a given key from two different `bucket`s may differ, even if +/// those `bucket` resources were opened using the same string identifier. In addition, consecutive +/// operations on a single `bucket` resource may produce temporarily inconsistent results if +/// e.g. the implementation is forced to reconnect to a different replica due to a connection +/// failure. For example, a write followed by a read may not return the value just written, even if +/// no other recent or subsequent writes have occurred. /// -/// The following pseudocode example illustrates this behavior. Note that we assume there is -/// initially no value set for any key and that no other writes are happening beyond what is shown -/// in the example. +/// Consider the following pseudocode example (and assume we start with an empty store and no other +/// concurrent activity): /// +/// ``` /// bucketA = open("foo") /// bucketB = open("foo") /// bucketA.set("bar", "a") -/// // The following are guaranteed to succeed: -/// assert bucketA.get("bar").equals("a") +/// +/// // These are guaranteed to succeed: +/// assert bucketA.get("bar").equals("a") or bucketA.get("bar") is None /// assert bucketB.get("bar").equals("a") or bucketB.get("bar") is None -/// // ...whereas this is NOT guaranteed to succeed immediately (but SHOULD eventually): -/// // assert bucketB.get("bar").equals("a") /// -/// Once a value is `set` for a given key on a given `bucket` resource, all subsequent `get` -/// requests on that same resource will reflect that write or any subsequent writes. `get` requests -/// using a different bucket may or may not immediately see the new value due to e.g. cache effects -/// and/or replication lag. +/// // This is likely to succeed, but not guaranteed; e.g. `bucketA` might need to reconnect to a +/// // different replica which hasn't received the above write yet. It will _eventually_ +/// // succeed, provided there are no irrecoverable errors which prevent the propagation of the +/// // write. +/// assert bucketA.get("bar").equals("a") /// -/// Continuing the above example: +/// // Likewise, this will _eventually_ succeed in the absence of irrecoverable errors: +/// assert bucketB.get("bar").equals("a") /// /// bucketB.set("bar", "b") /// bucketC = open("foo") /// value = bucketC.get("bar") +/// +/// // This is guaranteed to succeed: /// assert value.equals("a") or value.equals("b") or value is None +/// ``` /// /// In other words, the `bucketC` resource MAY reflect either the most recent write to the `bucketA` /// resource, or the one to the `bucketB` resource, or neither, depending on how quickly either of /// those writes reached the replica from which the `bucketC` resource is reading. However, -/// assuming there are no unrecoverable errors -- such that the state of a replica is irretrievably -/// lost before it can be propagated -- one of the values ("a" or "b") SHOULD eventually be -/// considered the "latest" and replicated across the system, at which point all three resources -/// will return that same value. +/// assuming there are no irrecoverable errors -- such that the state of a replica is irretrievably +/// lost before it can be propagated -- one of the values ("a" or "b") MUST eventually be considered +/// the "latest" and replicated across the system, at which point all three resources will return +/// that same value. /// /// ## Durability /// From 418d48fcbee746cfe03302302668e0f611b618b2 Mon Sep 17 00:00:00 2001 From: Joel Dice Date: Wed, 18 Jun 2025 14:53:59 -0600 Subject: [PATCH 8/9] update store.wit to discuss eventual consistency My mental model of what eventual consistency does and does not guarantee was inaccurate, meaning much of what I wrote earlier was incorrect. I _believe_ this new version gets it right; we'll see. Yay for PR review! Signed-off-by: Joel Dice --- imports.md | 88 ++++++++++++++++++++---------------------------- watch-service.md | 88 ++++++++++++++++++++---------------------------- wit/store.wit | 80 ++++++++++++++++--------------------------- 3 files changed, 103 insertions(+), 153 deletions(-) diff --git a/imports.md b/imports.md index 0c11a41..7e4679c 100644 --- a/imports.md +++ b/imports.md @@ -24,53 +24,39 @@ ensuring compatibility between different key-value stores. Note: the clients wil serialization/deserialization overhead to be handled by the key-value store. The value could be a serialized object from JSON, HTML or vendor-specific data types like AWS S3 objects.

    Consistency

    -

    An implementation of this interface MUST be eventually consistent, meaning that, after some time -with no further updates, all replicas in the (potentially distributed) system will eventually -converge on a consistent state for all values. This allows replicas to temporarily diverge to -ensure low latency and high availability. Implementations based on a centralized or local -backing store may provide a stronger consistency model, but guest components which are intended -to be portable to any wasi-keyvalue implementation should not rely on anything stronger than -eventual consistency.

    -

    Given that each bucket resource may represent a connection to a different replica in a -distributed system, values read for a given key from two different buckets may differ, even if -those bucket resources were opened using the same string identifier. In addition, consecutive -operations on a single bucket resource may produce temporarily inconsistent results if -e.g. the implementation is forced to reconnect to a different replica due to a connection -failure. For example, a write followed by a read may not return the value just written, even if -no other recent or subsequent writes have occurred.

    -

    Consider the following pseudocode example (and assume we start with an empty store and no other -concurrent activity):

    -
    bucketA = open("foo")
-bucketB = open("foo")
-bucketA.set("bar", "a")
-
-// These are guaranteed to succeed:
-assert bucketA.get("bar").equals("a") or bucketA.get("bar") is None
-assert bucketB.get("bar").equals("a") or bucketB.get("bar") is None
-
-// This is likely to succeed, but not guaranteed; e.g. `bucketA` might need to reconnect to a
-// different replica which hasn't received the above write yet.  It will _eventually_
-// succeed, provided there are no irrecoverable errors which prevent the propagation of the
-// write.
-assert bucketA.get("bar").equals("a")
-
-// Likewise, this will _eventually_ succeed in the absence of irrecoverable errors:
-assert bucketB.get("bar").equals("a")
-
-bucketB.set("bar", "b")
-bucketC = open("foo")
-value = bucketC.get("bar")
-
-// This is guaranteed to succeed:
-assert value.equals("a") or value.equals("b") or value is None
-
    -

    In other words, the bucketC resource MAY reflect either the most recent write to the bucketA -resource, or the one to the bucketB resource, or neither, depending on how quickly either of -those writes reached the replica from which the bucketC resource is reading. However, -assuming there are no irrecoverable errors -- such that the state of a replica is irretrievably -lost before it can be propagated -- one of the values ("a" or "b") MUST eventually be considered -the "latest" and replicated across the system, at which point all three resources will return -that same value.

    +

    An implementation of this interface MUST be eventually consistent, but is not required to +provide any consistency guaranteeds beyond that. Practically speaking, eventual consistency is +among the weakest of consistency models, guaranteeing only that values will not be produced +"from nowhere", i.e. any value read is guaranteed to have been written to that key at some +earlier time. Beyond that, there are no guarantees, and thus a portable component must neither +expect nor rely on anything else.

    +

    In the future, additional interfaces may be added to wasi:key-value with stronger guarantees, +which will allow components to express their requirements by importing whichever interface(s) +provides matching (or stronger) guarantees. For example, a component requiring strict +serializability might import a (currently hypothetical) strict-serializable-store interface +with a similar signature to store but with much stronger semantic guarantees. On the other +end, a host might either support implementations of both the store and +strict-serializable-store or just the former, in which case the host would immediately reject +a component which imports the unsupported interface.

    +

    Here are a few examples of behavior which an component developer might wish to rely on but which +are NOT guaranteed by an eventually consistent system (e.g. a distributed system composed of +multiple replicas, each of which may receive writes in a different order, making no attempt to +converge on a global consensus):

    +
      +
    • +

      Read-your-own-writes: eventual consistency does NOT guarantee that a write to a given key +followed by a read from the same key will retrieve the same or newer value.

      +
      • +
    • +

      Convergence: eventual consistency does NOT guarantee that any two replicas will agree on the +value for a given key -- even after all writes have had time to propagate to all replicas.

      +
      • +
    • +

      Last-write-wins: eventual consistency does NOT guarantee that the most recent write will +take precendence over an earlier one; old writes may overwrite newer ones temporarily or +permanently.

      +
      • +

    Durability

    This interface does not currently make any hard guarantees about the durability of values stored. A valid implementation might rely on an in-memory hash table, the contents of which are @@ -79,7 +65,7 @@ all writes to disk -- or even to a quorum of disk-backed nodes at multiple locat returning a result for a set call. Finally, a third implementation might persist values asynchronously on a best-effort basis without blocking set calls, in which case an I/O error could occur after the component instance which originally made the call has exited.

    -

    Future versions of the wasi-keyvalue package may provide ways to query and control the +

    Future versions of the wasi:keyvalue package may provide ways to query and control the durability and consistency provided by the backing implementation.

    Types

    @@ -118,7 +104,7 @@ there are no more keys to fetch.

    resource bucket

    A bucket is a collection of key-value pairs. Each key-value pair is stored as a entry in the bucket, and the bucket itself acts as a collection of all these entries.

    -

    It is worth noting that the exact terminology for bucket in key-value stores can very +

    It is worth noting that the exact terminology for bucket in key-value stores can vary depending on the specific implementation. For example:

    1. Amazon DynamoDB calls a collection of key-value pairs a table
      2. @@ -134,8 +120,8 @@ key-value pairs.

      Note that opening two bucket resources using the same identifier MAY result in connections to two separate replicas in a distributed database, and that writes to one of those resources are not guaranteed to be readable from the other resource promptly (or ever, in -the case of a replica failure). See the Consistency section of the store interface -documentation for details.

      +the case of a replica failure or message reordering). See the Consistency section of the +store interface documentation for details.

      Functions

      open: func

      Get the bucket with the specified identifier.

      diff --git a/watch-service.md b/watch-service.md index fe83324..496b694 100644 --- a/watch-service.md +++ b/watch-service.md @@ -22,53 +22,39 @@ ensuring compatibility between different key-value stores. Note: the clients wil serialization/deserialization overhead to be handled by the key-value store. The value could be a serialized object from JSON, HTML or vendor-specific data types like AWS S3 objects.

      Consistency

      -

      An implementation of this interface MUST be eventually consistent, meaning that, after some time -with no further updates, all replicas in the (potentially distributed) system will eventually -converge on a consistent state for all values. This allows replicas to temporarily diverge to -ensure low latency and high availability. Implementations based on a centralized or local -backing store may provide a stronger consistency model, but guest components which are intended -to be portable to any wasi-keyvalue implementation should not rely on anything stronger than -eventual consistency.

      -

      Given that each bucket resource may represent a connection to a different replica in a -distributed system, values read for a given key from two different buckets may differ, even if -those bucket resources were opened using the same string identifier. In addition, consecutive -operations on a single bucket resource may produce temporarily inconsistent results if -e.g. the implementation is forced to reconnect to a different replica due to a connection -failure. For example, a write followed by a read may not return the value just written, even if -no other recent or subsequent writes have occurred.

      -

      Consider the following pseudocode example (and assume we start with an empty store and no other -concurrent activity):

      -
      bucketA = open("foo")
-bucketB = open("foo")
-bucketA.set("bar", "a")
-
-// These are guaranteed to succeed:
-assert bucketA.get("bar").equals("a") or bucketA.get("bar") is None
-assert bucketB.get("bar").equals("a") or bucketB.get("bar") is None
-
-// This is likely to succeed, but not guaranteed; e.g. `bucketA` might need to reconnect to a
-// different replica which hasn't received the above write yet.  It will _eventually_
-// succeed, provided there are no irrecoverable errors which prevent the propagation of the
-// write.
-assert bucketA.get("bar").equals("a")
-
-// Likewise, this will _eventually_ succeed in the absence of irrecoverable errors:
-assert bucketB.get("bar").equals("a")
-
-bucketB.set("bar", "b")
-bucketC = open("foo")
-value = bucketC.get("bar")
-
-// This is guaranteed to succeed:
-assert value.equals("a") or value.equals("b") or value is None
-
      -

      In other words, the bucketC resource MAY reflect either the most recent write to the bucketA -resource, or the one to the bucketB resource, or neither, depending on how quickly either of -those writes reached the replica from which the bucketC resource is reading. However, -assuming there are no irrecoverable errors -- such that the state of a replica is irretrievably -lost before it can be propagated -- one of the values ("a" or "b") MUST eventually be considered -the "latest" and replicated across the system, at which point all three resources will return -that same value.

      +

      An implementation of this interface MUST be eventually consistent, but is not required to +provide any consistency guaranteeds beyond that. Practically speaking, eventual consistency is +among the weakest of consistency models, guaranteeing only that values will not be produced +"from nowhere", i.e. any value read is guaranteed to have been written to that key at some +earlier time. Beyond that, there are no guarantees, and thus a portable component must neither +expect nor rely on anything else.

      +

      In the future, additional interfaces may be added to wasi:key-value with stronger guarantees, +which will allow components to express their requirements by importing whichever interface(s) +provides matching (or stronger) guarantees. For example, a component requiring strict +serializability might import a (currently hypothetical) strict-serializable-store interface +with a similar signature to store but with much stronger semantic guarantees. On the other +end, a host might either support implementations of both the store and +strict-serializable-store or just the former, in which case the host would immediately reject +a component which imports the unsupported interface.

      +

      Here are a few examples of behavior which an component developer might wish to rely on but which +are NOT guaranteed by an eventually consistent system (e.g. a distributed system composed of +multiple replicas, each of which may receive writes in a different order, making no attempt to +converge on a global consensus):

      +
        +
      • +

        Read-your-own-writes: eventual consistency does NOT guarantee that a write to a given key +followed by a read from the same key will retrieve the same or newer value.

        +
        • +
      • +

        Convergence: eventual consistency does NOT guarantee that any two replicas will agree on the +value for a given key -- even after all writes have had time to propagate to all replicas.

        +
        • +
      • +

        Last-write-wins: eventual consistency does NOT guarantee that the most recent write will +take precendence over an earlier one; old writes may overwrite newer ones temporarily or +permanently.

        +
        • +

      Durability

      This interface does not currently make any hard guarantees about the durability of values stored. A valid implementation might rely on an in-memory hash table, the contents of which are @@ -77,7 +63,7 @@ all writes to disk -- or even to a quorum of disk-backed nodes at multiple locat returning a result for a set call. Finally, a third implementation might persist values asynchronously on a best-effort basis without blocking set calls, in which case an I/O error could occur after the component instance which originally made the call has exited.

      -

      Future versions of the wasi-keyvalue package may provide ways to query and control the +

      Future versions of the wasi:keyvalue package may provide ways to query and control the durability and consistency provided by the backing implementation.

      Types

      @@ -116,7 +102,7 @@ there are no more keys to fetch.

      resource bucket

      A bucket is a collection of key-value pairs. Each key-value pair is stored as a entry in the bucket, and the bucket itself acts as a collection of all these entries.

      -

      It is worth noting that the exact terminology for bucket in key-value stores can very +

      It is worth noting that the exact terminology for bucket in key-value stores can vary depending on the specific implementation. For example:

      1. Amazon DynamoDB calls a collection of key-value pairs a table
        2. @@ -132,8 +118,8 @@ key-value pairs.

        Note that opening two bucket resources using the same identifier MAY result in connections to two separate replicas in a distributed database, and that writes to one of those resources are not guaranteed to be readable from the other resource promptly (or ever, in -the case of a replica failure). See the Consistency section of the store interface -documentation for details.

        +the case of a replica failure or message reordering). See the Consistency section of the +store interface documentation for details.

        Functions

        open: func

        Get the bucket with the specified identifier.

        diff --git a/wit/store.wit b/wit/store.wit index 93e751a..251d72d 100644 --- a/wit/store.wit +++ b/wit/store.wit @@ -10,58 +10,36 @@ /// /// ## Consistency /// -/// An implementation of this interface MUST be eventually consistent, meaning that, after some time -/// with no further updates, all replicas in the (potentially distributed) system will eventually -/// converge on a consistent state for all values. This allows replicas to temporarily diverge to -/// ensure low latency and high availability. Implementations based on a centralized or local -/// backing store may provide a stronger consistency model, but guest components which are intended -/// to be portable to any `wasi-keyvalue` implementation should not rely on anything stronger than -/// eventual consistency. +/// An implementation of this interface MUST be eventually consistent, but is not required to +/// provide any consistency guaranteeds beyond that. Practically speaking, eventual consistency is +/// among the weakest of consistency models, guaranteeing only that values will not be produced +/// "from nowhere", i.e. any value read is guaranteed to have been written to that key at some +/// earlier time. Beyond that, there are no guarantees, and thus a portable component must neither +/// expect nor rely on anything else. /// -/// Given that each `bucket` resource may represent a connection to a different replica in a -/// distributed system, values read for a given key from two different `bucket`s may differ, even if -/// those `bucket` resources were opened using the same string identifier. In addition, consecutive -/// operations on a single `bucket` resource may produce temporarily inconsistent results if -/// e.g. the implementation is forced to reconnect to a different replica due to a connection -/// failure. For example, a write followed by a read may not return the value just written, even if -/// no other recent or subsequent writes have occurred. +/// In the future, additional interfaces may be added to `wasi:key-value` with stronger guarantees, +/// which will allow components to express their requirements by importing whichever interface(s) +/// provides matching (or stronger) guarantees. For example, a component requiring strict +/// serializability might import a (currently hypothetical) `strict-serializable-store` interface +/// with a similar signature to `store` but with much stronger semantic guarantees. On the other +/// end, a host might either support implementations of both the `store` and +/// `strict-serializable-store` or just the former, in which case the host would immediately reject +/// a component which imports the unsupported interface. /// -/// Consider the following pseudocode example (and assume we start with an empty store and no other -/// concurrent activity): +/// Here are a few examples of behavior which an component developer might wish to rely on but which +/// are _NOT_ guaranteed by an eventually consistent system (e.g. a distributed system composed of +/// multiple replicas, each of which may receive writes in a different order, making no attempt to +/// converge on a global consensus): /// -/// ``` -/// bucketA = open("foo") -/// bucketB = open("foo") -/// bucketA.set("bar", "a") +/// - Read-your-own-writes: eventual consistency does _NOT_ guarantee that a write to a given key +/// followed by a read from the same key will retrieve the same or newer value. /// -/// // These are guaranteed to succeed: -/// assert bucketA.get("bar").equals("a") or bucketA.get("bar") is None -/// assert bucketB.get("bar").equals("a") or bucketB.get("bar") is None +/// - Convergence: eventual consistency does _NOT_ guarantee that any two replicas will agree on the +/// value for a given key -- even after all writes have had time to propagate to all replicas. /// -/// // This is likely to succeed, but not guaranteed; e.g. `bucketA` might need to reconnect to a -/// // different replica which hasn't received the above write yet. It will _eventually_ -/// // succeed, provided there are no irrecoverable errors which prevent the propagation of the -/// // write. -/// assert bucketA.get("bar").equals("a") -/// -/// // Likewise, this will _eventually_ succeed in the absence of irrecoverable errors: -/// assert bucketB.get("bar").equals("a") -/// -/// bucketB.set("bar", "b") -/// bucketC = open("foo") -/// value = bucketC.get("bar") -/// -/// // This is guaranteed to succeed: -/// assert value.equals("a") or value.equals("b") or value is None -/// ``` -/// -/// In other words, the `bucketC` resource MAY reflect either the most recent write to the `bucketA` -/// resource, or the one to the `bucketB` resource, or neither, depending on how quickly either of -/// those writes reached the replica from which the `bucketC` resource is reading. However, -/// assuming there are no irrecoverable errors -- such that the state of a replica is irretrievably -/// lost before it can be propagated -- one of the values ("a" or "b") MUST eventually be considered -/// the "latest" and replicated across the system, at which point all three resources will return -/// that same value. +/// - Last-write-wins: eventual consistency does _NOT_ guarantee that the most recent write will +/// take precendence over an earlier one; old writes may overwrite newer ones temporarily or +/// permanently. /// /// ## Durability /// @@ -73,7 +51,7 @@ /// asynchronously on a best-effort basis without blocking `set` calls, in which case an I/O error /// could occur after the component instance which originally made the call has exited. /// -/// Future versions of the `wasi-keyvalue` package may provide ways to query and control the +/// Future versions of the `wasi:keyvalue` package may provide ways to query and control the /// durability and consistency provided by the backing implementation. interface store { /// The set of errors which may be raised by functions in this package @@ -108,7 +86,7 @@ interface store { /// A bucket is a collection of key-value pairs. Each key-value pair is stored as a entry in the /// bucket, and the bucket itself acts as a collection of all these entries. /// - /// It is worth noting that the exact terminology for bucket in key-value stores can very + /// It is worth noting that the exact terminology for bucket in key-value stores can vary /// depending on the specific implementation. For example: /// /// 1. Amazon DynamoDB calls a collection of key-value pairs a table @@ -125,8 +103,8 @@ interface store { /// Note that opening two `bucket` resources using the same identifier MAY result in connections /// to two separate replicas in a distributed database, and that writes to one of those /// resources are not guaranteed to be readable from the other resource promptly (or ever, in - /// the case of a replica failure). See the `Consistency` section of the `store` interface - /// documentation for details. + /// the case of a replica failure or message reordering). See the `Consistency` section of the + /// `store` interface documentation for details. resource bucket { /// Get the value associated with the specified `key` /// From a9f34a46e854736d85245da206cc2d31dd7dc72f Mon Sep 17 00:00:00 2001 From: Joel Dice Date: Mon, 23 Jun 2025 08:54:48 -0600 Subject: [PATCH 9/9] address review feedback Signed-off-by: Joel Dice --- imports.md | 8 ++++---- watch-service.md | 8 ++++---- wit/store.wit | 8 ++++---- 3 files changed, 12 insertions(+), 12 deletions(-) diff --git a/imports.md b/imports.md index 7e4679c..103db60 100644 --- a/imports.md +++ b/imports.md @@ -30,7 +30,7 @@ among the weakest of consistency models, guaranteeing only that values will not "from nowhere", i.e. any value read is guaranteed to have been written to that key at some earlier time. Beyond that, there are no guarantees, and thus a portable component must neither expect nor rely on anything else.

        -

        In the future, additional interfaces may be added to wasi:key-value with stronger guarantees, +

        In the future, additional interfaces may be added to wasi:keyvalue with stronger guarantees, which will allow components to express their requirements by importing whichever interface(s) provides matching (or stronger) guarantees. For example, a component requiring strict serializability might import a (currently hypothetical) strict-serializable-store interface @@ -38,7 +38,7 @@ with a similar signature to store but with much stronger semantic g end, a host might either support implementations of both the store and strict-serializable-store or just the former, in which case the host would immediately reject a component which imports the unsupported interface.

        -

        Here are a few examples of behavior which an component developer might wish to rely on but which +

        Here are a few examples of behavior which a component developer might wish to rely on but which are NOT guaranteed by an eventually consistent system (e.g. a distributed system composed of multiple replicas, each of which may receive writes in a different order, making no attempt to converge on a global consensus):

        @@ -65,8 +65,8 @@ all writes to disk -- or even to a quorum of disk-backed nodes at multiple locat returning a result for a set call. Finally, a third implementation might persist values asynchronously on a best-effort basis without blocking set calls, in which case an I/O error could occur after the component instance which originally made the call has exited.

        -

        Future versions of the wasi:keyvalue package may provide ways to query and control the -durability and consistency provided by the backing implementation.

        +

        Future versions of wasi:keyvalue may provide ways to query and control the durability and +consistency provided by the backing implementation.

        Types

        variant error

        diff --git a/watch-service.md b/watch-service.md index 496b694..05d90bf 100644 --- a/watch-service.md +++ b/watch-service.md @@ -28,7 +28,7 @@ among the weakest of consistency models, guaranteeing only that values will not "from nowhere", i.e. any value read is guaranteed to have been written to that key at some earlier time. Beyond that, there are no guarantees, and thus a portable component must neither expect nor rely on anything else.

        -

        In the future, additional interfaces may be added to wasi:key-value with stronger guarantees, +

        In the future, additional interfaces may be added to wasi:keyvalue with stronger guarantees, which will allow components to express their requirements by importing whichever interface(s) provides matching (or stronger) guarantees. For example, a component requiring strict serializability might import a (currently hypothetical) strict-serializable-store interface @@ -36,7 +36,7 @@ with a similar signature to store but with much stronger semantic g end, a host might either support implementations of both the store and strict-serializable-store or just the former, in which case the host would immediately reject a component which imports the unsupported interface.

        -

        Here are a few examples of behavior which an component developer might wish to rely on but which +

        Here are a few examples of behavior which a component developer might wish to rely on but which are NOT guaranteed by an eventually consistent system (e.g. a distributed system composed of multiple replicas, each of which may receive writes in a different order, making no attempt to converge on a global consensus):

        @@ -63,8 +63,8 @@ all writes to disk -- or even to a quorum of disk-backed nodes at multiple locat returning a result for a set call. Finally, a third implementation might persist values asynchronously on a best-effort basis without blocking set calls, in which case an I/O error could occur after the component instance which originally made the call has exited.

        -

        Future versions of the wasi:keyvalue package may provide ways to query and control the -durability and consistency provided by the backing implementation.

        +

        Future versions of wasi:keyvalue may provide ways to query and control the durability and +consistency provided by the backing implementation.

        Types

        variant error

        diff --git a/wit/store.wit b/wit/store.wit index 251d72d..5d999b8 100644 --- a/wit/store.wit +++ b/wit/store.wit @@ -17,7 +17,7 @@ /// earlier time. Beyond that, there are no guarantees, and thus a portable component must neither /// expect nor rely on anything else. /// -/// In the future, additional interfaces may be added to `wasi:key-value` with stronger guarantees, +/// In the future, additional interfaces may be added to `wasi:keyvalue` with stronger guarantees, /// which will allow components to express their requirements by importing whichever interface(s) /// provides matching (or stronger) guarantees. For example, a component requiring strict /// serializability might import a (currently hypothetical) `strict-serializable-store` interface @@ -26,7 +26,7 @@ /// `strict-serializable-store` or just the former, in which case the host would immediately reject /// a component which imports the unsupported interface. /// -/// Here are a few examples of behavior which an component developer might wish to rely on but which +/// Here are a few examples of behavior which a component developer might wish to rely on but which /// are _NOT_ guaranteed by an eventually consistent system (e.g. a distributed system composed of /// multiple replicas, each of which may receive writes in a different order, making no attempt to /// converge on a global consensus): @@ -51,8 +51,8 @@ /// asynchronously on a best-effort basis without blocking `set` calls, in which case an I/O error /// could occur after the component instance which originally made the call has exited. /// -/// Future versions of the `wasi:keyvalue` package may provide ways to query and control the -/// durability and consistency provided by the backing implementation. +/// Future versions of `wasi:keyvalue` may provide ways to query and control the durability and +/// consistency provided by the backing implementation. interface store { /// The set of errors which may be raised by functions in this package variant error {