Add composefs-ostree and some basic CLI tools #144

alexlarsson · 2025-06-16T16:42:07Z

Based on ideas from #141

This is an initial version of ostree support. This allows pulling
from local and remote ostree repos, which will create a set of
regular file content objects, as well as a blob containing all the
remaining ostree objects. From the blob we can create an image.

When pulling a commit, a base blob (i.e. "the previous version" can be
specified. Any objects in that base blob will not be downloaded. If a
name is given for the pulled commit, then pre-existing blobs with the
same name will automatically be used as a base blob.

This is an initial version and there are several things missing:

Pull operations are completely serial
There is no support for ostree summary files
There is no support for ostree delta files
There is no caching of local file availability (other than base blob)
Local ostree repos only support archive mode

allisonkarlitskaya

I love this! Thanks for working on it!

I made some comments on the first round of commits. Feel free to adjust those and PR them separately: we can merge those now without further discussion.

The blobs thing is going to need a call.

I didn't review the crate addition in any detail at all. That's probably also going to need a call :)

crates/composefs/src/repository.rs

crates/composefs/src/lib.rs

crates/cfsctl/Cargo.toml

crates/composefs/src/repository.rs

alexlarsson · 2025-06-19T07:41:11Z

Hmmm, thinking more about this. We probably want a "content type" magic thing in the splitstream header as well, so we can error out if the wrapped thing is of the wrong type.

alexlarsson · 2025-06-26T12:07:47Z

Ok. Reworked this to use splitstreams for object maps and commits. And, by using an object mapping to find the object map we make the content of the splitstream for the commit be just the commit data, and thus the sha256 of that splitstream matches the ostree commit id.

alexlarsson · 2025-06-26T12:08:17Z

@allisonkarlitskaya There is still lots to do here. But have a look at this approach and see what you think.

alexlarsson · 2025-06-27T16:19:58Z

Added some further changes. We now validate all objects when pulling and all non-file objects when creating images. Its hard to efficiently validate file objects during create-image though, we would like to avoid re-reading the external object files to compute the sha256.

Remaining things to do:

Stream larger objects into repo
Support summaries and summary branches for remote repos
Support deltas when remote pulling
Parallelize downloads of objects
Report pull progress in some sane way
Use some kind of local cache for available objects other than just those from "previous version"
Handle GPG validation of commit objects

alexlarsson · 2025-06-30T14:38:26Z

I started working on the delta support, but it failed because of an issue in gvariant-rs.

allisonkarlitskaya

It occurs to me that it might be interesting not to sort the table of fs-verity references, and it might also be interesting to permit duplicate items.

On the topic of deferring writing of objects to a background thread, this would allow us to write "external object #123" based on a sequential index to the splitstream without actually knowing the hash value yet, and then fill in the actual values in the header at the end when we're writing: it helps there that the fs-verity references aren't compressed and therefore not part of the stream...

cgwalters · 2025-09-05T12:58:46Z

It seems like we should get in the splitstream changes in 0f6d69e at least sooner rather than later? Can you file a separate PR?

This changes the splitstream format a bit, with the goal of allowing splitstreams to support ostree files as well (see containers#144) The primary differences are: * The header is not compressed * All referenced fs-verity objects are stored in the header, including external chunks, mapped splitstreams and (a new feature) references that are not used in chunks. * The mapping table is separate from the reference table (and generally smaller), and indexes into it. * There is a magic value to detect the file format. * There is a magic content type to detect the type wrapped in the stream. * We store a tag for what ObjectID format is used * The total size of the stream is stored in the header. The ability to reference file objects in the repo even if they are not part of the splitstream "content" will be useful for the ostree support to reference file content objects. This change also allows more efficient GC enumeration, because we don't have to parse the entire splitstream to find the referenced objects. Signed-off-by: Alexander Larsson <alexl@redhat.com>

alexlarsson · 2026-01-19T09:55:44Z

I rebased the code I had here on top of the splitstream changes that was merged in main. I removed all the code to give ostree commits some kind of named ref for now, as we need to figure out exactly how that will look. But, it has the basic minimal support to pull from a (remote or local) ostree repo into a ostree-commit-* splitstream object, and you can specify another such splitstream as a base name, and it will only download objects not in that version. (Which eventually we want to automatically pass what was the previous commit of the named ostree ref).

It also supports converting ostree commits to images, which can then be mounted.

There are tons of things that could be added on top of this:

Named refs support
Support summaries for remote repos
Use of static deltas when downloading from http
Some kind of object file map caching between commits other than the specified base name
Parallel pulling with progress
gpg validation
Handle more local repo modes (currently only archive and bare-user-only

To test this you can run for example:

$ cfsctl --repo repo ostree pull-local /var/lib/flatpak/repo remotes/flathub/app/org.gnome.gedit/x86_64/stable
$ cfsctl --repo repo pull --base-name ostree-commit-306eed5546b7af406377e4947a118bd942729d9b380099fb191728654e2c925b https://dl.flathub.org/repo/ app/org.gnome.gedit/aarch64/stable
$ cfsctl --repo repo ostree create-image --image-name gedit ostree-commit-306eed5546b7af406377e4947a118bd942729d9b380099fb191728654e2c925b
$ sudo cfsctl --repo repo mount refs/gedit mnt

alexlarsson · 2026-01-19T14:23:46Z

The CI failures seem unrelated to this change. Weird.

cgwalters · 2026-01-28T02:28:50Z

Can you explain a bit what the medium-term is for this? Is it flatpak using this? Would we try to create a shim C API? Or is it for something like bootc (or rpm-ostree?) fetching from ostree repos? Something else?

Something else I'm thinking about here is the intersection with #216 - this is a half-baked thought but...maybe we could always map ostree to OCI (instead of mapping ostree to splitstream), though depending on how we do that there could be nontrivial metadata overhead.

alexlarsson · 2026-01-29T11:12:49Z

@cgwalters The origins of this is the work that @allisonkarlitskaya was doing on a quickly moving flatpak-next experimental codebase (in rust). The idea for a flatpak-next would be to have a native composefs local on-disk format, but still allow installing from ostree based remotes.

I would not say there is any real, concrete usecase for this atm, but it seems important to me to at least have the new codebase and on-disk format flexible enough to be able to handle ostree if we need it for backwards compatibility reasons.

alexlarsson · 2026-01-29T11:16:54Z

Something else I'm thinking about here is the intersection with #216 - this is a half-baked thought but...maybe we could always map ostree to OCI (instead of mapping ostree to splitstream), though depending on how we do that there could be nontrivial metadata overhead.

I'm not sure about this. For sure we can convert to anything, but if we ever want to have efficient "delta ostree pull" based on a previous commit we can't lose fidelity of the ostree objects.

allisonkarlitskaya · 2026-01-29T11:28:48Z

I would actually like to see this merged. It's good code and will almost definitely be useful to somebody at some point. It also doesn't really hurt anyone since it's in a separate crate.

allisonkarlitskaya · 2026-01-29T11:30:21Z

I also think the fact that it has nothing to do with OCI is great. I'm not as much on the OCI train and one of the main things you hear from composefs detractors is "but OCI sucks!". Showing that composefs != OCI is good IMHO.

cgwalters · 2026-01-29T14:14:37Z

@cgwalters The origins of this is the work that @allisonkarlitskaya was doing on a quickly moving flatpak-next experimental codebase (in rust).

OK right! That is super exciting to me and a lot of potential there. If we're semi-serious about pushing it forward then it is indeed a strong argument against #213 as written (tangent but an important one).

For sure we can convert to anything, but if we ever want to have efficient "delta ostree pull" based on a previous commit we can't lose fidelity of the ostree objects.

Yes, of course - my proposal wouldn't lose that, but it's not the most important thread right now.

cgwalters · 2026-01-29T14:18:54Z

I also think the fact that it has nothing to do with OCI is great.

I think unless we prove out that composefs can be a very good way to store OCI, then it is not worth investing in. Thankfully that's not the case - I think it is (and I believe you do too!).

So it's not that it has "nothing to do with OCI" (right?) - how about "has the capability to easily/natively store any type of content that one would want to represent as read-only immutable versioned filesystem trees".

For example, today Android as far as I know uses fsverity on single zip files, and they've made it work quite well, but it's harder to get deduplication across apps that way, and maybe someday they go to a composefs-like model.

Signed-off-by: Alexander Larsson <alexl@redhat.com>

alexlarsson · 2026-01-29T14:59:12Z

I rebased this, lets see if CI passes now.

This lets you look up a ref digest from the splitstream by index and is needed by the ostree code. Signed-off-by: Alexander Larsson <alexl@redhat.com>

allisonkarlitskaya · 2026-01-29T15:23:34Z

I also think the fact that it has nothing to do with OCI is great.

I think unless we prove out that composefs can be a very good way to store OCI, then it is not worth investing in. Thankfully that's not the case - I think it is (and I believe you do too!).

So it's not that it has "nothing to do with OCI" (right?) - how about "has the capability to easily/natively store any type of content that one would want to represent as read-only immutable versioned filesystem trees".

Just to be clear, when I said "it has nothing to do with OCI" I specifically meant composefs-ostree, not composefs-rs generally (which very clearly was designed with OCI in mind).

Very obviously the main target of composefs-rs right now is bootc (OCI), probably followed by container storage (obviously also OCI). flatpak is probably a distant third at the moment, and indeed, even that has something to do with OCI (the current flatpak demo only works with OCI, in fact)...

For example, today Android as far as I know uses fsverity on single zip files, and they've made it work quite well, but it's harder to get deduplication across apps that way, and maybe someday they go to a composefs-like model.

Ya, that's sort of what I meant... it would be cool to show that you can really do a lot of different things with this stuff...

Based on ideas from containers#141 This is an initial version of ostree support. This allows pulling from local and remote ostree repos, which will create a set of regular file content objects, as well as a commit splitstream containing all the remaining ostree objects and file data. From the splitstream we can create an image. When pulling a commit, a base commit (i.e. "the previous version" can be specified. Any objects in that base commit will not be downloaded. If a name is given for the pulled commit, then pre-existing blobs with the same name will automatically be used as a base commit. This is an initial version and there are several things missing: * Pull operations are completely serial * There is no support for ostree summary files * There is no support for ostree delta files * There is no caching of local file availability (other than base commit) * Local ostree repos only support archive mode * There is no GPG validation on ostree pull Signed-off-by: Alexander Larsson <alexl@redhat.com>

cgwalters

Just an initial pass

cgwalters · 2026-01-29T19:42:23Z

crates/composefs-ostree/src/pull.rs

+        for d in dirs_data.iter() {
+            let (_name, tree_checksum, meta_checksum) = d.to_tuple();
+
+            self.maybe_fetch_dirmeta(meta_checksum.try_into().unwrap());


These should be ? since this is reachable over the network.

Hmm we probably want a clippy lint that uwraps in non-test code require Justification.

cgwalters · 2026-01-29T19:42:59Z

crates/composefs-ostree/src/pull.rs

+
+        self.enqueue_fetch(commit_id, ObjectType::Commit);
+
+        // TODO: Support deltas


Going to be fun...

cgwalters · 2026-01-29T19:43:22Z

crates/composefs-ostree/src/pull.rs

+        // TODO: At least for http we should make parallel fetches
+        while !self.outstanding.is_empty() {
+            let fetch = self.outstanding.pop_front().unwrap();
+            println!(


We shouldn't println! in a library. Let's use indicatif at least...

cgwalters · 2026-01-29T19:43:47Z