[server] parse as stream #133
Conversation
There was a problem hiding this comment.
Pull request overview
This PR refactors the RSS feed parsing in the ChannelService to use a streaming approach instead of loading the entire response into memory. The version is bumped from 2.3.2 to 2.3.3 to reflect this change.
- Replaces the byte array compilation approach with fs2 streaming using
toInputStream - Updates error handling to pattern match on HTTP status codes
- Changes from
IO.blockingtoIO.blockingfor the XmlReader resource
Reviewed changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| server/src/main/scala/ru/trett/rss/server/services/ChannelService.scala | Refactors XML parsing to use streaming via fs2.io.toInputStream, adds explicit status code handling, and modifies the HTTP client usage pattern |
| scripts/local-docker/docker-compose.yml | Updates the server Docker image version from 2.3.2 to 2.3.3 |
| build.sbt | Bumps the project version from 2.3.2 to 2.3.3 |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| .evalMap(is => { | ||
| Resource | ||
| .fromAutoCloseable(IO.blocking(new XmlReader(is))) | ||
| .use { reader => parse(reader, link) } | ||
| }) |
There was a problem hiding this comment.
The error handling for the parse operation has been removed. The old code had .handleErrorWith that logged parsing errors and returned IO.none. Without this error handling, if the parse operation fails (e.g., malformed XML), the error will propagate up and potentially crash the stream. Consider adding error handling using .handleErrorWith or .attempt to gracefully handle parsing failures.
| .map(b => | ||
| logger.error( | ||
| s"Request failed with status ${r.status.code} and body $b" | ||
| ) | ||
| ) *> IO.pure(None) |
There was a problem hiding this comment.
The logger.error call returns Unit, not IO[Unit], which makes this expression incorrect. The .map(b => logger.error(...)) will execute the logger call eagerly during the map, not as part of the IO effect. This should be flatMap or flatTap with the logger call wrapped in IO, such as: r.as[String].flatTap(b => logger.error(...))
| .map(b => | |
| logger.error( | |
| s"Request failed with status ${r.status.code} and body $b" | |
| ) | |
| ) *> IO.pure(None) | |
| .flatTap(b => | |
| logger.error( | |
| s"Request failed with status ${r.status.code} and body $b" | |
| ) | |
| ) | |
| .as(None) |
| } | ||
| } | ||
| .get[Option[Channel]](url) { | ||
| case Status.Successful(r) => { |
There was a problem hiding this comment.
Unnecessary braces around the case clause. In Scala 3, braces are optional for case clauses and the opening brace here is inconsistent with the following case clause at line 87 which doesn't use braces.
| client | ||
| .get(url) { response => | ||
| response.body.compile.to(Array).flatMap { bytes => | ||
| Resource | ||
| .fromAutoCloseable( | ||
| IO(new XmlReader(new java.io.ByteArrayInputStream(bytes))) | ||
| ) | ||
| .use { reader => | ||
| parse(reader, link).handleErrorWith { error => | ||
| logger.error(error)( | ||
| s"Failed to parse the feed: $link" | ||
| ) *> IO.none | ||
| } | ||
| } | ||
| .get[Option[Channel]](url) { | ||
| case Status.Successful(r) => { |
There was a problem hiding this comment.
The client.get[Option[Channel]](url) call uses url derived directly from the HTTP request body parameter link (see ChannelController), allowing any authenticated user to make the server perform HTTP requests to arbitrary hosts (including internal services), which is a classic SSRF risk. An attacker could add a channel URL such as http://127.0.0.1:... or a cloud metadata endpoint and cause the server to fetch internal resources and process their responses. To mitigate this, enforce an allowlist of schemes/hosts (e.g., only http/https and public domains), and optionally block link-local, loopback, and RFC1918 address ranges before invoking client.get.
No description provided.