Add virtual file support and logging

CLAUDE.md

@@ -122,9 +122,13 @@ SharpSync is a **pure .NET file synchronization library** with no native depende
 - **Large File Support**: Chunked/multipart uploads with platform-specific optimizations
 - **Network Resilience**: Retry logic and error handling with automatic reconnection
 - **Parallel Processing**: Configurable parallelism with intelligent prioritization
+- **Bandwidth Throttling**: Configurable transfer rate limits via `SyncOptions.MaxBytesPerSecond`
+- **Virtual File Support**: Callback hook for Windows Cloud Files API placeholder integration
+- **Structured Logging**: High-performance logging via `Microsoft.Extensions.Logging`
 
 ### Dependencies
 
+- `Microsoft.Extensions.Logging.Abstractions` (9.0.1) - Logging abstraction
 - `sqlite-net-pcl` (1.9.172) - SQLite database
 - `SQLitePCLRaw.bundle_e_sqlite3` (3.0.2) - SQLite native binaries
 - `WebDav.Client` (2.9.0) - WebDAV protocol
@@ -164,6 +168,7 @@ SharpSync is a **pure .NET file synchronization library** with no native depende
 │       ├── Auth/             # OAuth2 authentication
 │       ├── Core/             # Interfaces and models
 │       ├── Database/         # State persistence
+│       ├── Logging/          # High-performance logging
 │       ├── Storage/          # Storage backends
 │       └── Sync/             # Sync engine
 ├── tests/
@@ -290,11 +295,16 @@ _watcher.EnableRaisingEvents = true;
 | No selective folder sync API | Can't sync single folders on demand | Planned: `SyncFolderAsync()`, `SyncFilesAsync()` |
 | No pause/resume | Long syncs can't be paused | Planned: `PauseAsync()`, `ResumeAsync()` |
 | No incremental change notification | FileSystemWatcher triggers full scan | Planned: `NotifyLocalChangeAsync()` |
-| No virtual file awareness | Can't track placeholder vs downloaded | Planned: `VirtualFileCallback` |
 | Single-threaded engine | One sync at a time per instance | By design - create separate instances if needed |
-| ~~No bandwidth throttling~~ | ~~Can saturate network~~ | ✅ `SyncOptions.MaxBytesPerSecond` IMPLEMENTED |
 | OCIS TUS not implemented | Falls back to generic upload | Planned for v1.0 |
 
+### ✅ Resolved API Gaps
+
+| Feature | Implementation |
+|---------|----------------|
+| Bandwidth throttling | `SyncOptions.MaxBytesPerSecond` - limits transfer rate |
+| Virtual file awareness | `SyncOptions.VirtualFileCallback` - hook for Windows Cloud Files API integration |
+
 ### Required SharpSync API Additions (v1.0)
 
 These APIs are required for v1.0 release to support Nimbus desktop client:
@@ -308,16 +318,19 @@ These APIs are required for v1.0 release to support Nimbus desktop client:
 4. OCIS TUS protocol implementation (`WebDavStorage.cs:547` currently falls back)
 
 **Sync Control:**
-5. ~~`SyncOptions.MaxBytesPerSecond`~~ ✅ Built-in bandwidth throttling (IMPLEMENTED)
-6. `PauseAsync()` / `ResumeAsync()` - Pause and resume long-running syncs
-7. `GetPendingOperationsAsync()` - Inspect sync queue for UI display
+5. `PauseAsync()` / `ResumeAsync()` - Pause and resume long-running syncs
+6. `GetPendingOperationsAsync()` - Inspect sync queue for UI display
 
 **Progress & History:**
-8. Per-file progress events (currently only per-sync-operation)
-9. `GetRecentOperationsAsync()` - Operation history for activity feed
+7. Per-file progress events (currently only per-sync-operation)
+8. `GetRecentOperationsAsync()` - Operation history for activity feed
 
-**Virtual Files:**
-10. `SyncOptions.VirtualFileCallback` - Hook for virtual file systems (Windows Cloud Files API)
+**✅ Completed:**
+- `SyncOptions.MaxBytesPerSecond` - Built-in bandwidth throttling
+- `SyncOptions.VirtualFileCallback` - Hook for virtual file systems (Windows Cloud Files API)
+- `SyncOptions.CreateVirtualFilePlaceholders` - Enable/disable virtual file placeholder creation
+- `VirtualFileState` enum - Track placeholder state (None, Placeholder, Hydrated, Partial)
+- `SyncPlanAction.WillCreateVirtualPlaceholder` - Preview which downloads will create placeholders
 
 ### API Readiness Score for Nimbus
 
@@ -332,9 +345,9 @@ These APIs are required for v1.0 release to support Nimbus desktop client:
 | Conflict resolution | 9/10 | Rich analysis, extensible callbacks |
 | Selective sync | 4/10 | Filter-only, no folder/file API |
 | Pause/Resume | 2/10 | Not implemented |
-| Desktop integration hooks | 6/10 | Good abstraction, missing some hooks |
+| Desktop integration hooks | 8/10 | Virtual file callback, bandwidth throttling implemented |
 
-**Current Overall: 7/10** - Solid foundation, gaps identified
+**Current Overall: 7.25/10** - Solid foundation, key desktop hooks now available
 
 **Target for v1.0: 9.5/10** - All gaps resolved, ready for Nimbus development
 
@@ -485,6 +498,9 @@ The core library is production-ready, but several critical items must be address
 - ✅ FTP/FTPS storage implementation
 - ✅ S3 storage implementation with AWS S3 and S3-compatible services
 - ✅ Integration test infrastructure with Docker for SFTP, FTP, and S3/LocalStack
+- ✅ Bandwidth throttling (`SyncOptions.MaxBytesPerSecond`)
+- ✅ Virtual file placeholder support (`SyncOptions.VirtualFileCallback`) for Windows Cloud Files API
+- ✅ High-performance logging with `Microsoft.Extensions.Logging.Abstractions`
 
 **🚧 Required for v1.0 Release**
 
@@ -504,7 +520,7 @@ Desktop Client APIs (for Nimbus):
 - [ ] `PauseAsync()` / `ResumeAsync()` - Pause and resume long-running syncs
 - [ ] `GetPendingOperationsAsync()` - Inspect sync queue for UI display
 - [ ] Per-file progress events (currently only per-sync-operation)
-- [ ] `SyncOptions.VirtualFileCallback` - Hook for virtual file systems (Windows Cloud Files API)
+- [x] `SyncOptions.VirtualFileCallback` - Hook for virtual file systems (Windows Cloud Files API) ✅
 - [ ] `GetRecentOperationsAsync()` - Operation history for activity feed
 
 Performance & Polish:

src/SharpSync/Core/SyncItem.cs

@@ -48,4 +48,23 @@ public class SyncItem {
     /// Gets or sets the MIME type
     /// </summary>
     public string? MimeType { get; set; }
+
+    /// <summary>
+    /// Gets or sets the virtual file state for cloud file systems
+    /// </summary>
+    /// <remarks>
+    /// Used by desktop clients integrating with Windows Cloud Files API or similar
+    /// virtual file systems. When <see cref="VirtualFileState.Placeholder"/>, the file
+    /// exists only as metadata locally and content must be fetched on demand.
+    /// </remarks>
+    public VirtualFileState VirtualState { get; set; } = VirtualFileState.None;
+
+    /// <summary>
+    /// Gets or sets the cloud-specific item identifier for virtual file tracking
+    /// </summary>
+    /// <remarks>
+    /// Platform-specific identifier used by cloud file systems to track the file.
+    /// For Windows Cloud Files API, this could be the CF_PLACEHOLDER_BASIC_INFO identifier.
+    /// </remarks>
+    public string? CloudFileId { get; set; }
 }

src/SharpSync/Core/SyncOptions.cs

@@ -78,6 +78,35 @@ public class SyncOptions {
     /// </remarks>
     public long? MaxBytesPerSecond { get; set; }
 
+    /// <summary>
+    /// Gets or sets whether to create virtual file placeholders after downloading files.
+    /// </summary>
+    /// <remarks>
+    /// When enabled, the <see cref="VirtualFileCallback"/> will be invoked after each
+    /// successful file download. This allows desktop clients to integrate with
+    /// platform-specific virtual file systems like Windows Cloud Files API.
+    /// </remarks>
+    public bool CreateVirtualFilePlaceholders { get; set; }
+
+    /// <summary>
+    /// Gets or sets the callback invoked after a file is downloaded to create a virtual file placeholder.
+    /// </summary>
+    /// <remarks>
+    /// <para>
+    /// This callback is only invoked when <see cref="CreateVirtualFilePlaceholders"/> is true
+    /// and a file (not directory) is successfully downloaded to local storage.
+    /// </para>
+    /// <para>
+    /// The callback receives the relative path, full local path, and file metadata,
+    /// allowing the application to convert the file to a cloud files placeholder.
+    /// </para>
+    /// <para>
+    /// If the callback throws an exception, the error is logged but sync continues.
+    /// The file will remain fully hydrated (not converted to placeholder).
+    /// </para>
+    /// </remarks>
+    public VirtualFileCallbackDelegate? VirtualFileCallback { get; set; }
+
     /// <summary>
     /// Creates a copy of the sync options
     /// </summary>
@@ -96,7 +125,9 @@ public SyncOptions Clone() {
             ConflictResolution = ConflictResolution,
             TimeoutSeconds = TimeoutSeconds,
             ExcludePatterns = new List<string>(ExcludePatterns),
-            MaxBytesPerSecond = MaxBytesPerSecond
+            MaxBytesPerSecond = MaxBytesPerSecond,
+            CreateVirtualFilePlaceholders = CreateVirtualFilePlaceholders,
+            VirtualFileCallback = VirtualFileCallback
         };
     }
 }

src/SharpSync/Core/SyncPlanAction.cs

@@ -48,9 +48,10 @@ public string Description {
         get {
             var sizeStr = IsDirectory ? "folder" : FormatSize(Size);
             var pathDisplay = IsDirectory ? $"{Path}/" : Path;
+            var placeholderSuffix = WillCreateVirtualPlaceholder ? " [placeholder]" : "";
 
             return ActionType switch {
-                SyncActionType.Download => $"Download {pathDisplay}" + (IsDirectory ? "" : $" ({sizeStr})"),
+                SyncActionType.Download => $"Download {pathDisplay}" + (IsDirectory ? "" : $" ({sizeStr})") + placeholderSuffix,
                 SyncActionType.Upload => $"Upload {pathDisplay}" + (IsDirectory ? "" : $" ({sizeStr})"),
                 SyncActionType.DeleteLocal => $"Delete {pathDisplay} from local storage",
                 SyncActionType.DeleteRemote => $"Delete {pathDisplay} from remote storage",
@@ -69,6 +70,26 @@ public string Description {
     /// </remarks>
     public int Priority { get; init; }
 
+    /// <summary>
+    /// Gets whether this download action will create a virtual file placeholder.
+    /// </summary>
+    /// <remarks>
+    /// This is true when <see cref="SyncOptions.CreateVirtualFilePlaceholders"/> is enabled
+    /// and this action is a download operation for a file (not directory).
+    /// Desktop clients can use this to display placeholder indicators in their UI.
+    /// </remarks>
+    public bool WillCreateVirtualPlaceholder { get; init; }
+
+    /// <summary>
+    /// Gets the current virtual file state of the item (if applicable).
+    /// </summary>
+    /// <remarks>
+    /// For existing local files, this indicates whether the file is currently
+    /// a placeholder, hydrated, or a regular file. Useful for displaying
+    /// file status in desktop client UIs.
+    /// </remarks>
+    public VirtualFileState CurrentVirtualState { get; init; }
+
     private static string FormatSize(long bytes) {
         if (bytes < 1024) {
             return $"{bytes} B";

src/SharpSync/Core/VirtualFileCallback.cs

@@ -0,0 +1,36 @@
+namespace Oire.SharpSync.Core;
+
+/// <summary>
+/// Delegate for handling virtual file placeholder creation after a file is downloaded.
+/// </summary>
+/// <param name="relativePath">The relative path of the downloaded file.</param>
+/// <param name="localFullPath">The full local filesystem path where the file was written.</param>
+/// <param name="fileMetadata">Metadata about the downloaded file including size, modified time, etc.</param>
+/// <param name="cancellationToken">Cancellation token to cancel the operation.</param>
+/// <returns>A task that completes when the virtual file placeholder is created.</returns>
+/// <remarks>
+/// <para>
+/// This callback is invoked by the sync engine after a file is successfully downloaded
+/// to the local storage. Desktop clients can use this hook to integrate with platform-specific
+/// virtual file systems like Windows Cloud Files API.
+/// </para>
+/// <para>
+/// Example usage with Windows Cloud Files API:
+/// <code>
+/// options.VirtualFileCallback = async (path, fullPath, metadata, ct) =&gt; {
+///     // Convert the downloaded file to a cloud files placeholder
+///     await CloudFilesAPI.ConvertToPlaceholderAsync(fullPath, metadata.Size, ct);
+/// };
+/// </code>
+/// </para>
+/// <para>
+/// If the callback throws an exception, the sync engine will log the error but continue
+/// processing. The file will remain fully hydrated (not a placeholder) in that case.
+/// </para>
+/// </remarks>
+public delegate Task VirtualFileCallbackDelegate(
+    string relativePath,
+    string localFullPath,
+    SyncItem fileMetadata,
+    CancellationToken cancellationToken
+);

src/SharpSync/Core/VirtualFileState.cs

@@ -0,0 +1,34 @@
+namespace Oire.SharpSync.Core;
+
+/// <summary>
+/// Represents the virtual file state for cloud file systems (e.g., Windows Cloud Files API)
+/// </summary>
+public enum VirtualFileState {
+    /// <summary>
+    /// Not a virtual file - the file is fully present on disk
+    /// </summary>
+    None,
+
+    /// <summary>
+    /// Placeholder file - only metadata is stored locally, content is on remote storage
+    /// </summary>
+    /// <remarks>
+    /// The file appears in the file system but its content is not downloaded.
+    /// Accessing the file will trigger on-demand download (hydration).
+    /// </remarks>
+    Placeholder,
+
+    /// <summary>
+    /// Hydrated file - content has been downloaded and is available locally
+    /// </summary>
+    /// <remarks>
+    /// The file was previously a placeholder but has been fully downloaded.
+    /// It may still be tracked by the cloud file system for dehydration.
+    /// </remarks>
+    Hydrated,
+
+    /// <summary>
+    /// Partially hydrated - some content is available locally (e.g., during streaming)
+    /// </summary>
+    Partial
+}

src/SharpSync/Logging/LogMessages.cs

@@ -0,0 +1,44 @@
+using Microsoft.Extensions.Logging;
+
+namespace Oire.SharpSync.Logging;
+
+/// <summary>
+/// High-performance log messages using source generators.
+/// </summary>
+internal static partial class LogMessages {
+    [LoggerMessage(
+        EventId = 1,
+        Level = LogLevel.Warning,
+        Message = "Error scanning directory {DirectoryPath}")]
+    public static partial void DirectoryScanError(this ILogger logger, Exception ex, string directoryPath);
+
+    [LoggerMessage(
+        EventId = 2,
+        Level = LogLevel.Warning,
+        Message = "Error processing {FilePath}")]
+    public static partial void ProcessingError(this ILogger logger, Exception ex, string filePath);
+
+    [LoggerMessage(
+        EventId = 3,
+        Level = LogLevel.Warning,
+        Message = "Error processing large file {FilePath}")]
+    public static partial void LargeFileProcessingError(this ILogger logger, Exception ex, string filePath);
+
+    [LoggerMessage(
+        EventId = 4,
+        Level = LogLevel.Warning,
+        Message = "Error resolving conflict for {FilePath}")]
+    public static partial void ConflictResolutionError(this ILogger logger, Exception ex, string filePath);
+
+    [LoggerMessage(
+        EventId = 5,
+        Level = LogLevel.Warning,
+        Message = "Error deleting {FilePath}")]
+    public static partial void DeletionError(this ILogger logger, Exception ex, string filePath);
+
+    [LoggerMessage(
+        EventId = 6,
+        Level = LogLevel.Warning,
+        Message = "Virtual file callback failed for {FilePath}")]
+    public static partial void VirtualFileCallbackError(this ILogger logger, Exception ex, string filePath);
+}

src/SharpSync/SharpSync.csproj

@@ -34,6 +34,7 @@
   </ItemGroup>
 
   <ItemGroup>
+    <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="9.0.1" />
     <PackageReference Include="sqlite-net-pcl" Version="1.9.172" />
     <PackageReference Include="SQLitePCLRaw.bundle_e_sqlite3" Version="3.0.2" />
     <PackageReference Include="WebDav.Client" Version="2.9.0" />