Refactor v35 module restructure

sql-cli/docs/DATATABLE_IMPLEMENTATION_STRATEGY.md

@@ -22,7 +22,6 @@ Currently:
 - Data and view logic are intertwined
 - Row counts are incorrect due to multiple filter states
 - The TUI knows too much about data implementation
-
 Goal:
 - TUI only knows about views (what to display)
 - DataTable holds immutable source data

sql-cli/docs/MODULE_STRUCTURE.md

@@ -0,0 +1,188 @@
+# Module Structure
+
+## Overview
+Organizing the codebase into logical modules for better maintainability and clarity.
+
+## Proposed Structure
+
+```
+src/
+├── lib.rs                    # Public API exports
+├── main.rs                   # Binary entry point
+│
+├── core/                     # Core business logic
+│   ├── mod.rs
+│   ├── app_state_container.rs
+│   ├── buffer_manager.rs
+│   ├── service_container.rs
+│   └── global_state.rs
+│
+├── data/                     # Data layer (DataTable/DataView)
+│   ├── mod.rs
+│   ├── provider.rs          # DataProvider traits
+│   ├── table.rs             # DataTable implementation
+│   ├── view.rs              # DataView implementation
+│   ├── adapters/            # Adapters for existing data sources
+│   │   ├── mod.rs
+│   │   ├── buffer_adapter.rs
+│   │   ├── csv_adapter.rs
+│   │   └── api_adapter.rs
+│   └── converters/          # Data format converters
+│       ├── mod.rs
+│       ├── csv_converter.rs
+│       └── json_converter.rs
+│
+├── ui/                      # UI layer
+│   ├── mod.rs
+│   ├── enhanced_tui.rs     # Main TUI application
+│   ├── classic_cli.rs      # Classic CLI mode
+│   ├── key_dispatcher.rs   # Key event handling
+│   └── renderer.rs         # Rendering logic
+│
+├── widgets/                 # UI widgets
+│   ├── mod.rs
+│   ├── debug_widget.rs
+│   ├── editor_widget.rs
+│   ├── help_widget.rs
+│   ├── stats_widget.rs
+│   ├── search_modes_widget.rs
+│   ├── history_widget.rs
+│   └── table_widget.rs
+│
+├── state/                   # State management
+│   ├── mod.rs
+│   ├── selection_state.rs
+│   ├── filter_state.rs
+│   ├── sort_state.rs
+│   ├── search_state.rs
+│   ├── column_search_state.rs
+│   ├── clipboard_state.rs
+│   ├── chord_state.rs
+│   └── undo_redo_state.rs
+│
+├── sql/                     # SQL parsing and execution
+│   ├── mod.rs
+│   ├── parser.rs
+│   ├── executor.rs
+│   ├── optimizer.rs
+│   └── cache.rs
+│
+├── api/                     # External API interactions
+│   ├── mod.rs
+│   ├── client.rs
+│   ├── models.rs
+│   └── endpoints.rs
+│
+├── utils/                   # Utility functions
+│   ├── mod.rs
+│   ├── debouncer.rs
+│   ├── formatter.rs
+│   ├── logger.rs
+│   └── paths.rs
+│
+├── config/                  # Configuration
+│   ├── mod.rs
+│   ├── settings.rs
+│   ├── themes.rs
+│   └── keybindings.rs
+│
+└── tests/                   # Integration tests
+    ├── mod.rs
+    └── ...
+```
+
+## Migration Strategy
+
+### Phase 1: Create Directory Structure (V35)
+- Create directories
+- Add mod.rs files with re-exports
+- No code moves yet
+
+### Phase 2: Move Widgets (V36)
+- Move all *_widget.rs files to widgets/
+- Update imports
+
+### Phase 3: Move Data Layer (V37)
+- Move DataProvider trait to data/provider.rs
+- Move datatable* files to data/
+- Move converters and adapters
+
+### Phase 4: Move State Components (V38)
+- Extract state structs from app_state_container.rs
+- Create separate files in state/
+- Keep AppStateContainer as orchestrator
+
+### Phase 5: Move UI Components (V39)
+- Move enhanced_tui.rs to ui/
+- Move classic_cli.rs to ui/
+- Move key_dispatcher.rs to ui/
+
+### Phase 6: Move SQL Components (V40)
+- Move SQL-related files to sql/
+- Organize parser, executor, cache
+
+### Phase 7: Move Utils and Config (V41)
+- Move utility files to utils/
+- Move config files to config/
+
+## Benefits
+
+1. **Clearer Organization**: Related files grouped together
+2. **Easier Navigation**: Find files by functionality
+3. **Better Encapsulation**: Modules can have private internals
+4. **Scalability**: Easy to add new features in appropriate modules
+5. **Testing**: Can test modules in isolation
+6. **Documentation**: Each module can have its own README
+
+## Module Visibility Rules
+
+- Each module has a `mod.rs` that controls what's public
+- Internal implementation details stay private
+- Public API is explicitly exported
+- Cross-module dependencies are minimized
+
+## Example: widgets/mod.rs
+
+```rust
+// Re-export public widgets
+pub mod debug_widget;
+pub mod editor_widget;
+pub mod help_widget;
+pub mod stats_widget;
+
+// Common widget traits (if any)
+pub trait Widget {
+    fn render(&self, area: Rect, buf: &mut Buffer);
+}
+
+// Widget utilities
+mod utils;  // Private to widgets module
+```
+
+## Example: data/mod.rs
+
+```rust
+// Public API for data layer
+pub mod provider;
+pub use provider::{DataProvider, DataViewProvider};
+
+// DataTable and DataView will be public
+pub mod table;
+pub mod view;
+
+// Adapters are public for gradual migration
+pub mod adapters;
+
+// Internal converters
+mod converters;
+```
+
+## Gradual Migration
+
+Each phase is a separate PR that:
+1. Moves specific files
+2. Updates imports
+3. Ensures tests pass
+4. Maintains backward compatibility
+
+No breaking changes - just reorganization!

sql-cli/src/api/mod.rs

@@ -0,0 +1,9 @@
+//! External API client and models
+//!
+//! This module handles communication with external APIs
+//! and defines the data models for API requests/responses.
+
+// API components to be moved here:
+// - api_client.rs → client.rs
+// - API response models
+// - Endpoint definitions

sql-cli/src/buffer_handler.rs

@@ -1,5 +1,5 @@
 use crate::buffer::{Buffer, BufferAPI, BufferManager};
-use crate::config::Config;
+use crate::config::config::Config;
 use tracing::{debug, info};
 
 /// Handles all buffer-related operations

sql-cli/src/cell_renderer.rs

@@ -4,7 +4,7 @@ use ratatui::{
     widgets::{Block, Borders, Cell},
 };
 
-use crate::config::CellSelectionStyle;
+use crate::config::config::CellSelectionStyle;
 
 /// Different visual styles for rendering selected cells
 #[derive(Debug, Clone)]

sql-cli/src/completer.rs

@@ -1,7 +1,7 @@
 use reedline::{Completer, Span, Suggestion};
 use std::sync::{Arc, Mutex};
 
-use crate::parser::{ParseState, Schema, SqlParser};
+use sql_cli::sql::parser::{ParseState, Schema, SqlParser};
 
 pub struct SqlCompleter {
     parser: Arc<Mutex<SqlParser>>,

sql-cli/src/config/mod.rs

@@ -0,0 +1,8 @@
+//! Configuration module
+//!
+//! This module contains all configuration-related functionality
+//! including settings, key bindings, and schema configuration.
+
+pub mod config;
+pub mod key_bindings;
+pub mod schema_config;

sql-cli/src/core/mod.rs

@@ -0,0 +1,10 @@
+//! Core business logic and application state management
+//!
+//! This module contains the central components that manage application state,
+//! buffer management, and service orchestration.
+
+// Core components will be moved here:
+// - app_state_container.rs
+// - buffer_manager.rs
+// - service_container.rs
+// - global_state.rs

sql-cli/src/data/adapters/mod.rs

@@ -0,0 +1,9 @@
+//! Adapters for existing data sources
+//!
+//! These adapters implement the DataProvider trait for existing data sources,
+//! allowing gradual migration to the new architecture.
+
+// Future adapters:
+// - buffer_adapter.rs - Makes Buffer implement DataProvider
+// - csv_adapter.rs - Makes CSVClient implement DataProvider
+// - api_adapter.rs - Makes API responses implement DataProvider

sql-cli/src/data/converters/mod.rs

@@ -0,0 +1,9 @@
+//! Data format converters
+//!
+//! Converters transform data between different formats
+//! (CSV, JSON, DataTable, etc.)
+
+// Will contain:
+// - csv_converter.rs
+// - json_converter.rs
+// - datatable_converter.rs

sql-cli/src/data_provider.rs → sql-cli/src/data/data_provider.rs

@@ -5,6 +5,47 @@
 
 use std::fmt::Debug;
 
+/// Filter specification for DataView
+#[derive(Debug, Clone)]
+pub enum FilterSpec {
+    /// SQL WHERE clause filter
+    WhereClause(String),
+    /// Fuzzy text search across all columns
+    FuzzySearch(String),
+    /// Column-specific filter
+    ColumnFilter { column: usize, pattern: String },
+    /// Custom filter function
+    Custom(String),
+}
+
+/// Sort order for columns
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum SortOrder {
+    Ascending,
+    Descending,
+}
+
+/// Data type for columns
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum DataType {
+    Text,
+    Number,
+    Date,
+    Boolean,
+    Json,
+    Unknown,
+}
+
+/// Column statistics
+#[derive(Debug, Clone)]
+pub struct ColumnStats {
+    pub null_count: usize,
+    pub unique_count: usize,
+    pub min_value: Option<String>,
+    pub max_value: Option<String>,
+    pub mean_value: Option<f64>,
+}
+
 /// Core trait for read-only data access
 ///
 /// This trait defines the minimal interface that any data source must provide

sql-cli/src/data/mod.rs

@@ -0,0 +1,23 @@
+//! Data layer for DataTable/DataView architecture
+//!
+//! This module provides the data abstraction layer that separates
+//! data storage from presentation.
+
+pub mod adapters;
+pub mod converters;
+
+// Core data modules
+pub mod data_provider;
+pub mod datatable;
+pub mod datatable_buffer;
+pub mod datatable_converter;
+pub mod datatable_loaders;
+pub mod datatable_view;
+
+// Data source modules
+pub mod csv_datasource;
+pub mod csv_fixes;
+pub mod data_analyzer;
+pub mod data_exporter;
+pub mod datasource_adapter;
+pub mod datasource_trait;