Skip to content

Conversation

@nipsysdev
Copy link
Owner

@nipsysdev nipsysdev commented Sep 16, 2025

Pull Request: Refactor Crawler Type Implementation

Summary of Changes

This PR refactors the crawler type implementation to use a TypeScript enum (CrawlerType) instead of string literals. This change improves type safety and maintainability across the codebase.

Key Changes

1. Type Safety Enhancement

  • Replaced string-based crawler types ("listing") with CrawlerType enum
  • Updated import statements to include CrawlerType from core types

2. Configuration Structure Updates

  • Simplified source configuration structure by removing the nested items object
  • Moved container_selector and fields properties directly under listing

3. Core Type Definitions

  • Updated core/types.ts to use the new CrawlerType enum
  • Removed CRAWLER_TYPES constant in favor of the enum

4. Comprehensive Test Updates

  • Updated all test files to use the new CrawlerType enum
  • Modified test configurations to match the new structure
  • Fixed optional chaining in content field assertions

@nipsysdev nipsysdev self-assigned this Sep 16, 2025
@nipsysdev nipsysdev added the enhancement New feature or request label Sep 16, 2025
@nipsysdev nipsysdev merged commit 51c3dd9 into main Sep 16, 2025
5 checks passed
@nipsysdev nipsysdev deleted the refactor/crawler_type branch September 16, 2025 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants