Add end-to-end tests for extraction pipeline. Closes #636

[Padhmashikha]

Description

This PR introduces end-to-end tests for the extraction pipeline, ensuring that the full extraction flow works correctly from data ingestion to database persistence.

While there are existing unit tests for individual extraction-related classes, the extraction pipeline as a whole was not previously tested. This PR fills that gap by validating the complete execution path, including cronjob invocation, service-layer processing, and database object creation.

The tests are designed to work with the current main branch architecture, which no longer contains a standalone ExtractionPipeline class. Instead, the tests cover the real execution flow used in production

Related issues

Please add related issues.
#634

Type of change

Test improvement (adds end-to-end test coverage for the extraction pipeline).
Non-breaking change (no changes to production logic)
Improves reliability and maintainability.
Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue).
• New feature (non-breaking change which adds functionality).
• Breaking change (fix or feature that would cause existing functionality to not work as expected).

Checklist

• I have read and understood the rules about how to Contribute to this project.
• The pull request is for the branch develop.
• I have added documentation of the new features.
• Linters (Black, Flake, Isort) gave 0 errors. If you have correctly installed pre-commit, it does these checks and adjustments on your behalf.
• I have added tests for the feature/bug I solved. All the tests (new and old ones) gave 0 errors.
• If changes were made to an existing model/serializer/view, the docs were updated and regenerated (check CONTRIBUTE.md).
• If the GUI has been modified:
  • I have a provided a screenshot of the result in the PR.
  • I have created new frontend tests for the new component or updated existing ones.

Important Rules

• If you miss to compile the Checklist properly, your PR won't be reviewed by the maintainers.
• If your changes decrease the overall tests coverage (you will know after the Codecov CI job is done), you should add the required tests to fix the problem
• Everytime you make changes to the PR and you think the work is done, you should explicitly ask for a review. After being reviewed and received a "change request", you should explicitly ask for a review again once you have made the requested changes.

[regulartim]

Hey @Padhmashikha ! You made a lot of changes to already existing files. Is this really necessary for solving issue #634?

Also, in case that you missed it, we use Ruff instead of Black, Flake and Isort for linting. Please refer the contribution guidelines for further information.

[Padhmashikha]

hi @regulartim can u please go through the pr.

[regulartim]

Hey @Padhmashikha ! Sure, if you answer my question from five days ago! ;)

[regulartim]

Also: Why do you think that this is a breaking change?