Major refactor: security enhancements, modularization, config integration, reduced Parquet reliance

- Migrated codebase to class-based architecture for better modularity and maintainability
- Introduced system_config.json for centralized configuration (required for runtime)
- Added structured working directories for improved file organization
- Significantly reduced reliance on Parquet; replaced with alternative data handling
- Implemented security improvements across modules
- Several TODOs remain in the main script for future enhancements
- Linter formatting affected readability in some files (e.g., utils); cleanup is on the agenda

requirements.txt

@@ -1,29 +1,11 @@
-bson==0.5.10
-certifi==2025.8.3
-charset-normalizer==3.4.3
-colorama==0.4.6
-cramjam==2.11.0
-docopt==0.6.2
-dotenv==0.9.9
-fastparquet==2024.11.0
-fsspec==2025.9.0
-idna==3.10
-ijson==3.4.0
-lxml==6.0.0
-markdown-it-py==4.0.0
-mdurl==0.1.2
+cryptography==46.0.1
+keyring==25.6.0
 numpy==2.3.2
-packaging==25.0
 pandas==2.3.1
-pretty-tables==3.1.0
-pyarrow==21.0.0
-Pygments==2.19.2
-python-dateutil==2.9.0.post0
 python-dotenv==1.1.1
-pytz==2025.2
-requests==2.32.4
-six==1.17.0
+pymongo
+requests==2.32.5
+schedule==1.2.2
 tqdm==4.67.1
-tzdata==2025.2
 urllib3==2.5.0
-yarg==0.1.10
+bson==0.5.10