Development of a Cloud Data Pipeline Orchestration Portal
Data Science — 2025, Undergraduate
web applicationorchestrationschedulingcloud data pipeline
This study developed a cloud data pipeline orchestration portal that schedules, monitors, and retries data jobs across storage and processing stages. The portal visualises job graphs and tracks runtime and failures. Adopting a client-server architecture and a survey-based usability evaluation, the system was built using Python, Flask, and SQLite. The findings showed that the portal improved pipeline reliability and observability, while achieving a high System Usability Scale score. The study recommends deployment for cloud-based data teams.
Ad slot
Homepage screenshot
