Blog

Welcome to The Scalable Way Blog — your source for insights, strategies, and best practices in data platform engineering, analytics, and data science.

CI/CD for Data Workflows: Automating Prefect Deployments with GitHub Actions

prefect prefect worker github actions CI/CD data workflows data platform architecture

The final part of the Data Platform Infrastructure on GCP series covers CI/CD for Prefect deployments using GitHub Actions and Docker. Automate flow builds, worker updates, and streamline orchestration across environments.

Scaling Secure Data Access: A Systematic RBAC Approach Using Entra ID

Data Governance Access Management RBAC Entra ID Security Architecture

Establish scalable, secure access controls for your data platform with a systematic RBAC strategy built on Microsoft Entra ID. This article outlines a five-phase implementation—from user persona mapping to automated auditing—designed to balance flexibility, compliance, and operational efficiency.

Getting to Your First Flow Run: Prefect Worker & Deployment Setup

prefect data platform architecture workflow orchestration Kubernetes

Run your first data ingestion workflow with Prefect, Docker, and Kubernetes. This guide walks through containerized flow execution, Prefect worker deployment, and clean deployment configs, laying the foundation for a scalable, maintainable orchestration layer.

Roles in the Context of the Analytics Workflow

governance analytics workflow roles

Code-based analytics workflows offer a unique advantage - the ability to combine robust collaboration with strict governance. While technically straightforward to implement, this approach only reaches its full potential when aligned with well-adapted business and analytics processes.

How to Setup Data Platform Infrastructure on Google Cloud Platform with Terraform

google cloud platform terraform data platform architecture prefect

Learn how to set up a secure, scalable data platform infrastructure on Google Cloud Platform (GCP) using Terraform. This step-by-step guide covers VPC configuration, Compute Engine setup, firewall rules, Identity-Aware Proxy (IAP), Cloud NAT, and more, ensuring a cost-effective, flexible, and secure foundation for your data platform.

Organizing Networking for Data Platforms: Key Connectivity Options

networking data platform architecture elt process connectivity

Optimize your data platform by making informed networking decisions. This article explores how networking impacts ELT workflows, covering key connectivity options, security considerations, and best practices. Learn how to design a secure, scalable, and high-performing data platform architecture with the right networking.