Introduction 

For a sub prime lender who has been using Teradata and Data stage has been having constant maintenance costs as the business is growing. Scalability issues, long time to run critical queries. Ability to provide right set of data to partners and consumers.

• Slash data platform costs

• Single data source Truth

• Instant non-disruptive scalability

• Easy to use Governance and security

• Ability to extract data and run Azure ML models

• Ad-hoc queries support for new business needs

• Structured, Semi-structured data with transformations in multiple locations.

• Constant data streaming with delta changes

• Not a well architected existing system

• Initially designed a lift and shift model to migrate all data leveraging Databricks instead of native Snowpark for cost and efficiency.

• Rearchitected the system to remove dependencies from DataStage and use one tool and aggregated the data.

• Built entire platform as single source of truth using snowflake on Azure.

• The security is managed through Active Directory and tied corporate governance model.

Technological Framework:

Why this setup?

Snowflake's elastic compute and separation of storage from computation delivers instant, non-disruptive scalability and superior query performance compared to Teradata — significantly reducing total cost of ownership.

Why this setup?

Replacing DataStage dependencies with Databricks-based pipelines streamlines the transformation layer and enables real-time delta streaming at scale without requiring infrastructure team involvement.

Why this setup?

Active Directory integration enforces enterprise security and compliance, while Azure ML connectivity unlocks advanced analytics and predictive modelling directly on the unified Snowflake platform.

Takeaway

Zapcom migrated a sub-prime lender's entire data estate from Teradata to a unified Snowflake-on-Azure platform, achieving 55% operational efficiency gain and 3.5x improvement in query processing time.