Across industries, organizations are under growing pressure to modernize their data & AI infrastructure. Whether it’s to support cloud adoption, enable real-time analytics, integrate AI, or simply reduce costs—legacy systems need to evolve.
But modernizing these systems – especially migrating existing content to a new data platform and technology – is rarely straightforward. Many projects fail, exceed budgets, or stall indefinitely.
In this post, we’ll explore:
- Challenges of data platform migrations
- Why data platform migrations are necessary?
- Why data platform migration fail?
- How can Large Language Models (LLMs) start to help teams navigate these projects with more confidence?
We’ll also look at tools provided by Databricks, Snowflake, and compare those to platform-independent tools like ChatGPT.
Data Platform Migrations Challenges
Some of our customers operate with over 10,000 ETL scripts running on aging platforms. Others maintain multi-thousand-line SQL procedures buried in monolithic jobs. And honestly, you’d be surprised how many critical data workflows exist today on platforms that are barely running – or aren’t even being used.
Certain of our customers know they need to modernize – but the sheer scale, risk, and cost of migration often puts the brakes on. There’s no easy path from legacy to modern. Here are some of the key challenges we regularly encounter:
- Undocumented logic: Years of business rules live inside stored procedures no one wants to touch.
- Tight interdependencies: Dashboards, reports, APIs, even third-party systems depend on the same legacy database objects.
- Custom procedures and triggers: Migrating vendor specific procedural logic or custom UDFs is never 1:1.
- Technological diversity: You’re not just translating SQL. You’re connecting different data formats, scripts and coding languages, connectors, and business logic.
- Ownership confusion: Often it’s not even clear who built a process – or who owns it today.
- Outdated technology: Migrating content requires knowledge of the legacy system and the new platform. Usually the combination of these knowledge is quite rare.
Common Reasons Why a Platform Migrations Can Fail
Despite the pressure to modernize, many migration projects stall – or collapse entirely. Industry studies (McKinsey, SCIRP, Experian) show that the majority of data migration efforts exceed budgets, miss timelines, or fail altogether.
Here are some of the most common reasons why a platform migration can fail:
- Underestimating technical debt
- No proper assessment of the current system
- Weak migration strategy or no clear plan
- Constantly shifting requirements
- Lack of tooling, automation, or governance
- Confusion over who owns what: business or IT?
- Important Regulatory reports partly not being updated for months
- Injection of different coding languages into the SQL
- Highly vendor specific performance optimizations
- Differences between documentation (if any) and the productive ETL pipelines
How to Make Migrations Succeed?
From our own experience, we have seen the following approach work successfully.
Six steps for a successful data platform migration
1. Start with a platform assessment
Understand what’s actually running on the legacy platform: code volume, dependencies, usage patterns, etc.
2. Run a focused MVP
Start small and learn fast.
3. Identify all stakeholders early
Avoid last-minute surprises. Everyone from Compliance to BI needs to be in the loop.
4. Building detailed migration strategy
Plan all required steps of the migration and decouple applications from each other
5. Build an interdisciplinary team
Combine project managers, architects, and engineers who understand both legacy and modern platforms.
6. Plan the go-live and sunset carefully
Switchover dates, fallback plans, and communication matter as much as code.
Estimating the migration efforts
Even with a well-run migration strategy, the scale of work can be staggering. For large enterprise systems, rough estimates can easily exceed 10,000 person-days of effort.
- The considerable amount of effort required often appears not to be worth for many companies
- No budget allocated for this
- Running legacy infrastructure seems to be more attractive than a new “fancy” environment
- This leads to rethink the entire migration
- Results in being locked in at the current stage
- Limits Data & AI innovations
LLM-Assisted Migration Tool for Data Platform Migration
LLM-assisted migration tools may offer a breakthrough. They can speed up the entire process of migrating legacy code by providing:
- Parse and explain legacy code
- Suggest optimized rewrites for modern engines
- Automate translation with human-readable justifications
To provide some examples what is often considered as legacy toolstack:
- Oracle PL/SQL: procedural code embedded in the database; dense packages, triggers, and hidden side-effects
- Teradata: powerful MPP with proprietary SQL and utilities (BT/ET, FastLoad, BTEQ) that cant not easy migrated to Opensource
- Exasol: high-performance in-memory SQL with vendor-specific functions and optimizer steps
- Informatica: GUI-driven ETL with complex mappings, reusable transformations, and job orchestration baked into the tool
Why avoiding some LLM-assisted migration tool
These toolstack are hard to read/maintain and even harder to migrate, mostly due to:
- Proprietary coding languages or dialects
- Implicit logic (triggers, stored procedures, job parameters) that’s easy to miss in code reviews
- Scattered lineage across SQL, ETL jobs, and scheduler, hard to reconstruct intent
- Rare technical skills nowadays: expertise exists, but it’s increasingly hard to replace with hires
With thousands of scripts and projects, a migration becomes a massive, high-effort program. At enterprise scale with thousands of pipelines, jobs, and code artifacts, a migration turns into a heavy lift that demands rigorous planning and automation. Additionally, managing thousands of legacy components makes migration a costly, coordination-intensive initiative.
How LLM-assisted migration tool can help data platform migration?
LLM-supported migration tool takes a lot of the pain out of moving legacy code. Instead of spending days untangling old data pipelines or reverse-engineering legacy ETL logic, AI can now do the heavy lifting. That means teams can finally focus on what really matters: designing better data models, validating results, and tuning for performance.
A growing set of tools is emerging to make this work faster and smarter. General-purpose copilots like ChatGPT or GitHub Copilot can help refactor and explain legacy scripts, while specialized migration tools such as Databricks Lakebridge or Snowflake SnowConvert are specifically designed for the target architecture. Early tests indicate show time savings of up to 80% for the tedious “translation” tasks.
