14.76 Million Records Out of Windchill Product Analytics and Into 3DEXPERIENCE in a Single Load Window

The source system was PTC Windchill Product Analytics 11.0-M010. The data itself carried four problems that had to be resolved before anything could be loaded: EBOM structures held multiple versions with no obvious correct one, some parts appeared in the BOM structure table but were missing from the part table, duplicates existed where the supposedly unique value contained a space, and the IPC-1752 material declaration files came in several different formats.

The migration ran on SteepGraph’s ETL Data Migration Framework for extraction, transformation and loading, into 3DEXPERIENCE 2021x.FD06. History was not carried over, selective triggers were switched off for the duration, and SOV stamping was applied during the load. A dedicated Linux server carried the load with a server-only installation for MQL access, running 120 threads in parallel against a dedicated database server; the application server sat at 60–70% CPU throughout and the database server at 85–90%, against a normal baseline of 2–3%. The full 14.76 million records completed in five to six hours. Validation was done by extracting everything that had landed in 3DEXPERIENCE, applying the same transformation rules to the source data, and comparing the two sets. Duplicate data and special characters were cleaned as part of the load rather than before or after it. The material declaration files were processed through Dassault Systèmes out-of-the-box APIs. The licences required were MCM, MCY, PDE and PDM.

Go-live was incremental. The build was deployed to production a month ahead, with no user onboarding and no announcement that it was available, and a delta migration of that month’s data ran on the day itself. The customer gained an enhanced interface in 3DEXPERIENCE carrying more validation than before, and improved data accuracy through the duplicate and special-character handling.