The challenge
Neuhaus needed a dependable data flow across commerce and operational systems that did not share one contract or delivery mechanism. Orders arrived from Salesforce Commerce Cloud, Dynamics 365 and another retail channel, while product and price information used separate XML contracts. These sources had different identifiers, structures, update behaviour and failure modes, but downstream consumers needed consistent data for storage, reporting and further processing.
An event-driven integration layer
David designed and built a set of .NET Azure Functions around Azure Service Bus and blob-created events. Source-specific functions read JSON or XML payloads, validate and map them into shared contracts for orders, products and prices. A scheduled Salesforce Commerce Cloud flow retrieves changed orders and feeds them into the same normalized order pipeline.
Each translator keeps knowledge of its source at the edge. Once a message is normalized, downstream components no longer need to understand MATMAS product XML, price contracts or the individual order formats. This gives the platform a stable internal language while allowing each external integration to evolve independently.
Reliable processing and recovery
Processing state is recorded so duplicate or stale messages can be detected before they affect the data set. Message identifiers and source timestamps support idempotent behaviour, while failed transformations are logged separately and infrastructure failures can move messages to dead-letter queues. Scheduled recovery functions can replay those messages after the underlying issue is resolved.
Managed Azure identities provide access to Service Bus and storage services. Cancellation tokens, retry-enabled SQL access and centralized exception middleware help the functions behave predictably during shutdowns and temporary infrastructure failures.
From raw events to usable data
A separate ingestion application consumes the normalized messages. It retains source artefacts in Azure storage and builds structured destinations for analytical and operational use. Orders are separated into headers, addresses, lines, discounts and payments. Products are expanded into channels, descriptions, compositions and inner products, while prices receive their own history and detail structures.
The data is written to Azure SQL and transformed into Parquet datasets in bronze and silver Data Lake layers. Processing-history and error records make movement through the pipeline observable without coupling the translation functions to a single destination.
Configurable database synchronization
The platform later expanded with an ASP.NET Core and Hangfire service for configurable SQL Server synchronization. Each enabled table has its own schedule and connection configuration. Jobs can perform complete refreshes or incremental synchronization using date/time or row-version watermarks.
The service inspects schemas, validates supported SQL types, bulk-loads changed rows into temporary tables and merges them into their destination. Separate maintenance and synchronization queues, per-table concurrency control, structured SQL logging and a Hangfire dashboard make recurring work visible and controllable.
Result
Neuhaus gained a reusable data backbone rather than a collection of point-to-point scripts. Commerce events now become shared contracts and flow toward Data Lake and SQL destinations with traceable processing and recovery. Automated contract, scheduling, watermark and SQL-generation tests protect the most important transformations as the platform continues to grow.