Why The Sitecore Data Exchange Framework is Important

I recently had a discussion with a colleague around how we were best going to integrate a CRM and multiple external data lookup services with Sitecore Xdb for a full experience management merry go round.

The issue was a bit of a brain melter because the CRM is in transition between platforms, the external service lookups need to run through Azure Apps and the whole back end system is undergoing alignment to get a single view of the customer internally - fun fun fun.

So you can see that there is a lot of integration pieces that need to be created and managed in an orderly modular Helix-compliant fashion.

Now initially the integration pieces (CRM of any type - might as well be a black box) and numerous external lookup services (writing to the xDB too) might be small to middle sized but undoubtedly they will get added to and hopefully not in way that adds technical debt - that would be my preference. 

To my mind a lot of these processes can be built abstracted and modular and seem well suited to being implemented using Pipelines in the Data Exchange Framework. 

So why, I asked myself, add another layer to this bucket brigade?

Well being only a junior dev in terms C# experience, I can see we have come along way with MVC and NuGet and other methods to keep our code modularised and re-usable etc. etc., but the Data Exchange Framework offers something that no well structured solution can.. and that is..

The ability to model and organise processes as an ETL (extract, transform and load) layer for Sitecore. The parts of a synchronization process map directly to ETL functions.

The Data Exchange Framework (DEF) allows you to model the ETL functions using Sitecore Items in the content tree and this is where it's classic Sitecore. 

DEF also does other funky stuff, your Pipelines can be run on other machines with Sitecore acying as just a configuration server.

Lastly DEF is the precursor to xConnect (as I understand it), the coming integration framework for Sitecore. As such I assume it will be the Sitecore recommended approach to external integration going forward so it seems like a good plan to get on board now. 

The developers guide is a good place to start: 

http://integrationsdn.sitecore.net/DataExchangeFramework/v1.2/

And the full API is available here: 

https://dev.sitecore.net/~/media/D47877D168AE446CA05500F24AF7D9C1.ashx

Main Sitecore page and download here: 

https://dev.sitecore.net/Downloads/Data_Exchange_Framework.aspx