Bridging the Gap: How Data & Software Engineering Teams Can Work Together to Ensure Smooth Data Integrations

GOTO Chicago 2023

Imagine this scenario: The engineering team at your company is about to launch a new product or feature. The pressure is on the data team to deliver metrics and stats from the get-go, as everyone is keen to see how the new feature is performing. Behind the scenes, the data team is scrambling to understand how to connect to the new systems, transform the data, interpret it, and turn it into actionable insights.

In this talk, I will walk the audience through the different steps necessary to ensure both data teams and software engineers are ready to integrate with a new production system and deliver correct and reliable analytics from day 1. I will cover topics such as infrastructure and data access considerations, data models, data flow and state changes, and data quality checks. I will also touch upon “data contracts”, a concept that’s been talked about a lot recently in the data space, and how these can be established to preserve consistent high-quality data integrations.

Both data practitioners and software engineers will benefit from this talk and learn how to “bridge the gap” between the two worlds.