In this multi-part blog series, we will review all the integration points from the Google stack: Google Marketing Platform, Google Cloud Platform, Google Ad Manager.
Link to Part A: Data Collection
There are three out-of-the-box data connectors that can facilitate data ingestion:
- Google Pub/Sub
- Google Big Query
- Google Cloud Storage
Use case
As an enterprise, you make use of Google Cloud Platform (GCP) as your provider of data warehousing, streaming dataflows or files. Adobe Experience Platform has built data ingestion connectors to speed up the data flow by offering standardised workflows.
So what?
As an organisation, the IT team is using Google Cloud Platform services for existing services. These services transmit or store customer profiles or transactional data that is required for our customer experience management use cases:
- Increase the focus on the data rather than the technical integration
- Decrease developer work to manage middleware
- Decrease points of failure and focus on monitoring key integration points
Example:
- Event pipeline for customer profile creation or updates e.g. change of preferences/consent, form submission with profile data
- Customer interactions such as call centre events, chatbot events or in-store events
Glossary
Streaming or Batch Ingestion – Data Source/Data Connector
The type of data ingestion will determine what is the fastest Segment evaluation achievable. Understanding the Segment evaluation required to achieve the use case is key to assessing which combination of Data Source and Connector to integrate with.
Read more here: Segmentation Service Overview | Adobe Experience Platform
Category – Determines which connector configuration will apply e.g. a Database Category connector will connect at the table-level.
1. Google Cloud Storage & Adobe Experience Platform
One of the simplest ways of ingestion data into AEP is file-based ingestion.
| Streaming or Batch Ingestion | Category | Documentation Link |
|---|---|---|
| Batch | Cloud / File Store | Link |
Avoid if
- The data is used in Streaming or Edge Segments as this will be processed as a batch Segment evaluation
Best Practices
- Perform data cleanup before/while generating the files. Although there are data prep capabilities in AEP, it is best practice to get the data right from the source
- Form incremental data files to decrease the ingestion load and make use of the data storage management features in AEP such as Data Hygiene
- For time-series data, structure event data into a log sequential form i.e. instead of appending attributes to existing rows
2. Google Big Query & Adobe Experience Platform
Big Query is the swiss-army knife to get Google Marketing Platform (GMP) data into AEP. Most GMP applications will have a native integration to Google Big Query. Using this connector can facilitate this data ingestion.
Avoid if
- The data is used in Streaming or Edge Segments as this will be processed as a batch Segment evaluation
Best Practices
- Create separate tables with the fIltered data based on the use cases for segmentation and activation
- For profile attributes, select a convention to indicate incremental data changes: type, date, or time and update the least number of rows
- For time-series data, structure event data into a log sequential form i.e. instead of appending attributes to existing rows
3. Google Pub/Sub & Adobe Experience Platform
Avoid if
- There is little flexibility to pre-process the data
Best Practices
- Format the payload to the XDM Schema
- Send all the attributes of an Individual Profile (record) for a given new/existing Profile – although there is a way to handle partial payloads, this will decrease potential profile reconciliations
Conclusion
Work closely with your internal teams including IT and marketing to understand the end-to-end use cases in order to assess the suitability of using a connector or if you prefer a level of abstraction, the AEP HTTP API is always an option.
The number of out-of-box data connectors is growing and Google connectors are no different. AEP is also offering a self-serve sources SDK so that vendors can create and maintain their own Source Connector on the platform.
Photo by Daniel Olah on Unsplash