Platform
Build on the existing lakehouse or warehouse before buying a CDP.
Why. Lower cost and full control of data and logic for most mid-market organisations.
Trade-off. You build activation connectors that a CDP bundles.
CRM and business systems · Data platform
Customer 360 platform joining CRM, support, billing and analytics
One governed customer profile built from the systems you already have: identity resolution, consent-aware activation and privacy controls, with a clear line between what is analysed and what is acted on.
A reference architecture from EITSAA, not a named client system. Service limits, prices and product features change, so confirm current details before you commit to a design.
- Australian Privacy Principles
- Privacy Act 1988
- Spam Act 2003
- ACSC Essential Eight
Architecture
- SaaS and business systems
- Databases and data
- Object and file storage
- Security and identity
- Messaging, events and workflow
- Monitoring, audit and analytics
- Request or data flow
- Asynchronous or optional
- Boundary: region, account, network or trust zone
Request and data flow
Each source is ingested with its own keys and timestamps, using change data capture for databases and APIs or events for SaaS and web.
Data lands in the lakehouse with lineage. Nothing is merged yet, so every later decision stays explainable.
Identity resolution applies deterministic rules (verified email, customer number, phone) and records which rule produced each link.
The profile table holds one row per person or organisation, with source references, attributes and consent and preference flags.
Activation sends audiences back to CRM, email and support tools only where consent and purpose allow, and logs every export.
Analytics and ML use a pseudonymised view by default, for segmentation, churn and service demand.
Components and why they were chosen
| Component | What it does | Why this choice |
|---|---|---|
| Source connectors (DMS, AppFlow, Fivetran, Airbyte) | Land source data. | Managed connectors reduce maintenance. |
| Lakehouse on S3 with Iceberg, or Fabric OneLake | Storage with history and time travel. | Auditable and engine-neutral. |
| Identity resolution (SQL rules, dbt, Glue) | Links records for the same customer. | Deterministic first because it is explainable and reversible. |
| Profile store (warehouse table, Aurora or DynamoDB) | Serves lookups and audiences. | Match the store to the access pattern: analytics or low latency. |
| Consent and preference store | System of record for permissions. | Activation must check it every time. |
| Lake Formation or Microsoft Purview | Governance, tags and access policy. | Central policy for personal information. |
| Reverse ETL and APIs | Send audiences and attributes to tools. | Closes the loop. |
| BI and ML tooling | Dashboards and models. | Where the single view pays back. |
Design decisions and trade-offs
Matching
Deterministic rules with reviewable overrides.
Why. Wrong merges of people are harmful and hard to undo.
Trade-off. A lower match rate than fuzzy logic.
Consent
Store consent as data with timestamp and source, enforced at activation.
Why. Supports Spam Act and privacy duties.
Trade-off. Every activation path must call the check.
Analytics view
Pseudonymised by default.
Why. Most analysis does not need identity.
Trade-off. Re-identification needs an approved path.
Well-architected pillars
Security
Column-level controls, tokenisation, private connectivity, least-privilege roles and export logging.
Reliability
Replayable raw data and idempotent merges. Profiles can be rebuilt from raw.
Performance
Incremental processing and serving tables tuned to the lookup pattern.
Cost optimisation
Storage is cheap. Matching compute and syncs drive cost.
Operational excellence
Data owners per domain, quality tests on keys and a match review queue.
Sustainability
Incremental processing and removal of unused attributes.
Network, identity and data
Networking
Connectors run in private subnets with endpoints. SaaS sources are reached over TLS with allow-listed egress. BI tools connect privately to the warehouse.
Identity and least privilege
Role-based access by data classification. Analysts see pseudonymised data. A small named group can resolve identity under logged, approved access.
Data and encryption
Classify attributes, encrypt with KMS and apply retention per purpose. Support deletion on request through lineage-aware processes and record the purpose per attribute. Review overseas disclosure for each SaaS source (APP 8).
Security and compliance mapping
| Control | How this design addresses it |
|---|---|
| Collection, use and disclosure | Purpose tags and consent flags support APP 3, 6 and 7. |
| Access and correction | A profile makes APP 12 and 13 requests easier to answer across systems. |
| Security of personal information | Encryption, masking and access logging support APP 11. |
| Restrict administrative privileges | Separate admins from analysts. Essential Eight. |
Mappings show how the design supports each control. They are not certifications, and EITSAA does not claim IRAP assessment or compliance for any reference architecture.
Failure modes, targets and scaling
- Target RTOHours, rebuilding the profile from raw
- Target RPOPer source, from minutes to a day
- Scaling and limitsMillions of profiles are routine on a lakehouse. Real-time lookups need a serving store.
| Failure mode | Mitigation |
|---|---|
| Wrong merge of two customers | Rule provenance, an unmerge process and conservative thresholds. |
| Stale consent used | Check consent at send time, not at audience build time. |
| Source schema change breaks a pipeline | Data contracts and alerts. |
| Re-identification misuse | Access approval and logging. |
Observability and runbooks
- Match rate, merge conflicts and unmerge count.
- Freshness per source and per attribute.
- Audience export log with consent evidence.
Cost drivers and levers
Connector volume, matching compute, storage and BI.
- Process incrementally.
- Limit history on high-volume event data.
- Do not ingest sources with no use case.
Implementation and fit
Phased approach
- Pick two use cases and the two sources they need.
- Build ingestion and the first deterministic match.
- Add consent and one activation path.
- Add sources and ML once the profile is trusted.
Anti-patterns to avoid
- Fuzzy matching on day one.
- Activating without checking consent.
- A profile no team owns.
- Collecting data with no use.
When to use it
- Customer data spread across several systems.
- Teams needing one view for service and marketing.
When not to
- One system already holds everything.
- No clear use case or data owner.
More in crm and business systems
Zendesk and ServiceNow
Service desk with chatbot deflection and a knowledge base RAG
Zendesk or ServiceNow as the system of record for tickets, a chatbot that answers from approved knowledge with citations, and a clean hand-over to a person with full context when it cannot.
Entra ID and Okta
Identity and single sign-on across CRM and business apps
One identity provider for staff, partners and customers: SAML and OIDC federation, SCIM provisioning, Conditional Access, role mapping and a leaver process that really removes access.
Microsoft 365
Secure Microsoft 365 workplace, ready for Copilot
A Microsoft 365 tenant for councils and small organisations that is safe before AI arrives: Entra ID with Conditional Access, Intune-managed devices, Purview labels and DLP, oversharing cleaned up, and a backup copy outside the tenant.
Related insights
Discuss this architecture
Tell us your systems, constraints and timeline. We reply within one business day, and the initial conversation is free.