Protocol
SFTP or AS2 where partners require, offer an API for new partners.
Why. Meet partners where they are while steering them to better.
Trade-off. You run both for years.
Integration and multi-application · Partners
B2B file and EDI exchange with partners and suppliers
Secure, auditable exchange of orders, invoices and reports with partners that still send files: managed SFTP, validation, translation, acknowledgements and monitoring that proves nothing was lost.
A reference architecture from EITSAA, not a named client system. Service limits, prices and product features change, so confirm current details before you commit to a design.
- ACSC Essential Eight
- Australian Privacy Principles
- Partner security requirements
Architecture
- People and clients
- Object and file storage
- AI and ML
- Messaging, events and workflow
- SaaS and business systems
- Security and identity
- Monitoring, audit and analytics
- Request or data flow
- Asynchronous or optional
- Boundary: region, account, network or trust zone
Request and data flow
A partner connects to a managed endpoint with SSH key or certificate authentication, restricted to its own home directory and source IPs.
Files land in a per-partner prefix in an encrypted bucket. Versioning and Object Lock preserve the original as received.
An event triggers scanning for malware, size limits and schema or format validation.
Valid files are translated to the internal format by mapping rules. Invalid files are quarantined and a rejection is generated.
An acknowledgement (such as an EDI 997 or an AS2 MDN, or a status file) is produced for the sender.
The acknowledgement goes back to the partner.
Valid messages enter a queue and workflow with retries and a dead-letter queue.
A worker posts to ERP or CRM APIs with idempotency keys. File, message and result are correlated by one ID.
Components and why they were chosen
| Component | What it does | Why this choice |
|---|---|---|
| AWS Transfer Family (or Azure Storage SFTP) | Managed SFTP, FTPS and AS2 endpoints backed by object storage. | No SFTP servers to patch. Per-user scoping and logging. |
| S3 with SSE-KMS, Object Lock and lifecycle | Landing, archive and quarantine. | Immutable evidence of what was received. |
| Malware scanning (GuardDuty Malware Protection for S3 or Defender for Storage) | Scans uploads. | Partners are an attack path. |
| Validation and mapping (Lambda, Glue or B2B Data Interchange) | Schema checks and EDI translation. | Managed EDI translation is available if volume justifies it. |
| Workflow (Step Functions, SQS) | Orchestrates, retries and parks failures. | Visible state per message. |
| Partner registry | Keys, IPs, formats and contacts per partner. | Onboarding becomes configuration. |
| Monitoring and alerting | Expected-file checks and failure alerts. | Detect the file that never arrived. |
| Key and certificate management | Rotation schedule. | Expired certificates are a common outage. |
Design decisions and trade-offs
Evidence
Keep the original file immutably, process a copy.
Why. Disputes are settled by the original.
Trade-off. Storage cost.
Failure handling
Quarantine and notify, never silently drop.
Why. A missed order is a business failure.
Trade-off. Needs a team to handle rejections.
Onboarding
Configuration-driven per partner.
Why. Speeds up onboarding and reduces code.
Trade-off. Needs a well-designed registry.
Well-architected pillars
Security
Key auth, IP allow lists, scanning, encryption and scoped access per partner.
Reliability
Queues, retries, acknowledgements and expected-file alerts.
Performance
Parallel processing by file, with size limits and chunking for large files.
Cost optimisation
Transfer Family endpoint hours are the baseline. Storage and processing are small.
Operational excellence
A partner runbook for onboarding, key rotation and resend requests.
Sustainability
Lifecycle archive and avoid reprocessing.
Network, identity and data
Networking
Endpoint with fixed Elastic IPs so partners can allow-list, in a VPC with security groups limiting source ranges. Downstream workers in private subnets with endpoints.
Identity and least privilege
Each partner has its own user, key and S3 prefix restricted by IAM policy. Internal operators use SSO. Service roles are limited to specific prefixes.
Data and encryption
Files can contain personal and commercially sensitive data. Encrypt with KMS, restrict by prefix, apply retention and masking in lower environments, and log who accessed what.
Security and compliance mapping
| Control | How this design addresses it |
|---|---|
| Restrict administrative privileges | Scoped per-partner access. Essential Eight. |
| Logging | Transfer logs, S3 data events and workflow history. |
| Third-party risk | Partner security expectations documented and reviewed. |
Mappings show how the design supports each control. They are not certifications, and EITSAA does not claim IRAP assessment or compliance for any reference architecture.
Failure modes, targets and scaling
- Target RTOHours. Partners retry on failure
- Target RPOZero for accepted files held in S3
- Scaling and limitsThousands of files per day are routine. Check endpoint and AS2 message size limits.
| Failure mode | Mitigation |
|---|---|
| Partner sends malformed file | Validation, quarantine and a rejection sent promptly. |
| Expected file does not arrive | Schedule-based alert to the business owner. |
| Duplicate file sent twice | Hash and business-key de-duplication. |
| Certificate or key expiry | Expiry dashboard and 30-day alerts. |
Observability and runbooks
- Files received, accepted, rejected by partner.
- Time from receipt to ERP posting.
- Missing-file alerts.
Cost drivers and levers
Transfer endpoint hours and per-GB charges, translation, storage.
- Consolidate endpoints.
- Lifecycle old files.
- Move partners to APIs.
Implementation and fit
Phased approach
- Inventory partners and formats.
- Stand up the endpoint and landing zone with two partners.
- Add validation, acknowledgements and monitoring.
- Onboard remaining partners by configuration.
Anti-patterns to avoid
- Self-managed SFTP servers.
- Shared partner credentials.
- Silent file drops.
- No expected-file monitoring.
When to use it
- Partners and suppliers that exchange files or EDI.
- Regulatory or customer reporting by file.
When not to
- New partners who can use APIs.
- Real-time interactions.
More in integration and multi-application
AWS and Azure
API-led integration hub with an event bus, API gateway and iPaaS
A small number of governed APIs and an event backbone in front of many systems, so each application integrates once with the hub instead of with every other application.
Web, mobile and back office
Multi-channel API platform with a portal over several back-end apps
Web, mobile and staff back-office experiences on one API platform. A backend-for-frontend per channel and a micro-frontend portal let several applications look like one product without becoming one codebase.
Master data
Master data management and integration across HR, finance and CRM
Agree who owns each core entity (person, organisation, product, cost centre), publish the golden record to the systems that need it and reconcile the differences before they reach a report or an auditor.
Related insights
Discuss this architecture
Tell us your systems, constraints and timeline. We reply within one business day, and the initial conversation is free.