Home › AWS › AWS data services › Transfer Family

AWS data services · Ingestion and CDC · about 11 min read

AWS Transfer Family

A managed SFTP, FTPS, FTP and AS2 server, plus outbound connectors and browser web apps, that store files straight into Amazon S3.

RoleIngestion and CDC
StatusGenerally available. Web apps, VPC-reachable SFTP connectors and IPv6 for connectors and web apps were added in 2025 and 2026.
Verdict on this sitePursue
AzureSFTP support for Azure Blob Storage (or Azure Data Factory SFTP connector)
Google CloudNo managed SFTP server; a partner product from the Marketplace or a self-run server
Databricksnone: Auto Loader reads the files Transfer Family writes to S3; Lakeflow Connect has an SFTP source for pulling
Open sourceOpenSSH sftp-server or SFTPGo on your own virtual machine
Verdict: Pursue. The front door for supplier and partner files on this platform. Suppliers keep the SFTP they already use, every file lands in the S3 landing bucket, and the Databricks lander picks up only new files from its file log. There is no server for us to patch.

In plain words

What it is

Many companies still exchange data as files. A raw-material supplier sends a price list every morning, a logistics partner sends delivery confirmations every hour, and a retail chain sends point-of-sale extracts at night. The way they send them has barely changed in twenty years: they connect to a server with SFTP, which is the Secure File Transfer Protocol that runs over SSH, and they upload a CSV or an Excel file. Some older partners use FTPS, which is FTP wrapped in TLS, and some large retailers and manufacturers use AS2, a protocol for signed and encrypted business documents such as EDI purchase orders.

AWS Transfer Family gives you that server without running one. You create a server endpoint, choose which protocols it speaks, and tell it where files go. Instead of writing to a local disk, the server writes straight into an Amazon S3 bucket or an Amazon EFS file system. To the supplier it looks like any other SFTP server, with a host name, a user name and a key. To you it is a bucket that fills up with files, and AWS keeps the servers patched, scaled and highly available.

Transfer Family has grown beyond the inbound server. SFTP connectors let your platform push files to, or pull files from, a partner's own SFTP server, so you can fetch the daily price file instead of waiting for it. Web apps give business users a browser page to upload or download files in S3 without any SFTP client. Managed workflows can run steps after a file arrives, such as copying, tagging, decrypting a PGP file or calling a Lambda function.

A concrete example: a tyre maker buys natural rubber, carbon black and steel cord from twelve suppliers. Each supplier gets its own user and its own folder, which is really a prefix in the landing bucket. At 06:00 a supplier uploads prices_2026-11-11.csv. Transfer Family writes it to s3://landing/suppliers/acme/prices/, and the next platform run reads it into bronze.

A post office counter that accepts parcels in the old familiar way, but drops each one straight into the right locker in your warehouse.

Big picture

Where it fits

SourcesMove inStreamStoreProcessServeUseAcross everything: catalog · security · orchestration · observability and cost · infrastructure as codegets data fromhands data toSupplier and partner SFTP clientsRetail and EDI partners over AS2Business users in a browser (web apps)Transfer FamilyIngestion and CDCS3Databricks Auto Loader (the lander)Lambda

The lit lane is where Transfer Family works. Boxes on the left feed it; boxes on the right use what it produces.

Transfer Family sits in the move-in lane, on the files hop. On the reference platform the movers are split by source type: DMS for databases, AppFlow for SaaS and SAP OData, Transfer Family for supplier SFTP, and IoT Core with SiteWise Edge for plants. Each mover writes into the S3 landing area, and nothing else on the platform has to know which mover was used.

Downstream, the Common Ingestion Framework treats a supplier feed as one more YAML entry. The lander, running on Databricks, reads new files with Auto Loader in available-now mode on a schedule, records each file in its file log, and writes rows into bronze with the file name, arrival time and run id. Because Transfer Family only ever adds objects to S3, replay is simple: the files stay in landing, and the lander can read them again.

Transfer Family is there because suppliers will not change how they send files for you. A managed endpoint removes the patching and high-availability work of a self-run SFTP box, and it keeps the file in your own bucket, under your own KMS key, from the first byte. Macie then scans the landing bucket for personal data, and CloudTrail records who changed users and servers.

How it works

The ideas you need

IdeaWhat it means
Server endpointThe thing partners connect to. It has a host name, one or more protocols, and a type: public, or hosted inside your VPC with optional internet-facing Elastic IPs.
Service-managed userA user that Transfer Family stores itself, with an SSH public key, a home directory and an IAM role. Simplest for a handful of suppliers.
Custom identity providerA Lambda function or API Gateway method that checks a password or key against your own directory, used when you need passwords, many users, or Active Directory.
Logical home directoryA mapping that shows each user only their own folder, for example a user sees / while files really go to s3://landing/suppliers/acme/. It stops one supplier seeing another's files.
Managed workflowA short list of steps that runs after an upload completes: copy, tag, delete, PGP decrypt, or a custom Lambda step.
SFTP connectorAn outbound client run by AWS that sends files from S3 to a partner's SFTP server or retrieves files from it, on request. It can now reach servers that are only visible from your VPC.
AS2 connector and agreementThe partner, certificate and agreement records that let you exchange signed and encrypted EDI documents with acknowledgements, called MDNs.
Web appA browser front end for files in S3, tied to IAM Identity Center users and S3 Access Grants, billed per web app unit hour.

Choosing

When to use it, and when not

Use it when

  • Suppliers, carriers or retail partners already send files over SFTP, FTPS or AS2 and will not adopt an API.
  • You want files to land directly in an S3 landing bucket that your lakehouse already reads.
  • You need per-partner isolation, with each partner seeing only its own folder and every login recorded.
  • You need to pull files from a partner's SFTP server on a schedule without running your own SFTP client host, which is what SFTP connectors do.
  • Business users need to drop occasional files, such as a manual price adjustment, and you would rather give them a browser page than an SFTP client.

Avoid it when

  • The source is a database. Use change capture with DMS instead of nightly CSV dumps over SFTP.
  • The source is a SaaS application with an API that AppFlow or a Lakeflow Connect connector already supports.
  • You are doing a one-time bulk copy of terabytes from a file server; DataSync or Snowball is built for that and costs less per GB.
  • Only one internal team moves files between two AWS accounts; S3 replication or a bucket policy is simpler than an SFTP endpoint billed by the hour.

Production grade

What goes wrong in production

ProblemWhat breaksWhat to do
Half-written files are readThe lander starts reading a large file while the supplier is still uploading it, and bronze gets a truncated file.Read only completed objects: S3 only shows an object once the upload finishes, so do not have suppliers write in place over EFS. Ask suppliers to upload to a temp name and rename, or trigger on the workflow's completed upload.
Re-sent files create duplicatesA supplier re-sends yesterday's file with the same name or a new name, and the same rows load twice.Keep a file log keyed on bucket, key and ETag, dedupe on a business key in silver, and turn on S3 versioning so overwrites are kept.
Supplier IP allowlistsThe partner's firewall only allows connections to fixed IP addresses, and a public endpoint's addresses are not fixed.Use a VPC-hosted endpoint with Elastic IP addresses attached; for outbound connectors, use the connector's static IPs or the VPC connectivity option.
Key and password sprawlDozens of supplier keys, some shared between staff, none rotated, and no one knows who still uses which.One user per partner system, SSH keys only where possible, an owner and expiry date in the source register, and CloudTrail plus CloudWatch login logs reviewed monthly.
Schema drift in CSV filesA supplier adds a column or changes a date format and the downstream job fails or silently shifts values.Land files untouched, read them in bronze with rescued-data columns, and keep a data contract per feed that alerts the feed owner.
Endpoint hours add upEach protocol on each server is billed every hour, so a test server and a forgotten FTPS protocol cost money all month.Use one production server with several users rather than one server per supplier, enable only the protocols you need, and delete test servers.
Personal data arrives unaskedA partner file contains customer names or emails that were not in the agreement.Macie scans the landing bucket; the lander tokenises personal fields before bronze, and the contract names the allowed columns.

Alternatives

What to compare it with

AlternativePick the alternative whenPick Transfer Family when
Self-run SFTP on EC2 (OpenSSH or SFTPGo)You have one or two partners, need features Transfer Family lacks, and accept patching and failover yourself.You want no servers to patch, built-in high availability and native S3 storage.
AWS DataSyncYou are copying a large existing file share or another cloud's bucket into S3, often once or on a schedule you control.Outside partners need to push files to you over SFTP with their own logins.
Amazon AppFlowThe source is a SaaS app or SAP OData service with an API connector.The partner only offers files over SFTP.
Lakeflow Connect SFTP source or a Databricks job with an SFTP libraryYou only need to pull from a partner server and want everything inside Databricks.Partners must push to you, or you need AS2 and per-partner login records.
Azure Blob Storage SFTPYour platform is on Azure and files should land in ADLS Gen2.Your landing zone is S3 on AWS.
Google Cloud partner SFTP productsYour platform is on Google Cloud; there is no first-party managed SFTP server there.You are on AWS, where a first-party service exists.

Cost

How you pay

You pay for each protocol enabled on a server endpoint for every hour the endpoint exists, whether anyone connects or not, plus a charge per GB uploaded or downloaded over SFTP, FTPS or FTP. AS2 is billed per message. SFTP connectors are billed per call and per GB moved. Web apps are billed per web app unit per hour, where one unit covers a set number of sessions. PGP decryption in a workflow is billed per GB. S3 storage and requests, Lambda, CloudWatch and KMS are billed separately at their own rates. In US East the examples show about 0.30 US dollars an hour per protocol and 0.04 US dollars per GB; list price, check your region.

What drives the bill

  • Number of servers multiplied by protocols enabled, multiplied by hours in the month.
  • Volume of data uploaded and downloaded by partners.
  • SFTP connector volume, which is priced much higher per GB than inbound uploads.
  • Web app units left running for occasional uploaders.

Cost traps

  • One server per supplier multiplies the hourly charge; one server with many users is usually enough.
  • Leaving FTPS or FTP enabled alongside SFTP doubles or triples the endpoint charge.
  • Pulling large files through SFTP connectors instead of having the partner push them.
  • Test and development servers left running all month.

Prices change; check the AWS pricing page for your region before deciding.

Security and governance

Keeping it safe

  • Give each user an IAM role scoped to its own prefix only, and use logical home directories so a supplier cannot list or read another supplier's folder.
  • Prefer SSH keys over passwords; when passwords or Active Directory are needed, use a custom identity provider and keep credentials in Secrets Manager.
  • Encrypt the landing bucket with the catalog's KMS key, and pick a current security policy for the server so only modern ciphers and key exchanges are allowed.
  • Use a VPC-hosted endpoint with security groups that allow only known partner IP ranges, and keep the data path to S3 through a VPC gateway endpoint.
  • Turn on structured logging to CloudWatch for every login and file operation, and use CloudTrail for changes to servers and users.
  • Treat partner files as possibly holding personal data: Macie scans landing, and the lander tokenises before bronze.

Operating it

What to watch, and the limits

WatchWhy
BytesIn and BytesOutData moved per server; a sudden drop on a supplier's usual day means the feed did not arrive.
FilesIn and FilesOutCount of files transferred; compare with the expected files per feed in the source register.
OnUploadExecutionsStarted, Success and FailedWorkflow runs after uploads; any failure means a file did not reach its final place.
CloudWatch log events for authentication failuresRepeated failed logins point to an expired key or someone probing the endpoint.
Lander file log freshnessOn the platform side, the age of the newest file per feed; alert when a daily feed is more than a few hours late.
LimitValue and notes
Servers per account per Region50 by default, adjustable
Service-managed users per server10,000 by default, adjustable
Concurrent sessions per server10,000, not adjustable
VPC-hosted servers per account per Region10, not adjustable
Active Directory groups per server100 by default

Limits and quotas checked 2026-11-11; many can be raised through Service Quotas.

Decisions

Situations and verdicts

SituationChoiceVerdict
Twelve suppliers send daily price files over SFTPOne VPC-hosted SFTP server, one user per supplier, landing bucketPursue: no server to patch, files land where the lander already looks.
A partner only lets you pull from their SFTP serverTransfer Family SFTP connector on a schedule from EventBridgePursue with care: it works, but per-GB connector pricing is high, so check volume first.
A large retailer insists on AS2 for EDI ordersAS2 server with partner agreementsPursue with care: AS2 works well, but certificate renewals need an owner and a calendar.
You already run a stable SFTP server on premises that writes to a shareDataSync from the share to S3Keep if you have it: move only when the box needs replacing or patching becomes a burden.
Migrating ten years of archived supplier filesDataSync or Snowball, not Transfer FamilyDrop Transfer Family for this: it is priced and built for ongoing partner traffic, not bulk moves.

FAQ

Questions people ask

Does AWS Transfer Family store files itself?

No. It writes every file into your own Amazon S3 bucket or Amazon EFS file system. You own the storage, the encryption key and the lifecycle rules.

Can a supplier see other suppliers' files?

Not if you set it up properly. Give each user a logical home directory that maps to its own prefix, and an IAM role that only allows that prefix.

How do I give partners a fixed IP address?

Use a server endpoint hosted in your VPC and attach Elastic IP addresses to make it internet-facing. Partners then allowlist those addresses.

Is AWS Transfer Family expensive for a few files a day?

The main cost is the hourly charge per protocol, which applies even when no one connects. For one or two feeds that can be more than the data itself, so share one server across all partners.

How does Databricks read files from Transfer Family?

Databricks never talks to Transfer Family. It reads the S3 landing prefix with Auto Loader, which tracks which files it has already seen.

Sources

Checked against

Checked 2026-11-11.

Related

Where it connects

Common Ingestion Framework · Data platform on AWS and Databricks · Amazon S3 · AWS DataSync · Amazon Macie · Medallion architecture · All AWS data services

← IoT SiteWise · Zero-ETL →