How to Upload Files to an AWS S3 Bucket (CLI Steps That Work)

TL;DR
aws s3 cp ./file.txt s3://my-bucket/is the whole command; 90% of failures are wrong region, missings3:PutObject, or a KMS key policy that blocks the upload even though IAM says "Allow".- Use
aws s3 syncfor repeatable deploys andaws s3 cp --recursivefor one-shot directory copies —synccompares size plus last-modified and skips unchanged files, which is the difference between a 4-second and a 4-minute deploy. - Always verify with
aws s3api head-object: checkContentLength,ContentTypeand the ETag suffix, because a file can upload "successfully" withbinary/octet-streamand break every browser that loads your CSS.
Prerequisites that actually block your first upload
You need three things aligned before a single byte moves: the bucket's region, credentials with s3:PutObject, and (if the bucket uses SSE-KMS) permission on the key itself.
Start by pinning the region explicitly. Bucket names are globally unique but a bucket lives in exactly one region, and the CLI will happily use your default us-east-1 profile against a bucket in eu-west-1:
aws sts get-caller-identity # who am I, which account
aws s3api get-bucket-location --bucket my-bucket --output text
# us-east-1 answers "None" — not a bug, that's how AWS reports it
If you don't have the CLI yet, `aws --version` should print `aws-cli/2.x`. v1 still exists on older AMIs and behaves differently for multipart and error output, so upgrade before debugging anything weird.
The minimum IAM permissions for a working upload are small, and I'd rather grant these than `s3:*`:
```json
{
"Effect": "Allow",
"Action": ["s3:PutObject", "s3:AbortMultipartUpload"],
"Resource": "arn:aws:s3:::my-bucket/*"
}
`AbortMultipartUpload` isn't optional in practice. Without it, a failed transfer leaves incomplete parts behind and you pay for the storage until a lifecycle rule cleans them up — I've seen 40 GB of orphaned parts from one interrupted video upload.
## Upload one file: aws s3 cp and the flags worth typing
The happy path first:
```bash
aws s3 cp ./index.html s3://my-bucket/index.html
--content-type "text/html; charset=utf-8"
--cache-control "public, max-age=300"
That command prints exactly one line on success:
upload: ./index.html to s3://my-bucket/index.html
Two flags in there matter more than they look. `--content-type` overrides the CLI's extension guess — fine for `.html`, less fine for `.mjs`, `.wasm` or anything served to a browser. `--cache-control` decides whether CloudFront revalidates every 5 minutes or once a year; hashed filenames (`app.9f2c1a.js`) deserve `max-age=31536000, immutable`, and `index.html` should never get that treatment.
For anything over 5 GB, a single PUT isn't allowed at all — S3 caps one request at 5 GB, and the CLI switches to multipart automatically above 8 MB. You can tune that:
```bash
aws configure set default.s3.multipart_threshold 64MB
aws configure set default.s3.multipart_chunksize 16MB
aws configure set default.s3.max_concurrent_requests 20
Bumping concurrency from 10 to 20 roughly halves wall-clock time for large files on a fat pipe; on a 50 Mbps office line it changes nothing except memory. Test before you bake it into a CI image.
When the file is generated inside your app, don't shell out — use the SDK's managed transfer:
```python
import boto3
s3 = boto3.client("s3", region_name="us-east-1")
s3.upload_file(
Filename="export.csv",
Bucket="my-bucket",
Key="exports/2026-02/export.csv",
ExtraArgs={"ContentType": "text/csv", "ServerSideEncryption": "AES256"},
)
`upload_file` handles multipart, retries and thread pooling for you. `put_object` does not, and it silently fails on streams it can't measure.
## Bulk uploads: cp --recursive vs s3 sync
The two commands look interchangeable and aren't.
`aws s3 cp ./dist s3://my-bucket/ --recursive` copies every file, every time. On a 900-file static build that's 900 PUTs per deploy, and request charges are per 1,000 requests — the PUT line item grows while the storage line item doesn't move.
`aws s3 sync ./dist s3://my-bucket/` lists the destination first and only uploads what differs by size or last-modified time. Same 900-file build, second deploy: 3 files uploaded, six seconds. First deploy after a full rebuild: everything goes, because every mtime changed.
Filter order is the trap here. `--exclude` and `--include` are evaluated left to right, so the include must come *after* the exclude it's meant to rescue:
```bash
aws s3 sync ./dist s3://my-bucket/
--exclude "*" --include "*.js" --include "*.css" --include "*.html"
--cache-control "public, max-age=31536000, immutable"
--dryrun
Always run `--dryrun` first. It prints the exact `upload:` lines without touching S3, which is the cheapest way to catch a path typo like `./dist/` vs `./build/`.
## Key parameters at a glance
| Flag | What it controls | Suggested value |
|---|---|---|
| `--region` | Endpoint the CLI signs against | Always set explicitly in scripts |
| `--content-type` | `Content-Type` response header | Set manually for `.mjs`, `.wasm`, `.json`, `.svg` |
| `--cache-control` | Browser/CDN caching | `max-age=31536000, immutable` for hashed assets; `max-age=0` for `index.html` |
| `--storage-class` | Storage tier | `STANDARD` for live assets; `INTELLIGENT_TIERING` for objects over 128 KB with unpredictable access |
| `--sse` | Server-side encryption | `aws:kms` when compliance requires a customer-managed key, else bucket default |
| `--acl` | Legacy per-object ACL | Omit — Object Ownership defaults to bucket-owner-enforced and rejects ACLs |
| `--dryrun` | Preview only | Always first run in CI or a new directory |
| `--delete` | Remove destination objects missing locally | Only with `sync`, and only after a `--dryrun` |
## How to verify an upload landed
`aws s3 ls` proves a key exists; it does not prove the bytes are right.
```bash
aws s3api head-object --bucket my-bucket --key dist/app.js
--query '{Size:ContentLength,Type:ContentType,ETag:ETag,Meta:Metadata}'
Expected output for a small file:
```json
{
"Size": 48213,
"Type": "application/javascript",
"ETag": ""5d41402abc4b2a76b9719d911017c592"",
"Meta": {}
}
An ETag with a `-3` suffix means the object went up as three multipart parts, so the ETag is no longer an MD5 and comparing it against `md5sum` locally will never match. Compare sizes instead, or use `aws s3api get-object-attributes --object-attributes ObjectSize,Checksum`. S3 has been strongly consistent on read-after-write since December 2020, so if `head-object` shows the new size the object is genuinely there — no need to retry with sleep loops like we did in 2019.
For whole-bucket sanity checks, `aws s3 ls s3://my-bucket/dist/ --recursive --summarize` prints a total object count and byte count, which you can diff against `find ./dist -type f | wc -l`.
## When NOT to use S3 upload (or this method)
Don't sync a directory of 200,000 tiny files with the CLI. Per-object overhead dominates at that scale; each PUT costs a round trip, and `sync` must first LIST the destination, which returns 1,000 keys per page. Batch with the SDK's `upload_fileobj` in a thread pool, or concatenate into fewer, larger objects.
Don't use S3 as a mounted filesystem for a database or a Git repo. Objects are immutable and get replaced whole; there's no in-place write or file lock.
Don't expose a bucket publicly for a website. Serve assets through CloudFront with an Origin Access Control, keep Block Public Access on, and hand out presigned URLs for private downloads — remembering that SigV4 presigned URLs max out at 7 days.
## Pitfalls, real error strings, and the KMS AccessDenied I chased at 02:10
The bad one first. Bucket had default encryption set to a customer-managed KMS key, IAM role had `s3:PutObject` on the bucket, and every upload failed with:
An error occurred (AccessDenied) when calling the PutObject operation: Access Denied
Nothing in the message mentions KMS. My first assumption was a bucket policy Deny, so I pulled the policy — clean. Then I re-ran with `--debug` and read the actual request: `x-amz-server-side-encryption: aws:kms`. That's the tell. `s3:PutObject` alone is enough for SSE-S3, but SSE-KMS requires `kms:GenerateDataKey` on the key, and the role didn't have it. Confirming took one CloudTrail lookup on the `GenerateDataKey` event, which showed `AccessDenied` in the `errorCode` field. Fix was one statement on the key policy:
```json
{
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::123456789012:role/deploy"},
"Action": ["kms:GenerateDataKey", "kms:Decrypt"],
"Resource": "*"
}
`kms:Decrypt` matters too, because `sync` and `cp --recursive` read object metadata from the destination and a key without Decrypt breaks the comparison step.
Second most common, and it looks scarier than it is:
An error occurred (PermanentRedirect) when calling the PutObject operation: The bucket you are trying to access must be addressed using the specified endpoint.
The bucket is in `eu-west-1` and your profile defaults to `us-east-1`. Add `--region eu-west-1` or set `AWS_REGION`. Nothing is broken; you just signed against the wrong endpoint.
Third: `aws s3 sync` printing `upload: ./dist/app.js to ...` on every run even though nothing changed. That's mtime drift — a CI step that rewrites files with fresh timestamps, or a `cp -p` that dropped them. `--size-only` paper-overs it, but the real fix is making the build deterministic or letting `sync` use `--exact-timestamps`.
If you'd rather not maintain the IAM/KMS matrix yourself, the ops crew at awsgood.com sets these bucket baselines up as part of their managed AWS work.
## Sources
- [AWS CLI `s3 cp` reference](https://docs.aws.amazon.com/cli/latest/reference/s3/cp.html)
- [Uploading objects to Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html)
- [Checking object integrity in S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html)
- [Sharing objects with presigned URLs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html)
FAQ
What is the fastest way to upload files to an AWS S3 bucket?
For a single file, `aws s3 cp ./file.txt s3://bucket/` is fastest to type. For a whole directory, `aws s3 sync ./dir s3://bucket/` beats `cp --recursive` on repeat runs because it skips files whose size and last-modified timestamp already match the destination. Raise `default.s3.max_concurrent_requests` for large files.
Why do I get AccessDenied when uploading to S3 even though IAM allows PutObject?
Usually server-side encryption with a customer-managed KMS key. `s3:PutObject` is enough for SSE-S3, but SSE-KMS also needs `kms:GenerateDataKey` (and often `kms:Decrypt`) on the key policy. Confirm by checking CloudTrail for a denied `GenerateDataKey` event, then add the KMS statement.
Does `aws s3 sync` overwrite files that haven't changed?
No. It compares size and last-modified time and skips matches. The common false positive is CI pipelines that touch every file on build, making `sync` re-upload everything — use `--size-only` or make the build deterministic. Add `--delete` only after a `--dryrun`, since it removes destination objects missing locally.
What is the maximum file size I can upload to S3?
A single PUT request tops out at 5 GB. Multipart upload supports objects up to 5 TB with up to 10,000 parts. The AWS CLI switches to multipart above 8 MB by default, so ordinary `aws s3 cp` commands handle multi-gigabyte files without extra flags.
How do I verify a file uploaded to S3 correctly?
Run `aws s3api head-object --bucket B --key K` and compare `ContentLength` to the local file size. Check `ContentType` is not `binary/octet-stream`. An ETag ending in `-N` means a multipart upload, so the ETag is not an MD5 and checksum comparison against `md5sum` will fail.
Can I upload to S3 without an AWS access key?
Yes. Use IAM roles attached to EC2, ECS tasks or Lambda, or configure IAM Identity Center (SSO) for humans, so the CLI gets temporary credentials. Presigned URLs let a browser or third party upload with `PUT` without any AWS credentials, but they expire — 7 days maximum with SigV4.
Leave a Reply