www.awsgood.com
aws s3 cp ./file.txt s3://my-bucket/ is the whole command; 90% of failures are wrong region, missing s3:PutObject, or a KMS key policy that blocks the upload even though IAM says "Allow".aws s3 sync for repeatable deploys and aws s3 cp --recursive for one-shot directory copies — sync compares size plus last-modified and skips unchanged files, which is the difference between a 4-second and a 4-minute deploy.aws s3api head-object: check ContentLength, ContentType and the ETag suffix, because a file can upload "successfully" with binary/octet-stream and break every browser that loads your CSS.You need three things aligned before a single byte moves: the bucket's region, credentials with s3:PutObject, and (if the bucket uses SSE-KMS) permission on the key itself.
Start by pinning the region explicitly. Bucket names are globally unique but a bucket lives in exactly one region, and the CLI will happily use your default us-east-1 profile against a bucket in eu-west-1:
aws sts get-caller-identity # who am I, which account
aws s3api get-bucket-location --bucket my-bucket --output text
# us-east-1 answers "None" — not a bug, that's how AWS reports it
If you don't have the CLI yet, `aws --version` should print `aws-cli/2.x`. v1 still exists on older AMIs and behaves differently for multipart and error output, so upgrade before debugging anything weird.
The minimum IAM permissions for a working upload are small, and I'd rather grant these than `s3:*`:
```json
{
"Effect": "Allow",
"Action": ["s3:PutObject", "s3:AbortMultipartUpload"],
"Resource": "arn:aws:s3:::my-bucket/*"
}
`AbortMultipartUpload` isn't optional in practice. Without it, a failed transfer leaves incomplete parts behind and you pay for the storage until a lifecycle rule cleans them up — I've seen 40 GB of orphaned parts from one interrupted video upload.
## Upload one file: aws s3 cp and the flags worth typing
The happy path first:
```bash
aws s3 cp ./index.html s3://my-bucket/index.html
--content-type "text/html; charset=utf-8"
--cache-control "public, max-age=300"
That command prints exactly one line on success:
upload: ./index.html to s3://my-bucket/index.html
Two flags in there matter more than they look. `--content-type` overrides the CLI's extension guess — fine for `.html`, less fine for `.mjs`, `.wasm` or anything served to a browser. `--cache-control` decides whether CloudFront revalidates every 5 minutes or once a year; hashed filenames (`app.9f2c1a.js`) deserve `max-age=31536000, immutable`, and `index.html` should never get that treatment.
For anything over 5 GB, a single PUT isn't allowed at all — S3 caps one request at 5 GB, and the CLI switches to multipart automatically above 8 MB. You can tune that:
```bash
aws configure set default.s3.multipart_threshold 64MB
aws configure set default.s3.multipart_chunksize 16MB
aws configure set default.s3.max_concurrent_requests 20
Bumping concurrency from 10 to 20 roughly halves wall-clock time for large files on a fat pipe; on a 50 Mbps office line it changes nothing except memory. Test before you bake it into a CI image.
When the file is generated inside your app, don't shell out — use the SDK's managed transfer:
```python
import boto3
s3 = boto3.client("s3", region_name="us-east-1")
s3.upload_file(
Filename="export.csv",
Bucket="my-bucket",
Key="exports/2026-02/export.csv",
ExtraArgs={"ContentType": "text/csv", "ServerSideEncryption": "AES256"},
)
`upload_file` handles multipart, retries and thread pooling for you. `put_object` does not, and it silently fails on streams it can't measure.
## Bulk uploads: cp --recursive vs s3 sync
The two commands look interchangeable and aren't.
`aws s3 cp ./dist s3://my-bucket/ --recursive` copies every file, every time. On a 900-file static build that's 900 PUTs per deploy, and request charges are per 1,000 requests — the PUT line item grows while the storage line item doesn't move.
`aws s3 sync ./dist s3://my-bucket/` lists the destination first and only uploads what differs by size or last-modified time. Same 900-file build, second deploy: 3 files uploaded, six seconds. First deploy after a full rebuild: everything goes, because every mtime changed.
Filter order is the trap here. `--exclude` and `--include` are evaluated left to right, so the include must come *after* the exclude it's meant to rescue:
```bash
aws s3 sync ./dist s3://my-bucket/
--exclude "*" --include "*.js" --include "*.css" --include "*.html"
--cache-control "public, max-age=31536000, immutable"
--dryrun
Always run `--dryrun` first. It prints the exact `upload:` lines without touching S3, which is the cheapest way to catch a path typo like `./dist/` vs `./build/`.
## Key parameters at a glance
| Flag | What it controls | Suggested value |
|---|---|---|
| `--region` | Endpoint the CLI signs against | Always set explicitly in scripts |
| `--content-type` | `Content-Type` response header | Set manually for `.mjs`, `.wasm`, `.json`, `.svg` |
| `--cache-control` | Browser/CDN caching | `max-age=31536000, immutable` for hashed assets; `max-age=0` for `index.html` |
| `--storage-class` | Storage tier | `STANDARD` for live assets; `INTELLIGENT_TIERING` for objects over 128 KB with unpredictable access |
| `--sse` | Server-side encryption | `aws:kms` when compliance requires a customer-managed key, else bucket default |
| `--acl` | Legacy per-object ACL | Omit — Object Ownership defaults to bucket-owner-enforced and rejects ACLs |
| `--dryrun` | Preview only | Always first run in CI or a new directory |
| `--delete` | Remove destination objects missing locally | Only with `sync`, and only after a `--dryrun` |
## How to verify an upload landed
`aws s3 ls` proves a key exists; it does not prove the bytes are right.
```bash
aws s3api head-object --bucket my-bucket --key dist/app.js
--query '{Size:ContentLength,Type:ContentType,ETag:ETag,Meta:Metadata}'
Expected output for a small file:
```json
{
"Size": 48213,
"Type": "application/javascript",
"ETag": ""5d41402abc4b2a76b9719d911017c592"",
"Meta": {}
}
An ETag with a `-3` suffix means the object went up as three multipart parts, so the ETag is no longer an MD5 and comparing it against `md5sum` locally will never match. Compare sizes instead, or use `aws s3api get-object-attributes --object-attributes ObjectSize,Checksum`. S3 has been strongly consistent on read-after-write since December 2020, so if `head-object` shows the new size the object is genuinely there — no need to retry with sleep loops like we did in 2019.
For whole-bucket sanity checks, `aws s3 ls s3://my-bucket/dist/ --recursive --summarize` prints a total object count and byte count, which you can diff against `find ./dist -type f | wc -l`.
## When NOT to use S3 upload (or this method)
Don't sync a directory of 200,000 tiny files with the CLI. Per-object overhead dominates at that scale; each PUT costs a round trip, and `sync` must first LIST the destination, which returns 1,000 keys per page. Batch with the SDK's `upload_fileobj` in a thread pool, or concatenate into fewer, larger objects.
Don't use S3 as a mounted filesystem for a database or a Git repo. Objects are immutable and get replaced whole; there's no in-place write or file lock.
Don't expose a bucket publicly for a website. Serve assets through CloudFront with an Origin Access Control, keep Block Public Access on, and hand out presigned URLs for private downloads — remembering that SigV4 presigned URLs max out at 7 days.
## Pitfalls, real error strings, and the KMS AccessDenied I chased at 02:10
The bad one first. Bucket had default encryption set to a customer-managed KMS key, IAM role had `s3:PutObject` on the bucket, and every upload failed with:
An error occurred (AccessDenied) when calling the PutObject operation: Access Denied
Nothing in the message mentions KMS. My first assumption was a bucket policy Deny, so I pulled the policy — clean. Then I re-ran with `--debug` and read the actual request: `x-amz-server-side-encryption: aws:kms`. That's the tell. `s3:PutObject` alone is enough for SSE-S3, but SSE-KMS requires `kms:GenerateDataKey` on the key, and the role didn't have it. Confirming took one CloudTrail lookup on the `GenerateDataKey` event, which showed `AccessDenied` in the `errorCode` field. Fix was one statement on the key policy:
```json
{
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::123456789012:role/deploy"},
"Action": ["kms:GenerateDataKey", "kms:Decrypt"],
"Resource": "*"
}
`kms:Decrypt` matters too, because `sync` and `cp --recursive` read object metadata from the destination and a key without Decrypt breaks the comparison step.
Second most common, and it looks scarier than it is:
An error occurred (PermanentRedirect) when calling the PutObject operation: The bucket you are trying to access must be addressed using the specified endpoint.
The bucket is in `eu-west-1` and your profile defaults to `us-east-1`. Add `--region eu-west-1` or set `AWS_REGION`. Nothing is broken; you just signed against the wrong endpoint.
Third: `aws s3 sync` printing `upload: ./dist/app.js to ...` on every run even though nothing changed. That's mtime drift — a CI step that rewrites files with fresh timestamps, or a `cp -p` that dropped them. `--size-only` paper-overs it, but the real fix is making the build deterministic or letting `sync` use `--exact-timestamps`.
If you'd rather not maintain the IAM/KMS matrix yourself, the ops crew at awsgood.com sets these bucket baselines up as part of their managed AWS work.
## Sources
- [AWS CLI `s3 cp` reference](https://docs.aws.amazon.com/cli/latest/reference/s3/cp.html)
- [Uploading objects to Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html)
- [Checking object integrity in S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html)
- [Sharing objects with presigned URLs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html)