Amazon S3

Table of Contents

S3 Canned ACL discussion

S3 Access Control: IAM Policy vs ACL discussion

Overall: Can use both. But IAM is preferable, since it is more modern one.

S3 ACLs is a legacy access control mechanism that predates IAM.

If you want to manage permissions on individual objects within a bucket, S3 ACLs enable you to apply policies on the objects themselves, whereas bucket policies can only be applied at the bucket level.

Server Side Encryption discussion

There is no extra charge for encryption, and there’s no observable effect on PUT or GET performance.

It seems pointless as follows:

Wait a minute? What are we protecting against? Someone gaining access to Amazon's data centers and stealing hard disks? Or worse, a disgruntled Amazon employee (do such people exist?) plugging in a USB stick and sucking out your precious data.

aws s3 reference awscli

aws s3 mv test.txt s3://mybucket/test2.txt
aws s3 mv s3://mybucket/test.txt s3://mybucket/test2.txt
aws s3 mv s3://mybucket/test.txt test2.txt
aws s3 mv s3://mybucket . --recursive
aws s3 mv myDir s3://mybucket/ --recursive --exclude "*.jpg"
aws s3 mv s3://mybucket/ s3://mybucket2/ --recursive --exclude "mybucket/another/*"

# delete a bucket and all its contents
aws s3 rb s3://bucket-name --force

aws s3 cp MyFolder s3://bucket-name --recursive

# upload a stream from stdin to s3
aws s3 cp - s3://my-bucket/stream

# download an s3 object as a stdout stream
aws s3 cp s3://my-bucket/stream

# list the sizes of an S3 bucket and its contents
aws s3api list-objects --bucket BUCKETNAME --output json --query "[sum(Contents[].Size), length(Contents[])]"

# inter-region sync
aws s3 sync s3://oldbucket s3://newbucket --source-region us-west-1 --region us-west-2

aws s3 sync reference awscli

For syncing strictly, specify following flags:

--exact-timestamps (boolean)
When syncing from S3 to local, same-sized items will be ignored only when the timestamps match exactly. The default behavior is to ignore same-sized items unless the local version is newer than the S3 version.
--delete (boolean)
Files that exist in the destination but not in the source are deleted during sync.