Deleting and emptying Amazon S3 Buckets using the AWS CLI

Short answer

1. Run the following command to get a list of commands to delete each bucket that you want to delete. And then copy, paste and run the output for the buckets that you want to delete.

aws s3 ls | grep <my-regex> | awk '{print "aws s3 rb s3://" $3 " --force"}'

For example:

aws s3 ls | grep infrapipeline | awk '{print "aws s3 rb s3://" $3 " --force"}'

Might print out something like:

aws s3 rb s3://infrapipelinestack-1111111 --force
aws s3 rb s3://infrapipelinestack-2222222 --force
# etc

And then you can run the commands to actually empty and delete the buckets.

Note: you may still see the buckets for a short time in the console or S3 API after they're deleted since S3 is eventually consistent.

In my use case, I deleted buckets  because I had run into the max buckets limit when setting up a CI/CD pipeline and had to clean them out in order to create a new stack. I was still at the max limit for several hours, even after deleting about 100 buckets using this script.

Explanation

1. Get list of all buckets that you want to delete

aws s3 ls | grep <my-regex>

For example:

aws s3 ls | grep infrapipeline

2. Pipe that into the an AWK command to format the output into the remove bucket command for the AWS CLI. Note that we're adding the force flag to empty the contents as well.

awk '{print "aws s3 rb s3://" $3 " --force"}'

Bonus

Save this as a command for usage later so you don't have to look this up again.

# In ~/.zshrc or ~/.bashrc file

function getEmptyAndDeleteS3BucketsCommands() {
  REGEX="$1"
  FILENAME="$2"
  
  # Create file with commands to empty and delete s3 buckets that match the regex 
  aws s3 ls | grep "$REGEX" | awk '{print "aws s3 rb s3://" $3 " --force"}' > "$FILENAME"
  
  # Add permissions to run script
  chmod +x "$FILENAME"
}

And then you can call it like:

getEmptyAndDeleteS3BucketsCommands <my-regex> <my-filename>

# Example
getEmptyAndDeleteS3BucketsCommands infrapipeline ./bucket-delete-commands.sh

# Then you can view the file to see the buckets that matched the regex and run the file to delete them

./bucket-delete-commands.sh

Happy coding! SL