S3 Bucket-Wise Size Report with Excel Export Using Python

S3 Bucket-Wise Size Report with Excel Export Using Python

This Python script checks all S3 buckets in an AWS account, calculates the number of files and total storage used in each bucket, and exports the results to an Excel file.

What This Script Does

• Connects to AWS S3.
• Gets all buckets available to the account.
• Counts objects in each bucket.
• Calculates total size in bytes, GB, and TB.
• Creates an Excel report.

1. Install Required Packages

pip install boto3 pandas openpyxl

2. Configure AWS

Update these values in the script:

aws_access_key_id = "YOUR_S3_ACCESS_KEY"
aws_secret_access_key = "YOUR_SECRET_KEY"
region_name = "ap-south-1"

The script uses list_buckets(), so the AWS identity must have permission to list the account's S3 buckets and read object listings for the buckets you want to scan.

3. Complete Python Script

Save the following as s3_bucket_wise_size_excel.py:

import boto3
import pandas as pd

# AWS credentials
aws_access_key_id = "YOUR_S3_ACCESS_KEY"
aws_secret_access_key = "YOUR_SECRET_KEY"
region_name = "ap-south-1"

# Excel output
output_file = "s3bucket_size.xlsx"

# Connect to S3
s3 = boto3.client(
    "s3",
    aws_access_key_id=aws_access_key_id,
    aws_secret_access_key=aws_secret_access_key,
    region_name=region_name
)

# Get all S3 buckets
response = s3.list_buckets()
buckets = response.get("Buckets", [])

print(f"Found {len(buckets)} S3 buckets.\n")

report = []

# Process each bucket
for bucket in buckets:

    bucket_name = bucket["Name"]

    print(f"Scanning: {bucket_name}")

    file_count = 0
    total_size = 0

    try:
        paginator = s3.get_paginator("list_objects_v2")

        for page in paginator.paginate(Bucket=bucket_name):

            for obj in page.get("Contents", []):

                key = obj["Key"]

                # Ignore folder placeholders
                if key.endswith("/"):
                    continue

                file_count += 1
                total_size += obj["Size"]

        total_size_gb = total_size / (1024 ** 3)
        total_size_tb = total_size / (1024 ** 4)

        report.append({
            "Bucket": bucket_name,
            "File Count": file_count,
            "Total Size (Bytes)": total_size,
            "Total Size (GB)": round(total_size_gb, 2),
            "Total Size (TB)": round(total_size_tb, 4)
        })

        print(
            f"  Files: {file_count:,} | "
            f"Size: {total_size_gb:.2f} GB"
        )

    except Exception as e:

        print(f"  ERROR: {e}")

        report.append({
            "Bucket": bucket_name,
            "File Count": "ERROR",
            "Total Size (Bytes)": "ERROR",
            "Total Size (GB)": "ERROR",
            "Total Size (TB)": "ERROR"
        })

# Create Excel report
df = pd.DataFrame(report)
df = df.sort_values(by="Bucket")

df.to_excel(
    output_file,
    index=False
)

print("\n===================================")
print("S3 BUCKET REPORT COMPLETED")
print("===================================")
print(f"Report saved to: {output_file}")

4. How It Works

Get All Buckets

response = s3.list_buckets()
buckets = response.get("Buckets", [])

This gets the buckets visible to the AWS credentials used by the script.

Scan Bucket Objects

paginator = s3.get_paginator("list_objects_v2")

for page in paginator.paginate(Bucket=bucket_name):

The paginator processes object listings page by page, which is important for buckets containing many objects.

Calculate Size

total_size_gb = total_size / (1024 ** 3)
total_size_tb = total_size / (1024 ** 4)

The total object size is converted from bytes to GB and TB.

Export to Excel

df.to_excel(
    output_file,
    index=False
)

The final report is saved as an Excel file.

5. Run the Script

python s3_bucket_wise_size_excel.py

The Excel report will be created as:

s3bucket_size.xlsx

6. Excel Report

The report contains:

Bucket
File Count
Total Size (Bytes)
Total Size (GB)
Total Size (TB)

Example:

Bucket              File Count   Total Size (GB)   Total Size (TB)
---------------------------------------------------------------
production-data     125,430      842.56            0.8228
backup-data         98,210       512.32            0.5003

7. Useful Notes

• The script calculates size by reading the Size value from S3 object listings.
• Objects whose keys end with / are ignored as folder placeholders.
• Large buckets can take time because every object must be listed.
• If a bucket cannot be scanned, the report records ERROR for that bucket.

Quick Reference

# Install packages
pip install boto3 pandas openpyxl

# Run report
python s3_bucket_wise_size_excel.py

Conclusion

This script provides a simple way to create an account-level S3 inventory showing bucket-wise file counts and storage usage in an Excel report.