List Files from AWS S3 Bucket and Export to Excel Using Python

List Files from AWS S3 Bucket and Export to Excel Using Python

This Python script scans an AWS S3 folder, finds all PDF files, and creates an Excel report containing the folder name, PDF filename, and full S3 path.

Security: Never publish real AWS access keys or secret keys. Use secure AWS credentials in real environments.

What This Script Does

1. Connects to an AWS S3 bucket.
2. Scans a selected folder/prefix.
3. Finds PDF files.
4. Collects the folder name and file name.
5. Creates an Excel file with the results.

Example

If your S3 structure looks like:

my-bucket/ └── documents/    â”œâ”€â”€ 2025/    â”‚   ├── file1.pdf    â”‚   └── file2.pdf    â””── 2026/        â””── file3.pdf

The Excel report will contain:

Folder Name | PDF File Name | Full S3 Path 2025        | file1.pdf     | s3://my-bucket/documents/2025/file1.pdf 2025        | file2.pdf     | s3://my-bucket/documents/2025/file2.pdf 2026        | file3.pdf     | s3://my-bucket/documents/2026/file3.pdf

1. Install Required Packages

pip install boto3 pandas openpyxl

2. Configure the Script

aws_access_key_id = 'YOUR_ACCESS_KEY' aws_secret_access_key = 'YOUR_SECRET_KEY' bucket_name = 'YOUR_BUCKET_NAME' base_prefix = 'documents/' output_file = 's3_filename.xlsx'

base_prefix is the S3 folder you want to scan. For example:

documents/

3. Complete Python Script

Save the following as s3_file_list.py:

import boto3
import logging
import os
import pandas as pd

# Configuration
aws_access_key_id = 'YOUR_ACCESS_KEY'
aws_secret_access_key = 'YOUR_SECRET_KEY'
bucket_name = 'YOUR_BUCKET_NAME'
base_prefix = 'documents/'
output_file = 's3_filename.xlsx'

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s'
)

s3 = boto3.Session(
    aws_access_key_id=aws_access_key_id,
    aws_secret_access_key=aws_secret_access_key
).client('s3')


def generate_excel_report(bucket, prefix):
    data = []
    paginator = s3.get_paginator('list_objects_v2')

    logging.info(f'Scanning s3://{bucket}/{prefix}')

    for result in paginator.paginate(Bucket=bucket, Prefix=prefix):
        for obj in result.get('Contents', []):
            key = obj['Key']

            if not key.lower().endswith('.pdf'):
                continue

            folder = os.path.dirname(key).replace(prefix, '').strip('/')
            filename = os.path.basename(key)
            s3_path = f's3://{bucket}/{key}'
            data.append([folder, filename, s3_path])

    if not data:
        logging.info('No PDF files found.')
        return

    df = pd.DataFrame(
        data,
        columns=['Folder Name', 'PDF File Name', 'Full S3 Path']
    )

    df.to_excel(output_file, index=False, engine='openpyxl')
    print(f'Excel report saved: {output_file}')


if __name__ == '__main__':
    try:
        generate_excel_report(bucket_name, base_prefix)
    except Exception as e:
        logging.error(f'Failed to generate report: {str(e)}')

4. How It Works

Connect to S3

s3 = session.client('s3')

Creates the S3 client used to read objects from the bucket.

Scan the S3 Folder

paginator = s3.get_paginator('list_objects_v2')

The paginator allows the script to process large numbers of S3 objects.

Find PDF Files

if not key.lower().endswith('.pdf'):    continue

Only PDF files are added to the report.

Create the Excel Report

df.to_excel(    output_file,    index=False,    engine='openpyxl' )

Pandas writes the collected data to an Excel file.

5. Run the Script

Run:

python s3_file_list.py

If successful, you will see:

Excel report saved: s3_filename.xlsx

6. Excel Report

The generated Excel file contains three columns:

• Folder Name - S3 folder containing the PDF.
• PDF File Name - Name of the PDF file.
• Full S3 Path - Complete S3 object path.

Quick Reference

# Install packages pip install boto3 pandas openpyxl # Run script python s3_file_list.py

Conclusion

This script is useful when an S3 bucket contains many PDF files and you need a quick Excel list of the available files and their S3 paths.