Hasha's blog
Step-by-Step Guide to Building an Image Uploader Using AWS S3 and Django

Step-by-Step Guide to Building an Image Uploader Using AWS S3 and Django

Upload and Fetch Images on AWS S3 with Django using Pre-Signed URLs

Harsha Vardhan's photo
Harsha Vardhan
·

8 min read

Table of contents

Here we are going to build a Web Page, where we can Upload images and fetch back from S3 bucket, where S3 Bucket is a service offered by Amazon Web Services for storing Objects which can include all file types. Here the project being a Image Upload we will be using Images. For fetching back from S3 Bucket we will be using Pre-Signed URLs with time expiry.

Requirements

  • AWS Account

  • Python

  • pip

AWS Account

  • Login to your AWS Console - https://aws.amazon.com/ - as an IAM User or Root User

  • After login, you can change to Dark Theme using the settings icon beside Region

Creating Access Keys and Secret Access Keys

  • Click on the Security credentials in the Top right drop down menu - link - https://us-east-1.console.aws.amazon.com/iam/home#/security_credentials

  • If not you can use your root account to create Access Keys, but Best Practice is creating a IAM user and using their Access Keys

  • Click on the Create Access Key

  • Below is creating the Access Keys for Root user

  • Store the created Access Keys and Secret access key for Root User or you can Download then as .csv

  • Best Practice - Once the Project is completed, you should delete the created Access Keys and Secret access key for Root User

Creating a New IAM User

  • You can create IAM user and can give permissions to the IAM user by attaching

    S3 bucket Policy

  • Now, we are creating New User - type IAM in the Search bar and click the link

  • You will lead to this below page

  • Here Left side, click on Users Tab and click on the Create User in the Users page

  • It will lead to below image - add User name and select AWS Console Access

  • Click Next - select Attach Policies - here I had given S3 Full Access Policy - If needed you can fine grain them

  • Click Create User - it will shows you the Console link to login, User Name and Password - store them - you can download the them as .csv file

    • Go back to the Users tab - you can see a list of Users created - click on the User - created in the last step

    • You can see the Security Credentials tab - Click the Create Access Key

  • Select the Command Line Interface(CLI)

  • Add the Description click Create Access Key

  • Store the created Access Keys and Secret access key for IAM User or you can Download then as .csv

  • Best Practice - Once the Project is completed, you should delete the created Access Keys and Secret access key for IAM User - if you are not using the IAM user once project is done - you can delete the IAM user

S3 Bucket Creation

  • If you logging as IAM User, make sure you had S3 Bucket creation, list, delete access

  • Search for "S3" in the search bar - click the link

  • You can see the below Image

  • On the Top Right Corner - you can see Mumbai which is my region - you can change your region based on your location or you can host your S3 bucket over in the region (nearer the region faster access to the objects in S3 bucket)

  • Click on Create Bucket

  • In the General configuration - check the AWS Region Name - you can change the region if needed

  • Give an Unique Bucket name - it should be unique - I had given django-image-upload-2024

  • Then don't change any settings - scroll down and click Create Bucket

  • After creating the S3 Bucket, upload a file to check if the file can accessible from Internet or Outside - while creating the S3 Bucket we had blocked All Public access - click on the Permissions tab to check

  • Click on the Upload - Add Files - Select the Image you want to upload and click Upload

  • You can see Image is upload - click on the image - you can check all the Details

  • Here you need to consider Object URL and Open at the top - if you click Open the Image open in New tab with Pre signed URL and when Open with the Object URL you can't access

  • Below is with Object URL - if you are not Blocking Public Access the Bucket will available to Outside world or Internet - they can fetch your data. To overcome this issue we are using Pre-Signed URLs

  • Below is with Pre Signed URL - here you can see text after .jpg while compared to Object URL - Pre Signed URL is like token with Object URL - it is accessible for 12 Hours

  • In the Object Actions - click the Share with a Pre-Signed URL - here you can mention the number of interval Minutes - then after the Link expires you can't access them

  • So, using Pre-Signed URLs you can Access the objects in S3 Buckets with Block Public Access

  • So, now we move to our Project we implement Images Upload to S3 Bucket, using Pre-signed with certain time interval access using Django

Git Repository

Git clone this repo - https://github.com/harshakp06/django-image-upload.git

Make sure git installed on your system.

git clone https://github.com/harshakp06/django-image-upload.git

Open folder in Code Editor

Create Virtual Environment

For Unix/Linux

python3 -m venv .venv

source .venv/bin/activate

For Windows

python -m venv .venv

.venv\Scripts\activate

Deactivate a virtual environment

Don't use the command now - once the project is finished use this

deactivate

Install Dependencies

Dependencies list requirements.txt is present in src/ folder

cd src/

pip install -r requirements.txt

Once all the dependencies are installed

Run Django Server

python manage.py makemigrations

python manage.py migrate

python manage.py runserver

Default port of Django is :8000 - open http://127.0.0.1:8000/ on any browser - if you can see below image as Home Page - all the configuration went well and working

If you got any error/exception about app_image table - it means migration haven't happen for app folder where our models are present

Run below command to migrate app models

python manage.py makemigrations app

python manage.py migrate

python manage.py runserver

Now you can see the above Home Page Image

By now, our Django project is running. But you can't upload files using the upload.

We need to configure S3 bucket with Django

Configuring S3 Bucket with Django App

Adding required values to.envfile

  • Check .env.example file present in src/ folder

  • Rename .env.example file name to .env file name

  • Edit the values with which we got from AWS Account and the Save the file as .env

  •     # AWS settings
    AWS_ACCESS_KEY_ID = "AWS_ACCESS_KEY_ID"
    AWS_SECRET_ACCESS_KEY = "AWS_SECRET_ACCESS_KEY"
    AWS_STORAGE_BUCKET_NAME = "AWS_STORAGE_BUCKET_NAME"
    AWS_S3_REGION_NAME = "AWS_S3_REGION_NAME"
    AWS_S3_CUSTOM_DOMAIN = "AWS_STORAGE_BUCKET_NAME_REPLACE.s3.amazonaws.com"
    #replace storage bucket name above

  • If your Django App is running and stop the App by using Ctrl+c or you can Kill terminal

  • Start the Django App again

  •     python manage.py runserver

  • Now go the http://127.0.0.1:8000/ you can see the Home Page

  • Click the Browse Button, select the image to Upload and click Upload

  • Check the S3 Bucket in the IAM User or Root AWS Account - you can see a Folder name images/ - where all your uploaded Images are stored

  • Click on the folder media/ folder - you can see the upload image - you can check the Details of the object

  • After Uploading Image name changes - it is in the code you can check at src/app/models.py - unique_file_path

  • You can opening the Object/Image with Object URL - it will be access denied as mentioned

  • But with Open with Pre signed URL it is accessible

  • Open the same image in New tab on the Django App Home Page - you can observe the Object URL extends with AWS Access Key Id provided and added token with expiry time

  • Here we made it work with Access Key Id which one of the way to access S3 Objects from Outside

  • The Pre Signed URL will expire in 10mins - to access again reload the Home Page and then open the Image in New Tab

  • You can change the expiry time - check code at src/app/utils.py - def generate_presigned_url(object_name, expiration=600): change the expiration value to increase or decrease in expiry time, value is in secs

  • You can upload any number of Images to S3 Bucket

Using Code Spaces from GitHub

  • You can use the Code Spaces on GitHub and it's free for this project

  • Fork my repo on GitHub - https://github.com/harshakp06/django-image-upload

  • Then click on the Code after the Fork - click on Create Code Spaces

  • It will open in the VS Code editor in a container - the changes made in the Code Space won't reflect on the GitHub Repo - until you commit and Push to the Repo

  • You can steps steps from above Configuring S3 Bucket

  • You can delete the Code Space after completion of the project

Here the project came to an end, now you can deactivate your environment as mentioned above and you can remove, delete Access Key Id or IAM User.

Happy coding !

DjangoAWSAWS s3S3S3-bucketPythonGitHub