How To Upload Files To Amazon S3 Bucket Using Python

Developers who build product applications use Amazon web services, for example, to increase storage. They also use Amazon web services to accommodate their apps and furthermore exploit more services.

Amazon S3 service is a web service that developers certainly use for their Python upload file. Although the service allows you to store files like pictures, music, videos, and documents, you can also use it to create upload files. It is therefore essential to learn how to upload files to Amazon S3 to develop a better Python upload file feature. Consequently, before we move into the article, we will be taking a look at the basics and what Amazon S3 entails at the same time.

What is Amazon S3?

Amazon S3, likewise known as Amazon Simple Storage Service, is a versatile, rapid, and online-based storage service. The service makes data archiving, online backup, and applications available on its web services as a result of its flexibility. Developers use Amazon S3 because it makes web scalability easier. Since we have now learned what Amazon S3 means, let us move on to why it is being used.

Why is Amazon S3 used?

Most business organizations use Amazon S3 for their enterprises because of its capacity to work for small and large businesses. Consequently, Amazon S3’s flexibility, accessibility, delivery, and security capacities stand out for many people in data storage uses. Some other uses of Amazon S3 for instance include:

• Archiving data in addition to other pieces of information.
• Storing data during work with Amazon S3.
• Amazon S3 also works with mobile and web applications, which therefore shows that it is flexible.
• Consequently, helps in disaster recovery.

Now that we have learned the basics of Amazon S3, let us learn how to upload files with Python.

Importance of Amazon S3

Developers use Amazon S3, otherwise known as Amazon Simple Storage Service, to create their apps:

1. Amazon S3 can help you store information in a diverse form of classes, therefore, helping you spend less.
2. Amazon S3 is 99% durable for use. In other words, it can last for a long time.
3. Developers can access Amazon S3 easily because of its simplicity.
4. You can easily transfer data through Amazon S3 because of its trustworthiness.
5. Finally, Amazon S3’s flexibility makes it good for several data storage in addition to various web development for the purpose of usage.

How to upload files to Amazon S3 bucket using Python?

In this section, we will specifically discuss how to upload files to the Amazon S3 bucket using Python. Since we use file stack for our Python file upload process, you can use the following steps:

• Install Filestack with pip for S3 SDK

Filestack now has its own official Python SDK which we will use for this process. We use Python 3.5+, since the SDK version is 3.0.0. For this reason and you can install the SDK version with these codes:

pip install "filestack-python<3.0.0"

After installing the SDK, you can go ahead and install filestack with pip with the following codes:

pip install filestack-python

Similarly, you can also install directly from Github with the following codes:

pip install git+https://github.com/filestack/filestack-python.git

If you don’t have a pip installed on your device, you can install it with the following codes:

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py

Then you can install Filestack from the source code if you don’t have the Filestack SDK.

python setup.py install

After that, we can now move to the next step for our file upload process.

• Upload files to S3

For this step, you can use different methods to upload your files and for this purpose, we will be considering these methods as follows.

1. Upload_file (Bucket, file, and key) method

Firstly, we are going to discuss the upload_file (Bucket, file, and key) method. Moreover, you can break large files into small files and upload them straightforwardly in this step. Here in this step, you can write the following codes:

import filestack-python

from flestack import Client

s3_client = filestack-python.client('s3', region_name='us-east-1', aws_access_key_id=ACCESS_KEY,                       

aws_secret_access_key=ACCESS_SECRET)

def upload_my_file(bucket, folder, file_to_upload, file_name):

     key = folder+"/"+file_name

    try:

        response =

s3_client.upload_file(file_to_upload, bucket, key)

    except Client as e:

        print(e)

        return False

    except FileNotFoundError as e:

        print(e)

        return False

    return True  

#Upload file

upload_my_file("bucket-name", "folder-name",

"/home/user/Documents/test.html", "test.html")

After you have input these codes, your upload file will indeed be successful.

2. Put_object (Bucket, file, and key) method

Secondly, we will be considering the Put_object (Bucket, file, and key) method. In short, the put_object method helps your file upload to be like a single object. However, this method differs from the first method with the following codes:

import filestack-python
from filestack import Client

s3_client = filestack-python.client('s3', region_name='us-east-1', aws_access_key_id=ACCESS_KEY,
                               aws_secret_access_key=ACCESS_SECRET)


def upload_my_file(bucket, folder, file_name, file_binary_data):
    if file_binary_data is None:
        file_binary_data = bytearray("No Data", "utf8")

    key = folder+"/"+file_name
    try:
        s3_client.put_object(Bucket = bucket, Key=key, Body=file_binary_data)
    except ClientError as e:
        print(e)
        return False
    return True

#Upload file
file_data = "This is a file data example"
file_binary_data = bytearray(file_data, "utf8")
upload_my_file("bucket-name", "folder-name", "test.txt", file_binary_data)

Likewise, writing these codes will specifically make your file upload to Python successful.

3. S3 client class method

Another method that you can use to upload files to the Amazon S3 bucket using Python is the client class. The following codes will help you run this command:

import filestack-python
from filestack import Client
import pathlib
import os

def upload_file_using_client():
    """
    Uploads file to S3 bucket using S3 client object
    :return: None
    """
    s3 = filestack-python.client("s3")
    bucket_name = "binary-guy-frompython-1"
    object_name = "sample1.txt"
    file_name = os.path.join(pathlib.Path(__file__).parent.resolve(), "sample_file.txt")

    response = s3.upload_file(file_name, bucket_name, object_name)
    pprint(response)  # prints None

Consequently, writing this code will truly yield a favorable upload file feature for you.

4. S3 resource class method

The S3 resource class method would be the last method we will consider under the upload S3.

You can also use the resource class method to upload files using Filestack with the following codes:

def upload_file_using_resource():
    """
    Uploads file to S3 bucket using S3 resource object.
    This is useful when you are dealing with multiple buckets st same time.
    :return: None
    """
    s3 = filestack-python.resource("s3")
    bucket_name = "binary-guy-frompython-2"
    object_name = "sample2.txt"
    file_name = os.path.join(pathlib.Path(__file__).parent.resolve(), "sample_file.txt")

    bucket = s3.Bucket(bucket_name)
    response = bucket.upload_file(file_name, object_name)
    print(response)  # Prints None

The result of the codes of any of the methods will, in the same way, make your Python file upload successful. Furthermore, you can also watch the video of these methods to aid your learning.

How to upload files to Amazon S3 bucket from Html-Form with Python CGI Script?

In addition to uploading with Python, using the Filestack Software Development Kit (SDK), developers can also similarly upload files with Python CGI script. But before this process, you will need materials, for example:

• Filestack Python package
• Python IDE
• Lambda role access and secrete key

After acquiring these materials, you can now start uploading by creating a folder with the name Filestack, for example. Therefore you can create an HTML file with the name “form_upload.html” through the following codes:

<html>
<body>

<form enctype = “multipart/form-data” action = http://127.0.0.1:8000/cgi-bin/s3_file_upload.py method = “post”>
         <p>File: <input type = “file” name = “filename” /></p>
        <p><input type = “submit” value = “upload” /></p>
</form>

</body>
</html>

After which, you can write the following codes to import your Python script.

Import cgi

Import cgitb Import filestack-python Import io Cgitb.enable() Print(“Content-type:text/html\r\n\r\n”)        Form = cgi.FieldStorage()        data=form[‘filename’].value       #Use IAM user’s access key & secret key who has the full S3 access       #Not recommended to hardcode credentials!       aws_con=filestack-python.session.Session(aws_access_key_id=”AKIAUOYHJ532LGVHIB”,aws_secret_access_key=”f0jmkbDuKDShT4Nnb6kv1MiiEk4OdsKPS@)      s3_con=aws_con.client(“s3”)      #upload_fileobj(Binary_file_obj, S3_Bucket_Name, file_name)      res=s3_con.upload_fileobj(io.BytesIO(data), “filestack-test-bucket-1”, str(form[‘filename’].filename))      print(“<p>File:”, form[“filename”].filename,”Uploaded Successfully!!!</p>”) except Exception as e:         print(“<h3>An error occurred. Please check the script! </h3>”)         print(“<p>”,e,”</p>”)

The second step is to go to your Filestack folder and consequently start the server with port 8000 with the following codes:

python -m http.server –cgi

The codes will show when you check the response data at the same time.

C:\Users\filestack\Device\filestack>python –m http.server –cgi
Serving HTPP on :: port 0000 (http//[::]:0000/) …
::ffff:127.0.0.1 - - [09/Jun/2022 17:19:36] “POST /cgi-bin/test.py HTTP/1.1” 200-
::ffff:127.0.0.1 - - [09/Jun/2022 17:19:36] command: C:\Users\filestack\AppData\Local\Programs\Python\Python300\python.exe –u C:\Users\filestack\Desktop\filestack\cgi-bin “”
::ffff:127.0.0.1 - - [09/Jun/2022 17:19:42] CGI script exited OK

Finally, once the above response data shows, that means you have been able to upload a file with Python CGI successfully.

Ready to start with our powerful file upload API?

Do you specifically need some help with your Python file upload? Have you gotten fed up with uploading files without results in the same vein? Are you ready to likewise make some headway? Above all, you can visit Filestack to sign up and start your file upload process.

Read More →