Tech 3 minutes read

Using REST API to upload files to the S3 bucket

This article came into existence due to a task we had for connecting the internal piece of the platform with S3 as a backup. In this specific task, we had to manage the PDF files so, this article will revolve around this example. TL;DR: we have some files in our internal system and we need this piece of the internal platform using REST API to upload files to the S3 bucket.

Requirements:

AWS IAM (AWS Identity and Access Management)
AWS API Gateway
AWS S3 Bucket 

First step would be to create the IAM role. We will start by creating the IAM policy to allow the API to upload binary files to S3:

Creating IAM Policy
Creating IAM Policy

This policy gives a minimal amount of permissions that are required to get a job done. Make sure to replace bluegrid-test with your S3 bucket name. Now create the API Gateway IAM role and attach the policy above:

Creating IAM Role
Creating IAM Role

Next, we’re going to create a Folder and Item resources to represent the S3 bucket and S3 object. Both parameters will be specified as part of a request URL by the client. 

  • In the API Gateway Console, create an API named test-api.
  • Under the API's root resource, create a child resource named folder and set the required Resource Path as /{folder}
Creating Folder Resource
Creating Folder Resource
  • For the folder resource, create a child resource named item and set the required Resource Path as /{item} 
Creating Item Resource
Creating Item Resource
  • On the /folder resource from the Resources tree, create the PUT method as follows. Make sure to select your AWS Region and replace the Execution role which was made at the beginning of this tutorial. 
Creating PUT HTTP Method for Folder Resource
Creating PUT HTTP Method for Folder Resource
  • Map the {folder}/{item} path parameters of the method request to the {bucket}/{object} path parameters of the integration request.
    • Choose Method Execution and then Integration Request.
    • Expand URL Path Parameters and choose Add path
    • Type bucket and object in the Name column and method.request.path.folder and method.request.path.item in the Mapped from the column. Choose the check-mark icon to save the mapping.
Mapping the {folder}/{item} path parameters of the method request to the {bucket}/{object} path parameters of the integration request
Mapping the {folder}/{item} path parameters of the method request to the {bucket}/{object} path parameters of the integration request
  • In the API Gateway console, register the pdf type to the API’s binaryMediaTypes.
    • Choose Settings for the API.
    • Under Binary Media Types, choose Add Binary Media Type.
    • Enter the required media type, in our case application/pdf
    • Choose Save Changes to save the setting.
Registering the pdf type to the API’s binary Media Types
Registering the pdf type to the API’s binary Media Types
  • Deploy the API and call the integration endpoint using Postman or the command line
Deploying API
Deploying API

Finally, let's test this setup using CURL and sending the PUT HTTP request with a file we want on the S3 bucket:

$ curl --request PUT -H "Content-Type: application/pdf"  --data-binary "@test.pdf" https://4mpwzgqkxf.execute-api.eu-central-1.amazonaws.com/dev/bluegrid-test/test.pdf
Upload process via CURL
Upload process via CURL

That's it! We'll roll on to the nest task and you feel free to write to us on dev@bluegrid.io!