Build AWS Lambda Layers for Python runtime

Intro – Why are we doing this

This article has been made from a real-world scenario on a project that had a goal to build AWS Lambda Layers for python runtime. This project consisted of the python codebase and DynamoDB for certain data to be held in. The basic data flow is pretty simple:

• The API request is received and communicates with Lambda Layer responsible for delivering required libraries for different project functions
• Backend crunches data from within DB and returns the response
• Saves some data in DB

In this particular scenario, DB holds some sensitive information that when accessed to DynamoDB is decrypted for whoever has access to DB itself. However, given the data type and scenarios in which unprivileged persons are allowed to access the DB we need to make sure sensitive data is still encrypted even for privileged access.

To solve this glitch we decided to add an encryption layer that will make sure all sensitive information in the DB is encrypted regardless of access level. Most handy way to add an encryption layer in the existing python project is by including a “cryptography” package in it and applying the type of encryption you need/want. 

The project is being developed on Mac OS which slipped us following error while trying to use Lambda Layers with “cryptography” library:

ImportError: /var/task/cryptography/hazmat/bindings/_constant_time.so: invalid ELF header

cryptography contains native code and that code is compiled for the architecture of the current machine. AWS Lambda needs Layers compiled as Linux ELF shared objects.

Resolution

At the time of writing this article there wasn’t a known work-around to this problem so, the most appropriate resolution was, also very acceptable due to long term plans with this project, to put it in a container structure. At the same time it seemed like the perfect opportunity to write up an article about this 🙂

Requirements: Docker – needs to be installed on your local machine.

Installation process: Clone the following github repo to your local machine:

$ git clone https://github.com/bluegridio/build-lambda-layer.git
$ cd build-lambda-layer

Usage: Run the script and provide the package name as the first argument.

$ ./build-lambda-layer.sh curl

Note that this solution is designed so it can be used for generating layers in general, regardless of scenario:

Ok, let’s do the work now 🙂

That’s all folks! Check out the tech blog posts section for more (hopefully 😁) useful articles.