Introduction

Getting Python libraries like yfinance to work on AWS Lambda can be a real challenge, especially when you're developing on an Apple Silicon (M1/M2) Mac. I recently spent a weekend struggling to create a compatible Lambda Layer, a task that surprisingly consumed most of my project time.

After numerous attempts, I found a reliable method that not only works but also provides a deeper understanding of the deployment package. As a bonus, this approach makes it easy to test the Lambda function locally before deploying. This guide shares that exact process to save you the headache.

Approach

Build using AWS Lambda base image (I used for python 3.12 dockerfile from AWS Dev guide)
[Optional] Obtain interactive shell on the running image and review environment
[Optional] Run docker image and test lambda function locally
Copy files from container to mac
Prune files and zip into the layer

Upload the zipped layer to AWS and use it! With heavy pruning, the zipped layer goes below 50 Mb and can even be uploaded to AWS directly.

Prerequisites

Docker (Docker Desktop 4.43.2) installed on mac to build and run containers
Curl - To call lambda function from mac terminal
Assume one knows how to run basic lambda functions on AWS.

Steps

1 Build Docker image

1.1 Create a working directory for docker build on your mac

mkdir lambda_layer_yfin

cd lambda_layer_yfin

1.2 Create Dockerfile, requirements.txt and lambda_function.py in that directory

Dockerfile

FROM public.ecr.aws/lambda/python:3.12

# Copy requirements.txt
COPY requirements.txt ${LAMBDA_TASK_ROOT}

# Install the specified packages
RUN pip install -r requirements.txt

# Copy function code
COPY lambda_function.py ${LAMBDA_TASK_ROOT}

# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
CMD [ "lambda_function.handler" ]

Note: Other python versions should work as well with appropriate changes in other files. The only other version I have personally tried is python 3.10.

requirements.txt

# ***Based on "pip freeze" of working docker environment***
# Using AWS Lambda base image with python 3.12 + yfinance
# site_package dir: /var/lang/lib/python3.12/site-packages
beautifulsoup4==4.13.4
certifi==2025.7.14
cffi==1.17.1
charset-normalizer==3.4.2
curl_cffi==0.12.0
frozendict==2.4.6
idna==3.10
multitasking==0.0.12
numpy==2.3.2
pandas==2.3.1
peewee==3.18.2
platformdirs==4.3.8
protobuf==6.31.1
pycparser==2.22
pytz==2025.2
requests==2.32.4
soupsieve==2.7
typing_extensions==4.14.1
tzdata==2025.2
websockets==15.0.1
yfinance==0.2.65
# below commented as already present in 3.12 base image
# awslambdaric==3.1.1
# boto3==1.38.36
# botocore==1.38.36
# jmespath==1.0.1
# python-dateutil==2.9.0.post0
# s3transfer==0.13.0
# simplejson==3.20.1
# six==1.17.0
# snapshot-restore-py==1.0.0
# urllib3==1.26.19

Note: This is essentially a dump from a working environment with packages already in 3.12 base manually removed. A one line requirements.txt with just yfinance should work as well. Make sure to do that if you have changed python version from 3.12 in dockerfile above.

lambda_handler.py

import yfinance as yf
from datetime import datetime, timedelta
def handler(event, context):
   aapl = yf.Ticker("AAPL")
   return "AAPL: " + str(aapl.info["currentPrice"])

1.3 Build docker image based on the above files

docker buildx build --platform linux/amd64 --provenance=false -t docker-awsbaseimage:test .

2 Start a container from docker image and obtain interactive shell

# ***Run image and get an interactive shell for funsies***
docker run --platform linux/amd64 -it --entrypoint /bin/bash docker-awsbaseimage:test 
python --version # check python version is 3.12.x
command -v python # check path /var/lang/bin/python ; type -a also works
pip show yfinance   # copy Location: /var/lang/lib/python3.12/site-packages
pip freeze > requirements.txt # dump all packages, check versions etc
exit # exit the container

3 Start a container and Test Lambda locally

# ***Test our image locally
# --platform linux/amd64: Needed as Apple silicon macs like mine are ARM-based
#                 but running an image built for Intel/AMD (x86_64) processors
# -p 9000:8000 forwards traffic on mac port 9000 to port 8080 on container
docker run --platform linux/amd64 -p 9000:8080 docker-awsbaseimage:test

# Invoke lambda from a different terminal
# Lambda Runtime Interface Emulator (RIE) standard URL ->
# http://<host>:<port>/2015-03-31/functions/function/invocations
# -d '{}' is to POST an empty JSON
curl "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'

4 Build lambda layer from the files from the tested docker image

4.1 Create a clean layer_build directory on mac and copy packages from container (./layer_build/site-packages)

mkdir -p layer_build

docker ps -a # find container id

docker cp <container_id>:/var/lang/lib/python3.12/site-packages layer_build

4.2 prune files for optimizing size of layer

cd layer_build/site-packages

# Remove packages that are already part of base image

rm -rf boto3 botocore jmespath six s3transfer urllib3 dateutil simplejson

rm -rf awslambdaric pip README.txt snapshot_restore_py.py

rm -rf boto3*.dist-info botocore*.dist-info jmespath*.dist-info six.py 

rm -rf s3transfer*.dist-info awslambdaric*.dist-info six*.dist-info

rm -rf urllib3*info snapshot_restore_py*info simplejson*info pip*info


# Remove cache files

find . -type d -name "__pycache__" -exec rm -rf {} +

find . -type f -name "*.pyc" -delete

There are some hard restrictions from AWS on the layer size (at the moment). e.g. files above 50Mb cannot be directly uploaded and have to be uploaded to S3 first. Also, the unzipped layer cannot exceed 250Mb (which my first attempt was going beyond!).

4.3 Structure the files into the right directory structure and zip

# Go back to the parent directory “layer_build”
cd ..

# move the pruned packages into a directory structure the layer needs
mkdir -p python/lib/python3.12

mv site-packages python/lib/python3.12/
# zip python/lib/python3.12/site-packages
zip -r yfin-python-layer.zip python

The lambda layer file(yfin-python-layer.zip) is ready!

Conclusion

Create a layer through AWS console by uploading this (or command line). Once the layer is created successfully on AWS, we can assign it to lambda functions that require yfinance. This approach can be easily adapted to other libraries with minor variations in requirements.txt.

How to Create AWS Lambda Layer for yfinance on Apple Silicon Macs

Table of contents