How to Use Rosetta 2 and Native ARM64 for Virtual Environments on Apple Silicon

Hey there, fellow data scientists!

Are you excited to dive into the world of Apple Silicon, but unsure about how to set up your development environment? Look no further! In this post, we'll walk you through the process of creating and activating virtual environments using Rosetta 2 emulation and native ARM64 with Conda.

Why do we need Rosetta 2 emulation and native ARM64 environments?

As you may know, Apple's M1 chip is based on ARM64 architecture, which is different from the traditional x86 architecture used by Intel-based Macs. While many packages have been ported to ARM64, some still require Rosetta 2 emulation to run. By setting up both environments, you'll be able to take advantage of the latest packages optimized for ARM64 and still be able to use packages that require emulation.

Creating a Rosetta 2 Emulation Environment with Conda

To create a Rosetta 2 emulation environment with Conda, follow these steps:

1. Open a terminal and run the following command to create a new environment:

conda create --platform osx-64 --name rosetta_env python

2. Activate the environment:

conda activate rosetta_env

3. Verify that you're running in Rosetta 2 emulation mode:

arch

This should output i386.

Creating a Native ARM64 Environment with Conda

To create a native ARM64 environment with Conda, follow these steps:

1. Open a terminal and run the following command to create a new environment:

conda create --platform osx-arm64 --name arm64_env python

2. Activate the environment:

conda activate arm64_env

3. Verify that you're running in native ARM64 mode:

arch

This should output arm64.

Usingzshwith Homebrew,pyenv, pyenv-virtualenv, and pipx

If you're using zsh as your shell, you can set up your environment with Homebrew, pyenv, pyenv-virtualenv, and pipx. Here's an example configuration:

# ~/.zshrc.x86_64
# `brew` setup
local brew_path="/usr/local/bin"
local brew_opt_path="/usr/local/opt"
export PATH="${brew_path}:${PATH}"
eval "$(${brew_path}/brew shellenv)"

# `pyenv` setup
export PYENV_ROOT="$HOME/.pyenv86"
command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"

# `pipx` setup
export PATH="$PATH:/Users/lynn/.local/x86_64/bin"
export PIPX_BIN_DIR="$HOME/.local/x86_64/bin"
export PIPX_HOME="$HOME/.local/x86_64/pipx"

# pretty stuff
function rosetta {
  echo "%{$fg_bold[blue]%}(%{$FG[205]%}x86%{$fg_bold[blue]%})%{$reset_color%}"
}
PROMPT='$(rosetta)$(virtualenv_info) $(collapse_pwd)$(prompt_char)$(git_prompt_info)'

# ~/.zshrc.arm64
# `brew` setup
local brew_path="/opt/homebrew/bin"
local brew_opt_path="/opt/homebrew/opt"
export PATH="${brew_path}:${PATH}"
eval "$(${brew_path}/brew shellenv)"

# `pyenv` setup
export PYENV_ROOT="$HOME/.pyenv"
command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"

# `pipx` setup
export PATH="$PATH:/Users/lynn/.local/bin"
export PIPX_BIN_DIR="$HOME/.local/bin"
export PIPX_HOME="$HOME/.local/pipx"

# pretty stuff
PROMPT='$(virtualenv_info)$(collapse_pwd)$(prompt_char)$(git_prompt_info)'

Using Docker

If you need to use Docker, you can create a Dockerfile that builds a TensorFlow environment with Rosetta 2 emulation. Here's an example:

# This builds tensorflow v2.7.0 from source to be able to run on M1 within Docker
# as they do not provide binaries for arm64 platforms.
# https://github.com/tensorflow/tensorflow/issues/52845
# 
# Adapted from https://www.tensorflow.org/install/source
# 
# Warning!! this can take 1 - 2 hours to build.
# 
# If a different version of tf is needed, you'll need to figure out the min
# bazel version (see doc link above). You may also need to figure out the
# build dependency version limitations - I learned `numpy<1.18` and `numba<0.55`
# the hard way :-!.
# 
# --platform isn't needed particularly if building on M1, but it's more
# for info purposes, and as a safe guard if a non-M1 arch tries to build this
# Dockerfile
FROM --platform=linux/arm64 python:3.8-buster

# Set working directory to /app
WORKDIR /app

# Copy requirements file
COPY requirements.txt .

# Install dependencies
RUN pip install -r requirements.txt

# Copy application code
COPY . .

# Expose port 8888 for Jupyter Notebook
EXPOSE 8888

# Run command to start Jupyter Notebook
CMD ["jupyter", "notebook", "--allow-root", "--port=8888"]

Building the Docker Image

To build the Docker image, run the following command:

docker build -t tensorflow-m1 .

Running the Docker Container

To run the Docker container, run the following command:

docker run -p 8888:8888 tensorflow-m1

This will start a Jupyter Notebook server on port 8888. You can access it by visiting http://localhost:8888 in your web browser.

Conclusion

In this post, we've shown you how to create and activate virtual environments using Rosetta 2 emulation and native ARM64 with Conda. We've also provided an example of how to use Docker to build a TensorFlow environment with Rosetta 2 emulation. With these tools, you'll be able to take advantage of the latest packages optimized for ARM64 and still be able to use packages that require emulation.

References

• Apple Silicon documentation

• Conda documentation

• Docker documentation

• TensorFlow documentation

We hope this post has been helpful! Let us know if you have any questions or need further assistance.