Skip to content

Commit

Permalink
feat: improve dependency installation experience (lambdaclass#1298)
Browse files Browse the repository at this point in the history
* Add installation instructions to README

* Add compilation step to installation testing

* Add script for universal dependency installation

* Update changelog

* Enable test_install workflow only on main

* Update optional dependencies

Just having PyEnv and make installed suffices for running `make deps`.
I don't think we need to list cairo-lang as a dependency.

* Don't compile, only check

* Push again

* Use swatinem cache

* Cache all crates even on failure

* Fix debian rust cache

* Use manual cache

* Test cache

* Cache cairo compiler

* Add pyenv dependency caching

* Cache pip instead of pyenv

* Try fixing cache

* Use setup-python action

* Use ~ instead of ${HOME}

* Use manual cache in Debian container

* Add some comments

* Make cache owned by current user and add logs

* Remove python install from script
  • Loading branch information
MegaRedHand authored and kariy committed Jul 25, 2023
1 parent 59c1b76 commit 5d2fc97
Show file tree
Hide file tree
Showing 8 changed files with 169 additions and 63 deletions.
64 changes: 56 additions & 8 deletions .github/workflows/test_install.yml
Expand Up @@ -17,22 +17,70 @@ jobs:
steps:
- uses: actions/checkout@v3

- if: matrix.os == 'ubuntu-22.04'
name: "Install on ubuntu-22.04"
run: install-scripts/install-ubuntu.sh
- name: Set up Cargo cache
uses: Swatinem/rust-cache@v2
with:
cache-all-crates: 'true'
cache-on-failure: 'true'
cache-directories: ./cairo/target/

- if: matrix.os == 'macos-12'
name: "Install on macos-12"
run: install-scripts/install-macos.sh
# just to cache dependencies
- name: Install python
uses: actions/setup-python@v4
with:
python-version: '3.9'
cache: 'pip'

- name: Install dependencies
run: bash install.sh

- name: Compile project
run: cargo check

install_debian:
name: "Install on debian-11"
runs-on: ubuntu-22.04
container:
image: debian:11
defaults:
run:
shell: bash {0}

steps:
- uses: actions/checkout@v3

- name: "Install on debian-11"
run: install-scripts/install-debian.sh
# we don't use swatinem because rustc isn't installed yet
- name: Cache Rust dependencies
uses: actions/cache@v3
with:
path: |
~/.cargo/
./target/
./cairo/target/
key: ${{ runner.os }}-install-${{ github.sha }}
restore-keys: ${{ runner.os }}-install-

- name: Enable pip cache
run: |
mkdir -p ~/.cache/pip
chown -v -R $USER ~/.cache
chmod -v -R 770 ~/.cache
ls -al ~/
ls -al ~/.cache
# NOTE: we don't use install-python because lsb_release isn't installed
- name: Cache Python dependencies
uses: actions/cache@v3
with:
path: |
~/.cache/pip/wheels
key: ${{ runner.os }}-python-install-${{ github.sha }}
restore-keys: ${{ runner.os }}-python-install-

- name: Install dependencies
run: bash install.sh

- name: Compile project
run: |
. "$HOME/.cargo/env"
cargo check
5 changes: 3 additions & 2 deletions CHANGELOG.md
Expand Up @@ -2,12 +2,13 @@

#### Upcoming Changes

* feat: add dependency installation script `install.sh` [#1298](https://github.com/lambdaclass/cairo-vm/pull/1298)

* fix: specify resolver version 2 in the virtual workspace's manifest [#1311](https://github.com/lambdaclass/cairo-vm/pull/1311)

* feat: add `lambdaworks-felt` feature to `cairo-vm-cli` [#1308](https://github.com/lambdaclass/cairo-vm/pull/1308)

* chore: update dependencies, particularly clap 3.2 -> 4.3 [#1309](https://github.com/lambdaclass/cairo-vm/pull/1309)

* this removes dependency on _atty_, that's no longer mantained

* chore: remove unused dependencies [#1307](https://github.com/lambdaclass/cairo-vm/pull/1307)
Expand All @@ -23,7 +24,7 @@

* rename github repo from https://github.com/lambdaclass/cairo-rs to https://github.com/lambdaclass/cairo-vm [#1289](https://github.com/lambdaclass/cairo-vm/pull/1289)

* fix(security): avoid OOM crashes when programs jump to very high invalid addresses.
* fix(security): avoid OOM crashes when programs jump to very high invalid addresses [#1285](https://github.com/lambdaclass/cairo-vm/pull/1285)

* fix: add `to_bytes_be` to the felt when `lambdaworks-felt` feature is active [#1290](https://github.com/lambdaclass/cairo-vm/pull/1290)

Expand Down
5 changes: 2 additions & 3 deletions Makefile
Expand Up @@ -124,17 +124,16 @@ cargo-deps:
cargo install --version 0.11.0 wasm-pack

deps: cargo-deps build-cairo-1-compiler
pyenv install -s pypy3.9-7.3.9
pyenv install -s pypy3.9-7.3.9
PYENV_VERSION=pypy3.9-7.3.9 python -m venv cairo-vm-pypy-env
. cairo-vm-pypy-env/bin/activate ; \
pip install -r requirements.txt ; \
pyenv install -s 3.9.15
pyenv install -s 3.9.15
PYENV_VERSION=3.9.15 python -m venv cairo-vm-env
. cairo-vm-env/bin/activate ; \
pip install -r requirements.txt ; \

deps-macos: cargo-deps build-cairo-1-compiler
brew install gmp
arch -x86_64 pyenv install -s pypy3.9-7.3.9
PYENV_VERSION=pypy3.9-7.3.9 python -m venv cairo-vm-pypy-env
. cairo-vm-pypy-env/bin/activate ; \
Expand Down
19 changes: 13 additions & 6 deletions README.md
Expand Up @@ -27,7 +27,8 @@ A faster and safer implementation of the Cairo VM in Rust
- [Getting Started](#-getting-started)
- [Dependencies](#dependencies)
- [Usage](#-usage)
- [Running cairo-vm](#running-cairo-vm)
- [Adding cairo-vm as a dependency](#adding-cairo-vm-as-a-dependency)
- [Running cairo-vm from the CLI](#running-cairo-vm-from-cli)
- [Using hints](#using-hints)
- [Running a function in a Cairo program with arguments](#running-a-function-in-a-cairo-program-with-arguments)
- [WebAssembly Demo](#webassembly-demo)
Expand Down Expand Up @@ -67,17 +68,23 @@ It's Turing-complete and it was created by [Starkware](https://starkware.co/) as

### Dependencies

**Required**
#### Required

- [Rust 1.66.1](https://www.rust-lang.org/tools/install)
These are needed in order to compile and use the project.

- [Rust 1.69.0 or newer](https://www.rust-lang.org/tools/install)
- Cargo

**Optional**
#### Optional

These dependencies are only necessary in order to run the original VM, compile Cairo programs, and run tests.

- PyEnv with Python 3.9
- cairo-lang
- make
- PyEnv

#### Installation script

You can install all of the required and optional dependencies by running the script `install.sh` while in the repository root.

## 🚀 Usage

Expand Down
19 changes: 4 additions & 15 deletions install-scripts/install-debian.sh
Expand Up @@ -17,7 +17,9 @@ source "$HOME/.cargo/env"
rustc --version

# Install pyenv dependencies
apt-get install -y git make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev
apt-get install -y git make build-essential libssl-dev zlib1g-dev libbz2-dev \
libreadline-dev libsqlite3-dev wget llvm libncurses5-dev libncursesw5-dev \
xz-utils tk-dev libffi-dev liblzma-dev libgmp3-dev

# Install pyenv
curl https://pyenv.run | bash
Expand All @@ -28,17 +30,4 @@ eval "$(pyenv init -)"
# Make sure pyenv has been installed correctly
pyenv -v

# Installing python 3.9 with pyenv
pyenv install 3.9

# Setting python 3.9 as the default local version
pyenv local 3.9

# Create and enter a virtual environment
python3.9 -m venv ~/cairo_venv
source ~/cairo_venv/bin/activate

# Install cairo & its dependencies
apt install -y libgmp3-dev
pip3 install -r requirements.txt

make deps
16 changes: 2 additions & 14 deletions install-scripts/install-macos.sh
Expand Up @@ -9,21 +9,9 @@ source "$HOME/.cargo/env"
rustc --version

# Install pyenv
brew install pyenv
brew install pyenv gmp

# Make sure pyenv has been installed correctly
pyenv -v

# Installing python 3.9 with pyenv
pyenv install 3.9

# Setting python 3.9 as the default local version
pyenv local 3

# Create and enter a virtual environment
python3 -m venv ~/cairo_venv
source ~/cairo_venv/bin/activate

# Install cairo & its dependencies
pip3 install -r requirements.txt

make deps-macos
22 changes: 7 additions & 15 deletions install-scripts/install-ubuntu.sh
@@ -1,6 +1,9 @@
#!/usr/bin/env bash
set -ex

PYTHON_VERSION=3.9.15
PYTHON_MINOR_VERSION=${PYTHON_VERSION%.*}

sudo apt update -y

# Install Rust and Cargo
Expand All @@ -11,7 +14,9 @@ source "$HOME/.cargo/env"
rustc --version

# Install pyenv dependencies
sudo apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget llvm libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev
sudo apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev \
libreadline-dev libsqlite3-dev wget llvm libncurses5-dev libncursesw5-dev \
xz-utils tk-dev libffi-dev liblzma-dev libgmp3-dev

# Install pyenv
curl https://pyenv.run | bash
Expand All @@ -23,17 +28,4 @@ eval "$(pyenv init -)"
# Make sure pyenv has been installed correctly
pyenv -v

# Installing python 3.9 with pyenv
pyenv install 3.9.16

# Setting python 3.9 as the default local version
pyenv local 3.9.16

# Create and enter a virtual environment

python3.9 -m venv ~/cairo_venv
source ~/cairo_venv/bin/activate

# Install cairo & its dependencies
sudo apt install -y libgmp3-dev
pip3 install -r requirements.txt
make deps
82 changes: 82 additions & 0 deletions install.sh
@@ -0,0 +1,82 @@
#!/usr/bin/env bash

# Print an error message and exit with 1
unsupported_os () {
echo "Detected OS ($1) is unsupported."
echo "Please open an issue (PRs welcome ❤️) on:"
echo " https://github.com/lambdaclass/cairo-vm/issues"
echo ""
echo "NOTE: you can still try installing dependencies manually"
echo "If your OS differs from the detected one, you can look \
for the installation script for your OS in the install-scripts folder."
exit 1
}

# Print the detected OS
print_os() {
echo "Detected OS: $1"
}

# Print a message and run the script
run_script() {
echo "Running $1..."
. $1
}

# Detect Linux distro
install_linux() {
# taken from: https://unix.stackexchange.com/a/6348
# tries different methods to detect the Linux distro
if [ -f /etc/os-release ]; then
# freedesktop.org and systemd
. /etc/os-release
OS=$NAME
VER=$VERSION_ID
elif type lsb_release >/dev/null 2>&1; then
# linuxbase.org
OS=$(lsb_release -si)
VER=$(lsb_release -sr)
elif [ -f /etc/lsb-release ]; then
# For some versions of Debian/Ubuntu without lsb_release command
. /etc/lsb-release
OS=$DISTRIB_ID
VER=$DISTRIB_RELEASE
elif [ -f /etc/debian_version ]; then
# Older Debian/Ubuntu/etc.
OS=Debian
VER=$(cat /etc/debian_version)
elif [ -f /etc/SuSe-release ]; then
# Older SuSE/etc.
OS="Old SuSE"
elif [ -f /etc/redhat-release ]; then
# Older Red Hat, CentOS, etc.
OS="Old RedHat"
else
# Fall back to uname, e.g. "Linux <version>", also works for BSD, etc.
OS=$(uname -s)
VER=$(uname -r)
fi

print_os $OS

# NOTE: we don't use $VER for now, but this might change
case "$OS" in
Ubuntu*) run_script "install-scripts/install-ubuntu.sh" ;;
Debian*) run_script "install-scripts/install-debian.sh" ;;
*) unsupported_os "linux: $OS" ;;
esac
}

install_macos() {
print_os "MacOS"
run_script install-scripts/install-macos.sh
}

case "$OSTYPE" in
linux*) install_linux ;;
darwin*) install_macos ;;
msys*|cygwin*) unsupported_os "Windows" ;;
solaris*) unsupported_os "Solaris" ;;
bsd*) unsupported_os "BSD" ;;
*) unsupported_os "unknown: ${OSTYPE}" ;;
esac

0 comments on commit 5d2fc97

Please sign in to comment.