Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: improve dependency installation experience #1298

Merged
merged 26 commits into from Jul 4, 2023
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
c64cea4
Add installation instructions to README
MegaRedHand Jun 29, 2023
32c1680
Add compilation step to installation testing
MegaRedHand Jun 29, 2023
db1c9a3
Add script for universal dependency installation
MegaRedHand Jun 29, 2023
dfd1c8f
Update changelog
MegaRedHand Jun 29, 2023
3a52a42
Enable test_install workflow only on main
MegaRedHand Jun 29, 2023
04ae709
Update optional dependencies
MegaRedHand Jun 29, 2023
22d2ac6
Don't compile, only check
MegaRedHand Jun 30, 2023
98de9e8
Push again
MegaRedHand Jun 30, 2023
bfe9f88
Use swatinem cache
MegaRedHand Jun 30, 2023
e72b176
Cache all crates even on failure
MegaRedHand Jun 30, 2023
1cbc041
Fix debian rust cache
MegaRedHand Jun 30, 2023
28e9ca5
Use manual cache
MegaRedHand Jun 30, 2023
6a68c9a
Test cache
MegaRedHand Jun 30, 2023
cf9f9a7
Merge branch 'main' into better-installation-experience
MegaRedHand Jun 30, 2023
9af1c08
Cache cairo compiler
MegaRedHand Jun 30, 2023
d2a1728
Add pyenv dependency caching
MegaRedHand Jun 30, 2023
6030c72
Cache pip instead of pyenv
MegaRedHand Jun 30, 2023
60ac9ac
Try fixing cache
MegaRedHand Jun 30, 2023
7362bd9
Use setup-python action
MegaRedHand Jun 30, 2023
9c4631e
Use ~ instead of ${HOME}
MegaRedHand Jun 30, 2023
b15d1d8
Use manual cache in Debian container
MegaRedHand Jun 30, 2023
d0d413e
Add some comments
MegaRedHand Jun 30, 2023
8aedff4
Make cache owned by current user and add logs
MegaRedHand Jun 30, 2023
eaf9891
Merge branch 'main' into better-installation-experience
MegaRedHand Jul 3, 2023
ee7ce53
Remove python install from script
MegaRedHand Jul 4, 2023
30da4a4
Merge branch 'main' into better-installation-experience
MegaRedHand Jul 4, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
64 changes: 56 additions & 8 deletions .github/workflows/test_install.yml
Expand Up @@ -17,22 +17,70 @@ jobs:
steps:
- uses: actions/checkout@v3

- if: matrix.os == 'ubuntu-22.04'
name: "Install on ubuntu-22.04"
run: install-scripts/install-ubuntu.sh
- name: Set up Cargo cache
uses: Swatinem/rust-cache@v2
with:
cache-all-crates: 'true'
cache-on-failure: 'true'
cache-directories: ./cairo/target/

- if: matrix.os == 'macos-12'
name: "Install on macos-12"
run: install-scripts/install-macos.sh
# just to cache dependencies
- name: Install python
uses: actions/setup-python@v4
with:
python-version: '3.9'
cache: 'pip'

- name: Install dependencies
run: bash install.sh

- name: Compile project
run: cargo check

install_debian:
name: "Install on debian-11"
runs-on: ubuntu-22.04
container:
image: debian:11
defaults:
run:
shell: bash {0}

steps:
- uses: actions/checkout@v3

- name: "Install on debian-11"
run: install-scripts/install-debian.sh
# we don't use swatinem because rustc isn't installed yet
- name: Cache Rust dependencies
uses: actions/cache@v3
with:
path: |
~/.cargo/
./target/
./cairo/target/
key: ${{ runner.os }}-install-${{ github.sha }}
restore-keys: ${{ runner.os }}-install-

- name: Enable pip cache
run: |
mkdir -p ~/.cache/pip
chown -v -R $USER ~/.cache
chmod -v -R 770 ~/.cache
ls -al ~/
ls -al ~/.cache

# NOTE: we don't use install-python because lsb_release isn't installed
- name: Cache Python dependencies
uses: actions/cache@v3
with:
path: |
~/.cache/pip/wheels
key: ${{ runner.os }}-python-install-${{ github.sha }}
restore-keys: ${{ runner.os }}-python-install-

- name: Install dependencies
run: bash install.sh

- name: Compile project
run: |
. "$HOME/.cargo/env"
cargo check
4 changes: 3 additions & 1 deletion CHANGELOG.md
Expand Up @@ -2,6 +2,8 @@

#### Upcoming Changes

* feat: add dependency installation script `install.sh` [#1298](https://github.com/lambdaclass/cairo-vm/pull/1298)

* chore: remove unused dependencies [#1307](https://github.com/lambdaclass/cairo-vm/pull/1307)
* rand_core
* serde_bytes
Expand All @@ -13,7 +15,7 @@

* rename github repo from https://github.com/lambdaclass/cairo-rs to https://github.com/lambdaclass/cairo-vm [#1289](https://github.com/lambdaclass/cairo-vm/pull/1289)

* fix(security): avoid OOM crashes when programs jump to very high invalid addresses.
* fix(security): avoid OOM crashes when programs jump to very high invalid addresses [#1285](https://github.com/lambdaclass/cairo-vm/pull/1285)

* fix: add `to_bytes_be` to the felt when `lambdaworks-felt` feature is active [#1290](https://github.com/lambdaclass/cairo-vm/pull/1290)

Expand Down
4 changes: 2 additions & 2 deletions Makefile
Expand Up @@ -124,11 +124,11 @@ cargo-deps:
cargo install --version 0.11.0 wasm-pack

deps: cargo-deps build-cairo-1-compiler
pyenv install -s pypy3.9-7.3.9
pyenv install -s pypy3.9-7.3.9
PYENV_VERSION=pypy3.9-7.3.9 python -m venv cairo-vm-pypy-env
. cairo-vm-pypy-env/bin/activate ; \
pip install -r requirements.txt ; \
pyenv install -s 3.9.15
pyenv install -s 3.9.15
PYENV_VERSION=3.9.15 python -m venv cairo-vm-env
. cairo-vm-env/bin/activate ; \
pip install -r requirements.txt ; \
Expand Down
19 changes: 13 additions & 6 deletions README.md
Expand Up @@ -27,7 +27,8 @@ A faster and safer implementation of the Cairo VM in Rust
- [Getting Started](#-getting-started)
- [Dependencies](#dependencies)
- [Usage](#-usage)
- [Running cairo-vm](#running-cairo-vm)
- [Adding cairo-vm as a dependency](#adding-cairo-vm-as-a-dependency)
- [Running cairo-vm from the CLI](#running-cairo-vm-from-cli)
- [Using hints](#using-hints)
- [Running a function in a Cairo program with arguments](#running-a-function-in-a-cairo-program-with-arguments)
- [WebAssembly Demo](#webassembly-demo)
Expand Down Expand Up @@ -67,17 +68,23 @@ It's Turing-complete and it was created by [Starkware](https://starkware.co/) as

### Dependencies

**Required**
#### Required

- [Rust 1.66.1](https://www.rust-lang.org/tools/install)
These are needed in order to compile and use the project.

- [Rust 1.69.0 or newer](https://www.rust-lang.org/tools/install)
- Cargo

**Optional**
#### Optional

These dependencies are only necessary in order to run the original VM, compile Cairo programs, and run tests.

- PyEnv with Python 3.9
- cairo-lang
- make
- PyEnv

#### Installation script

You can install all of the required and optional dependencies by running the script `install.sh` while in the repository root.

## 🚀 Usage

Expand Down
1 change: 0 additions & 1 deletion install-scripts/install-debian.sh
Expand Up @@ -41,4 +41,3 @@ source ~/cairo_venv/bin/activate
# Install cairo & its dependencies
apt install -y libgmp3-dev
pip3 install -r requirements.txt

1 change: 0 additions & 1 deletion install-scripts/install-macos.sh
Expand Up @@ -26,4 +26,3 @@ source ~/cairo_venv/bin/activate

# Install cairo & its dependencies
pip3 install -r requirements.txt

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should install pyenv install 3.9.15

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, this isn't duplicated? Since then install_macos runs make deps-macos
But we can leave it duplicate because make deps-macos won't fail.

What we need to check is to add the CFLAGS=-I/opt/homebrew/opt/gmp/include LDFLAGS=-L/opt/homebrew/opt/gmp/lib before the pip install

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! Removed the whole Python install 👍

86 changes: 86 additions & 0 deletions install.sh
@@ -0,0 +1,86 @@
#!/usr/bin/env bash

# Print an error message and exit with 1
unsupported_os () {
echo "Detected OS ($1) is unsupported."
echo "Please open an issue (PRs welcome ❤️) on:"
echo " https://github.com/lambdaclass/cairo-vm/issues"
echo ""
echo "NOTE: you can still try installing dependencies manually"
echo "If your OS differs from the detected one, you can look \
for the installation script for your OS in the install-scripts folder."
exit 1
}

# Print the detected OS
print_os() {
echo "Detected OS: $1"
}

# Print a message and run the script
run_script() {
echo "Running $1..."
. $1
}

# Detect Linux distro
install_linux() {
# taken from: https://unix.stackexchange.com/a/6348
# tries different methods to detect the Linux distro
if [ -f /etc/os-release ]; then
# freedesktop.org and systemd
. /etc/os-release
OS=$NAME
VER=$VERSION_ID
elif type lsb_release >/dev/null 2>&1; then
# linuxbase.org
OS=$(lsb_release -si)
VER=$(lsb_release -sr)
elif [ -f /etc/lsb-release ]; then
# For some versions of Debian/Ubuntu without lsb_release command
. /etc/lsb-release
OS=$DISTRIB_ID
VER=$DISTRIB_RELEASE
elif [ -f /etc/debian_version ]; then
# Older Debian/Ubuntu/etc.
OS=Debian
VER=$(cat /etc/debian_version)
elif [ -f /etc/SuSe-release ]; then
# Older SuSE/etc.
OS="Old SuSE"
elif [ -f /etc/redhat-release ]; then
# Older Red Hat, CentOS, etc.
OS="Old RedHat"
else
# Fall back to uname, e.g. "Linux <version>", also works for BSD, etc.
OS=$(uname -s)
VER=$(uname -r)
fi

print_os $OS

# NOTE: we don't use $VER for now, but this might change
case "$OS" in
Ubuntu*) run_script "install-scripts/install-ubuntu.sh" ;;
Debian*) run_script "install-scripts/install-debian.sh" ;;
*) unsupported_os "linux: $OS" ;;
esac

make deps
}

install_macos() {
print_os "MacOS"
run_script install-scripts/install-macos.sh

make deps-macos
}

case "$OSTYPE" in
linux*) install_linux ;;
darwin*) install_macos ;;
msys*|cygwin*) unsupported_os "Windows" ;;
solaris*) unsupported_os "Solaris" ;;
bsd*) unsupported_os "BSD" ;;
*) unsupported_os "unknown: ${OSTYPE}" ;;
esac