-
Notifications
You must be signed in to change notification settings - Fork 180
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Scikit-learn benchmark takes very long time to finish (~18 hrs) when EDMM is enabled #1612
Comments
@anjalirai-intel Have you tried the perf optimizations for EDMM? In particular, PR #1513 |
@dimakuv The above issue was first tested and observed on PR #1513. Later on, it was tested with master and Gramine v1.5 with only |
@vasanth-intel So PR #1513 doesn't help, right? The performance overhead is still huge? |
With |
I wonder if this workload does a lot of dynamic allocation AND deallocation. Maybe we will need "lazy free" optimization as well. |
Description of the problem
When EDMM is enabled,
scikit-learn
benchmark takes about 18 hours to finish 1 iteration with Gramine-SGX. Otherwise, when EDMM is disabled the benchmark executes successfully for 10 iterations in ~20 hours with Linux Native, Gramine-Direct and Gramine-SGX execution modes.Steps to reproduce
Git clone the benchmark from https://github.com/IntelPython/scikit-learn_bench.
Install the benchmark using pip install command using
requirements-common.txt
andsklearn_bench/requirements.txt
present within the above github link.OR
Save the below lines in requirements.txt and install using the command
pip install -r requirements.txt
.tqdm numpy==1.24.1 scipy==1.10.0 daal==2023.0.1 daal4py==2023.0.1 pandas==1.5.2 scikit-learn==1.2.0 dpcpp-cpp-rt==2023.0.0 scikit-learn-intelex==2023.0.1
Edit
configs/skl_config.json
to includekmeans
andknn_clsf
algorithms only.Update the manifest to enable EDMM and generate the relevant SGX manifest.
Create a new
results
directory for the benchmark output.Execute the below benchmark command.
gramine-sgx sklearnex runner.py --configs configs/skl_config.json --output-file results/sgx_output.json
Expected results
When EDMM is enabled and with Gramine-SGX execution mode, the benchmark should complete it's 10 iterations of execution within ~20 hours.
Actual results
When EDMM is enabled and with Gramine-SGX execution mode, the benchmark takes ~18 hours for 1 iteration to complete.
Gramine commit hash
master
The text was updated successfully, but these errors were encountered: