[JK] Portable/Relocatable Python Linking #275
Open
+33
−2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Open
5 tasks
jaswanthikolla
force-pushed
the
portable_python
branch
from
7e7b102
to
e36e941
Compare
kurst03
approved these changes
May 20, 2024
nicolaswebb
approved these changes
May 20, 2024
mayeut
reviewed
Jun 1, 2024
Comment on lines
+97
to
+120
|
|
||
| It "Relocatable Python" { | ||
| $semversion = [semver] $Version | ||
| $pyfilename = "python$($semversion.Major).$($semversion.Minor)" | ||
| $artifactPath = Join-Path "Python" $Version | Join-Path -ChildPath $Architecture | ||
|
|
||
| $relocatedPython = Join-Path $HOME "relocated_python" | ||
| $relocatedPythonTool = Join-Path -Path $relocatedPython -ChildPath $artifactPath | ||
| $relocatedFullPath = Join-Path $relocatedPythonTool "bin" | Join-Path -ChildPath $pyfilename | ||
|
|
||
| # copy the current build to relocated_python | ||
| $toolCacheArtifact = Join-Path $env:RUNNER_TOOL_CACHE $artifactPath | ||
| moveAssets -source $toolCacheArtifact -destination $relocatedPythonTool | ||
| try { | ||
| # Verify that relocated Python works | ||
| $relocatedFullPath | Should -Exist | ||
| "$relocatedFullPath --version" | Should -ReturnZeroExitCode | ||
| "sudo $relocatedFullPath --version" | Should -ReturnZeroExitCode | ||
| } | ||
| finally { | ||
| # Revert the changes for other tests | ||
| moveAssets -source $relocatedPythonTool -destination $toolCacheArtifact | ||
| } | ||
| } |
There was a problem hiding this comment.
I think this might fail on macOS.
@jaswanthikolla, can you run the full build in your fork to make sure this passes ?
It might save some time if it requires some modifications before maintainers validate the workflow / review the PR.
It might only fail when using a version not using the universal2 installer so worth checking a build of python < 3.11
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What's the Issue?
Python binary is compiled with
rpaththat's/opt/hostedtoolcache/Python. Now, latest github runners defineRUNNER_TOOL_CACHE/AGENT_TOOLSDIRECTORYdifferently than/opt/hostedtoolcacheand that installs python at/home/runner/_work/_tool/Python/. So, with this, there are 2 issues.sudo python --versiondoesn't work ( See Error section) because most systems's doesn't allow passing LD_LIBRARY_PATH due to security issues.python --versiondoesn't work without setting environment variable LD_LIBRARY_PATHoutput of ldd :
Error:
Solution:
$ORIGIN is used to as reference to the binary path. So, we can use rpath that references relative path instead of absolute compile time path. With this relative path, Python binaries become relocatable.
Change:
--rpathchanged to-rpathas it's the standard documented one.How this is tested?
Why bother about this issue?
LD_LIBRARY_PATH can't be used with
sudo pythondue to security concerns. But, more details are here actions/setup-python#871