dev-lang/rust: fix the build with musl-1.2.4 #31205

orbea · 2023-05-28T23:28:23Z

This fixes the build with musl-1.2.4, may need -system-bootstrap after updating musl.

Closes: https://bugs.gentoo.org/903607
Upstream-PR: rust-lang/rust#106246
Upstream-Issue: rust-lang/libc#2934
Upstream-PR: rust-lang/libc#2935
Upstream-Commit: rust-lang/libc@1e8c55c
Upstream-PR: rust-random/getrandom#326
Upstream-Commit: rust-random/getrandom@7f73e3c

gentoo-bot · 2023-05-28T23:32:04Z

Pull Request assignment

Submitter: @orbea
Areas affected: ebuilds
Packages affected: dev-lang/rust

dev-lang/rust: @gyakovlev, @gentoo/rust

Linked bugs

Bugs linked: 903607

In order to force reassignment and/or bug reference scan, please append [please reassign] to the pull request title.

Docs: Code of Conduct ● Copyright policy (expl.) ● Devmanual ● GitHub PRs ● Proxy-maint guide

gentoo-repo-qa-bot · 2023-05-28T23:43:50Z

Pull request CI report

Report generated at: 2023-05-28 23:43 UTC
Newest commit scanned: 088976d
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/9284601217/output.html

vimproved · 2023-06-01T07:49:47Z

Would it potentially be a good idea to have a dependency along the lines of

elibc_musl? (
    !system-bootstrap? (
        <sys-libs/musl-1.2.4
    )
)

to ensure people aren't trying to build it without system-bootstrap on 1.2.4?
This might be better suited to a check in pre_build_checks though.

orbea · 2023-06-01T12:57:59Z

Good idea, I updated the PR. It seems there is already a elibc_musl check in DEPEND and if USE=-system-boostrap is set then emerge will try to downgrade to musl-1.2.3. I think the remaining foot gun is that when rust was built successfully without these patches before upgrading to musl-1.2.4, but it can be manually resolved and is still probably better than not being able to build rust at all.

gentoo-repo-qa-bot · 2023-06-01T13:24:15Z

Pull request CI report

Report generated at: 2023-06-01 13:24 UTC
Newest commit scanned: fc0a2b6
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/8227195a7b/output.html

orbea · 2023-06-05T00:20:16Z

I updated this PR for rust-1.70.0.

@gyakovlev Can you please review this PR, its a blocking issue for me.

gentoo-repo-qa-bot · 2023-06-05T00:29:01Z

Pull request CI report

Report generated at: 2023-06-05 00:29 UTC
Newest commit scanned: 45b1686
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/0b4d6f7466/output.html

gentoo-repo-qa-bot · 2023-06-05T17:24:23Z

Pull request CI report

Report generated at: 2023-06-05 17:24 UTC
Newest commit scanned: d2b0ef0
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/e73e9b4e35/output.html

gentoo-repo-qa-bot · 2023-06-08T18:44:07Z

Pull request CI report

Report generated at: 2023-06-08 18:44 UTC
Newest commit scanned: f8bcc5d
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/49bb719cbd/output.html

orbea · 2023-06-09T16:25:47Z

@gyakovlev Can you please review this? Its a pain to rebase this over and over again.

gentoo-repo-qa-bot · 2023-06-09T16:39:22Z

Pull request CI report

Report generated at: 2023-06-09 16:39 UTC
Newest commit scanned: a82e6e2
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/5645309970/output.html

vimproved · 2023-06-10T22:59:13Z

Would probably be a good idea to re-add 1.69 since 1.70 is now unkeyworded and masked.

orbea · 2023-06-10T23:46:51Z

Downgrading rust is unfortunately a lot of effort to test.

vimproved · 2023-06-11T00:30:45Z

Did it myself, working fine for me.

vimproved · 2023-06-11T00:31:00Z

(Plus it was working fine before, so it should be fine.)

orbea · 2023-06-11T01:05:34Z

If @gyakovlev wants it I can add it back, but until I hear something one way or another I'd prefer to do as little work as possible.

gentoo-repo-qa-bot · 2023-06-29T15:35:28Z

Pull request CI report

Report generated at: 2023-06-29 15:35 UTC
Newest commit scanned: 3166125
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/7c0aa46e7c/output.html

LinuxUserGD · 2023-07-08T15:33:01Z

if USE=-system-boostrap is set then emerge will try to downgrade to musl-1.2.3.

Downgrading musl works if only a few packages have been compiled with musl-1.2.4, otherwise the Gentoo installation might break.
https://www.openwall.com/lists/musl/2023/05/02/1

The dynamic linker (and static-PIE entry point code) adds support for
the new compact "RELR" format for relative relocations which recent
linkers can generate. Use of this linker feature for dynamic-linked
programs will make them depend on having musl 1.2.4 or later available
at runtime.

vimproved · 2023-07-08T18:19:01Z

if USE=-system-boostrap is set then emerge will try to downgrade to musl-1.2.3.

Downgrading musl works if only a few packages have been compiled with musl-1.2.4, otherwise the Gentoo installation might break. https://www.openwall.com/lists/musl/2023/05/02/1

The dynamic linker (and static-PIE entry point code) adds support for
the new compact "RELR" format for relative relocations which recent
linkers can generate. Use of this linker feature for dynamic-linked
programs will make them depend on having musl 1.2.4 or later available
at runtime.

What I've been doing is reverting the musl commits that removed lfs64 support before rebuilding rust. I don't know if this would be feasible, however the only thing I can think of that would be able to fix it is adding a default disabled USE flag to musl that reverts the LFS64 removal until the issue gets fixed upstream, and then adding something like

elibc_musl? (
    !system-bootstrap? (
        || (
            <sys-libs/musl-1.2.4
            sys-libs/musl[revert-lfs64]
        )
    )
)

Obviously this would be an inelegant fix but I don't think there's a particularly good solution to this until Rust gets it together and finally fixes it upstream.

orbea · 2023-07-09T14:19:18Z

Fwiw I believe this is the only rust PR we are waiting on. rust-lang/rust#106246

Conditional patches for musl sounds like an even bigger mess to me.

vimproved · 2023-07-09T14:22:28Z

Yeah, at this point it's probably best to just wait on upstream until the fixes make it into the bootstrap tarballs.

orbea · 2023-07-09T14:26:57Z

Yeah, at this point it's probably best to just wait on upstream until the fixes make it into the bootstrap tarballs.

That only makes it harder for anyone with musl-1.2.4 and downgrading your system could be problematic regardless if rust is fixed or not. Also we might be waiting a long time...

12101111 · 2023-07-09T16:03:48Z

I'm using this wrapper to workaround linking issue for upstream release

#include <unistd.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <dirent.h>

int fstat64(int fd, struct stat *statbuf) {
  return fstat(fd, statbuf);
}
off_t lseek64(int fd, off_t offset, int whence) {
  return lseek(fd, offset, whence);
}
int stat64(const char* pathname, struct stat* statbuf) {
  return stat(pathname, statbuf);
}
int open64(const char* pathname, int flags) {
  return open(pathname, flags);
}
int openat64(int dirfd, const char* pathname, int flags, mode_t mode) {
  return openat(dirfd, pathname, flags, mode);
}
int fstatat64(int dirfd, const char* pathname, struct stat* statbuf, int flags) {
  return fstatat(dirfd, pathname, statbuf, flags);
}
struct dirent* readdir64(DIR *dirp) {
  return readdir(dirp);
}
int lstat64(const char* pathname, struct stat* statbuf) {
  return lstat(pathname, statbuf);
}

Compile this to archive file (eg. libalias.a) and add -L/usr/local/lib -lalias to RUSTFLAGS.

cc -O2 -c -o alias.o alias.c
ar -crs libalias.a alias.o
sudo cp libalias.a /usr/local/lib

vimproved · 2023-07-12T18:07:09Z

Here's a very hacky solution that allows bootstrapping from the stage0 tarball without downgrading or patching musl, using objcopy to redefine symbols in the rust stdlib:

src_prepare() {
	eapply_crate vendor/getrandom "${FILESDIR}"/1.69.0-musl-1.2.4-getrandom.patch
	eapply_crate vendor/libc-0.2.138 "${FILESDIR}"/1.69.0-musl-1.2.4-libc-0.2.138.patch
	eapply_crate vendor/libc-0.2.139 "${FILESDIR}"/1.69.0-musl-1.2.4-libc.patch
	eapply_crate vendor/libc "${FILESDIR}"/1.69.0-musl-1.2.4-libc.patch

	if ! use system-bootstrap; then
		has_version sys-devel/gcc || esetup_unwind_hack
		local rust_stage0_root="${WORKDIR}"/rust-stage0
		local rust_stage0="rust-${RUST_STAGE0_VERSION}-$(rust_abi)"

		if use elibc_musl; then
			LIBSTD="${WORKDIR}/${rust_stage0}/rust-std-$(rust_abi)/lib/rustlib/$(rust_abi)/lib/libstd-"*".rlib"

			"$(tc-getOBJCOPY)" \
				--redefine-sym=stat64=stat \
				--redefine-sym=fstat64=fstat \
				--redefine-sym=lstat64=lstat \
				--redefine-sym=open64=open \
				--redefine-sym=readdir64=readdir \
				--redefine-sym=fstatat64=fstatat \
				--redefine-sym=lseek64=lseek \
				--redefine-sym=ftruncate64=ftruncate \
				"${LIBSTD}" \
				"${LIBSTD}.out" || die

			mv "${LIBSTD}.out" "${LIBSTD}" || die
		fi

		"${WORKDIR}/${rust_stage0}"/install.sh --disable-ldconfig \
			--without=rust-docs-json-preview,rust-docs --destdir="${rust_stage0_root}" --prefix=/ || die
	fi

	default
}

orbea · 2023-07-13T17:20:50Z

Would that be compatible with both gcc and clang? Also < musl-1.2.4?

vimproved · 2023-07-13T17:29:11Z

Would that be compatible with both gcc and clang? Also < musl-1.2.4?

Yes. All functions on >=musl-1.2 are automatically 64-bit, and this works with both LLVM and GNU binutils.

orbea · 2023-07-16T16:46:44Z

@vimproved I pushed your suggestion with minor changes.

gentoo-repo-qa-bot · 2023-07-16T17:10:31Z

Pull request CI report

Report generated at: 2023-07-16 17:10 UTC
Newest commit scanned: 1851598
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/80181023df/output.html

gentoo-repo-qa-bot · 2023-07-17T00:45:21Z

Pull request CI report

Report generated at: 2023-07-17 00:45 UTC
Newest commit scanned: b1378ad
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/1ab525cc05/output.html

richfelker · 2023-07-20T19:15:08Z

Yes. All functions on >=musl-1.2 are automatically 64-bit, and this works with both LLVM and GNU binutils.

You mean >=musl-0.5.0.

gentoo-repo-qa-bot · 2023-07-20T19:40:37Z

Pull request CI report

Report generated at: 2023-07-20 19:40 UTC
Newest commit scanned: 11b7aed
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/beb13639d8/output.html

vimproved · 2023-07-21T01:09:53Z

Yes. All functions on >=musl-1.2 are automatically 64-bit, and this works with both LLVM and GNU binutils.

You mean >=musl-0.5.0.

Didn't know that, thanks for the correction. Was thinking of 64-bit time_t probably.

gentoo-repo-qa-bot · 2023-07-21T22:11:46Z

Pull request CI report

Report generated at: 2023-07-21 22:11 UTC
Newest commit scanned: 14da7e8
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/7bd5a4027a/output.html

gentoo-repo-qa-bot · 2023-07-24T19:40:26Z

Pull request CI report

Report generated at: 2023-07-24 19:40 UTC
Newest commit scanned: d13271b
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/3e2c3ccb2b/output.html

gentoo-repo-qa-bot · 2023-08-04T16:20:36Z

Pull request CI report

Report generated at: 2023-08-04 16:20 UTC
Newest commit scanned: 81fc190
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/4d1e6d3f2b/output.html

orbea · 2023-08-04T16:21:46Z

@xen0n Can you please review this PR?

This fixes the build with musl-1.2.4, may need -system-bootstrap after updating musl. Closes: https://bugs.gentoo.org/903607 Upstream-PR: rust-lang/rust#106246 Upstream-Issue: rust-lang/libc#2934 Upstream-PR: rust-lang/libc#2935 Upstream-Commit: rust-lang/libc@1e8c55c Upstream-PR: rust-random/getrandom#326 Upstream-Commit: rust-random/getrandom@7f73e3c Signed-off-by: orbea <orbea@riseup.net>

gentoo-repo-qa-bot · 2023-08-07T01:05:52Z

Pull request CI report

Report generated at: 2023-08-07 01:05 UTC
Newest commit scanned: 5033d91
Status: ✅ good

There are existing issues already. Please look into the report to make sure none of them affect the packages in question:
https://qa-reports.gentoo.org/output/gentoo-ci/e9ecb86905/output.html

gentoo-bot added assigned PR successfully assigned to the package maintainer(s). bug linked Bug/Closes found in footer, and cross-linked with the PR. labels May 28, 2023

orbea force-pushed the rust-musl-1.2.4 branch from 088976d to fc0a2b6 Compare June 1, 2023 12:55

orbea force-pushed the rust-musl-1.2.4 branch from fc0a2b6 to 45b1686 Compare June 5, 2023 00:19

orbea mentioned this pull request Jun 5, 2023

www-client/firefox: fix the build with musl-1.2.4 #31310

Closed

thesamesam requested a review from gyakovlev June 5, 2023 10:53

orbea force-pushed the rust-musl-1.2.4 branch from 45b1686 to d2b0ef0 Compare June 5, 2023 17:13

orbea force-pushed the rust-musl-1.2.4 branch 2 times, most recently from 783738e to f8bcc5d Compare June 8, 2023 18:33

orbea force-pushed the rust-musl-1.2.4 branch from f8bcc5d to a82e6e2 Compare June 9, 2023 16:25

orbea force-pushed the rust-musl-1.2.4 branch from a82e6e2 to 3166125 Compare June 29, 2023 15:05

orbea force-pushed the rust-musl-1.2.4 branch from 3166125 to 1851598 Compare July 16, 2023 16:46

orbea force-pushed the rust-musl-1.2.4 branch from 1851598 to b1378ad Compare July 17, 2023 00:32

orbea force-pushed the rust-musl-1.2.4 branch from b1378ad to 11b7aed Compare July 20, 2023 19:06

thesamesam requested a review from xen0n July 20, 2023 23:23

orbea force-pushed the rust-musl-1.2.4 branch from 11b7aed to 14da7e8 Compare July 21, 2023 21:58

orbea force-pushed the rust-musl-1.2.4 branch from 14da7e8 to d13271b Compare July 24, 2023 19:29

orbea force-pushed the rust-musl-1.2.4 branch from d13271b to 81fc190 Compare August 4, 2023 16:05

orbea force-pushed the rust-musl-1.2.4 branch from 81fc190 to 5033d91 Compare August 7, 2023 00:53

orbea closed this Aug 18, 2023

dev-lang/rust: fix the build with musl-1.2.4 #31205

dev-lang/rust: fix the build with musl-1.2.4 #31205

Conversation

orbea commented May 28, 2023 • edited

gentoo-bot commented May 28, 2023

Pull Request assignment

Linked bugs

gentoo-repo-qa-bot commented May 28, 2023

Pull request CI report

vimproved commented Jun 1, 2023 • edited

orbea commented Jun 1, 2023 • edited

gentoo-repo-qa-bot commented Jun 1, 2023

Pull request CI report

orbea commented Jun 5, 2023

gentoo-repo-qa-bot commented Jun 5, 2023

Pull request CI report

gentoo-repo-qa-bot commented Jun 5, 2023

Pull request CI report

gentoo-repo-qa-bot commented Jun 8, 2023

Pull request CI report

orbea commented Jun 9, 2023

gentoo-repo-qa-bot commented Jun 9, 2023

Pull request CI report

vimproved commented Jun 10, 2023

orbea commented Jun 10, 2023

vimproved commented Jun 11, 2023

vimproved commented Jun 11, 2023

orbea commented Jun 11, 2023

gentoo-repo-qa-bot commented Jun 29, 2023

Pull request CI report

LinuxUserGD commented Jul 8, 2023

vimproved commented Jul 8, 2023

orbea commented Jul 9, 2023

vimproved commented Jul 9, 2023

orbea commented Jul 9, 2023 • edited

12101111 commented Jul 9, 2023

vimproved commented Jul 12, 2023 • edited

orbea commented Jul 13, 2023

vimproved commented Jul 13, 2023 • edited

orbea commented Jul 16, 2023

gentoo-repo-qa-bot commented Jul 16, 2023

Pull request CI report

gentoo-repo-qa-bot commented Jul 17, 2023

Pull request CI report

richfelker commented Jul 20, 2023

gentoo-repo-qa-bot commented Jul 20, 2023

Pull request CI report

vimproved commented Jul 21, 2023

gentoo-repo-qa-bot commented Jul 21, 2023

Pull request CI report

gentoo-repo-qa-bot commented Jul 24, 2023

Pull request CI report

gentoo-repo-qa-bot commented Aug 4, 2023

Pull request CI report

orbea commented Aug 4, 2023

gentoo-repo-qa-bot commented Aug 7, 2023

Pull request CI report

orbea commented May 28, 2023 •

edited

vimproved commented Jun 1, 2023 •

edited

orbea commented Jun 1, 2023 •

edited

orbea commented Jul 9, 2023 •

edited

vimproved commented Jul 12, 2023 •

edited

vimproved commented Jul 13, 2023 •

edited