Add in a RtlGenRandom fallback for non-UWP Windows #337

chutten · 2023-02-16T20:14:25Z

In some instances BCryptRandom will fail when RtlGenRandom will work. On UWP, we might be unable to actually use RtlGenRandom.

Thread the needle and use RtlGenRandom when we have to, when we're able.

See also rust-lang/rust#108060

Fixes #314

chutten · 2023-02-16T20:15:03Z

Builds and tests on my Windows machine, fwiw. Gonna try patching Firefox and see where that leaves us.

chutten · 2023-02-16T21:38:15Z

(Whoops. Built and tested it on my Windows machine within WSL which isn't at all Windows. Lemme just tweak and rebuild that real quick.)

yjugl

Thank you @chutten for working on this! Here are my remarks on the Windows aspects of this change.

yjugl · 2023-02-17T08:54:22Z

src/windows.rs

-            // system does not have a way to express this yet.
-            let code = unsafe { NonZeroU32::new_unchecked(code) };
-            return Err(Error::from(code));
+        if ret > 0 {


if ret > 0 is not properly checking for failure of BCryptGenRandom here. NTSTATUS is a signed 32-bit value and the two most significant bits encode whether it is a success, information, warning, or error. Checking for success is typically "success or information" (which translates to if ret >= 0) while failure is typically "warning or error" (which translates to if ret < 0).

Be careful that the code before this patch has defined the return value from BCryptGenRandom as u32 instead of i32, which I would say is incorrect (wrt C definitions) or at best confusing (but this is what allowed ret >> 30 to compare with 0b11 instead of -1, I guess). So unless you also change that u32 to i32 (which I would recommend), my if ret < 0 suggestion would actually not work written like that here.

For more details check https://learn.microsoft.com/en-us/windows-hardware/drivers/kernel/using-ntstatus-values and/or https://searchfox.org/mozilla-central/source/__GENERATED__/__RUST_STDLIB__/std/src/sys/windows/c.rs#292-296 .

Ah, and here I thought I was clever by adapting rust-lang's nt_success... but without thinking about it first. Lemme give this another go, paying more close attention to the types and values (that's what I get for rushing it at the end of my day yesterday)

yjugl · 2023-02-17T09:03:43Z

src/windows.rs

+
+#[cfg(target_vendor = "uwp")]
+#[inline(never)]
+fn fallback_rng(_chunk: &mut [MaybeUninit<u8>]) -> u8 {


The return type does not seem to match the return value here? RtlGenRandom return no more information than TRUE or FALSE (respectively a non-zero or a zero u8 value), so I think fallback_rng could return bool or Maybe<bool> depending on whether or not we want to be able to distinguish "RtlGenRandom is unavailable" from "RtlGenRandom failed". Given that at the moment we do not need to be able to distinguish since the error we propagate will be the one from BCryptGenRandom, maybe bool is enough here.

Note: TRUE in RtlGenRandom means success so its return value is currently incorrectly checked on line 46.

yjugl · 2023-02-17T09:25:31Z

src/windows.rs

+                // We zeroize the highest bit, so the error code will reside
+                // inside the range designated for OS codes.
+                let code = ret ^ (1 << 31);
+                // SAFETY: the second highest bit is always equal to one,


The code before this patch considered that warning values for NTSTATUS are not a failure (that's why it was checking for the two most significant bits of the return value from BCryptGenRandom). This is what this remark explains (although I think it's a mistake to consider that warning values are not a failure).

If in this patch we consider that warning values for NTSTATUS are a failure (and I think we should), then this remark does not hold anymore and should be updated. But I think we can still use NonZeroU32::new_unchecked, because NTSTATUS warning values start with 0x80000001 STATUS_GUARD_PAGE_VIOLATION, so there should be no 0x80000000. Or, for extra safety guarantees, we could map a theoretical 0x80000000 to 0xC0000001 STATUS_UNSUCCESSFUL.

I'm a fan of patches doing one thing at a time, so I'd prefer to restrain ourselves to not having opinions about the return values (like I accidentally did). If we wish to propose a change so warnings are failures as well (now that we have a fallback to attempt), I feel like that ought to be its own Issue and PR.

OK, good for me then.

I agree that our handling of error values here is not ideal. We should mimic what NT_SUCCESS(Status) does and only treat values 0 − 0x7FFFFFFF as "succeeding". However, this is better addressed in a followup PR.

yjugl · 2023-02-17T09:28:46Z

src/windows.rs

+        if ret > 0 {
+            // Failed. Try RtlGenRandom as a fallback.
+            let fallback_ret = fallback_rng(chunk);
+            // NTSTATUS codes use the two highest bits for severity status.


This comment is outdated / does not match with what the code does.

chutten · 2023-02-17T15:01:33Z

And to complete the see also links, see also: https://bugzilla.mozilla.org/show_bug.cgi?id=1816953

josephlr · 2023-02-18T01:13:31Z

src/windows.rs

@@ -21,6 +21,12 @@ extern "system" {
    ) -> u32;
 }

+extern "system" {
+    // Forbidden when targetting UWP


Instead of a comment, we should have a #[cfg(not(target_vendor = "uwp"))] attribute. Something like:

#[link(name = "bcrypt")] extern "system" { fn BCryptGenRandom( hAlgorithm: *mut c_void, pBuffer: *mut u8, cbBuffer: u32, dwFlags: u32, ) -> i32; } #[cfg(not(target_vendor = "uwp"))] #[link(name = "advapi32")] extern "system" { #[link_name = "SystemFunction036"] pub fn RtlGenRandom(RandomBuffer: *mut c_void, RandomBufferLength: u32) -> u8; }

Note that the buffer is of type PVOID and we need to make sure we link to Advapi32

josephlr · 2023-02-18T01:17:51Z

src/windows.rs

+            // Failed. Try RtlGenRandom as a fallback.
+            let success = fallback_rng(chunk);


This introduces a larger diff and more indentation than needed. Could we just do:

// NTSTATUS codes use the two highest bits for severity status. if ret >> 30 == 0b11 { // Failed. Try RtlGenRandom as a fallback. #[cfg(not(target_vendor = "uwp"))] if unsafe { RtlGenRandom(chunk.as_mut_ptr() as *mut c_void, chunk.len() as u32) } != 0 { continue; } // We zeroize the highest bit, so the error code will reside // inside the range designated for OS codes. let code = ret ^ (1 << 31); ...

chutten · 2023-02-21T19:32:29Z

Pushed to run a few tests on moz's tryservers here: https://treeherder.mozilla.org/jobs?repo=try&revision=81ff7509fa334c3bf3d2fe0fcce8ca9f2376a6a7

(I expect some oranges here and there from flaky tests, but for it to overall work out).

Attn @yjugl for if you want to doublecheck that this more-final version fixes your particular testcase.

newpavlov · 2023-02-22T11:24:34Z

@chutten
You need to wrap the if in a block to pass our MSRV tests.

chutten · 2023-02-22T14:13:35Z

@chutten You need to wrap the if in a block to pass our MSRV tests.

Oh, because of the cfg? Will do.

In some instances BCryptRandom will fail when RtlGenRandom will work. On UWP, we might be unable to actually use RtlGenRandom. Thread the needle and use RtlGenRandom when we have to, when we're able. See also rust-lang/rust#108060 Fixes rust-random#314

chutten · 2023-02-22T15:46:31Z

There, that ought to do it. I pulled down 1.36 and it compiles with the extra block. (For anyone reading, I did it via rustup install 1.36.0 then rm Cargo.lock (because 1.36 can't parse the modern lockfile) then cargo +1.36.0 build)

src/windows.rs

Co-authored-by: Artyom Pavlov <newpavlov@gmail.com>

src/windows.rs

yjugl · 2023-02-23T11:53:23Z

Attn @yjugl for if you want to doublecheck that this more-final version fixes your particular testcase.

I was able to confirm that on a Windows 7 machine where bcryptprimitives.dll is unavailable, Uuid::new_v4() no longer panics and successfully generates a random uuid. Thank you!

chutten · 2023-02-23T14:46:56Z

@newpavlov @josephlr sooooo... when would it be polite to ask about your plans vis a vis cutting a v0.2.9...? : D

notgull · 2023-02-25T23:12:23Z

Bit late to the party, but you probably shouldn't be matching on target_vendor. See rust-lang/rust#100343

chutten · 2023-02-27T14:34:05Z

Bit late to the party, but you probably shouldn't be matching on target_vendor. See rust-lang/rust#100343

TIL. The specific proposed guidance for UWP will be to replace target_vendor = "uwp" with target_abi = "uwp". Plenty of time for that yet, by the looks of things. e.g. the code we're adapting still uses target_vendor for now, at least.

chutten force-pushed the RtlGenRandom branch 2 times, most recently from b3cea33 to ab8c2d5 Compare February 16, 2023 21:38

yjugl suggested changes Feb 17, 2023

View reviewed changes

chutten force-pushed the RtlGenRandom branch from ab8c2d5 to 16250fd Compare February 17, 2023 14:47

chutten requested a review from yjugl February 17, 2023 14:48

josephlr reviewed Feb 18, 2023

View reviewed changes

chutten force-pushed the RtlGenRandom branch from 16250fd to ed34e2b Compare February 21, 2023 19:25

chutten requested review from josephlr and removed request for yjugl February 21, 2023 19:30

josephlr approved these changes Feb 21, 2023

View reviewed changes

chutten force-pushed the RtlGenRandom branch from ed34e2b to 6609385 Compare February 22, 2023 15:43

newpavlov reviewed Feb 22, 2023

View reviewed changes

src/windows.rs Outdated Show resolved Hide resolved

style suggestion

8060d8e

Co-authored-by: Artyom Pavlov <newpavlov@gmail.com>

chutten commented Feb 22, 2023

View reviewed changes

src/windows.rs Outdated Show resolved Hide resolved

appease clippy

baf37d9

josephlr merged commit c526c89 into rust-random:master Feb 22, 2023

chutten deleted the RtlGenRandom branch February 22, 2023 20:44

This was referenced Feb 22, 2023

Make the RNG fall back to using an algorithm handle if BCryptGenRandom fails #330

Closed

windows: move BCryptGenRandom to helper function #318

Closed

Windows: should we fall back to RtlGenRandom if BCryptGenRandom fails #314

Closed

newpavlov mentioned this pull request Apr 2, 2023

Release v0.2.9 #354

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add in a RtlGenRandom fallback for non-UWP Windows #337

Add in a RtlGenRandom fallback for non-UWP Windows #337

chutten commented Feb 16, 2023

chutten commented Feb 16, 2023

chutten commented Feb 16, 2023

yjugl left a comment

yjugl Feb 17, 2023

chutten Feb 17, 2023

yjugl Feb 17, 2023 •

edited

yjugl Feb 17, 2023 •

edited

chutten Feb 17, 2023

yjugl Feb 17, 2023

josephlr Feb 18, 2023

yjugl Feb 17, 2023

chutten commented Feb 17, 2023

josephlr Feb 18, 2023 •

edited

josephlr Feb 18, 2023

chutten commented Feb 21, 2023

newpavlov commented Feb 22, 2023

chutten commented Feb 22, 2023

chutten commented Feb 22, 2023

yjugl commented Feb 23, 2023

chutten commented Feb 23, 2023

notgull commented Feb 25, 2023

chutten commented Feb 27, 2023

		// Failed. Try RtlGenRandom as a fallback.
		let success = fallback_rng(chunk);

Add in a RtlGenRandom fallback for non-UWP Windows #337

Add in a RtlGenRandom fallback for non-UWP Windows #337

Conversation

chutten commented Feb 16, 2023

chutten commented Feb 16, 2023

chutten commented Feb 16, 2023

yjugl left a comment

Choose a reason for hiding this comment

yjugl Feb 17, 2023

Choose a reason for hiding this comment

chutten Feb 17, 2023

Choose a reason for hiding this comment

yjugl Feb 17, 2023 • edited

Choose a reason for hiding this comment

yjugl Feb 17, 2023 • edited

Choose a reason for hiding this comment

chutten Feb 17, 2023

Choose a reason for hiding this comment

yjugl Feb 17, 2023

Choose a reason for hiding this comment

josephlr Feb 18, 2023

Choose a reason for hiding this comment

yjugl Feb 17, 2023

Choose a reason for hiding this comment

chutten commented Feb 17, 2023

josephlr Feb 18, 2023 • edited

Choose a reason for hiding this comment

josephlr Feb 18, 2023

Choose a reason for hiding this comment

chutten commented Feb 21, 2023

newpavlov commented Feb 22, 2023

chutten commented Feb 22, 2023

chutten commented Feb 22, 2023

yjugl commented Feb 23, 2023

chutten commented Feb 23, 2023

notgull commented Feb 25, 2023

chutten commented Feb 27, 2023

yjugl Feb 17, 2023 •

edited

yjugl Feb 17, 2023 •

edited

josephlr Feb 18, 2023 •

edited