Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Account for existing but empty cpus file #2151

Merged
merged 2 commits into from Feb 20, 2024

Conversation

angelapwen
Copy link
Contributor

@angelapwen angelapwen commented Feb 15, 2024

If a CPU file exists but is empty, previously we reported this file with a CPU count of 1, which resulted in a single-threaded run.

Fixes #2139.

Merge / deployment checklist

  • Confirm this change is backwards compatible with existing workflows.
  • Confirm the readme has been updated if necessary.
  • Confirm the changelog has been updated if necessary.

@angelapwen angelapwen changed the title Account for non-empty cpus file with whitespace chars Account for existing but empty cpus file Feb 15, 2024
@angelapwen angelapwen marked this pull request as ready for review February 15, 2024 14:42
@angelapwen angelapwen requested a review from a team as a code owner February 15, 2024 14:42
Copy link
Contributor

@henrymercer henrymercer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing this. Couple of comments, and additionally, could we add a unit test?

@@ -453,9 +453,9 @@ function getCgroupCpuCountFromCpus(

let cpuCount = 0;
// Comma-separated numbers and ranges, for eg. 0-1,3
const cpusString = fs.readFileSync(cpusFile, "utf-8");
const cpusString = fs.readFileSync(cpusFile, "utf-8").trim();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that if the file is empty, we'll return 0, but I think we probably want to return undefined?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That could work too — we're filtering out all counts that are undefined, or <= 0, so they kind of have the same meaning. But I don't mind making it return undefined if that's more clear.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, that's good that we were already filtering out <=0. I think I'd find undefined slightly more clear, but it's not critical.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The updated logic returns undefined 😄

src/util.ts Outdated
for (const token of cpusString.split(",")) {
if (!token.includes("-")) {
if (token.length > 0 && !token.includes("-")) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does it mean if the token is empty here? If it's not valid, we might want to log then bail out and return undefined. If it's valid, is the else block doing the right thing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the file is empty here (or just has a newline character as in the linked issue) the trim should make it an empty string. Then .split() will also just return a single string that's empty. Previously we were incrementing the count because an empty string doesn't include -. You're right that we shouldn't be entering the else block though... we should just be bailing out. I'll update this and look at adding the unit test 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay — I added unit tests for the case where the file exists and the number of CPUs should be calculated, the case where the file doesn't exist, and the case where the file exists but only has a newline char in it!

If a CPU file exists but is empty, previously we reported this file with a CPU count of 1, which resulted in a single-threaded run.
Copy link
Contributor

@henrymercer henrymercer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@angelapwen angelapwen merged commit 592977e into github:main Feb 20, 2024
297 checks passed
@angelapwen angelapwen deleted the fix-cpu-group-bug branch February 20, 2024 11:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants