Perf | avoid boxing of SqlGuid in SqlBuffer (#2300) #2306

wilbit · 2024-01-18T13:05:01Z

fixes #2300

wilbit · 2024-01-18T13:18:30Z

@dotnet-policy-service agree company="Intrigma"

Wraith2

As long as the tests pass this looks good to me.

codecov · 2024-01-18T20:56:16Z

Codecov Report

Attention: 3 lines in your changes are missing coverage. Please review.

Comparison is base (900d051) 72.49% compared to head (54328bd) 72.44%.

Files	Patch %	Lines
...qlClient/src/Microsoft/Data/SqlClient/SqlBuffer.cs	0.00%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2306      +/-   ##
==========================================
- Coverage   72.49%   72.44%   -0.06%     
==========================================
  Files         310      310              
  Lines       61868    61867       -1     
==========================================
- Hits        44854    44821      -33     
- Misses      17014    17046      +32

Flag	Coverage Δ
addons	`92.88% <ø> (ø)`
netcore	`76.63% <25.00%> (-0.07%)`	⬇️
netfx	`69.89% <25.00%> (-0.06%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

David-Engel

My only concern is that SqlBuffer.cs lines 818 and 825-828 aren't exercised by tests and I don't know enough about the changes to know if they are correct or not. Are those lines easily covered by extending one of the existing tests?

wilbit · 2024-01-19T06:38:14Z

Honestly, I have not found any unit tests for SqlBuffer, but Codecov says it is 85.45% covered in main branch) I suppose I can add unit tests for writing/reading of values, but I'm not sure how to deal with boxing (over reflection and checking of object _object field, probably 🙃). Do you know any alternatives, @Wraith2?

I prepare another PR for the similar issue (#2300 (comment)) with boxed SqlBinary. Is it acceptable if I add unit tests for SqlBuffer.Guid, SqlBuffer.SqlGuid and SqlBuffer.SqlBinary in the upcoming PR, @David-Engel ?

Wraith2 · 2024-01-19T08:32:55Z

My only concern is that SqlBuffer.cs lines 818 and 825-828 aren't exercised by tests and I don't know enough about the changes to know if they are correct or not. Are those lines easily covered by extending one of the existing tests?

The logic to set the unmanaged _value buffer is not currently needed and we can't write a test for it without causing the condition this PR fixes to be possible through the library or through the use of complex reflection. The code is a safety net in case someone calls the setter in future because without this code being present (even if unused) if someone adds a call to the setter with a non-null value it'll compile and run but won't do the correct operation. An exception would require a lot of plumbing to get from here to the tds parser correctly and an assertion would be of no used because we run tests in Release mode.

David-Engel · 2024-01-19T17:21:58Z

I prepare another PR for the similar issue (#2300 (comment)) with boxed SqlBinary. Is it acceptable if I add unit tests for SqlBuffer.Guid, SqlBuffer.SqlGuid and SqlBuffer.SqlBinary in the upcoming PR?

If not too difficult to test, yes. Not sure if it's easy (or completely necessary) based on what Wraith2 said.

The logic to set the unmanaged _value buffer is not currently needed and we can't write a test for it without causing the condition this PR fixes to be possible through the library or through the use of complex reflection. The code is a safety net in case someone calls the setter in future because without this code being present (even if unused) if someone adds a call to the setter with a non-null value it'll compile and run but won't do the correct operation. An exception would require a lot of plumbing to get from here to the tds parser correctly and an assertion would be of no used because we run tests in Release mode.

Thanks for the clarification. I'm fine with the changes, then.

DavoudEshtehari · 2024-01-19T20:52:06Z

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBuffer.cs

@@ -815,14 +815,17 @@ internal SqlGuid SqlGuid
                }
                else if (StorageType.SqlGuid == _type)
                {
-                    return IsNull ? SqlGuid.Null : (SqlGuid)_object;
+                    return IsNull ? SqlGuid.Null : new SqlGuid(_value._guid);


Have you validated the outcome of this change by comparing the impact of creating a new instance versus unboxing? Overall, I'm not seeing a notable increase in efficiency.

The efficiency isn't gained here, This is just a knockon effect. The efficiency is gained by not assigning a struct SqlNull.Null to an object which causes an unavoidable box every time it happens.

I just did perf testing and there are some performance and memory degradations brought to this getter by the PR, so, @DavoudEshtehari highlighted a valid point 👍.
I will do more perf testing and try to find a better solution in terms of memory and performance.

DavoudEshtehari · 2024-01-19T21:04:00Z

src/Microsoft.Data.SqlClient/src/Microsoft/Data/SqlClient/SqlBuffer.cs

                }
                return (SqlGuid)SqlValue; // anything else we haven't thought of goes through boxing.
            }
            set
            {
                Debug.Assert(IsEmpty, "setting value a second time?");
-                _object = value;
+                if (!value.IsNull)


If _value is already set and the new value is set to null, there might be an inconsistency where _value still points to a non-null value while _isNull indicates that it's null. This could lead to confusion and unexpected behavior in your code. It's crucial to ensure that _value and _isNull are consistent and accurately represent the state of the variable. IMO, If the intention is to set the variable to null, both _value and _isNull should be appropriately updated to reflect this.

_value is a value type. it is the value it doesn't point to it. If the value is non-empty and then is updated to null it doesn't matter if there is a non-empty value in the variable because it won't be read through any of the code paths.

The class definitely requires its own unit tests.
I have introduced the bug in this PR in SqlBuffer.get_SqlValue 😥:

case StorageType.SqlBinary: case StorageType.SqlGuid: return _object;

I don't understand what is the right place to put unit tests.
I thought it is src\Microsoft.Data.SqlClient\tests\FunctionalTests\Microsoft.Data.SqlClient.Tests.csproj but this project does not target all targets of Microsoft.Data.SqlClient. Particularly, it targets net60 but misses net80 (I'm sure you are aware, but just want to remind that SqlGuid class has differences between these platforms).
May you point me to the right direction? I want to fix the bug ASAP, @Wraith2 , @David-Engel , @DavoudEshtehari

We don't have a net8 build currently. Write the test and use the correct attribute or preprocessor definition so that when the net8 build does exist (possibly soon-ish) it will be enabled and work.

Functional tests are run without a database connection. Anything that can be checked without running a query can go in there. Manual tests are run with a database connection, anything which can't be in functional tests belongs there.

I vote for a revert given that a release is planned for 24 Jan - just my 2 cents

Here the PR to fix that: #2310

@Wraith2 suggested to revert this PR too. See #2310 (comment)

I just did a PR to revert.

Now we just have to wait for the MS team to work their way through the email notifications, wonder what we've all been doing all weekend, and then approve it 😁

introduced by PR dotnet#2306

This reverts commit 9602271.

Perf | avoid boxing of SqlGuid in SqlBuffer (dotnet#2300)

54328bd

wilbit mentioned this pull request Jan 18, 2024

High memory traffic because of boxed SqlGuid struct #2300

Open

Wraith2 approved these changes Jan 18, 2024

View reviewed changes

DavoudEshtehari added this to the 5.2.0-preview5 milestone Jan 19, 2024

DavoudEshtehari added the 📈 Performance Use this label for performance improvement activities label Jan 19, 2024

DavoudEshtehari added this to In progress in SqlClient v5.2 via automation Jan 19, 2024

DavoudEshtehari requested review from David-Engel, JRahnama, DavoudEshtehari and H-Yeo January 19, 2024 00:26

David-Engel reviewed Jan 19, 2024

View reviewed changes

David-Engel approved these changes Jan 19, 2024

View reviewed changes

DavoudEshtehari moved this from In progress to Review in progress in SqlClient v5.2 Jan 19, 2024

DavoudEshtehari reviewed Jan 19, 2024

View reviewed changes

DavoudEshtehari approved these changes Jan 19, 2024

View reviewed changes

SqlClient v5.2 automation moved this from Review in progress to Reviewer approved Jan 19, 2024

DavoudEshtehari merged commit 9602271 into dotnet:main Jan 19, 2024
148 checks passed

SqlClient v5.2 automation moved this from Reviewer approved to Done Jan 19, 2024

wilbit deleted the issue-2300 branch January 21, 2024 15:08

wilbit added a commit to Intrigma/SqlClient that referenced this pull request Jan 21, 2024

Fix | bugs in SqlBuffer.get_Guid & get_SqlValue (#TBD)

0744802

introduced by PR dotnet#2306

wilbit added a commit to Intrigma/SqlClient that referenced this pull request Jan 21, 2024

Fix | bugs in SqlBuffer.get_Guid & get_SqlValue (dotnet#2300)

daec178

introduced by PR dotnet#2306

wilbit mentioned this pull request Jan 21, 2024

Fix | SqlBuffer.SqlGuid #2310

Merged

ErikEJ added a commit to ErikEJ/SqlClient that referenced this pull request Jan 22, 2024

Revert "Perf | avoid boxing of SqlGuid in SqlBuffer (dotnet#2306)"

a270a02

This reverts commit 9602271.

DavoudEshtehari pushed a commit that referenced this pull request Jan 23, 2024

Revert "Perf | avoid boxing of SqlGuid in SqlBuffer (#2306)" (#2311)

4ca71d9

This reverts commit 9602271.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Perf | avoid boxing of SqlGuid in SqlBuffer (#2300) #2306

Perf | avoid boxing of SqlGuid in SqlBuffer (#2300) #2306

wilbit commented Jan 18, 2024 •

edited

wilbit commented Jan 18, 2024

Wraith2 left a comment

codecov bot commented Jan 18, 2024 •

edited

David-Engel left a comment

wilbit commented Jan 19, 2024 •

edited

Wraith2 commented Jan 19, 2024

David-Engel commented Jan 19, 2024

DavoudEshtehari Jan 19, 2024

Wraith2 Jan 19, 2024

wilbit Jan 20, 2024 •

edited

DavoudEshtehari Jan 19, 2024

Wraith2 Jan 19, 2024

wilbit Jan 20, 2024

wilbit Jan 20, 2024 •

edited

Wraith2 Jan 20, 2024

ErikEJ Jan 21, 2024

wilbit Jan 21, 2024

wilbit Jan 22, 2024

ErikEJ Jan 22, 2024

Wraith2 Jan 22, 2024

Perf | avoid boxing of SqlGuid in SqlBuffer (#2300) #2306

Perf | avoid boxing of SqlGuid in SqlBuffer (#2300) #2306

Conversation

wilbit commented Jan 18, 2024 • edited

wilbit commented Jan 18, 2024

Wraith2 left a comment

Choose a reason for hiding this comment

codecov bot commented Jan 18, 2024 • edited

Codecov Report

David-Engel left a comment

Choose a reason for hiding this comment

wilbit commented Jan 19, 2024 • edited

Wraith2 commented Jan 19, 2024

David-Engel commented Jan 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wilbit Jan 20, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wilbit Jan 20, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wilbit commented Jan 18, 2024 •

edited

codecov bot commented Jan 18, 2024 •

edited

wilbit commented Jan 19, 2024 •

edited

wilbit Jan 20, 2024 •

edited

wilbit Jan 20, 2024 •

edited