gh-119182: Add PyUnicodeWriter C API

[vstinner]

• Issue: [C API] Add an efficient public PyUnicodeWriter API #119182

[vstinner]

@erlend-aasland @serhiy-storchaka @encukou: Do you want to review this change?

[erlend-aasland]

Nit; I prefer to use SemBr for paragraphs like this.

Suggested change

	*str* must be Python :class:`str` object. *start* must be greater than or
	equal to 0, and less than or equal to *end*. *end* must be less than or
	equal to *str* length.
	*str* must be Python :class:`str` object.
	*start* must be greater than or equal to 0,
	and less than or equal to *end*.
	*end* must be less than or equal to *str* length.

[serhiy-storchaka]

TIL that this is called SemBr!

Breaking on comma may be too much, but I prefer to break at the sentence boundary.

[vstinner]

@erlend-aasland: I addressed your review. Would you mind to review the updated PR?

[serhiy-storchaka]

Would not be better to raise SystemError to distinguish programming errors from from data driven errors? In all current use cases of _PyUnicodeWriter_WriteSubstring() it is impossible to pass wrong arguments, and if this happens, it is a programming error.

[vstinner]

TypeError is when the user pass the wrong type. Previously, we used SystemError more often. But it seems like the new trend (C API Working Group recommendations) is more to accept a generic PyObject* and raises TypeError if the function gets the wrong type.

[vstinner]

Recent example: PyLong_GetSign() raises TypeError, not SystemError.

[serhiy-storchaka]

Except using TypeError and ValueError instead of SystemError, LGTM.

But I do not insist if other core developers prefer TypeError and ValueError. I just think that SystemError is more appropriate for programming errors in using the C API.

[malemburg]

A more general question in the light of the free threading patches: Are these writer APIs thread-safe ?

[vstinner]

A more general question in the light of the free threading patches: Are these writer APIs thread-safe ?

I did nothing to make them safe, so I would say that they are not thread safe. You must not share a writer between two threads. Use one writer per thread.

[vstinner]

@malemburg @serhiy-storchaka: I modified the implementation to make write operations atomic. Either the whole string is written, or "nothing is written".

In the doc, I wrote "On error, leave the writer unchanged". In practice, it's more subtle, the internal buffer can be enlarged, or its kind (UCS1, UCS2 or UCS4) can change. But from the API consumer point of view, it's as if nothing was written.

I added two unit tests to show that it's still possible to use a writer after an error.

The hack is just to save/restore writer->pos internally in the 2 functions which are not atomic: WriteUTF8() and WriteFormat().

I withdraw my approve because writing cannot be atomic.