QTextCodec: fix int overflow and infinite loop in ICU
The ICU codec computes the required size of the encoded data using
the following code:
int requiredLength =
UCNV_GET_MAX_BYTES_FOR_STRING(length,
ucnv_getMaxCharSize(conv));
The macro expands into:
(((int32_t)(length)+10)*(int32_t)(maxCharSize))
The maxCharSize parameter depends on the used encoding, and can be
up to 8 chars per a code point - that is the case for ISO-2022-KR
codec. Several other codecs also have maxCharSize == 6.
As a result, for such codecs the requiredLength calculation can
overflow even with relatively small input data.
E.g., for maxCharSize == 8, it will be only around 250M input
QChars, so roughly 500 Mb of text. Not even close to INT_MAX.
The overflow itself is already UB, so should be fixed. In practice,
all compilers these days simply provide a negative result. This
negative requiredLength is then passed to the QByteArray
constructor, which creates an empty QByteArray in response.
As a result, the loop that tries to do the encoding keeps generating
U_BUFFER_OVERFLOW_ERROR and tries to resize the bytearray with
size = size * 2. But that’s obviously a no-op when the size is
initially zero. So, we get an infinite loop.
Fix it by expanding the macro and doing the initial requiredLength
calculation in qint64. If the result is larger than INT_MAX, simply
raise a warning and return an empty QByteArray immediately.
We need to discard sizes larger than INT_MAX even in Qt 6, because
ICU internally supports input and output buffers up to INT_MAX
bytes.
Obviously, for Qt 6 we could actually allocate the proper size,
and then rewrite the encoding logic to do it in chunks.
But considering that it's a compatibility module, I opted for a
simpler approach, that can also be picked to 5.15 with minimal
(if any) changes. The chunking can be implemented at the user side.
The size * 2 calculation in the while() loop can overflow or exceed
INT_MAX as well, so the patch applies the similar approach there.
The same problem exists in the convertToUnicode() method: the
output string unconditionally allocates length + 2 QChars, so the
length cannot be larger than INT_MAX / sizeof(QChar) - 2, otherwise
it'll cross the ICU buffer size limitation. Fix it similarly.
[ChangeLog][QTextCodec][ICU] Fixed a bug in an ICU codec that could
loop infinitely when the input to toUnicode() or fromUnicode() was
too long. The updated code now raises a warning and returns an
empty QString/QByteArray immediately. To process long inputs,
implement the chunking of the input data on the client side, using
the toUnicode() and fromUniocode() overloads that take the
ConverterState as the last parameter.
Fixes: QTBUG-149408
Pick-to: 6.12 6.11 6.8 6.5 5.15
Change-Id: Id523b9421b7cb364e48d6fcac2bbce98f968bd32
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>