Number overflow while rejecting a too-long length

- The length reader checked its 4-byte limit after doing the maths.
So, a 5th byte was multiplied in and overflows before the value
was thrown away. But it was not noticed, because we thrown it away
and returned -1.

- The fix moves iteration++ and if (iteration > 4) before the
calculation, preventing the overflow.

Pick-to: 6.11 6.8 6.12
Task-number: QTBUG-149571
Change-Id: I78c2063c7e01af3e8f196f824216d31de92fab48
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>