With GSM7 encoding, packedlen is the length of the unpacked string
after expanding septets to octets so it will be ~14% bigger than
the original string length. This means we have to be careful not
to copy too much data into the PDU buffer.
Similar issues exist in other branches of the same function.
Thanks rhezashan@gmail.com for the report.
[debug] parsing PDU (0)...
[debug] no SMSC address given
[debug] submit type PDU detected
[debug] message reference: 1
[debug] address size: 1 digits (1 bytes)
[debug] number parsed: 00
[debug] validity available, format relative
[debug] PID: 0
[debug] user data encoding is GSM7
[debug] user data length: 0 elements
[debug] user data length: 0 bytes
==125780== Command: ./build/test/mmsmspdu --pdu=00F101010C0000000000 --verbose
==125780==
==125780== Invalid read of size 1
==125780== at 0x10B422: mm_sms_part_3gpp_new_from_binary_pdu (mm-sms-part-3gpp.c:698)
==125780== by 0x10BF57: mm_sms_part_3gpp_new_from_pdu (mm-sms-part-3gpp.c:368)
==125780== by 0x10A44D: main (mmsmspdu.c:242)
==125780== Address 0x519988a is 0 bytes after a block of size 10 alloc'd
==125780== at 0x48455EF: calloc (vg_replace_malloc.c:1328)
==125780== by 0x49DF6C0: g_malloc0 (in /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7400.2)
==125780== by 0x48ABD24: mm_utils_hexstr2bin (mm-common-helpers.c:1884)
==125780== by 0x10BF36: mm_sms_part_3gpp_new_from_pdu (mm-sms-part-3gpp.c:362)
==125780== by 0x10A44D: main (mmsmspdu.c:242)
==101461== Command: ./build/test/mmsmspdu --pdu=004100010100014B00002E --verbose
==101461==
[debug] parsing PDU (0)...
[debug] no SMSC address given
[debug] submit type PDU detected
[debug] message reference: 0
[debug] address size: 1 digits (1 bytes)
[debug] number parsed: 00
[debug] PID: 1
[debug] user data encoding is GSM7
[debug] user data length: 0 elements
[debug] user data length: 0 bytes
[debug] decoding SMS text with 4294967294 elements
Based on a patch from Michal Mazur <mkm@semihalf.com>.
Before the actual number digits there is always a Type of Address byte
that we were not considering during the size check.
[debug] parsing PDU (0)...
[debug] no SMSC address given
[debug] deliver type PDU detected
[debug] address size: 1 digits (1 bytes)
==90832== Command: ./build/test/mmsmspdu --pdu=001C011C --verbose
==90832==
==90832== Invalid read of size 1
==90832== at 0x10AC90: sms_semi_octets_to_bcd_string (mm-sms-part-3gpp.c:71)
==90832== by 0x10AC90: sms_decode_address (mm-sms-part-3gpp.c:157)
==90832== by 0x10B0C5: mm_sms_part_3gpp_new_from_binary_pdu (mm-sms-part-3gpp.c:512)
==90832== by 0x10BF77: mm_sms_part_3gpp_new_from_pdu (mm-sms-part-3gpp.c:368)
==90832== by 0x10A44D: main (mmsmspdu.c:242)
==90832== Address 0x5199874 is 0 bytes after a block of size 4 alloc'd
==90832== at 0x48455EF: calloc (vg_replace_malloc.c:1328)
==90832== by 0x49DF6C0: g_malloc0 (in /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7400.2)
==90832== by 0x48ABD24: mm_utils_hexstr2bin (mm-common-helpers.c:1884)
==90832== by 0x10BF56: mm_sms_part_3gpp_new_from_pdu (mm-sms-part-3gpp.c:362)
==90832== by 0x10A44D: main (mmsmspdu.c:242)
[debug] parsing PDU (0)...
[debug] no SMSC address given
[debug] status report type PDU detected
[debug] message reference: 191
[debug] address size: 0 digits (0 bytes)
==78906== Command: ./build/test/mmsmspdu --pdu=000ABF00 --verbose
==78906==
==78906== Invalid read of size 1
==78906== at 0x10AA80: sms_decode_address (mm-sms-part-3gpp.c:132)
==78906== by 0x10AF7C: mm_sms_part_3gpp_new_from_binary_pdu (mm-sms-part-3gpp.c:507)
==78906== by 0x10BE17: mm_sms_part_3gpp_new_from_pdu (mm-sms-part-3gpp.c:368)
==78906== by 0x10A44D: main (mmsmspdu.c:202)
==78906== Address 0x5199874 is 0 bytes after a block of size 4 alloc'd
==78906== at 0x48455EF: calloc (vg_replace_malloc.c:1328)
==78906== by 0x49DF6C0: g_malloc0 (in /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7400.2)
==78906== by 0x48ABD24: mm_utils_hexstr2bin (mm-common-helpers.c:1884)
==78906== by 0x10BDF6: mm_sms_part_3gpp_new_from_pdu (mm-sms-part-3gpp.c:362)
==78906== by 0x10A44D: main (mmsmspdu.c:202)
g_date_time_new, and g_date_time_new_utc return NULL if inputs are out
of range, and currently mm_new_iso8601_time passes the GDateTime created
by those two functions to date_time_format_iso8601 without checking for
NULL values, causing a g_date_time_format_iso8601 crash if PDU data is
corrupted with wrong date.
To prevent this, mm_new_iso8601_time now can return NULL and set a new
GError if GDateTime created by g_date_time_new is NULL.
Fixes#546
If the conversion is not fully compatible, the user of the method
needs to request transliteration enabled explicitly in order to avoid
returning errors in this method.
Until now, this method would automatically apply transliteration;
i.e. replacing characters with '?' when no direct translation was
available.
We can attempt to do that transliteration on strings that are not
critical, e.g. the operator name reported by the network. But we
should not do that on other types of strings, e.g. on SMS contents
that may really have additional purposes than just being
human-readable.
This commit makes the transliteration option to be explicitly
requested by the caller.
There's no point in adding a quoting option to this method; if the
caller needs the appended string quoted, it should quote it before
passing it to this method.
It was nowhere used anyway.
Despite 3GPP TS 23.038 specifies that Unicode SMS messages are
encoded in UCS-2, UTF-16 encoding is commonly used instead on many
modern platforms to allow encoding code points that fall outside the
Basic Multilingual Plane (BMP), such as Emoji.
Update the logic to always use UTF-16 instead of UCS-2 when creating
or parsing PDUs (even if we always report as sending or receiving
UCS-2). For all purposes, UCS-2 is considered a subset of UTF-16
(assuming that code points out of the [U+0000,U+D7FF] and
[U+E000,U+FFFF] ranges are not applicable in UCS-2).
Fixes https://gitlab.freedesktop.org/mobile-broadband/ModemManager/-/issues/250
mm-sms-part-3gpp.c: In function ‘mm_sms_part_3gpp_new_from_binary_pdu’:
mm-sms-part-3gpp.c:721:9: error: enumeration value ‘MM_SMS_ENCODING_8BIT’ not handled in switch [-Werror=switch-enum]
721 | switch (user_data_encoding) {
| ^~~~~~
mm-sms-part-3gpp.c: In function ‘mm_sms_part_3gpp_get_submit_pdu’:
mm-sms-part-3gpp.c:917:5: error: enumeration value ‘MM_SMS_ENCODING_UNKNOWN’ not handled in switch [-Werror=switch-enum]
917 | switch (mm_sms_part_get_encoding (part)) {
| ^~~~~~
mm-sms-part-3gpp.c:917:5: error: enumeration value ‘MM_SMS_ENCODING_8BIT’ not handled in switch [-Werror=switch-enum]
mm-sms-part-3gpp.c: In function ‘mm_sms_part_3gpp_new_from_binary_pdu’:
mm-sms-part-3gpp.c:607:9: error: enumeration value ‘MM_SMS_ENCODING_UNKNOWN’ not handled in switch [-Werror=switch-enum]
607 | switch (user_data_encoding) {
| ^~~~~~
mm-sms-part-3gpp.c:714:9: error: enumeration value ‘MM_SMS_ENCODING_UNKNOWN’ not handled in switch [-Werror=switch-enum]
714 | switch (user_data_encoding) {
| ^~~~~~
Depsite 3GPP TS 23.038 specifies that Unicode SMS messages are encoded in
UCS-2, UTF-16 encoding is commonly used instead on many modern platforms to
allow encoding code points that fall outside the Basic Multilingual Plane
(BMP), such as Emoji. Most of the UCS-2 code points are identical to their
equivalent UTF-16 code points. In UTF-16, non-BMP code points are encoded in a
pair of surrogate code points (i.e. a high surrogate in 0xD800..0xDBFF,
followed by a low surrogate in 0xDC00..0xDFFF). An isolated surrogate code
point has no general interpretation in UTF-16, but could be a valid (though
unmapped) code point in UCS-2.
This patch modifies the 3GPP SMS decoding to first try UTF-16BE and then fall
back to UCS-2BE on failure. If both fail, an empty string is returned
instead of a NULL pointer.
Instead of having a method that returns the expected length after the
conversion and the amount of input UTF-8 characters that couldn't be
converted to the given charset, simplify the logic and just define a
method that returns a boolean specifying whether the conversion is
possible or not.
Also, include unit tests.
Guido Günther
Kyle Evans
Eric Caruso
Aleksander Morgado
Andrey Skvortsov
Dmitry Skorykh
Carlo Lobrano
Frederic Martinsons
Clayton Craft
Aleksander Morgado
Maxim Anisimov
amol.lad
Ville Skyttä
Ben Chan
Aleksander Morgado