g_date_time_new, and g_date_time_new_utc return NULL if inputs are out
of range, and currently mm_new_iso8601_time passes the GDateTime created
by those two functions to date_time_format_iso8601 without checking for
NULL values, causing a g_date_time_format_iso8601 crash if PDU data is
corrupted with wrong date.
To prevent this, mm_new_iso8601_time now can return NULL and set a new
GError if GDateTime created by g_date_time_new is NULL.
Fixes#546
If the conversion is not fully compatible, the user of the method
needs to request transliteration enabled explicitly in order to avoid
returning errors in this method.
Until now, this method would automatically apply transliteration;
i.e. replacing characters with '?' when no direct translation was
available.
We can attempt to do that transliteration on strings that are not
critical, e.g. the operator name reported by the network. But we
should not do that on other types of strings, e.g. on SMS contents
that may really have additional purposes than just being
human-readable.
This commit makes the transliteration option to be explicitly
requested by the caller.
There's no point in adding a quoting option to this method; if the
caller needs the appended string quoted, it should quote it before
passing it to this method.
It was nowhere used anyway.
Despite 3GPP TS 23.038 specifies that Unicode SMS messages are
encoded in UCS-2, UTF-16 encoding is commonly used instead on many
modern platforms to allow encoding code points that fall outside the
Basic Multilingual Plane (BMP), such as Emoji.
Update the logic to always use UTF-16 instead of UCS-2 when creating
or parsing PDUs (even if we always report as sending or receiving
UCS-2). For all purposes, UCS-2 is considered a subset of UTF-16
(assuming that code points out of the [U+0000,U+D7FF] and
[U+E000,U+FFFF] ranges are not applicable in UCS-2).
Fixes https://gitlab.freedesktop.org/mobile-broadband/ModemManager/-/issues/250
mm-sms-part-3gpp.c: In function ‘mm_sms_part_3gpp_new_from_binary_pdu’:
mm-sms-part-3gpp.c:721:9: error: enumeration value ‘MM_SMS_ENCODING_8BIT’ not handled in switch [-Werror=switch-enum]
721 | switch (user_data_encoding) {
| ^~~~~~
mm-sms-part-3gpp.c: In function ‘mm_sms_part_3gpp_get_submit_pdu’:
mm-sms-part-3gpp.c:917:5: error: enumeration value ‘MM_SMS_ENCODING_UNKNOWN’ not handled in switch [-Werror=switch-enum]
917 | switch (mm_sms_part_get_encoding (part)) {
| ^~~~~~
mm-sms-part-3gpp.c:917:5: error: enumeration value ‘MM_SMS_ENCODING_8BIT’ not handled in switch [-Werror=switch-enum]
mm-sms-part-3gpp.c: In function ‘mm_sms_part_3gpp_new_from_binary_pdu’:
mm-sms-part-3gpp.c:607:9: error: enumeration value ‘MM_SMS_ENCODING_UNKNOWN’ not handled in switch [-Werror=switch-enum]
607 | switch (user_data_encoding) {
| ^~~~~~
mm-sms-part-3gpp.c:714:9: error: enumeration value ‘MM_SMS_ENCODING_UNKNOWN’ not handled in switch [-Werror=switch-enum]
714 | switch (user_data_encoding) {
| ^~~~~~
Depsite 3GPP TS 23.038 specifies that Unicode SMS messages are encoded in
UCS-2, UTF-16 encoding is commonly used instead on many modern platforms to
allow encoding code points that fall outside the Basic Multilingual Plane
(BMP), such as Emoji. Most of the UCS-2 code points are identical to their
equivalent UTF-16 code points. In UTF-16, non-BMP code points are encoded in a
pair of surrogate code points (i.e. a high surrogate in 0xD800..0xDBFF,
followed by a low surrogate in 0xDC00..0xDFFF). An isolated surrogate code
point has no general interpretation in UTF-16, but could be a valid (though
unmapped) code point in UCS-2.
This patch modifies the 3GPP SMS decoding to first try UTF-16BE and then fall
back to UCS-2BE on failure. If both fail, an empty string is returned
instead of a NULL pointer.
Instead of having a method that returns the expected length after the
conversion and the amount of input UTF-8 characters that couldn't be
converted to the given charset, simplify the logic and just define a
method that returns a boolean specifying whether the conversion is
possible or not.
Also, include unit tests.
Andrey Skvortsov
Dmitry Skorykh
Carlo Lobrano
Frederic Martinsons
Clayton Craft
Aleksander Morgado
Maxim Anisimov
amol.lad
Ville Skyttä
Ben Chan
Aleksander Morgado