C++ strings also can contain and produce invalid UTF8. Does it mean C++ not support UTF8? =)
2
Conversation
std::string is just a dynamic array of bytes able to manage a terminating NUL for C string compatibility. It doesn't have any specific encoding. Being able to stick UTF-8 in std::string doesn't mean it provides any form of UTF-8 support or that it's a UTF-8 string type.
1
You still haven't answered, is it possible to conclude that C++ does not support UTF8? Yes or no?
1
C strings and std::string are not Unicode string types and while they can hold UTF-8 they are not UTF-8 string types and do not provide any actual UTF-8 support. They're arrays of bytes. C++ std::string is implemented as a library and other libraries can implement string types.
1
There's no point of making this disingenuous strawman argument. The fact is that JavaScript has legacy, broken strings and hasn't been fixed to support what Unicode has been for the past 21 years. JavaScript can be easily fixed rather than propagating brokenness further.
2
I agree that a new language inspired by JavaScript can have really unicode-aware operations for all the remaining USC2 methods. But changing the format from UTF16/WTF16 to UTF8 would cause a complete change in string API. Even Dart couldn't do that.
1
JavaScript strings are not UTF-16 or Unicode strings. Every UTF-16 string can be represented as UTF-8. If JavaScript had UTF-16 strings, there would never be one you couldn't convert to UTF-8.
UTF-8 is what's used for input/output and JavaScript is already forcing conversions.
1
Dart made the mistake of supporting compiling to JavaScript and heavily orienting itself around that ecosystem. I don't see why WebAssembly should inherit JavaScript's flaws rather than JavaScript being improved to support proper interoperability with nearly everything else.
1
I want to emphasize that nobody forces Wasm to support only UTF16. The point is that Wasm or more precisely Interface Types, which is an external layer, should also understand UTF16 along with UTF8, so that IT can be effectively used not only by Rust/Go, but also by Java, JS, C#
1
JavaScript strings aren't UTF-16. JavaScript is perfectly capable of supporting actual UTF-16 or supporting much more sensible UTF-8 strings. WTF-16 is not UTF-16 and doesn't belong in new languages / environments.
JavaScript strings are arrays of 16-bit integers, not Unicode.
2
WebAssembly has support for arrays of 16-bit integers and it can be turned into UTF-8 with the programmer deciding how to deal with the invalid Unicode such as by using replacement characters.
JavaScript strings can't simply be turned into UTF-16 strings without error handling.
No one forces JavaScript to stick with legacy strings from 1999 rather than moving on to how Unicode has worked for the past 21 years.
UTF-16 isn't just wasteful but dangerous since many people misunderstand what it is and think what languages like JavaScript have is UTF-16...
Show more replies