Repository navigation
Convert NSStrings returned to JS to UTF-8 only once - #59325
hryhoriiK97 wants to merge 2 commits into
Conversation
`convertNSStringToJSIString` called `UTF8String` twice and copied the result into a `std::string` to measure it. A non-ASCII string was converted to UTF-8 twice, a `\0` cut the string off, and a lone surrogate turned it into "". Pass the string's own ASCII characters when CoreFoundation exposes them, and otherwise convert it to UTF-8 once with `CFStringGetBytes` and pass it with its length, so Hermes keeps a `\0` (JSC still stops at it). Strings that UTF-8 can't encode (lone surrogates) are passed as UTF-16. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@christophpurrer From the git history, it looks like you've worked a lot on this file recently, so you seem like the right person to ask. This converts an Passing UTF-16 directly would be ~15× faster on Hermes, but JSC has no native |
| } | ||
| // UTF-8 can't encode unpaired surrogates, so pass those strings as UTF-16. | ||
| std::u16string utf16(length, u'\0'); | ||
| CFStringGetCharacters(string, CFRangeMake(0, length), reinterpret_cast<UniChar *>(utf16.data())); |
There was a problem hiding this comment.
CFStringGetCharactersPtr would still be preferred here, although also claim it may return null...
There was a problem hiding this comment.
Done. It now tries CFStringGetCharactersPtr first and only copies when that returns null. Each branch has its own test.
|
It worth adding |
| std::string utf8(CFStringGetMaximumSizeForEncoding(length, kCFStringEncodingUTF8), '\0'); | ||
| CFIndex utf8Length = 0; | ||
| CFIndex convertedLength = CFStringGetBytes( | ||
| string, | ||
| CFRangeMake(0, length), | ||
| kCFStringEncodingUTF8, | ||
| 0, | ||
| false, | ||
| reinterpret_cast<UInt8 *>(utf8.data()), | ||
| static_cast<CFIndex>(utf8.size()), | ||
| &utf8Length); | ||
| if (convertedLength == length) { | ||
| return jsi::String::createFromUtf8( | ||
| runtime, reinterpret_cast<const uint8_t *>(utf8.data()), static_cast<size_t>(utf8Length)); | ||
| } |
There was a problem hiding this comment.
I don't think this pass is worth it in Hermes - when giving it utf8 input, it will copy to utf16 anyway if it's not all ASCII https://github.com/facebook/hermes/blob/static_h/lib/VM/StringPrimitive.cpp#L169
There was a problem hiding this comment.
Good point, removed it. It now follows the order you suggested.
javache
left a comment
There was a problem hiding this comment.
Ordering that should we look, based on Hermes optimization
- CFStringGetCStringPtr(..., kCFStringEncodingASCII) → createFromAscii(pointer, length).
- Otherwise, CFStringGetCharactersPtr → createFromUtf16(pointer, length).
- If neither pointer is available, copy with CFStringGetCharacters into a UTF-16 buffer, then call createFromUtf16.
|
I wish we could do similar optimizations on Android too, but I think we're constrained there with folly::dynamic being the intermediate representation |
…atively on JSC Following review: Hermes converts non-ASCII UTF-8 input to UTF-16 anyway, so converting to UTF-8 first only adds work. Strings stored as ASCII are still passed with createFromAscii; other strings now pass their own UTF-16 characters (CFStringGetCharactersPtr), or one CFStringGetCharacters copy when CoreFoundation can't expose them. JSCRuntime only overrode the ASCII and UTF-8 string constructors, so createFromUtf16 went through JSI's default, which escapes every character and calls eval. It now uses JSStringCreateWithCharacters. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@javache added it to |
Thanks for the review @javache! Updated to the order you suggested, and added |
|
@javache has imported this pull request. If you are a Meta employee, you can view this in D126905172. |
|
This pull request was successfully merged by @hryhoriiK97 in 4fac9c8 When will my fix make it into a release? | How to file a pick request? |
Summary:
On iOS, every
NSStringa TurboModule returns to JS goes throughconvertNSStringToJSIString(ReactCommon/react/nativemodule/core/platform/ios/ReactCommon/RCTTurboModule.mm). That covers promise results, callbacks, and every string key and value inside returned objects. It also covers the body of everyfetch(...).text()and.json()response, which RN reads throughFileReader.Current behavior.
UTF8Stringtwice, once to check it againstnullptrand once for the value.std::stringforjsi::String::createFromUtf8, which measures it withstrlen.UTF8Stringcall converts the whole string to UTF-8, and Hermes then converts it back to UTF-16. For a 1 MB JSON response with accented letters this takes about 7 ms on the JS thread.strlen, a string with a\0character is cut off at it.UTF8Stringcan't encode, arrives as"".With this change.
CFStringGetCStringPtr→createFromAscii).CFStringGetCharactersPtr→createFromUtf16), which Hermes stores without converting.CFStringGetCharactersand passed as UTF-16.JSCRuntimegets a nativecreateStringFromUtf16(JSStringCreateWithCharacters). Before, JSC used JSI's default, which escapes every character as\uXXXXand callseval.Every string converts to the same JS string as before, with two exceptions:
\0no longer truncates strings passed as UTF-16. This fixes, on iOS, the native → JS side of JavaScript strings with NULL character are not handled properly #24129, which Fix Text truncation at NULL character on iOS and Android (#24129) #57906 fixed for JS → native. On Hermes this covers every string; on JSC, strings stored as ASCII still stop at the\0inside JSC's owncreateFromAscii.\0characters in JS instead of a cut-off string."".Changelog:
[IOS] [FIXED] - Strings returned from native modules to JS convert faster, are no longer truncated at a
\0character, and are no longer replaced with an empty string when they contain a lone surrogateTest Plan:
RCTTurboModuleStringConversionTests(new,RNTesterUnitTests, run byyarn test-ios).TurboModuleConvertUtils::convertObjCObjectToJSIValuewith a Hermes runtime and checks that the JS string has the same UTF-16 code units.NSStringsubclass).\0(including one with an ASCII pointer), and strings with a lone surrogate (with and without a UTF-16 pointer).\0and both lone-surrogate tests fail and the other 7 pass.clang-format(scripts/clang-format.js --check) passes on all three files.JSCRuntime.cppandjsi.cppconverts ASCII, Polish, CJK with emoji, empty,\0and lone-surrogate strings with the new function and reads every character back in JS. All of them match. With the old function, the\0and lone-surrogate strings don't. Speed on JSC, old function → new function, median of 9 rounds: accented letters and emoji −86% at 128 characters and −98% to −99% from 1 KB to 1 MB (1 MB: 4.1 ms → 26 µs); ASCII −24% to −33%.xcodebuildin Release, at sizes from 16 characters to 1 MB.readAsTextin JS: the same binary with the conversion switched by a launch environment variable, in two cycles of before, after, after, before, following a warmup launch. Each value is the median of 500 reads (60 for 1 MB, first 2 left out as warmups), averaged over the four launches per arm. Every read returned the original string in both arms.Conversion alone, time per string:
* At 16 characters, the text from the start of the JSON is still ASCII in all three inputs.
FileReader.readAsTextin JS (the pathfetch'stext()andjson()take), median per read:At 1 KB, a
readAsTextcall is mostly the fixed cost of the module call and the promise, so the gain there is smaller. The 1 KB ASCII input is the first one read in each launch, so its time also includes warmup.fetchcalls. I also profiled RNTester (Release, before this change) with Time Profiler on the same iPhone while it fetched 10 KB to 1 MB JSON files from a local HTTP server over Wi-Fi with.text()and.json().convertNSStringToJSIStringwas about 29% of all JS-thread time, the largest React Native cost on that thread, and Hermes converting that UTF-8 back to UTF-16 was another 12%.🤖 Generated with Claude Code