Repository navigation
feat(ai): Firebase AI Logic C++ SDK with LiteRT / LiteRT-LM Hybrid Inference - #1930
AustinBenoit wants to merge 2 commits into
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces the firebase::ai C++ SDK for Firebase AI Logic, enabling hybrid cloud and local on-device inference powered by Google AI Edge LiteRT and LiteRT-LM. It includes platform-specific HTTP senders, a C ABI bridge for Unity integration, a multi-turn chat sample with automatic context compaction, and unit tests. The review feedback suggests several robustness improvements: using LoadLibraryW with UTF-8 to UTF-16 conversion on Windows to support non-ASCII paths, utilizing idx.is_numeric() during JSON parsing to safely handle varying numeric types for grounding chunk indices, and making CMake download failures fatal to prevent downstream compilation errors.
| void* LoadSharedLibrary(const std::string& path) { | ||
| if (path.empty()) return nullptr; | ||
| #if defined(_WIN32) | ||
| return reinterpret_cast<void*>(LoadLibraryA(path.c_str())); | ||
| #else | ||
| return dlopen(path.c_str(), RTLD_NOW | RTLD_LOCAL); | ||
| #endif | ||
| } |
There was a problem hiding this comment.
On Windows, LoadSharedLibrary uses LoadLibraryA which expects an ANSI string. If the installation path or runtime library path contains non-ASCII characters (e.g., in non-English locales or user directories), LoadLibraryA will fail to load the library. It is highly recommended to use LoadLibraryW with a UTF-8 to UTF-16 conversion helper to ensure robust cross-platform path support.
void* LoadSharedLibrary(const std::string& path) {
if (path.empty()) return nullptr;
#if defined(_WIN32)
int len = MultiByteToWideChar(CP_UTF8, 0, path.c_str(), -1, nullptr, 0);
if (len <= 0) return nullptr;
std::vector<wchar_t> wpath(len);
MultiByteToWideChar(CP_UTF8, 0, path.c_str(), -1, wpath.data(), len);
return reinterpret_cast<void*>(LoadLibraryW(wpath.data()));
#else
return dlopen(path.c_str(), RTLD_NOW | RTLD_LOCAL);
#endif
}| if (idx.is_int64()) { | ||
| gs.grounding_chunk_indices.push_back( | ||
| static_cast<int>(idx.int64_value())); | ||
| } |
There was a problem hiding this comment.
In ParseGroundingMetadata, the code checks idx.is_int64() before extracting the grounding chunk index. However, depending on the JSON parser implementation and the platform, small integer values in JSON might be parsed as double or other numeric types. To prevent silently skipping valid indices, use idx.is_numeric() and convert it safely using idx.AsInt64().int64_value().
| if (idx.is_int64()) { | |
| gs.grounding_chunk_indices.push_back( | |
| static_cast<int>(idx.int64_value())); | |
| } | |
| if (idx.is_numeric()) { | |
| gs.grounding_chunk_indices.push_back( | |
| static_cast<int>(idx.AsInt64().int64_value())); | |
| } |
| else() | ||
| message(WARNING "Firebase AI: Failed to download LiteRT C/C++ SDK (${LITERT_SDK_DL_STATUS})") | ||
| endif() |
There was a problem hiding this comment.
If the download of the LiteRT C/C++ SDK fails, CMake currently prints a warning and continues configuration. This will lead to confusing compilation errors later in the build process when headers or libraries are missing. If FIREBASE_AI_DOWNLOAD_LITERT is enabled, a download failure should be treated as a fatal error so the user is notified immediately during the configuration step.
else()
message(FATAL_ERROR "Firebase AI: Failed to download LiteRT C/C++ SDK (${LITERT_SDK_DL_STATUS})")
endif()
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| return false; | ||
| } | ||
| state_->cc_compiled_model.reset( | ||
| new litert::CompiledModel(std::move(*model_res))); |
There was a problem hiding this comment.
Add #include <utility> for move
| size_t probe_len = | ||
| (end_line == std::string::npos) | ||
| ? std::min<size_t>(24, last_user_text.size() - first_non_ws) | ||
| : std::min<size_t>(24, end_line - first_non_ws); |
There was a problem hiding this comment.
Add #include <algorithm> for min
| size_t total = size * nmemb; | ||
| if (total == 0 || !userdata) return 0; | ||
| CurlWriteContext* ctx = static_cast<CurlWriteContext*>(userdata); | ||
| long http_code = 0; |
There was a problem hiding this comment.
Use int16/int64/etc, rather than the C type long
|
|
||
| if (req.timeout_ms > 0) { | ||
| curl_easy_setopt(curl, CURLOPT_TIMEOUT_MS, | ||
| static_cast<long>(req.timeout_ms)); |
There was a problem hiding this comment.
Use int16/int64/etc, rather than the C type long
| curl_easy_setopt(curl, CURLOPT_POST, 1L); | ||
| curl_easy_setopt(curl, CURLOPT_POSTFIELDS, req.body.data()); | ||
| curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, | ||
| static_cast<long>(req.body.size())); |
There was a problem hiding this comment.
Use int16/int64/etc, rather than the C type long
| curl_easy_setopt(curl, CURLOPT_WRITEDATA, &write_ctx); | ||
|
|
||
| CURLcode res = curl_easy_perform(curl); | ||
| long http_status = 0; |
There was a problem hiding this comment.
Use int16/int64/etc, rather than the C type long
1 participant
Summary
firebase::aiC++ SDK (FirebaseAI,GenerativeModel,Chat,TemplateGenerativeModel,TemplateChatSession,Schema,JsonSchema, Function Calling, Safety, and SSE Streaming) across Desktop, Android, and iOS/macOS.ONLY_ON_DEVICE,ONLY_IN_CLOUD,PREFER_ON_DEVICE,PREFER_IN_CLOUD) powered by Google AI Edge LiteRT (litert::CompiledModelfor.tflite) and LiteRT-LM (CLiteRTLMfor.litertlmGemma models).v2.2.0),libLiteRt, andCLiteRTLM(v0.18.0) via CMake (FIREBASE_AI_DOWNLOAD_LITERT=ON).firebase_ai_litert_bridgeC ABI shared library for Unity P/Invoke hybrid integration.firebase_ai_hybrid_chatinteractive multi-turn CLI sample with live mode toggling (/toggle,/mode),/compact,/clear, automatic context window detection & compaction, and repetition/4-gram loop prevention.ai/README.mdfor quick start and build instructions.