Refactor multiple flows to enhance performance, boost scalability, and ensure stability. by luuquangvu · Pull Request #131 · Nativu5/Gemini-FastAPI

luuquangvu · 2026-03-21T04:10:07Z

Feel free to try it out and share any feedback or report any issues you encounter. Thanks!

Here are some highlights of the changes:

The entire logic for storing conversation history has been rewritten, aiming for compatibility with various endpoints and easy scalability in the future.
The logic of the endpoints has been rewritten, and now all endpoints work correctly with both streaming and non-streaming flows.
Compatible with the latest library updates, including the ability to download full-size images and enable video or music generation.
All cookie-related errors will be fully resolved, and users will get a clear notification if the server invalidates cookies, making it simple to know when to manually refresh a new one.

Close #103 , #113 , #129 , #130 , #133 , #139, #143 , #149 , #150 , #154

…ystem instruction when reusing a session to save tokens.

… text file attachment - When multiple chunks are sent simultaneously, Google will immediately invalidate the access token and reject the request - When a prompt contains a structured format like JSON, splitting it can break the format and may cause the model to misunderstand the context - Another minor tweak as Copilot suggested

…e sessions. - Ensure that PR HanaokaYuzu/Gemini-API#220 is merged before proceeding with this PR.

…ith reusable sessions. - Ensure that PR HanaokaYuzu/Gemini-API#220 is merged before proceeding with this PR. - Introducing a new feature for real-time streaming responses. - Fully resolve the problem with reusable sessions. - Break down similar flow logic into helper functions. - All endpoints now support inline Markdown images. - Switch large prompts to use BytesIO to avoid reading and writing to disk.

…ith reusable sessions. - Ensure that PR HanaokaYuzu/Gemini-API#220 is merged before proceeding with this PR. - Introducing a new feature for real-time streaming responses. - Fully resolve the problem with reusable sessions. - Break down similar flow logic into helper functions. - All endpoints now support inline Markdown images. - Switch large prompts to use BytesIO to avoid reading and writing to disk. - Remove duplicate images when saving and responding.

…quest

… streaming responses

Caused by Gemini Web's post-processing mechanism.

…uments

Vigno04 · 2026-07-31T10:22:46Z

config are copied from the repo with just api key and stuff like that added, this is the logs:

gemini-fastapi-sanitized.log

luuquangvu · 2026-07-31T16:13:45Z

@Vigno04 I’ve made a small update to address the issue. Please try updating now to confirm if it resolves the problem.

Vigno04 · 2026-07-31T16:23:28Z

unfortunatelly not:
error.log

luuquangvu · 2026-07-31T16:30:28Z

@Vigno04 The two log files show different errors. Which client are you using to connect?

Vigno04 · 2026-07-31T16:32:55Z

I'm using openwebui sending "ciao" with no extra tool activa other than the base ones, i did docker compose down and docker compose up -d then sent the message and registered the logs

luuquangvu · 2026-07-31T17:16:45Z

@Vigno04 I’ve made a quick update to fix the issue. Go ahead and update now to see if it solves the problem.

Vigno04 · 2026-07-31T18:47:19Z

it was still broken but i may have found a fix, the problem was just with the responses format and i was able to fix it by changing the line 1562 in chat .py from:
"tools": request.tools or [],
to
"tools": [t.model_dump(exclude_none=True) if hasattr(t, "model_dump") else t.dict(exclude_none=True) if hasattr(t, "dict") else t for t in request.tools] if request.tools else [],

luuquangvu · 2026-08-01T01:45:58Z

@Vigno04 I’ve made an update to fix another issue with the strict schema validator rejecting null fields. Go ahead and update now to see if it resolves the problem.

Vigno04 · 2026-08-01T08:44:46Z

fixed thx

…=false to avoid cold start delay and update workflows

…pstream

…tch entirely to using dynamic models

…dynamic `AvailableModel`

…ion recovery, script restore (#6) * Survivors from reconciled fork (PR #5 follow-up): C1 v1beta, SSRF guard, truncation recovery, model-resolution fix, start script vs luuquangvu/main: - app/server/gemini.py + app/models/gemini_models.py + app/models/__init__.py + app/main.py + app/server/middleware.py: native Gemini /v1beta REST surface (models, generateContent, streamGenerateContent, x-goog-api-key auth, Google error envelopes) - unobjected in review - app/utils/helper.py + app/utils/config.py + config/config.yaml: SSRF guard (reject_unsafe_url, allow_private_url_fetch, url_fetch_timeout); recovery_timeout knob - app/services/client.py + app/server/chat.py: truncated-stream recovery via LIST_CONVERSATION_TURNS polling; fix chat.py:688 calling nonexistent client.resolve_model (lib: _resolve_model_by_name) - scripts/start-gemini-api.sh: restore local start/stop helper (deleted in 71c0db2 with no replacement) - app/services/pool.py: extract _init_one/_init_attempt (behavior-preserving) * Drop unused video-generation helper from PR surface (dead against luuquangvu models) * Remove VideoGeneration import (dead on PR surface) * Fix CI: move MIME registry below imports and use contextlib.suppress Ruff E402/SIM105 in app/main.py (self-introduced by the survivors main.py edit): the mimetypes loop ran before module imports and used try/except-pass. Imports first, loop after (still at import time, before any upload), no noqa needed. All 4 CI steps now pass: ruff check, ruff format, ty, pyright. * Update the PR to align with the current flow: - removing _recover_stream_output, as it’s now handled upstream. - removing fetch_last_model_turn since it’s already included upstream. - eliminating the unnecessary looks_like_html function.

…cing behavior while keeping them concise

luuquangvu · 2026-08-16T17:13:30Z

@Nativu5 The PR is ready to be merged, as the upstream library has incorporated new features related to it (HanaokaYuzu/Gemini-API#310). Since it was created some time ago, I’ve updated and improved it multiple times, which has made it a bit large. I also removed one feature from main, compaction, as it turned out to be less efficient than expected. Feel free to review the PR anytime. Thank you very much!

luuquangvu added 30 commits January 24, 2026 17:57

Refactor: Update the _prepare_messages_for_model helper to omit the s…

2eb9f05

…ystem instruction when reusing a session to save tokens.

Enable streaming responses and fully resolve the problem with reusabl…

bdd893f

…e sessions. - Ensure that PR HanaokaYuzu/Gemini-API#220 is merged before proceeding with this PR.

Merge branch 'Nativu5:main' into main

a51f75c

Merge branch 'main' of https://github.com/luuquangvu/Gemini-FastAPI

767f0b3

build: update dependencies

7716c62

Refactor: Use strip_system_hints to standardize the content.

61672cc

Refactor: Only inject code block hint if NOT a structured response re…

cc0b13f

…quest

Refactor: Remove the code block hint entirely

6b90e5d

Refactor: Remove the code block hint entirely

553bd94

Refactor: fix missing whitespace in the streaming response.

fd767da

Merge remote-tracking branch 'upstream/main'

f6b60aa

Refactor: remove unnecessary code

4beb33b

Refactor: Update StreamingOutputFilter logic to improve handling of…

6b8dd4e

… streaming responses

Refactor: Adjust function call format to prevent streaming issues

d86ae59

Caused by Gemini Web's post-processing mechanism.

Refactor: Adjust function call format to prevent streaming issues

db39ad1

Caused by Gemini Web's post-processing mechanism.

Refactor: Adjust function call format to prevent streaming issues

556a638

Caused by Gemini Web's post-processing mechanism.

Refactor: Adjust function call format to prevent streaming issues

d5fec7a

Caused by Gemini Web's post-processing mechanism.

Refactor: Enhance prompt to prevent issues with parsing tool call arg…

dbc553d

…uments

Refactor: Enhance prompt to prevent issues with parsing tool call arg…

ca721cf

…uments

Merge branch 'Nativu5:main' into main

d3b60c8

Refactor: enhance system prompts

263158e

Refactor: Enhance system prompts

68ce2df

Refactor: Enhance system prompts

77f7210

Refactor: Enhance system prompts

3addb2b

Resolve QUIC Idle Timeout

cc837f9

Fix the incorrect implementation of the Responses API

0a88e67

Fix strict schema validator rejecting null fields

926befc

Auto detect HTTP version and centralize message roles

7b2b32f

luuquangvu and others added 12 commits August 12, 2026 23:50

Revise the get_access_token logic in gemini-webapi, change auto_close…

5e96f7f

…=false to avoid cold start delay and update workflows

Update the Build and Push Docker Image workflow

01fe60a

Update the Gemini Web API checkpoint and minor optimize workflows

40225eb

Integrate temporary chat mode with metadata fallback logic from the u…

d2c023e

…pstream

Merge commit 'febafc8f3605932cd8a474065b267c7948e7bc70'

261030c

Update the logic to gracefully handle temporary or guest mode and swi…

71c0db2

…tch entirely to using dynamic models

Deprecate the old hardcoded Model and switch entirely to using the …

c8e0a83

…dynamic `AvailableModel`

Optimize system hints for tool calling

09f3204

Revise the prompts for greater accuracy and precision, strictly enfor…

dfcca63

…cing behavior while keeping them concise

Refine the logic flows to enhance accuracy, precision, and performance

1115894

Optimize Dockerfile and CI workflows

4464d63

luuquangvu marked this pull request as ready for review August 16, 2026 17:13

luuquangvu added 3 commits August 17, 2026 09:02

Add guest_mode health policy

2c3cbe5

Optimize Dockerfile and track gemini-webapi workflow

24ab2f0

Update Troubleshooting Cookie Section

7eb7a74

Navigation Menu

Refactor multiple flows to enhance performance, boost scalability, and ensure stability. - #131

Refactor multiple flows to enhance performance, boost scalability, and ensure stability.#131
luuquangvu wants to merge 310 commits into
Nativu5:mainfrom
luuquangvu:main

luuquangvu commented Mar 21, 2026 •

edited

Loading

Uh oh!

Vigno04 commented Jul 31, 2026 •

edited

Loading

Uh oh!

luuquangvu commented Jul 31, 2026

Uh oh!

Vigno04 commented Jul 31, 2026

Uh oh!

luuquangvu commented Jul 31, 2026

Uh oh!

Vigno04 commented Jul 31, 2026

Uh oh!

luuquangvu commented Jul 31, 2026

Uh oh!

Vigno04 commented Jul 31, 2026

Uh oh!

luuquangvu commented Aug 1, 2026

Uh oh!

Vigno04 commented Aug 1, 2026

Uh oh!

luuquangvu commented Aug 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

luuquangvu commented Mar 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Vigno04 commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

luuquangvu commented Jul 31, 2026

Uh oh!

Vigno04 commented Jul 31, 2026

Uh oh!

luuquangvu commented Jul 31, 2026

Uh oh!

Vigno04 commented Jul 31, 2026

Uh oh!

luuquangvu commented Jul 31, 2026

Uh oh!

Vigno04 commented Jul 31, 2026

Uh oh!

luuquangvu commented Aug 1, 2026

Uh oh!

Vigno04 commented Aug 1, 2026

Uh oh!

luuquangvu commented Aug 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

luuquangvu commented Mar 21, 2026 •

edited

Loading

Vigno04 commented Jul 31, 2026 •

edited

Loading