fix(context): keep trimmed MEMORY.md valid UTF-8 and on a line boundary

[kriss39]

What

Context.MemoryMd keeps the last max_bytes of MEMORY.md with binary_part(body, byte_size(body) - max_bytes, max_bytes), in both read_tail/2 (the before_turn/4 context) and trim/2 (which writes the tail back as the new file). Once the file grows past the limit (24 000 bytes by default — a dozen long turns), that byte offset regularly lands inside a multi-byte UTF-8 sequence: accented letters, €, emoji — ordinary Telegram traffic.

Consequences:

• the context returned by before_turn/4 is not valid UTF-8, so the ingress object's prefix <> turn.text reaches AgentProtocol.encode_task/2 and Jason.encode!/1 raises inside the agent's task path — the conversation stops answering until the file happens to be cut on a boundary again;
• trim/2 persists the damaged tail, so the file itself is invalid from then on.

Replaying 400 turns of mixed Spanish/emoji text through a line-for-line port of the two functions gave an invalid context on 61 turns (15 %).

Change

lib/genswarms/telegram/context/memory_md.ex: one tail/2 used by both call sites. It still keeps at most max_bytes, but starts the kept text on the first line break inside the window (so it is valid UTF-8 and begins with a whole - role: … turn or a template heading; leading blank lines are dropped), and when the window holds no line break it drops the leading UTF-8 continuation bytes so the text starts on a character boundary.

test/client_store_memory_test.exs: 40 turns of ñandú €N 🦤 ok (2-, 3- and 4-byte sequences) with limits cycling 61–70 bytes; after every turn the file and the before_turn/4 context must be valid UTF-8, within the limit, and start on a whole line.

Testing done

• mix test (Elixir 1.18.4 / OTP 27): 257 tests, 0 failures.
• The new test fails on main (turn N: context is not valid UTF-8).
• The existing trim/sanitize test (max_bytes: 120 / 50) still passes: the size bounds are unchanged, the tail only gets shorter.
• mix format on the touched files.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 9af4bc1a-c5b4-4700-bef2-6dfa8594ebcb

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[kriss39]

@jmlago gentle ping on this one when you have time.