Text outside Latin-1 in aux items, messages and status is handled three ways

littlefox commented

2026-10-24 22:21:29 +00:00

Owner

Interop audit, 2026-10-24.

Characters outside Latin-1 are handled three different ways.

Aux items (fast replies, cross-reference descriptions, image file names): compose::aux_data writes UTF-8 when the text doesn't fit Latin-1. The Emacs client decodes aux data as lyskom-server-coding-system (Latin-1; lyskom-aux-item-decode-data, aux-items.el), so its users see mojibake. httpkom/jskom try UTF-8 first, then Latin-1 (komserialization.py), so jskom is fine.
Chat messages: refused outside Latin-1 (Kom::send_message).
"Vad gör du?": such characters become ? (Kom::set_doing).
Names: refused outside Latin-1.
Text bodies are fine everywhere: UTF-8 with charset=utf-8, which Emacs honours (komtypes.el) and httpkom too.

Reactions already store Latin-1 emoticons for this reason (#101).

Options, strict to permissive:

Latin-1 everywhere outside bodies: refuse, or say what will change before sending. Consistent and safe for Emacs.
As now, made deliberate: UTF-8 where only Emacs suffers (aux items), refused where it reaches other users' screens directly (messages, names).
UTF-8 everywhere it's possible: modern, but Emacs users see mojibake.

Interop audit, 2026-10-24. **Characters outside Latin-1 are handled three different ways.** - **Aux items** (fast replies, cross-reference descriptions, image file names): `compose::aux_data` writes UTF-8 when the text doesn't fit Latin-1. The Emacs client decodes aux data as `lyskom-server-coding-system` (Latin-1; `lyskom-aux-item-decode-data`, `aux-items.el`), so its users see mojibake. httpkom/jskom try UTF-8 first, then Latin-1 (`komserialization.py`), so jskom is fine. - **Chat messages:** refused outside Latin-1 (`Kom::send_message`). - **"Vad gör du?":** such characters become `?` (`Kom::set_doing`). - **Names:** refused outside Latin-1. - **Text bodies** are fine everywhere: UTF-8 with `charset=utf-8`, which Emacs honours (`komtypes.el`) and httpkom too. Reactions already store Latin-1 emoticons for this reason (#101). Options, strict to permissive: 1. **Latin-1 everywhere outside bodies:** refuse, or say what will change before sending. Consistent and safe for Emacs. 2. **As now, made deliberate:** UTF-8 where only Emacs suffers (aux items), refused where it reaches other users' screens directly (messages, names). 3. **UTF-8 everywhere it's possible:** modern, but Emacs users see mojibake.

littlefox added the

area/writing

area/protocol

needs/user

labels

2026-10-24 22:21:29 +00:00

littlefox commented

2026-10-28 08:56:11 +00:00

Author

Owner

We will go with option 2. It is a feature that is not used often, and we have a strong case that the emacs client is the one not interpreting text correctly since it remains to be the odd one out.

littlefox removed the

needs/user

label

2026-10-28 08:56:18 +00:00

littlefox added this to the 0.8: Administration milestone

2026-10-28 09:08:41 +00:00

littlefox referenced this issue from a commit

2026-10-28 09:48:51 +00:00

What you're doing: refuse characters outside Latin-1, don't send '?'

littlefox referenced this issue from a pull request that will close it,

2026-10-28 09:49:03 +00:00

Text outside Latin-1: refuse it in the doing line too (#333) #384

littlefox closed this issue

2026-10-28 10:25:31 +00:00

littlefox referenced this issue from a commit

2026-10-28 10:25:32 +00:00

Merge pull request 'Text outside Latin-1: refuse it in the doing line too (#333)' (#384) from issue-333-latin1 into main

littlefox commented

2026-10-28 11:05:47 +00:00

Author

Owner

Merged in #384; main run #725 green. Option 2: "Vad gör du?" now refuses characters outside Latin-1, with a message and a disabled button (it used to send '?'). Names and chat messages were already refused; aux data stays UTF-8 when needed. The policy is in DESIGN §3. presence.spec checks it on all profiles.

littlefox referenced this issue

2026-10-28 15:15:35 +00:00

Release 0.8.0 #392

littlefox referenced this issue from a commit

2026-10-28 15:15:37 +00:00

Release 0.8.0

Rows
Columns

Text outside Latin-1 in aux items, messages and status is handled three ways #333