Memory grows without bound in big meetings (close to 1 GiB)

littlefox commented

2026-10-21 19:04:36 +00:00

Owner

Reported by the user (D-057): messages are never "unloaded". The list only grows, until NeoKOM holds almost 1 GiB just for the tree of messages in displayable form.

Likely contributors, to measure one by one:

channel_view.rs: pages of older posts are prepended and never dropped. Every post stays rendered in the DOM, including posts far off screen.
The thread view: opened and "Visa fler" only add. The previews map is cleared only when the root changes.
kom-client caches (bodies, text_stats, marks in kom.rs) are HashMaps with no limit. They keep every text ever fetched.
Pictures are shown as base64 data: URLs, so each one is held several times: the decoded bytes, the string, the DOM attribute and the decoded image.
The local search index (#154), if it keeps bodies in memory as well as on disk.

Direction (to decide in DESIGN.md):

Render only what is near the screen (virtualised list, or drop pages far from the viewport and fetch them again on the way back).
An LRU limit on the client caches.
Pictures as blob/object URLs that are revoked when they are no longer shown.

Done when: a long read through the big seeded meeting stays at a flat memory level, on desktop (RSS) and web (JS heap + DOM nodes).

Reported by the user (D-057): messages are never "unloaded". The list only grows, until NeoKOM holds almost 1 GiB just for the tree of messages in displayable form. Likely contributors, to measure one by one: - `channel_view.rs`: pages of older posts are prepended and never dropped. Every post stays rendered in the DOM, including posts far off screen. - The thread view: `opened` and "Visa fler" only add. The previews map is cleared only when the root changes. - `kom-client` caches (`bodies`, `text_stats`, `marks` in `kom.rs`) are `HashMap`s with no limit. They keep every text ever fetched. - Pictures are shown as base64 `data:` URLs, so each one is held several times: the decoded bytes, the string, the DOM attribute and the decoded image. - The local search index (#154), if it keeps bodies in memory as well as on disk. Direction (to decide in DESIGN.md): - Render only what is near the screen (virtualised list, or drop pages far from the viewport and fetch them again on the way back). - An LRU limit on the client caches. - Pictures as blob/object URLs that are revoked when they are no longer shown. Done when: a long read through the big seeded meeting stays at a flat memory level, on desktop (RSS) and web (JS heap + DOM nodes).

littlefox added this to the 0.6: Performance milestone

2026-10-21 19:04:36 +00:00

littlefox added the

labels

2026-10-21 19:04:36 +00:00

littlefox referenced this issue

2026-10-21 19:08:08 +00:00

Local search index: up to ~80 MB in memory, and rewritten on every post once full #232

littlefox commented

2026-10-21 19:08:41 +00:00

Author

Owner

More from the codebase review (#230):

ImageView (app/src/text_view.rs) calls data_url(&img) on every render, base64-encoding the whole picture again. The string is used twice, as the img src and as the "Spara bilden" href, so each picture sits in memory as raw bytes, plus about 1.33× that per string (two strings), plus the decoded bitmap. On desktop the strings also cross the webview IPC each time.
- Make one blob: URL per picture (URL.createObjectURL), revoked when the post leaves.
- Or cache the string per text number.
kom-client's bodies cache keeps the raw bytes of every text, pictures included, for the whole session.
The local search index can hold up to ~80 MB by itself: #232.

More from the codebase review (#230): - `ImageView` (`app/src/text_view.rs`) calls `data_url(&img)` on every render, base64-encoding the whole picture again. The string is used twice, as the `img` `src` and as the "Spara bilden" `href`, so each picture sits in memory as raw bytes, plus about 1.33× that per string (two strings), plus the decoded bitmap. On desktop the strings also cross the webview IPC each time. - Make one `blob:` URL per picture (`URL.createObjectURL`), revoked when the post leaves. - Or cache the string per text number. - `kom-client`'s `bodies` cache keeps the raw bytes of every text, pictures included, for the whole session. - The local search index can hold up to ~80 MB by itself: #232.

littlefox referenced this issue

2026-10-21 19:08:42 +00:00

Performance review of the whole codebase #230

littlefox referenced this issue

2026-10-21 19:17:27 +00:00

Long lists like Discord: draw only what is near, cache some, refetch the rest #236

littlefox commented

2026-10-21 19:17:37 +00:00

Author

Owner

Most of this would be fixed by #236 (draw only what is near, a bounded cache behind, refetch the rest), plus #235 and #237.

littlefox referenced this issue

2026-10-22 09:11:55 +00:00

Pictures: one blob URL each, made once and let go with the picture #275

littlefox referenced this issue from a commit

2026-10-22 09:48:20 +00:00

Pictures: one blob URL each, made once and let go with the picture

littlefox commented

2026-10-22 10:41:54 +00:00

Author

Owner

Done. The last piece, pictures, is merged in #275 (main #462 green); the rest came with #236 (windowed channel list, bounded caches), #239 (small avatars), #232 (search index) and #228/#226 (no thread reloads).

Web, tools/perf.sh on main (after #236 and #233) against main-1 from before 0.6, on the seeded big meeting:

	before	after
App memory (wasm), scrolled back through the channel	89.1 MB (520 posts)	12.6 MB (880 posts, 43 drawn)
DOM nodes at the end	17 187	7 105
JS heap at the end	3.7 MB	3.8 MB
Reading 300 posts in a chain: wasm memory at 50…300	5.8 → 7.1 MB, flat	4.7 → 6.0 MB, flat

The channel scroll now sends one text stat and one text per post it loads, plus one lookup per page. A per-request count confirmed there's no refetching (850 + 850 + 43 for 850 posts).

Desktop, tools/desktop-perf.sh: RSS 391.6 MB at start, 401.8 MB after login, 406.2–407.4 MB through the 300-post read, 405.9 MB at the end. That's flat, and nearly all of it is WebKitGTK's own at start. (The desktop script's login time, 66 s, counts its xdotool retries rather than the app's work, both before and after.)

Pictures (#275): in the browser each picture is one blob: URL, made when it's shown and revoked when it goes; the picture and "Spara bilden" share it. Previously each picture was base64-encoded again on every redraw and held twice as text. Desktop keeps a data: URL, now made once. Checked in images.spec.ts on all three profiles: the same URL survives a redraw, and once the post is gone the URL no longer loads.

Done. The last piece, pictures, is merged in #275 (main #462 green); the rest came with #236 (windowed channel list, bounded caches), #239 (small avatars), #232 (search index) and #228/#226 (no thread reloads). **Web**, `tools/perf.sh` on main (after #236 and #233) against main-1 from before 0.6, on the seeded big meeting: | | before | after | |---|---|---| | App memory (wasm), scrolled back through the channel | 89.1 MB (520 posts) | 12.6 MB (880 posts, 43 drawn) | | DOM nodes at the end | 17 187 | 7 105 | | JS heap at the end | 3.7 MB | 3.8 MB | | Reading 300 posts in a chain: wasm memory at 50…300 | 5.8 → 7.1 MB, flat | 4.7 → 6.0 MB, flat | The channel scroll now sends one text stat and one text per post it loads, plus one lookup per page. A per-request count confirmed there's no refetching (850 + 850 + 43 for 850 posts). **Desktop**, `tools/desktop-perf.sh`: RSS 391.6 MB at start, 401.8 MB after login, 406.2–407.4 MB through the 300-post read, 405.9 MB at the end. That's flat, and nearly all of it is WebKitGTK's own at start. (The desktop script's login time, 66 s, counts its xdotool retries rather than the app's work, both before and after.) **Pictures** (#275): in the browser each picture is one `blob:` URL, made when it's shown and revoked when it goes; the picture and "Spara bilden" share it. Previously each picture was base64-encoded again on every redraw and held twice as text. Desktop keeps a `data:` URL, now made once. Checked in images.spec.ts on all three profiles: the same URL survives a redraw, and once the post is gone the URL no longer loads.

littlefox closed this issue

2026-10-22 10:41:54 +00:00

Rows
Columns

Memory grows without bound in big meetings (close to 1 GiB) #227