Summary: The hard part of browser-side multiplayer 3D was never "can it connect". It is three things: how position sync feels, whether two players can still see each other after a reconnect, and how bandwidth and CPU are accounted for once a crowd shows up. This article takes those three in turn, then adds the implementation idea behind treating an external AI as a player on the same broadcast path, plus the measured bill for 100 AIs in one world.

Most multiplayer bugs are not in the first connection. They are in the tenth-thousandth reconnect.

1. Defining the problem: three real issues

The textbook structure of a multiplayer 3D room is "clients send positions, the server broadcasts to everyone else". In practice it lands on three concrete problems:

The third is especially sharp in the browser: the CPU budget lives on the player's device, so adding servers does not help. The bandwidth budget lives on your server, so money helps but costs a lot. They have to be handled separately.

2. Position sync: lag is an acceptable cost, jumping is not

Server authority plus client-side smoothing is the standard answer here. The key is to separate "lag" from "jump" — players tolerate lag far better than they tolerate jumping. A steady 200 ms lag reads as "he is slightly behind"; one hard coordinate assignment reads as "he teleported".

Our trade-off: keep real-player position lag under 3 metres, and in exchange movement stays smooth throughout with no stepping. Smaller is not automatically better — squeeze the interpolation window too tight and network jitter is transmitted straight into the picture, which looks worse.

One easily missed detail: AIs and humans update at different frequencies. Human clients report per frame; AI entities are pushed by the server at a chosen tier (see section 6). Those two streams have to land in the same position table and the same broadcast, or the AI will appear to move to a different rhythm than everyone else.

3. Weak networks: the genuinely hard part is "still visible after reconnecting"

This is where we spent the most time, and it is the part a demo hides most easily — because demos do not drop connections.

The classic version of the symptom: A and B each drop once, each reconnects successfully, and then they can no longer see each other, until both manually reload the page. The root cause reduces to one sentence:

On reconnect, the server re-registers you as a new connection, but the fact that "I am already present" is never broadcast again.

So we built a presence guard with four actions:

1. Cache the PLAYER_JOIN event and replay it automatically after every reconnect — this is the core line.
2. Reconnect indefinitely with backoff, rather than giving up after N attempts.
3. An application-level PING/PONG watchdog — a TCP connection can still be alive while the application layer is dead, and onclose will not tell you.
4. Server-side "unregistered connection" warnings — to catch the corners that item 1 does not cover.

The acceptance criterion is blunt: after a reconnect, the other side can still see me move — 9/9 passed.

The logic is short. An illustration follows (illustrative pseudocode, not project source):

// Client: keep the "I am here" fact, replay it on every reconnect
let pendingJoin = null;          // cached PLAYER_JOIN payload

function onLocalJoin(payload) {
  pendingJoin = payload;         // local join event, held for later
}

function onOpen() {              // every successful reconnect lands here
  if (pendingJoin) send(pendingJoin);   // <- the core line: replay presence
  startPingPong();                      // application-level watchdog
}

function onClose() {
  reconnectWithBackoff();        // no retry ceiling, with backoff
}

// Server: catch "connected but not registered" so the branch above is not the only net
setInterval(() => {
  for (const c of connections) {
    if (!registry.has(c.id)) warn('unregistered connection', c.id);
  }
}, 30_000);

The other category of bug is "visible but accumulating": video DOM nodes, pooled light rentals, chat bubbles, zombie connections. None of these should survive a long session — we ran a memory audit and fixed four leaks. Memory leaks in multiplayer compound: a leak in a single-player UI gets multiplied by the headcount.

4. Voice: from "mute the whole zone" to a metered slot

Proximity voice (push-to-talk, audible within 30 metres) has two degenerate designs and both are bad:

• Everyone open all the time: once a few people are within 30 metres, server bandwidth goes out of control.
• Mute the zone when it gets busy: crude, and it kills the feature — what the user experiences is "this stops working as soon as it gets crowded".

We moved to a cap on simultaneous speakers: push-to-talk, half-duplex, relayed within 30 metres with distance attenuation, but with a ceiling on how many people can speak at once, selectable in the admin panel according to server spec.

The value of the change is that it converts "unusable when crowded" into "queued when crowded". Queueing is acceptable; unusable is not.

There is also a compliance choice here: the voice path is processed on the server not at all — no speech recognition, no speech synthesis, audio is only relayed and never persisted. Transcription is left to the AI client, which keeps ownership clear and removes the most expensive part from the server.

5. Bringing an external AI in: reuse the same broadcast path

The world later opened up to AI access — an external AI can enter as a humanoid character, visible to real people, able to walk and talk. The engineering point worth telling is not the AI itself but the choice of how to attach it:

Do not give the AI a separate visibility system. Make it pose as a player and reuse the existing player broadcast.

Concretely there is a single weld point: write the AI's position into the player position table, then reuse the existing PLAYER_JOINED / POSITION_UPDATE / CHAT broadcasts. The result is that real players' browsers need almost no changes to see the AI's 3D body — it travels the same path as another human player.

The benefits run both ways:

• Zero disruption to the live path: the dedicated route splits by path (/ws/agent for AI, everything else falls through to humans), so human behaviour does not change at all.
• Extremely cheap: the AI's model is never sent to the AI. Its avatar is rendered by real players' browsers instead. What the AI receives is a structured JSON radar — not pixels, not screenshots.

One product decision is worth recording too: the gate is inverted. Without an API key you can still take a 30-minute public ticket by presenting a domain (pull-only). Only the scarce "push" resource requires a key. The reason is plain — early on, the risk is that nobody comes, not that you get abused. Put the barrier on the scarce resource, not on "taking a look".

Push itself is tiered, because "every AI receives every world change in real time" is an unnecessary cost:

6. Doing the arithmetic: 100 AIs in one world

This is the set of numbers that shows the architecture's value most clearly. Measured scenario: 100 guest AIs + 3 key agents + 2 human players, together for 60 seconds.

Note the point of the final row: it is not measuring performance, it is proving the red line actually holds — guest identities genuinely received no push at all. A safety design that cannot be falsified is just documentation.

A few built-in hard constraints explain why the cost stays down: guests are never pushed to; the observation radius is clamped at 30 metres; position traffic does not draw on the token bucket; back-pressure warns at 1 MB and disconnects at 4 MB; a 30-second heartbeat; idle eviction; and one connection per agent, with a newer one silently replacing the older so real players never see a flicker.

7. Current shortcomings, stated plainly

1. Three-tier player rendering is not implemented. Distance-based degradation of player avatars when a crowd appears is an obvious saving, and the module is currently not in the codebase — it needs to be rebuilt. In other words, performance under crowd conditions currently rests on model-side LOD; player avatars themselves are not yet tiered.
2. Jitter suppression on the position stream is naive. On low-frame-rate devices the feel of movement suffers (player.js still advances a fixed increment per frame rather than decoupling from frame rate), and that is perceptible on low-end phones.

Both entries are known, planned and unfinished. Writing them down beats hiding them.

8. Frequently asked questions

WebSocket or WebRTC for browser multiplayer 3D?

Small, server-authoritative messages such as positions are simpler and more controllable over WebSocket; continuous media such as voice suits WebRTC or a dedicated media path. Our voice relays within 30 metres in half-duplex, with the server only forwarding.

When a crowd shows up, is adding servers enough?

Position sync is actually cheap (100 entities under 0.1 cores). What really consumes resources is voice and rendering. Rendering happens on player devices, so more servers do not help — you need model LOD and player-avatar tiering. Voice is the part with a clear server-spec relationship.

How do you test weak-network behaviour?

Do not test "can it connect". Test "after reconnecting, can each side still see the other". Our acceptance criterion is to have A and B each drop and self-heal, then check that the other side still sees my movement. Only that counts as a pass.

Is a watchdog necessary, or is onclose enough?

Not enough. Intermediate network equipment can leave a connection half-dead — TCP alive, application layer unresponsive — and onclose never fires. You need the application-level PING/PONG and the "unregistered connection" warning as two additional nets.

Can AIs and humans share one world?

Yes, and they travel the same broadcast path. Real players' browsers render the AI's avatar while the AI itself receives only structured spatial data. That is why it is cheap.

About Genesis

Genesis is a self-hosted 3D virtual world system built on Three.js + WebGL, helping individuals and businesses build their own 3D spaces. Accessible directly from a browser, compatible with both PC and mobile, it supports multiplayer online, federated teleportation, a shop system, and Agent integration—where an AI can enter your world as an embodied character. Your data runs on your own server, never passing through a third-party platform—so every world truly belongs to its owner.

If you are building browser-based multiplayer 3D, these notes come from Genesis — a browser-side Three.js 3D virtual world foundation that runs on your own servers. The numbers come from re-runnable acceptance scripts, and the world also accepts AI agents as embodied characters.

These notes come from our own development work. The numbers are produced by in-repo acceptance scripts and can be re-run. Items labelled as current shortcomings are genuinely unfinished.

Source Code and Repositories

• Gitee (faster from mainland China): https://gitee.com/miduoxinxijeji/miduo.git
• GitCode (mirror for mainland China): https://gitcode.com/qq_35054471/virtual-world
• GitHub: https://github.com/miduo100/3d-virtual-world

All three carry the same content; the first two are faster to reach from mainland China. The repositories contain the deployment guide and a demo entry point.