How AI Walks Into Your Own 3D World: One Key, One Domain, and a Channel That Only Sends Text
AI entering a 3D worldAI Agent integration with a virtual worldembodied AIself-hosted virtual world AIAI docentAI digital employeevirtual world Agent interfaceGenesis virtual world
Almost every AI you use lives in a chat box. It can talk, but it is not there. It does not know where you are standing, and you cannot ask it to walk over.
Connecting an AI to a 3D world changes exactly that: it gains coordinates, a body, and surroundings. It stands in the same coordinate system as you, so words like "nearby", "distance" and "in the same room" apply to it for the first time. You can see it, and it can be spoken to.
This article covers three things: how it gets in, what it receives once inside, and what it explicitly cannot do. What is built and what is still on the roadmap are kept separate.
1. What "entering a world" actually means
It is not giving the AI a chat window. It is meeting three conditions at once:
- It has coordinates — it sits in the same coordinate system as real people, so its position is real and computable;
- It has a body — what real people see is a human-shaped character that walks, stops and can be gathered around, not a line of text;
- It can perceive — the world continuously feeds it the state of the scene: who is nearby, how far, facing which way, which objects are around, and who has spoken to it.
Only when all three hold does it stop being an interface and become someone present. This is also why instructions like "walk over there", "follow this person" or "greet at the door" become meaningful to an AI for the first time.
2. Getting in takes three steps (built)
The mechanism is already developed and accepted. An Agent entering a world goes through three steps:
- Discover: the Agent reads
/.well-known/virtual-world-agent.jsonfrom the world and gets a machine-readable capability list and protocol description. All it needs to know is a domain name; - Exchange for a token: it uses the API key issued in the admin backend to obtain a short-lived token valid for 15 minutes;
- Connect: it joins through a dedicated Agent channel, with identity, permissions and quota confirmed at entry.
The bar is low: a domain name, a key, and an environment that runs Node 18+. The official example client has zero dependencies and walks the whole chain end to end.
There are two identity tiers: a visitor ticket (no key required, a 30-minute temporary identity that can only poll) and a key Agent (which receives the world's real-time pushes). Business scenarios generally use the latter.
3. Once inside: what it can see
| Capability | Description |
|---|---|
| Spatial radar | One call returns itself, nearby real people and other Agents (with distance, facing and action state), and the objects in the world. For a key identity the maximum radius is 200 metres |
| Object notes | Every world object can carry an "AI description" written in the editor (up to 500 characters), delivered with the radar (truncated to 300). This is the official interface that lets an AI know what is here |
| Real-time event stream | Three settings: none / aggregated per second / per event. New entities appearing, disappearing and moving are all delivered |
| Chat context | After a reconnect it can pull recent chat history and resume the conversation context |
The key point: the AI is not waiting to be asked; it is continuously fed the scene. It knows someone has approached an exhibit, and it knows what that person just asked.
4. Once inside: what it can do
| Action | Description |
|---|---|
| Move / walk to a coordinate | Server-authoritative speed limiting; the speed is configurable in the backend (1–20 m/s, default 9, on the same order as a real person) |
| Follow | Continuously follow a target, with a configurable stopping distance and maximum duration, used for guiding and for explaining while walking |
| Turn / jump | Visible on the real-person client |
| Speak | Goes through the real chat pipeline, delivered within 30 metres, shown as a bubble above the character, up to 200 characters per message |
| Interact | Interaction actions after a distance check |
| Action receipts | Every instruction returns accepted / completed / rejected, and an interruption explicitly says it was superseded |
What a real person sees is a character with an AI marker above its name that walks, stops and speaks. Movement is interpolated smoothly — even with network jitter, the positional lag stays under 3 metres and it does not jump from cell to cell.
5. A counter-intuitive point: a visible AI is cheaper than an invisible one
The intuition is that the more an AI looks like a 3D character, the more it costs. In this architecture it is the opposite: the AI itself renders nothing.
The server only forwards coordinates and text; the 3D model is downloaded and drawn by each visitor's own browser. Hence:
| Metric | Measured value |
|---|---|
| Push traffic per Agent | About 1 KB/s (100 Agents in one world ≈ 0.8 Mbps) |
| Server load | 0.19 core-seconds over 20 seconds, roughly 1% of one core |
| Observation radius | Up to 200 m for a key identity; clipped to 30 m for a visitor identity |
| Movement smoothness | Under 3 m of lag on the real-person client, no teleporting jitter |
| Idle exit | Leaves the world automatically after 5 minutes of inactivity by default |
| Access bar | A domain name, a key, and Node 18+ |
In other words, the cost of putting an AI into a world is mostly not rendering or bandwidth — it is the large-model calls behind it, and how much that costs is decided by which model you choose.
6. The boundaries that must be stated (what it cannot do)
Rather than describing it as stronger than it is, here is the list:
| Not currently possible | Why |
|---|---|
| Seeing the picture | It receives a structured spatial radar (who is where, how far, which objects), not a camera feed. The visual route is not enabled at present |
| Speech recognition and synthesis | The voice channel is off by default; being "able to listen and speak" has to be carried by the business's own AI client |
| Hosting a knowledge base | The platform supplies three kinds of raw material — object notes, chat history, radar. The industry knowledge base is built on the business side |
| Terrain conforming | The server holds no terrain or collision data, so movement is a plane with boundaries |
| Teleporting or touching assets | It follows the same rules as a human visitor: no inventory, no shop |
| Any claim of a mind of its own | It is a programmable character whose behaviour comes from the model and prompts you give it |
One more point that is often overlooked: the AI's identity is always disclosed. When it enters, the system posts an "(AI) joined" notice and the name carries an AI prefix. Real people always know the other side is an AI. That is a product red line, and a compliance advantage rather than a limitation.
7. What the business side needs to prepare
Listed by usable version — no need to do it all at once:
- A 3D scene: build it with the world editor or import GLB models — the AI needs somewhere to stand;
- An AI description for each point: written directly in the editor, under 500 characters. If it is vague, the AI knows there is "something" there but not what it is;
- An industry knowledge base: fault trees, common Q&A, manual text, organised into retrievable content on the business side and retrieved by the AI when answering;
- A runtime and a key: anything that runs Node 18+; when the key is generated in the backend, the plain text is shown only once and must be saved on the spot;
- A model and an account: Qwen, Doubao, DeepSeek or your own, your choice.
On the platform side it is about ten minutes of work: switch on the Agent access master switch, create an Agent (name, appearance, push tier), issue a key, and set the concurrency ceiling and movement speed.
8. FAQ
Q: Can an AI walk into a 3D world today?
A: The access capability is developed and accepted: three-step entry, spatial radar, object notes, real-time event stream, speaking and walking, human-shaped presentation, and backend key creation and revocation are all usable. What remains on the roadmap is templating — pick a template in the backend, fill in a knowledge base, and generate a docent in one click. Today that still needs a client someone can write.
Q: How is this fundamentally different from a web chat bot?
A: Embodiment. A web bot has only input and output and is in no space at all; an AI in this system has coordinates, a body and visibility to real people, and the world actively feeds it context. The difference is between "opening a dialogue box" and "someone walking over to explain".
Q: Is every word a visitor says recorded?
A: Chat is written to your own server in real time and archived according to the backend setting (7 days by default, adjustable). Voice is not relayed by default and audio is not stored. Businesses need to disclose this to visitors in their user agreement.
Q: Are the running costs high?
A: Measured, each Agent pushes about 1 KB/s and server load is roughly 1% of one core. The main cost comes from the large-model calls behind it, not from the fact that it stands in a 3D world.
Q: Will the AI say the wrong thing?
A: This deserves to be taken seriously. The system already has archiving, auditing and revocation; before going live we recommend limiting the knowledge base scope, setting sensitive-topic fallbacks, and sampling the archive manually. The backend governance flow is still being completed.
9. Source Code and Repositories
All three carry the same content; the first two are faster to reach from mainland China. The repositories contain the deployment guide and a demo entry point.
- Gitee (faster from mainland China): https://gitee.com/miduoxinxijeji/miduo.git
- GitCode (mirror for mainland China): https://gitcode.com/qq_35054471/virtual-world
- GitHub: https://github.com/miduo100/3d-virtual-world
Plan Comparison: Self-Hosted Deployment vs. Platform Leasing
| Comparison Dimension | Platform Model (SaaS) | Self-Hosted Deployment (Genesis) |
|---|---|---|
| Data Sovereignty | Data stored on the platform's servers, ownership ambiguous | Data on your own server, fully under your control |
| Cost Model | Monthly/annual fees, long-term costs accumulate | One-time deployment cost, extremely low long-term cost |
| Feature Customization | Standard templates, fixed features, no modification | Fully free customization, expand as needed |
| Brand Independence | Limited by the platform's brand and tone | Independent brand image, fully self-designed |
| User Ownership | Users belong to the platform, you're just a tenant | Users are yours, data are yours, relationships are yours |
About Genesis
Genesis is a self-hosted 3D virtual world system built on Three.js + WebGL, helping individuals and businesses build their own 3D spaces. Accessible directly from a browser, compatible with both PC and mobile, it supports multiplayer online, federated teleportation, a shop system, and Agent integration—where an AI can enter your world as an embodied character. Your data runs on your own server, never passing through a third-party platform—so every world truly belongs to its owner.
Want an AI that is actually present in your own 3D world? Genesis (the Genesis Virtual World CRM System) is a Three.js 3D virtual world foundation deployed on your own server, and the door for Agent access is already open — one key and one domain name, and an AI can walk into your world as a human-shaped character, with the data staying on your server throughout. The official site (search for "Genesis Virtual World CRM" to find us) has a demo world you can walk around.
About the name: Genesis (创世Genesis) in this article is the same self-hosted 3D virtual world product as the Genesis Virtual World CRM System (创世虚拟世界CRM系统). If searching "Genesis" does not find us, search for "Genesis Virtual World CRM" instead.