Bit by bit the engine is becoming usable after all sorts of changes. As usual, every time I read about an interesting idea it feels like the whole thing needs rewriting :). The latest ideas were concurrency, memory debugging, fragmentation and reference counting. Each round still leaves the engine safer and easier to use.
A screenshot from the “Deadly Light” demo:
When you’re building a game level you need to see debugging information about the level and its models. I used to place every object, lights included, in 3ds Max, and it was slow going: tweak the lighting in 3ds Max, switch to the engine, check the lighting, and round again.
The Glow engine now has several debug modes: wireframe, world-space normals, pixel depth, lights, physics, AI (triggers) and navigation meshes. You can turn on several at once, except normals, depth and diffuse colour, which can’t be combined. To pick a mode, press the “layers” button and select it.
A room from the “Deadly Light” demo, lit by 6 lights:
The same room in wireframe mode:
Normals mode:
The lights debugger:
Physics:
Depth:
I’ve updated www.glow3d.com so it reflects the recent changes to the engine.
Most of those changes are about concurrency. The Glow engine now runs its tasks on Intel Threading Building Blocks instead of boost::thread_pool, and the scene graph of sectors and portals is a template that several threads can read at once. The renderer and physics each keep their own copy of it. None of that was on the site yet, so it needed a refresh.
Intel Threading Building Blocks is now built into the engine. It’s a good library with a lot of features.
I’ve started on a concurrent scene graph. The scene graph (sectors and portals) is now a template, so it can hold different types of objects, and several threads (or tasks) can read it at the same time. Some engine modules also keep their own copy of the scene graph so they only deal with what they need: the renderer holds one with the visible objects, and physics holds one with just the physical objects, which keeps the simulation cheaper.
I’m getting deeper into concurrency and multithreading, and I’ve decided to use Intel Threading Building Blocks instead of the experimental boost::thread_pool.
All computation in the Glow engine is organised into tasks, so in theory it should scale well as the number of hardware threads (CPU cores) goes up. Until now I ran those tasks on boost::thread_pool. Coming back to the concurrency code, I realised that approach can’t scale, because the pool relies heavily on spin-based mutexes.
Later I’ll run tests to measure how both methods scale and post the results here.
I am thinking about a concurrent scene graph.
Several modules in the engine need to query the scene graph: physics, audio, rendering. In the future, several threads (tasks) inside a single module will also need concurrent access to it.
The Glow engine now has high dynamic range lighting with eye adaptation, so the exposure adjusts over time as the brightness of the scene changes. Rendering is also gamma-correct now.
I have also started on colour transformations (hue, saturation, brightness and so on). These are post-processing effects, like the motion blur from last month.
Memory management matters a lot in the console world. On a PC we can mostly forget about it, at least for a few hours of running without leaks. Consoles are restricted, so we have to assume that at some point a memory allocation will return NULL (or throw an exception, depending on the compiler). Fragmentation is another serious problem once many small objects have been allocated. We plan to move to mobile devices, so we decided to build a proper memory management system.
I am working on memory management for the engine at the moment. A lot is done already, and I will write about it in detail later. This post is about arrays.
Dynamic allocation is a bad thing, especially in optimised applications. You get fragmentation, locks, and time spent searching for a free block. The general advice is to avoid any kind of dynamic allocation in the update loop, so I decided to remove it everywhere and move all dynamic data to the stack.
I went to SIGGRAPH Asia 2010 and found it interesting. I attended several courses and met a number of interesting people.
Tessellation is still the hot topic. It was discussed at length in both the DirectX 11 and the OpenGL 4.0 courses. Real-time ray tracing and stereoscopic rendering also came up a lot.
The timing suits my own work. The engine has just got a post-processing pipeline with motion blur, and deferred reflections are next on the list.