{wf}shadowspawn
he took away our vise-grips, and gave us a dream-set of metric tools from snap-on.

Originally posted by ydnar:
Q3Map, circa 2002-02-13:                 

The past few weeks I’ve been working on the innards of Q3Map. Whole sections have been gutted or rewritten from scratch, and new additions have been made. Prior to this, the main concern has been compiling efficiency (time). The past couple weeks have been focused on bsp efficiency (file size/rendering performance). Sweeping changes have been made to how surfaces are generated and the bsp is populated:

So, I was able to add some chrome:

While performance and functionality were the primary goals, I wasn’t too keen on the idea of new bugs, especially when introducing new functionality. Most of my time has been spent running regression tests, comparing old apples to new apples. With the previous stable builds as a baseline, any new version should mimic their behavior as closely as possible. While laborious, it forced me to fix all the known existing bugs. This is a good thing.

That out of the way, I could start comparing apples to oranges, and focusing on drawsurface generation. The entire surface pipeline has been given a new fork, which I term metasurfaces.

Quake3 still segregates various types of renderable surfaces in a map to some degree. Brush faces, patches, misc_models, terrain, etc. They have all have one thing in common though, except patches, but more on that later:

They can all be decomposed into their component triangles.

Previously, brush faces were each reduced into a single drawsurface, with triangles in strip or fan order. Terrain was reduced to a set of triangles, ideally in strip order. Model surfaces were only as good as the tool that generated them.

The new metasurface path eliminates the distinction between triangle surfaces. All triangles generated from brushfaces & terrain, etc., are thrown into a global pool of metatriangles. Drawsurfaces are then stitched together from these triangles. What this means is that the bsp contains fewer drawsurfaces, fewer drawverts, and, potentially, fewer triangles. This translates into a smaller bsp, lower r_speeds, and higher performance while in-game. My tests have shown surface count reduction on the order of 30-50%, and a drawvert reduction of 10-30% because of collapsed vertices. (Keep in mind that this doesn’t include non-planar surface merging) By default, neighboring triangles will not be merged into a metasurface if they do not lie in the same plane. This can be overridden with a new shader directive, q3map_nonplanar. This directive instructs Q3Map to merge any adjacent triangles with this shader into a non-planar triangle soup. Previously, a rocky wall or some ground, constructed of brushes, would reduce into a large number of drawsurfaces and coincident verts. The q3map_nonplanar directive allows Q3Map to merge them together into a single surface. This tends to reduce vertex count by a factor of 60% or more, depending on how many brush faces share a single point.

This new metasurface can be vertexlit or lightmapped.

If you’ve been reading between the lines, you’ll note that means anything can be lightmapped, including misc_model surfaces. There are restrictions, however. Lightmaps are only projected on a single x, y, or z-axis. All the triangles in a model must generally face in one direction, or the results are undefined.

Naturally, like all restrictions, this is subject to change, so don’t quote me on it. I have to tie up a few remaining loose ends, test it across a wider variety of maps, and shake any remaining bugs out.

Cheers,

y

 

Q3Map update 2002-02-21

The merging algorithm went through a few rewrites, and I’m confident its solid now. It does the right thing almost all the time, preferring edge sharing, plane and brush matching, as well as a few other criteria. However, combining fragmented brushes into larger surfaces has had a few drawbacks. Lightmap allocation was blown out by a factor of 150-200% because of the larger, non-convex surfaces. Raytracing was slower, taking nearly twice as long as 1.2.4-y3 to light maps. Sometimes surfaces would disappear when walking around a map. I addressed the disappearing surface problem first, with an overhauling the surface>bsp filtering code. It’s a little more robust now. More references are generated, as the larger surfaces touch more leaves, but throughput is still higher, because of vertex collapsing. Today I addressed the lightmap allocation and raytracing performance. I rewrote a section of the raytracer to take advantage of planar consistency across large surfaces, which resulted in lighting performance comparable to earlier versions. The lightmap allocation was a bit more interesting. As it turns out, maps have quite a bit of similar lightmaps that can be collapsed. As the new metasurfaces are no longer convex, there is potentially a lot of wasted lightmap space in the gaps. The solution was a cheap 2d memory allocator that searches for pixel-perfect gaps in a lightmap page. There may be tradeoffs, with lightmaps no longer being coarsely ordered by shader, but I don’t imagine it’d be any worse than what larger maps already do to vidcards with limited texture memory. My primary test map, q3dm17, has gone from about 2.3mb to 1.2mb with no loss of visual quality. There are roughly 35% fewer drawsurfaces, and about 1000 fewer verts. Q3dm7 showed a lot of benefit from the new lightmap allocator, going from 22 lightmaps to 14.

Also added a few new things for debugging purposes, which mappers may find useful:

Debugportals, Debugsurfaces, and the importing and exporting of lightmaps.

There have been many other subtle changes made, on vertex lighting, filtering, surface construction, indexing, etc. It’s semi-stable now, and works with all the weird cases I throw at it. About all that’s left are a few modifications to the terrain code to use metasurfaces, now that the metatriangle path has been proven, and fixing radiosity to work properly with the new lightmap code. As a side note, vlight is probably broken. Updating it to be aware of all the new things would probably be a nightmare. I can split it up so that it works with old-style bsp’s (which Q3Map can still generate), but that’s about all the effort I care to commit to it.

Cheers,

y

Tell Lunaran to try out 2.0 with -light -fast -super 2 –filter. He may be pleasantly surprised.

Aphax: If we could only have the bat file of an rtcw mapper… :(

Vexar: I love this. You’re going into the special thanks section of the map txt file; I’ll even create a shrine for you somewhere in the final version of the level.

{wf}shadowspawn: Yeah I put thx in txt file too. I swear I was able to learn more about lighting in one hour than what used to take days. It’s like using tweezers to pick up leaves then being handed a rake.

cloudscapes: What takes an hour now, on that big canyon map of mine, used to take a whole weekend.  :)

dotExE: You can’t imagine how many mappers you are making happy.
ydnar: :)
dotExE: Is it realistic for the compile to take 63 seconds?
ydnar: Yeah, it’s really fast

dotExE: I didn’t know how to compile like a man before I came here.

Cardigan: OOO! How very exciting! What with ydnar’s lightmapped mapmodels, I feel a completely new range of possibilities coming on!

Lord Daimos: This is pure fact! Q3Map ydnar ownz me, ownz you, and should own all other mappers out there! It’s great, and even greater than that! It just saved me some hours of compiling on q3f_openfield! :) btw: did I say that it ownz? http://lorddaimos.railbait.com/default.asp