MapServer Update – Toronto Code Sprint 2009

This past weekend, a group of developers joined forces at the OSGeo Code Sprint in Toronto.  The hope was to attract members of the ”C Tribe” to come together and work on the C-based projects that are part of the OSGeo stack of software.  Christopher Schmidt [1,2,3] and Paul Ramsey [1,2,3,4] gave daily updates in their blogs.

Although many projects got some attention because many of the developers work on several of the represented projects, MapServer had a very strong showing. I thought I would take this opportunity to summarize the changes coming out of the code sprint for MapServer.

While we won’t see any of these changes in the forthcoming 5.4 release, it is likely that some will be incorporated into the next release, either 5.6 or 6.0.  I’ve heard rumours of targeting a 6.0 release in time for the next FOSS4G conference in September which would be excellent!  And some of the performance related changes don’t impact map file structure or syntax and could be packed into a quick release prior to that.

The EPSG Code Lookup Problem

One of the biggest problems we have run into is the use of EPSG codes to define projections in Map files.  MapServer uses the excellent PROJ library to handle projections and a long standing issue has been the performance of converting EPSG codes into PROJ parameters.  While it is possible to avoid EPSG codes in some cases, they are integral to OGC services such as WMS and WFS.  Frank Warmerdam has solved this in PROJ and we should see substantial improvement in common cases that use EPSG codes for defining projects.

Large Shapefile Access

Shapefiles have often been considered the fastest spatial format to use with MapServer but when it comes to large volumes of data, this has relied on dividing the shapefiles into many smaller files and then indexing the lot using a tile index.  Theoretically, shapefiles have a spatial index file that should make the divide and conquer technique unnecessary but Paul Ramsey spent some time last year digging into why this was not so.  While looking at ways to improve MapServer performance this week, Julien Samuel Lacroix of MapGears realized that more could be done with Paul’s code and he managed to get 20% better performance in his test case with a bit of tweaking.

Ghosted Labels

One of the common things that we do (everyone does) in MapServer to improve the readability of labels is to render them with an outline color.  It turns out that the current implementation of this involves drawing the label multiple times in the outline color.  While this doesn’t sound optimal, it was the only way to achieve the effect with the tools at hand when first implemented.  Technology has advanced and FreeType, in particular, now supports ghosting of text in a much more efficient way.  Unfortunately, MapServer uses FreeType through the GD library to render text and this requires a change in the GD library.  It may take some time for this change to make its way into stock distributions of MapServer because of this, but custom builds will probably be able to take advantage sooner by patching GD.

Single Pass Query

MapServer currently implements a 2-pass query system for identifying features that lay within a particular geographic area and then collecting their attributes.  The OGS WFS service, in particular, had performance problems related to this 2-pass system.  Steve Lime has been talking about replacing this with a single pass query that would theoretically provide substantial improvements.  At the code sprint, he spent quite a bit of time evaluating alternatives and discussing them with other developers and settled on one approach that he roughed out some code for.  Paul Ramsey then modified the PostGIS data provider in MapServer to use the new code and achieved a substantial 24 x improvement in query response time (from 12 seconds to 0.5 seconds!)  Although they are still in the investigative stage, this is a very promising improvement in query speed and directly affects users of the WFS OGC service.

Raster Access Improvements

Based on some profiling of MapServer performance, access to raster images apparently stuck out as a potential problem.  Frank Warmerdam has been galvanized to dig into these problems and was able to achieve some immediate benefits by accessing multiple bands simultaneously.  He also spent some quality time with Michael Gerlek (LizardTech) and improved GDAL’s use of the MrSID SDK.

XML Map Files

Moving away from performance related changes, support for Map files in XML has been a long discussed feature that seems to be coming closer to fruition.  Although the benefits of an XML Map file have always been somewhat controversial, one of the driving factors this time around seems to be that it would ease the implementation of Map file editors.  Daniel Morissette of MapGears is leading this effort and has some funding to complete the work, always one of the major factors in getting things done in Open Source projects.

Named Styles

Another new feature to MapServer coming out of the code sprint will be the ability to create STYLE objects at the MAP level and then reference them, and extend them, within CLASS objects.  This will have a significant impact on map file maintenance in large map files with complex styling rules and will provide the basis for supporting the named style portion of the WMS specification, something MapServer has previously not been able to support.  I’ve been hoping for something like this for a while and am very happy to see it finally in the queue.

Output Resolution

Recently, there has been a lot of discussion around creating high resolution output from MapServer that is suitable for printing.  There are two solutions for this:

• using a resolution independent output format such as PDF (discussed below in Standardizing Output Renderers),
• creating a proportionally larger image suited for the desired output resolution.

Since the PDF output driver doesn’t (yet) support all the features of the image output drivers, the most practical approach at this time is to create a larger image.  MapServer assumes an output resolution of 72 dots per inch (DPI), a standard assumption for many images intended for the web.  To create a 300 DPI image for printing, the same geographic extent has to be rendered into an image about 4 times larger.  The problem with this approach is that labels and lines appear 4 times smaller/thinner in the printed output.  Using a scripting language and the appropriate mapscript library, it is possible to programmatically modify many of the sizes and achieve reasonable results.  However, there are some things that simply cannot be changed through mapscript.  The solution that has been discussed and adopted at the code sprint is to support this scaling through

Standardizing Output Renderers

With the increased interest in different output resolutions for high quality printed maps has come a much needed discussion over the state of the output drivers in MapServer.  The current implementation is unique for each output format that is supported, with a lot of code duplication and missing features in each of the output formats.  Work is underway now to define a standard interface to output drivers similar to the approach taken for spatial data sources that will alleviate this problem and make it much easier for developers to target new output formats and rendering engines going forward.  A couple of interesting items to note on what could be coming:

• Cairo is a rendering abstraction layer that can output to a number of different formats including PDF, SVG, Postscript and image buffers making it an ideal candidate for replacing the current PDF and SVG drivers.  It even supports hardware-accelerated rendering on platforms that have the appropriate graphics hardware.
• OpenGL is a well known and supported technology typically associated with dynamic displays but it is perfectly capable of rendering to image buffers and takes advantage of hardware acceleration if avaialable.