Virtual 3D city models provide the ability to communicate complex 2D and 3D geographical and geospatial data in an effective way (Fig. 1) . Visualization systems that are based on virtual 3D city models serve as enabling technology in GIS and in manifold fields of application, for example, as a platform for presentation and communication of geodata (e.g., city marketing, urban planning, public participation) and as visualization used in navigation. Also, they serve as a basis for 3D computational analysis  such as solar potential analysis of roof surfaces (Fig. 2)  or urban noise pollution. Due to the massive amounts of data to be processed, managed and rendered (ranging from several GB to TB), common applications for 3D geovisualization, running on specialized, high-performance workstations, take advantage of large main memory, extensive disk capacity and high-performance 3D graphics hardware. As a classic kind of desktop application, the software architecture of such systems is characterized by its monolithic structure. Consequently, design, implementation and deployment of 3D applications and systems face a high degree of complexity regarding implementation, portability, robustness and deployment issues.
Figure 1: Examples of 3D visualization of the Berlin virtual 3D city model.
Figure 2: Visualization of a solar potential analysis based on a virtual 3D city model. Service-based visualization of virtual 3D city models
The service-based visualization represents a key prerequisite for the use of virtual 3D city models in future applications and systems. The classic approach of a monolithic application architecture and resulting "fat” client applications is not suited for this purpose for several reasons:
- Data transmission between servers and clients that is limited by network bandwidth
- Massive 3D datasets that can hardly be efficiently stored and managed on clients
- 3D rendering techniques that are restricted to 3D graphics capabilities of clients
- Advanced 3D rendering techniques that can only be partially implemented on clients
- Energy consumption that is drastically increased by 3D rendering computations
A general solution to these challenges can be designed following a service-oriented approach for visualizing virtual 3D city models. In general, service-oriented architectures for 3D geovisualization foster the separation of concerns into management and rendering of virtual 3D city models and their interactive provision by client applications. While services for geodata access (Web Feature Service or WFS), processing (Web Processing Service or WPS) and 2D portrayal (Web Map Service or WMS) are standardized and are successfully used in the context of spatial data infrastructures,service-based management and rendering of 3D models as well as corresponding 3D client applications have not matured to a similar degree. We refer to these approaches as 3D portrayal services because the portrayal in the sense of visualization represents their main functionality.
Two types of 3D portrayal services, currently discussed in the Open Geospatial Consortium (OGC) context, can be distinguished:
- Web 3D Service (W3DS): It handles geodata access and mapping to renderable computer graphics primitives (e.g., textured 3D geometry represented by scene graphs) and their delivery to client applications .
- Web View Service (WVS): It encapsulates the process of generating images of 3D models and scenes (i.e., it implements the geovisualization pipeline), delivering rendered image representations ("portrayals") to client applications [6, 7].
In the W3DS case, client applications implement and perform 3D rendering, that is, clients have to process geometry and texture data of 3D city models. Consequently, W3DS clients rely on and are limited by their own memory and graphics resources.
In the WVS case, the server implementation encapsulates the whole 3D rendering process, which has to cope with geometry and texture data of virtual 3D city models; the implementation and deployment can be optimized for a given (controlled) hardware and software configuration. The WVS approach decouples a client from the complexity of 3D city models – the clients have to handle the image sequences but are not affected by the complexity of the underlying 3D model. Furthermore, the WVS approach allows us to use and optimize advanced 3D rendering techniques in contrast to implementations that have to be compatible to a potentially large hardware diversity of client devices. Client applications, in general, require only moderate 3D graphics capabilities to reconstruct or display the virtual environment encoded in the image sequences or 3D panoramas. This way, the WVS approach facilitates and simplifies the implementation of 3D visualization systems.
Basic 3D client applications (e.g., a browser-based application to request a single view) retrieve a single image containing a perspective view of the virtual 3D city model from a WVS and provide user interface elements for stepwise 3D navigation, requesting new images for the changed camera parameters. Compared to 3D clients that directly manage and interactively render 3D models (e.g., retrieved from a W3DS), 3D clients based on perspective views can only provide a limited interactive 3D experience to the user. We are generally faced with a trade-off between interactivity (based on client-side 3D rendering) and model complexity (costs for streaming 3D model data from the server to the client) in the scope of service-oriented visualization.
In our approach, we solve that trade-off to a certain degree by generating images of a six-sided box, the cube map panorama, transferring these six images to the client, where a specialized 3D rendering application allows users to interactively visualize and explore the virtual 3D world. For this, the 3D rendering process, running on the server, creates cube maps of virtual 3D city models. The 3D client application uses the cube map to simulate a discretized version of the virtual 3D city model. If the user zooms in or navigates through the 3D space, the client requests a new version of the cube map. For the 3D client application, the virtual 3D panorama generally represents a lightweight 3D model.
The key advantages of this approach are:
- The 3D client application doesn’t have to cope with the original model's complexity because it receives a stream of cube maps with a fixed resolution.
- The 3D server process can implement advanced 3D rendering techniques and use specialized 3D graphics hardware but doesn’t have to consider the graphics hardware profiles of the clients.
- The client and server only interchange a sequence of cube maps, encoded by standard image formats (or video formats), and clients can compensate for delays in the delivery of cube maps using previous, possibly non-optimal cube maps as an intermediate approximation.
Our implementation  of such a cube map-based client-server 3D rendering approach fetches different image layers (color, camera distance and object ID), referred to as G-Buffer in computer graphics, from a WVS and incorporates two different techniques to build a partial representation of the complete scene using this image data:
- A textured cube to enable a free look around (change of viewing direction)
- A 3D triangle mesh built out of camera distance information from the depth-image layer as intermediate representation of the 3D world during movements of the virtual camera
3D meshes approximately the server-side 3D model for one specific viewpoint (Fig. 3). They are used to maintain user orientation in case a user moves the virtual camera. After camera motion has stopped, a new cube map representation is requested, providing an optimal view on the scene for the new camera position and orientation. Such reduced, lightweight representations of 3D city models allow for interactive client applications to also run on mobile devices, such as smartphones and tablets (Fig 4).
Figure 3: Side view (left) and view from the requested camera position (right) of a mesh reconstruction for one side of a cube map. Depth and color information are used to generate a textured 3D mesh. Areas where no data for reconstruction are available from the retrieved image layers are colored black.
Figure 4: Mobile interactive 3D client application using a WVS to visualize the 3D city model of Paris. BATI3D data courtesy of Institut Géographique National, France. Integration of geodata in image-based 3D geovisualization applications
Typical applications using virtual 3D city models need to integrate different kinds of 2D and 3D geodata and to visualize georeferenced data to fulfill these application-specific and task-specific needs:
- Thematic maps provided by WMS can be used as terrain texture overlays.
- Planning variants retrieved as 3D models from WFS can be rendered in addition to the city model.
- Georeferenced data originating from real-world sensors or the social Web can be mapped on graphics variables of the virtual 3D city model.
Different strategies can be used to integrate those data into the visualization process of virtual 3D city models. In general, the data can be visualized by means of additional 3D model objects or by changing the appearance or geometry of existing 3D model objects. There are three principal ways to implement these strategies (Fig. 5.):
- The data are mapped to 3D model elements added to the 3D scene by the 3D server system (server-based object-space mapping).
- The server maps and renders the data based on image post processing which is applied as an additional step to its rendering pipeline (server-based image-space mapping) .
- The data can be infiltrated directly into the client's 3D rendering process (client-based mapping).
The integration of application-specific data into the service-oriented visualization of virtual 3D city models can be managed by dedicated integration services. In particular, the image-based WVS is well suited as a core service to be used by such integration services, as it supports server-based object-space and image-space mappings. Client-based mapping, however, requires modifications of the 3D client application and increases the costs for client-side 3D rendering.
In summary, the service-oriented visualization approach based on streaming of 3D panoramas represents a promising strategy for future GIS and 3D geoinformation solutions. In particular, this approach encapsulates model storage, management and rendering on the server side, relies on streaming fixed-size cube-maps, and enables interactive use of virtual 3D panoramas in client applications. It can be extended by data integration services to adapt the visualization to the specific needs of an application. Most importantly, the approach allows us to take advantage of cloud computing for geodata processing, management and rendering. Therefore, it is a fundamental building block of future scalable 3D geovisualization systems.
Figure 5: Types of data that can be used for integration into a visualization process. Server side integration is shown, but similar techniques can be implemented on the client side. References:
1. Kolbe, Thomas H. “Representing and Exchanging 3D City Models with CityGML.” In 3D Geo-Information Sciences, edited by Jiyeong Lee, Sisi Zlatanova, William Cartwright, Georg Gartner, Liqiu Meng, and Michael P Peterson, 15-31. Springer Berlin Heidelberg, 2009.
2. J. Engel and J. Döllner, “Approaches Towards Visual 3D Analysis for Digital Landscapes and Its Applications”, Digital Landscape Architecture Proceedings 2009, 2009, pp. 33-41
3. Homepage of the solar atlas Berlin, http://www.businesslocationcenter.de/en/3d-model-of-berlin/the-project/project-examples/solar-atlas
4. D. Hildebrandt, J. Klimke, B. Hagedorn, and J. Döllner, “Service-oriented interactive 3D visualization of massive 3D city models on thin clients” in Proceedings of the 2nd International Conference on Computing for Geospatial Research & Applications - COM.Geo ’11, 2011, pp. 1-1.
5. Web 3D Service Homepage, http://www.w3ds.org
6. B. Hagedorn, D. Hildebrandt, and J. Döllner, “Towards Advanced and Interactive Web Perspective View Services” in Developments in 3D Geo-Information Sciences, 2009, pp. 33-51.
7. Web View Service Homepage, http://www.webviewservice.org/
8. D. Hildebrandt, “Towards Service-Oriented, Standards- and Image-Based Styling of 3D Geovirtual Environments” in Proceedings of the 5th Ph.D. Retreat of the HPI Research School on Service-oriented Systems Engineering, 2011, pp. 133 - 147