The Summer Palace is one of the most popular tourist spots in Beijing, China. However, the name "Summer Palace" used to belong to Yuan Ming Yuan ( Figure 1
), one of the three most important architectural projects in Chinese history together with the Great Wall and the Dun Huang Caves. Translated into English, Yuan Ming Yuan means "the Garden of Perfect Brightness." In October 1860, at the peak of the Second Opium War (also known as the Arrow War), the British and French joint army set Yuan Ming Yuan on fire. The garden of gardens was burned to the ground in one of the worst acts of cultural vandalism in recorded history. To bring Yuan Ming Yuan back to life, we're building a digital version using computer graphics.
Figure 1. Yuan Ming Yuan overview painting.
Emperor Kang Xi started Yuan Ming Yuan's construction in the early Qing dynasty around 1700. Six generations of Qing emperors took 150 years to finish it. In its heyday, it covered 350 hectares and included more than 100 scenic sites ( Figure 2
), hundreds of lakes interconnected through waterways, 2,000 architectural structures, millions of pieces of furniture and precious objects, and countless plants, trees, rocks, animals, and birds from all over the country. Yuan Ming Yuan was more than an imperial playland, it was the largest and richest museum China ever had.
Figure 2. Lian Xi Le Chu (the Scholar's Delight), one of the 100 scenic sites in Yuan Ming Yuan.
The high art of gardening evolved over 5,000 years of Chinese culture and tradition. Architecture, gardening, landscape design, calligraphy, poetry, and painting are composed together to offer many layers of discovery and wonderment and to elicit both the surprise of novelty and the pleasure of recognition. Gardens were designed to be a totally integrated expression of heaven and people (nature and culture) seen through each other's perspective. Yuan Ming Yuan was the most successful achievement of this art form in mankind's history.
Since 1949, when the People's Republic of China was founded, people have tried to protect the Yuan Ming Yuan ruins. They built a 10-kilometer wall around its land. They restored the landscape and waterways as close as possible to their original forms and planted thousands of trees. Thousands of peasants who lived and farmed in Yuan Ming Yuan were emigrated outside the garden. Today, the garden is called The Yuan Ming Yuan Ruins Park, and people are encouraged to rest, wonder, meditate, and learn there.
Research on Yuan Ming Yuan
Decades of work by scholars in China and the rest of the world has produced much research material on Yuan Ming Yuan. For example, researchers have discovered more than 1,000 pieces of blueprints of the original garden plan. During Yuan Ming Yuan's construction, architects built small miniature models for the emperor's approval. Many of these models still exist in the Forbidden City Museum.
Visitors to Yuan Ming Yuan have described it in detail through letters and diaries. Poems written by emperors and imperial activity schedules recorded by officials provide a detailed record of many garden aspects.
After the garden was relatively complete, the emperors commissioned artists to paint the garden and imperial daily life. These paintings offer an authentic look at the garden. The most famous paintings are the "Forty Views of Yuan Ming Yuan," which show the most famous forty scenes of Yuan Ming Yuan from an overview perspective. Another set of paintings shows the imperial daily life in the garden. These paintings present a close-up view of the garden, including interior furnishings and imperial family members ( Figure 3
Figure 3. Painting of Emperor Qian Long resting in Yuan Ming Yuan.
Immediately before the garden was set on fire, the British and French soldiers and the locals looted it. Although most precious objects were destroyed, many were taken from the garden and scattered in various museums around the world. Many large items such as sculptures were later moved to other locations in Beijing after the garden was left unattended. We used the pictures of these existing objects to create the digital replicas.
The research on Yuan Ming Yuan extends to imperial life, stories, and legends. We feel an empty garden without life and history is not the complete Yuan Ming Yuan people would like to see. So we used the research to create products based on the garden and its stories. Many imperial archives recorded the emperor's daily schedule, which linked Yuan Ming Yuan's different sites to various imperial activities. We tailored these stories and built our products around them.
We believe we can produce a meaningful reconstruction of Yuan Ming Yuan so that people can glimpse its original beauty, the imperial life, and its history, even if the digital version doesn't fully match the original.
The Digital Yuan Ming Yuan
A complete reconstruction of the garden is impossible for many reasons—the cost of the reconstruction as well as the loss of workmanship, furniture, and antique objects.
We believe a digital version of Yuan Ming Yuan is the best and most feasible way of restoration ( Figure 4
). One of us (Lifeng Wang), a graduate from the Computer Graphics Research Group at the University of British Columbia, initiated the project. The project's cultural and historical significance as well as its potential in other areas of research made it possible for the Media and Graphics Interdisciplinary Centre (Magic) and Imager—computer graphics groups at UBC—to offer their support. Xing Xing Computer Graphics, an independent company founded to commercialize the project's results, now manages the project.
Figure 4. The digital Lian Xi Le Chu.
Initially, we aimed to build as complete, detailed, and accurate a computer database as possible to reconstruct a digital Yuan Ming Yuan. However, a substantial amount of funding is required to accomplish the project. Our strategy is to commercialize the database to create products and use the profits to support the project. Given the project's scope and the reality of public funding, this is the only feasible way for us to continue the project.
We used Alias|Wavefront Studio software (from Version 3.2 to Version 8.5) as the basic tool for this project. It's a perfect modeling tool for intricate and organic-looking Chinese classical architecture and landscape. Studio's full suite of tools for texture mapping and rendering helped us generate the imagery we wanted.
The main drawbacks of the Studio software were an inefficient rendering speed, the inability to handle large databases, an unstable animation module, and a limited ability to model natural phenomena. Fortunately, Alias|Wavefront's next-generation software, Maya, provides advanced animation tools and open architectures. With Maya, we can develop plug-ins to address some of the problems mentioned and meet specific modeling needs. For example, we could write a tool to model Chinese classical architecture procedurally per the detailed rule specifications. With such a system, we could create a full building in seconds instead of days after knowing a few key structural parameters.
We used composite software extensively in this project. Initially we wrote a compositing program to do very simple work. After the scene became more complicated, we switched to Alias|Wavefront's Composer.
We used Macromedia Director to create our first product, Lotus Spring, an interactive adventure CD-ROM based on a true love story that happened in Yuan Ming Yuan. With Director 6.5 we incorporated the images, animations (QuickTime movies and .wav files), music, and sound (.wav file) to create interactivity. Although Director had most of the tools we needed, its efficiency couldn't match custom-made engines, which we did not have the resources to develop.
For hardware, we started out using IBM/6000 workstations. However, because of slow hardware updates and Alias' inadequate support for IBM machines, we switched to SGI workstations running the current version of Studio. We routinely used five workstations in this project, but can have up to 25 workstations rendering concurrently. One SGI Origin 2000 workstation with 512 Mbytes of RAM and four CPUs helped solve the bottleneck of complex scenes that require large amounts of memory. For multimedia authoring, we used a PC running Windows 98 and a PowerMac running MacOS 8.1 to ensure Lotus Spring would run on both platforms.
Throughout this project, we encountered several modeling and rendering challenges.
Reconstructing the many scenes of Yuan Ming Yuan proved a monumental task.Architecture. After such extensive destruction, we had limited information on the garden's architecture despite the availability of blueprint pieces. Since Chinese classical architecture has distinct styles, each style has strict rules that architects must follow. Traditionally, given less than five parameters of a building, an architect would derive the rest of the details based on the rules. For example, the door's size can be derived from the pillar's diameter. Given the known parameters of each architecture, we followed the same set of rules for the digital reconstruction.
Another challenge was the size of data for each architecture. Each Chinese classical architecture has hundreds of roof tiles and the same number of wood beams below the roof. Even using instances instead of copies, these tiles and beams still more than triple the architecture's data size. It would be nice if Alias|Wavefront had a good displacement map algorithm to create them using texture mapping instead of real geometry ( Figure 5).
Figure 5. One pavilion modeled.
Landscape. For the digital models, we measured the garden sites one by one. Many building and pillar bases still exist and offer accurate data about the size and shape of each building. Satellite and aerial survey data, combined with on-site measurement and description material, provide close-to-original data on the landscape.Interior furnishings. Yuan Ming Yuan contained millions of pieces of decorative furniture. Most were intricate and detailed. We spent much time modeling these objects because we didn't want to lose the most important part—the details. With good 2D paint and texture-mapping tools, we were able to create some impressive art objects to furnish Yuan Ming Yuan ( Figure 6).
Figure 6. An incense burner set.
We could have used 3D scanners to scan some existing objects for texture and geometry. However, then we would have difficulty reconstructing them in a way that would look comparable.Human figures and clothes. We experienced great difficulty in character modeling and animation. Realistic human models have always been on the front edge of computer graphics research and remain a difficult problem to solve. Realistic facial modeling and expressions are even more difficult to achieve.
Emperors and empresses in Yuan Ming Yuan wore costumes with detailed texture and patterns that matched their status. Human models wore the costumes during the animation process. However, this proved difficult because when the model changed positions, the position of the clothes shifted ( Figure 7).
Figure 7. Animated figures in Lotus Spring.
Plants, trees, flowers. As you might expect, a Chinese garden has plants, trees, and flowers everywhere. Not only are they outdoors, but many are also put indoors as bonsai. Based on research, we estimated that Yuan Ming Yuan featured at least 500 types of plants in the garden—some rare, some even artificial—including hemp, a favorite ground cover. Botanical accuracy was of course necessary, and many of the trunks, leaves, and flowers had complex shapes.
Many software packages specialize in plant modeling, and some of them produce impressive results. Przemyslaw Prusinkiewicz is well known for the software he developed based on L-systems to model plants and trees. He and his colleagues provided the software (and training) for our project. Its modeling power is good, but our main problem was its data size—a single tree could be as large as 10 Mbytes in DXF format. Another problem is texture mapping, which can't be done in Studio because the plant data is polygonal. Polygonal leaves are copies of each other instead of instances. During camera movement, geometry data is not automatically simplified. All these problems prevented us from modeling plants effectively and efficiently.Tai Lake rocks. Many Chinese gardens feature miniature mountains and caves made of rocks from Tai Lake in Southern China. These rocks have been eroded by the lake water for hundreds of years. They have thousands of holes of various sizes and are often of beautiful shapes.
Tai Lake rocks are complicated both topologically and geometrically. Modeling a Tai Lake rock proved challenging, because we tried to model a specific appearance and often a specific rock, but at the same time we wanted to achieve a natural look.
Therefore, we developed our own models. We designed and modeled the rock's overall shape in Studio using parametric spline surfaces. Then we scanned the shape and converted it onto a 3D grid. The grid size was chosen as a function of the level of details to be achieved for the resulting rock. Then spheres modeled the holes (actually, they can be modeled by any volume that can be scan-converted, but spheres suffice). These spheres were subtracted from the overall shape by a constructive solid geometry operation. The result was an object defined in a volumetric grid. We then tessellated the object into polygons and imported it into Studio.
Figure 8 shows the digital rock (right) next to a picture of a real rock we used for the overall shape (left). In the example, the model has about 3,000 triangles and topological holes. When rendering, we applied a fractal 3D texture to the surface to make it appear more realistic. To create different types of rocks, we changed the shape and distribution of the holes procedurally.
Figure 8. Ornamental rocks in Chinese gardens (left) and its digital replica (right).
Water. Water is a key element of a Chinese garden. From the Sea of Fortune, the Yuan Ming Yuan's central body of water, to the hundreds of ponds, streams, and waterfalls, water covers more than half the garden. It plays a static role (as a reflecting surface) as well as a dynamic role in streams, currents, and falls. The main modeling challenge was including the dynamics of water. For example, when a boat runs through the water, the waves and splashes it produces are difficult to create by hand. These effects require physics-based modeling systems that the Studio software doesn't currently support.
Even though good models for waves, wakes, and moving water exist, we faced the problem of incorporating them together with the 3D models created in Studio. Therefore, we have not yet included them in the Yuan Ming Yuan model.
Rendering time, especially when producing animations, represented a real bottleneck for this project considering our limited computing resources. This is because the Alias|Wavefront renderer wasn't efficient, we had a large database, and we demanded high-quality rendering. On average, one good quality image at NTSC resolution (645 × 486) requires a 200-MHz SGI workstation with 180 Mbytes of RAM and takes 20 to 30 minutes to raycast. If we used the raytracer, the rendering would take close to an hour.
We used Alias|Wavefront Composer to achieve manageable rendering time, to filter down images for better quality, and to process the images to give them a unified look. Some scenes were composed of up to 24 different layers of compositing ( Figure 9
Figure 9. One scene with more than 20 layers of compositing.
Water reflection presented another rendering challenge. The Alias|Wavefront raytracer was unrealistically slow for this purpose so we wrote codes to create fake reflections from water surfaces. For example, we took an image containing only the objects above water, flipped it vertically, then ran image processing algorithms on the flipped image to create the reflected version of the original image. Then we composited the two images to create the final ( Figure 10
Figure 10. Scene with water.
We used texture mapping extensively to reduce the geometric count. One trick we developed was to replace leaves inside a tree head (versus on the outer perimeter) with 3D-textured spheres.
We achieved relatively smooth and realistic animation for animals, birds, and butterflies in the project. Studio didn't provide powerful tools for animating human figures. Plus, motion capture was too expensive for the project. We manually animated the characters and their clothes using Alias|Wavefront tools and went as far as we could. The new Maya software features the Maya Cloth module that would have met our modeling and animation needs.
We've created a large database of detailed and accurate models, texture maps, images, and animations. The models are mostly NURBS-based spline models with some polygonal models for plants and rocks. All are stored in the Alias native wire format. Our database currently has more than 1,500 models, taking up more than 8 Gbytes of disk space. These models include about 500 architecture models, 500 furniture and object models, plus models for people, animals, trees, flowers, rocks, and so on. A standard architectural model will have more than 100 spline patches, which convert to more than 500,000 polygons during rendering.
In terms of the ultimate goal of reconstructing the entire Yuan Ming Yuan, we've rebuilt about 10 percent during the last four years (with three people working on it full time).
With the help of procedural modeling methods, modeling the architecture, landscape, and plants might take another 20 person-years to complete, even considering that we're far along our learning curve. Research on interior furnishings and modeling thousands of pieces of objects will be the most time consuming. Creating products based on Yuan Ming Yuan models and stories represents another set of efforts. We estimate that each product will take another 20 person-years to finish. In short, this project has a long way to go, assuming it will ever get finished.
Xing Xing Computer Graphics has completed the first product, Lotus Spring, a story-based multimedia adventure in Yuan Ming Yuan. The product has been released in Europe, North America, and China, and received a positive response. It provides an alternative to those who are interested in cultural games with deep content and real history versus the violent games mostly available today. Xing Xing is also planning several other multimedia products based on true stories or legends from the garden. Each product will use a section of the garden as its setting. Thus, after developing approximately five to eight products, the exterior of the digital Yuan Ming Yuan will be complete.
Xing Xing is also planning an Internet virtual world based on Yuan Ming Yuan. Although real-time rendering of the models and Internet bandwidth are not expected to be feasible anytime soon, both are improving rapidly. Virtual Reality Modeling Language (VRML) and Java 3D are possible technologies to support the virtual world. The Web site will let visitors travel 200 years back in time to the greatest garden ever built. They'll be able to meet with the imperial family and get involved in their daily lives and stories based on the true history and legends. They'll experience different festivals, seasons, weather, and times of day. Plus, they could make friends with the digital characters in Yuan Ming Yuan, all of which have artificial intelligence and personalities.
Unlike the Lotus Spring CD-ROM, the virtual world will eventually be set in real time. It will have a much larger hard disk to store the entire garden. Visiting the virtual garden and learning about it through an interactive experience will be more interesting and entertaining for people.
Though many believe technology only destroys culture and history, in our case applying new technology to restore lost culture and history has proven highly valuable. In many ways modeling the Yuan Ming Yuan resembles modeling the world—the challenges we met are the challenges every similar project must face in some form.
The quality of current commercial animation software lets us accomplish much of our work. However, during this project we identified some of the difficulties, especially when modeling natural phenomena.
The original goals we established are all attainable. The models have scientific value and were implemented in the CD-ROM product, Lotus Spring. At the research level, Fred Brooks and his collaborators at the University of North Carolina at Chapel Hill are using some of our models in virtual reality environments.
Visit our Web site at http://www.xing-xing.com to see what we've accomplished thus far and for more background information on Yuan Ming Yuan. The site also provides many images from and information about Lotus Spring.