David Nilsson

Daniel Pletinckx Editor: Halina Gottlieb

The Interactive Institute
Luc Van Gool
Maarten Vergauwen

The ARC 3D
Webservice
How to turn images of an object or scene
into 3D models for exhibitions and archives
David Nilsson
Daniel Pletinckx
Luc Van Gool
Maarten Vergauwen

The ARC 3D Webservice

Editor: Halina Gottlieb
How to turn images of an object or scene
The Interactive Institute into 3D models for exhibitions and archives
 The ARC 3D Webservice

2
 The ARC 3D Webservice

Summary

From the cultural heritage field comes upload these images to the web service and
a large demand for cheaper and more download the resulting 3D reconstruction.
flexible ways of 3D reconstruction. Aiming to The service is free of charge for non-
fulfil this need, the VISICS Research Group of commercial use and the only equipment you
the University of Leuven, Belgium (partners need is a digital camera and a PC with access
of the EPOCH network) have developed to the Internet.
a web-based 3D reconstruction tool. This
service is called the ARC 3D Webservice, The service consists of a pipeline that starts
‘ARC’ standing for Automatic Reconstruction with the user uploading images of an
Conduit, and enables its users to transform object or scene(s) to be reconstructed in
images of scenes or objects into 3D models. 3D. The automatic reconstruction process,
The idea behind the service is very simple: running on a server connected to a cluster
You take a series of pictures of the object of computers, then determines the camera
or scene you want to reconstruct in 3D, calibration and depth maps (containing

3
 The ARC 3D Webservice

information on the distance of each pixel to

the camera). The result can be downloaded The development of 3D reconstruction
from a server and visualised with a specific
tool running on the user’s PC. Until the early 1990’s, 3D reconstruction research team was one of the leading
from images was mainly carried out teams in this field and developed a fully
Once you have downloaded the result, with photogrammetric methods, automatic 3D reconstruction pipeline.
you have the choice between two different using calibrated stereo setups, special
tools that can be used for visualising the cameras, and quite a lot of manual An important advantage that comes with
result. The built-in Model Viewer is a simple interaction. The increasing speed the ability to work from uncalibrated
tool that comes with the ARC 3D installer. and power of computer systems, images is that consumer-grade cameras
It lets you view your result in 3D and gives however, lead researchers in the can be used, and the images can be
you the option to export the model into computer vision field to develop taken while holding the camera in one’s
several common 3D formats. The other tool calculation intensive methods that hand. These advantages make the image
that can be used is called MeshLab and was could deal with uncalibrated imagery capturing process easy and efficient.
developed by ISTI-CNR, Italy. This tool has (i.e. no information about the different The possibility to use consumer-grade
more advanced features and provides you positions or the parameter settings still or video cameras also helps to keep
with a wide range of options in editing and of the camera(s) is given). Moreover, costs manageable. The 3D results will
visualising your result. ‘dense matching techniques’ and ‘wide get better and better as the quality
baseline matching techniques’ were of such cameras improve, which is
This KNOWHOW booklet guides you step- developed that allowed these systems already happening at a blistering speed.
by-step through the process of creating a 3D to detect corresponding points in the Of course, not having to go through
model with the use of ARC 3D. It also looks different images automatically, thereby painstaking camera calibration is an
into future uses of 3D models by giving some eliminating most of the tedious manual important advantage in its own right.
tips on how museums can expand their interaction. The K.U.Leuven VISICS
website, archives and exhibitions by adding
3D interactive content.

4
 The ARC 3D Webservice

The top image shows the archaeological

remains of the roof of the Nymphaeum at
Sagalassos, digitised through ARC 3D and
structured light techniques, and puzzled
together digitally. The bottom image shows the
fully reconstructed roof.

Background
The cultural heritage community has always as the 3D shape. In contrast to several Sagalassos, Turkey (www.sagalassos.be) was
had a keen interest in the creation of 3D alternative 3D acquisition methods, the virtually reconstructed from images.
models of objects, monuments, scenes, shape and the surface texture are not
etc. Such models not only help in creating captured separately. This would lead to the A webservice is an attractive formula for
a more vivid picture of the past in the mind need for a shape and texture alignment, several reasons: the limited bandwidth
of the general public, but they are also of which may turn out to be tricky as both necessary to upload the images, the fact that
great value for training and research. Yet are rarely captured from exactly the same the computer-intensive processing is done
3D modelling technology has often been viewpoints. ARC 3D uses the same images on the servers and the possibility to update
expensive and time consuming to use. It to build the 3D shape as well as extract with improved versions without disturbing
is the goal of the ARC 3D Webservice to the necessary textures, which avoids any the users. These were all elements that spoke
provide the community with an affordable, alignment issues. in favour of a remote computing strategy.
easy-to-use and yet effective alternative. The ARC 3D Webservice has been operational
This figure shows an example result obtained since the fall of 2006 and is under constant
With cultural artefacts, the surface colour with ARC 3D technology. The roof of the development.
and texture are often at least as important nymphaeum at the excavation site of

5
 The ARC 3D Webservice

Prepare your recording:

How it works: The process of the service
Problems, tips & tricks
To minimise frustration it is best to take
The service and its process can be heed of some simple guidelines, so we will
explained in five steps: explain some dos and don’ts. All of the
information is retrieved from the images, so
1. Record a sequence of images of it goes without saying that the first and most
an object important guideline is to take good care
2. Upload the images to the ARC when recording them. It is better to spend
3D server two more minutes capturing the images than
3. The server computes the 3D to have to go back and redo the job.
reconstruction
4. Download the results from the Secondly, objects with a lot of surface detail
ARC 3D website are much more suitable than objects with less
5. Visualise the results detailed areas, such as for example homogene-
ous, white marble objects. Such objects should
Equipment needed ARC 3D: be scanned through other techniques such as
– A digital camera structured light or laser scanning.
– A computer
– Access to the Internet The third thing to remember is to avoid
objects blocking the scene since it is
impossible to reconstruct hidden parts.
Sometimes the object is self-occluding (when
parts are sticking out and thus hide other
parts behind them). Try to record the pictures
in such a way that self-occlusion is minimal.

6
 The ARC 3D Webservice

Often the first thing people try is to upload

their baby pictures. This will not work. Human
skin is notorious for changing appearance
when viewed from different angles, and it
lacks textural variations. This will make it
difficult for the system to identify pixels in
the different images as corresponding to the
same point on the surface. A blank, evenly
A good sequence of a statue of Mercator on Leuven city hall
plastered wall is a nightmare scenario for the
same reason. With all points looking alike,
the system cannot find the projections of the 1. Record an object – Centre the camera to one point of the
same point in the different images, and the object. Always take pictures with the same
3D reconstruction fails miserably. To obtain a good sequence: point centred in the frame of the camera
– Choose the object to be reconstructed while walking around the object. Always
For the same reason, the appearance of the in 3D and make sure that the whole object keep the same object in the frame of the
object must be consistent while recording fits inside the frame of the camera. Try camera and view the same object from
the photographs. Glare causes problems using a wide-angle lens if you can’t keep different sides.
due to conspicuous bright regions on the enough distance to fit the whole object in
object’s surface that move around as the the frame. It is also important not to record – Take very small steps while recording. For
camera is displaced. For the same reasons, the sequence too far away from the object an object that is 2 metres away from the
shiny surfaces, like metals, also prove to – the object should fill up the frame as much camera, take a picture every 60cm (or 30 to
be difficult. Transparent or translucent as possible. Do not use the ‘zoom’ to fill out 40 pictures all around the object, which is
materials like glass are even worse, as the the frame if not absolutely necessary. From an easy way in the field to know if you did
intensities and colours of the image pixels close range with the image zoomed out is a good job). In any case, you will develop
are determined more by what is behind the better than from a distance with a strong a feeling for the required density of images
object than by the object itself. zoom. as you experiment with the system.

7
 The ARC 3D Webservice

– Use fixed exposure time on your camera If a person stands still next to a statue, it is – Do not zoom in or out during recording (as
while taking photographs. impossible for the system to know that it is the camera parameters should optimally be
not part of a sculpted group and the person constant during a set of photographs). Avoid
To avoid bad sequences: ends up in the 3D model. Post-processing can large changes in focus as this can result in
– It is better to record too many pictures than remedy this, but the problem can be avoid a change of camera parameters.
too few. A minimum of five or six images is at the time when the images are captured,
required for a good reconstruction. Less than by making sure the person has moved in – Do not use a flash, as this changes the
four images and the reconstruction will fail. between takes. appearance of the object and creates

– The viewing angle between images should Required equipment

not be too large, i.e. adjacent images should
not be too far apart. 10 degrees is sufficient.
A good camera will give you better Your computer should be fairly modern
results than a cheap camera with a and with Windows OS installed. The
– The scene to be reconstructed in 3D should
subpar lens and sensor. minimum configuration is 512 MB RAM
be static. Independently moving objects
and a 64 MB graphics card. More memory
can be discarded by the system if the static
A 5MP camera (2500x2000) is typically and a better graphics card make viewing
scene is dominant in the image, so it is
good enough. More pixels mean more the constructed model more efficient and
recommended to avoid moving objects in the
3D points, but increasing the amount of will be necessary if you want to process
images. If there are independently moving
pixels also leads to a longer upload and your model afterwards (merging, cleaning,
objects (e.g. tourists walking around on your
processing time. Most importantly, the etc., as explained in Chapter 5 – Visualise
site), it is advisable to take more images
camera should be equipped with a good your results). An internet connection
than usual and to take images from nearby
lens. The better the lens, the more light (broadband strongly recommended) is
positions with larger time lapses in between,
it allows to fall onto the CCD, resulting necessary since the reconstruction of your
as to maximise the chance that all of the
in less distortion. Both factors lead to a photographs is done on the ARC 3D server
moving parts have sufficiently changed their
better final result. at the University of Leuven.
position in order not to confuse the system.

8
 The ARC 3D Webservice

changing shadows. Flashes also tend to Lighting conditions and setup your own. Using a tripod is a good option if
create glare, which, as mentioned above, will The lighting conditions when photographing no extra lighting can be arranged and long
cause problems. are crucial to obtain a good result. When exposure times are needed in order to obtain
photographing objects indoors you are good image quality. If your museum has an
– Do not use a turntable with a static camera most likely to suffer from insufficient light in-house photographer it might be a good
and an object that rotates in front of the when you are recording handheld images idea to involve this person in the process.
camera. Because of this very specific motion, without using the flash. Many digital cameras Take advantage of and learn from each
there is not enough information in the automatically enable their flash when the other!
images to allow camera calibration. light is insufficient, so remember to turn off
this feature for better results. One could consider creating an even, dark
– Do not pan from the same location, as if background when shooting the object,
you were recording a panorama. When the camera moves during recording, but this is neither strictly necessary nor
It is not possible to determine enough the image becomes blurred if a long always advisable (see later comment on the
3D information from such a sequence. exposure time is used. Most cameras have positive effect background structures can
a ‘night mode’ setting for bad lighting have on the automatic camera calibration).
– Do not walk in a straight line towards the situations. This feature increases the In any case, this differs from the typical
scene you want to reconstruct. In general, do camera’s exposure time in order to catch setup that a museum photographer would
not walk along a straight line without rotating more light through the lens. Do not use such use. In most cases you will photograph the
the camera. Unfortunately, this is a typical features because they will most likely lead object all around, which means that you
camera motion people tend to do. Again, to blurry images and cause problems for the will need a dark background all around. Of
there is not enough 3D information embedded reconstruction and its quality. course, this could be a partial background
in such images. The result will typically be that you move accordingly as you take the
deformed, as if it were skewed. Holding the To achieve a good result it might be worth images. If this is not possible, you could
camera in your hand typically means that the setting up some lights to provide a soft, create a similar environment by shooting
camera will not perfectly follow such path, but even illumination of the object. Be aware of in the evening or at night with sufficient
it is wise be aware of this movement. unwanted shadows on the object, such as and appropriate lighting on the object.

9
 The ARC 3D Webservice

In practice this means that in most cases, 2. Upload the images Image Uploader
the object does not need to be touched to the ARC 3D server Upload the images of the object you want to
or to leave the museum. By creating The first thing to do is to obtain an account reconstruct in 3D using the Image Uploader.
a dark background you will also simplify from the website that enables you to log in Thumbnails of these images are shown on
‘the masking process’ used to mask out and use the service. Go to www.arc3d.be and the left and a large version of the selected
unwanted information or background in the navigate to Request Login. image is shown in the main panel. Images
visualisation process (see Chapter 5). can be removed from the list or extra images
When you fill in your name and email can be added, but make sure that the images
It is important to note if the recorded object address, your login information will be sent are in the right order to make a continuous
does not fill the image, the calibration of the to your email address. Download and install movement around the object.
camera will not be very accurate. In these the latest version (Windows only) of the tool
cases it is better to not hide the background, from the ARC 3D homepage. The service On the bottom left, information about the
but instead keep it as it is because the contains two programs, the Image Uploader image such as the number of pixels, the
calibration process will actually use these and the Model Viewer. size on disk and the format are shown.
areas to improve the result. You can subsample the images to a smaller
size for faster upload and computation,
To avoid colour changes from one picture but at the cost of losing some detail in
to another (for example due to a dominant the final result. You can also organise
colour of a part of the object), we also your uploads by selecting or creating
recommend using a fixed white balance `scenes’, supply a sequence label or add
while shooting the photographs. Most digital an academic reference, if desired. (In cases
cameras today have this feature. in which the ARC 3D team would use your
images in a publication, they will add the
academic reference. For information about
the conditions, please see the ARC 3D
homepage.)

10
 The ARC 3D Webservice

When satisfied with the selected images, vision techniques that are used to compute that are visible and recognizable in at least
you upload them to the reconstruction the results are automatic matching of 2 images are useful. Note that a sufficient
server. We recommend uploading at least 6 images, automatic computation of camera number of points will actually need to be
images. In order to do so, you first need to calibration and automatic dense matching visible and recognisable in at least 3 images
authenticate yourself with your username for depth map generation. These are the in order for the automatic calibration process
and password. The upload is started and major steps in the pipeline and may be to be successful.
a progress dialog shows the speed of the fragile when supplied with an inappropriate
upload. type of imagery. 4. Download the results
from the ARC 3D server
Once the uploading begins, have a coffee! The entire process is based on finding pixels The ARC 3D service reports its result to
The process time can take from 15 minutes corresponding to the same 3D points in you by email. If the reconstruction has
up to several hours, depending on the size of the images, so-called ‘matches’. All of the succeeded, you are sent a URL link to where
the upload and the traffic on the server. information is recovered from them. This you can download a zip file containing the
means that the camera viewpoints need results. The zip file contains the original
3. Computing the 3D to be laid out in such a way that each part images, the calibration of the cameras, the
reconstruction of the object is captured without losing dense depth maps and quality maps for
An automatic reconstruction process the general view of the object. Only points every image. The quality map shows for
running on a server connected to a cluster of
computers, computes the camera calibration,
as well as depth maps for the images. When
the server has finished processing, the user
is notified by email and the results can be
downloaded from the ARC 3D server.

The unique feature with ARC 3D is that it

only uses images as input. The computer

11
 The ARC 3D Webservice

every pixel in how many images that pixel Visualise with the Model Viewer version of the image is shown in the main
could be detected, which gives a rather Each zip bundle contains textures, camera window. Since all images have their own
good idea of the reliability and quality files, depth maps and quality maps and corresponding range-map, a model can be
of the result for that point. MeshLab (see a config file named model.XXXX.v3d. Unzip created for any image in the set. This `model’
below) uses a colour coding of the quality the file and load the data in the application is not a complete object model yet, but a 3D
maps. If for example most of your object is Model Viewer. A thumbnail is shown for reconstruction for that part of the object that
blue, the reconstruction will be fine, but if every image in the set. If selected, a large was visible to the selected camera.
it tends to green or yellow the result will be
poor and peppered with holes or artefacts. Creating a mask
If the reconstruction failed, some more
information is given, including tips on how Often images will contain other for dragging rectangles), erase (the same
to improve the result. objects than the one of interest (e.g. as paint, except select the eraser) or grow
in the background). So for a ‘clean’ (by holding down Shift-Left button) black
5. Visualise the results reconstruction, it is necessary to discard regions to mask out the image parts
It is now time to visualise your result and this extraneous information. considered irrelevant.
there are two tools available for this task. Typically, it is useful to remove Once satisfied with the mask, or if no
The ARC 3D team recommends MeshLab, backgrounds, floors or support structures, mask is required, a model can be created
a tool developed by CNR-ISTI in Pisa, Italy as they are in most cases not part of the by selecting Create Model. A dialog will
that is downloadable from http://meshlab. object that one wants to reconstruct. prompt the user for the various model
sf.net/. MeshLab is a powerful tool but it A simple shift-click inside such a region parameters.
takes a little while to master. For a first starts a mask-growing algorithm that Note that masking takes place at the
visualisation to inspect the results you might covers a homogeneous region. visualisation stage. The masked objects
consider using the Model Viewer tool that It is also possible to paint (select the or background could still be used for the
comes with the ARC installer. pen; use the left mouse button for free camera calibration in the earlier stages,
motion or use the right mouse button which is a good thing.

12
 The ARC 3D Webservice

To create a model, begin by selecting an thresholds yield more complete models, includ- Zooming: Clicking the right mouse button
image. We suggest you use the image that ing areas with a lower reconstruction quality. and dragging
best represents the visible parts of the object, Picking: Clicking and releasing the left mouse
in other words the best view of the object. Once the reconstruction process is complete, button without dragging. This centres the
These partial reconstructions or `models’ the model is displayed in the model widget, view on the point that was clicked.
come as surface meshes. which can be accessed by selecting the Model
tab in the thumbnail panel. The key and The models (i.e. camera centred, partial
The user can select a quality threshold, which mouse actions are described below: reconstructions) can be exported in different
means that the parts in the reconstruction that 3D file formats, for use with several 3D
have a lower quality (see above) are discarded Panning: Clicking the middle mouse button software programs. Formats that are
when creating the model. High thresholds yield and dragging supported are VRML2, Open Inventor, OBJ or
models with higher quality, but more holes Rotating: Clicking the left mouse button and OpenSG’s native file format. OpenSG is the
where low quality parts are absent. Lower dragging standard choice for the EPOCH consortium.

By holding down the Shift key while left clicking on the blue background in this The images show a reconstruction of the Makalös statue at the Stockholm city
image, the mask-growing algorithm starts to mask the background museum with and without texture (surface details) in the Model Viewer tool.
13
 The ARC 3D Webservice

Visualise with MeshLab Ideas for implementation are suitable for 3D reconstruction and add
MeshLab is an open source, portable and We will not go through any third-party web- another dimension to your museum’s archive,
extendible system for the processing and publishing tools for 3D interactive content on both for staff and the interested public.
editing of unstructured 3D triangular web pages in this book, but there are several
downloadable tools that can be used for An appealing way of presenting valuable
this. For example, ISTI-CNR has created the objects to audiences and putting them into
Visual Inspector that can create an interactive a larger context is to use the VIRTEX tool.
application for large 3D models. VIRTEX is a presentation methodology for
museums that allows for a radical change in
You could record a series of objects on dealing with valuable pieces. VIRTEX is based
a special theme like your museum’s statues or upon a replica of an object that serves as the
other suitable objects and present these on the interface to explore this object. To make the
museum’s website. Publishing such content on replica interactive, an orientation sensor is
the website can spark visitors’ curiosity about integrated in the replica, so that the object
the museum even before their visit. can be visualised on a computer screen
meshes. The system is aimed at the by precisely following all the orientation
processing of the typically not-so-small The webservice can also be a good way to in- changes of the replica. The replica is made
unstructured models arising in 3D scanning, crease accessibility of objects that may not fit in from a digital model of the object, made
and provides a set of tools for editing, the museum’s current exhibitions. To solve this for example through ARC 3D. In this way,
cleaning, healing, inspecting, rendering and problem and present it in a fine way, one could the user senses the shape and details of the
converting these kind of meshes. create a 3D archive that would be accessible object, sees the virtual representation of the
for the public via a website. All too often, ob- object and triggers stories by touching parts
For more information about this tool and jects kept in archives never get to be seen and of the replica. You can read more about this
its features, go to the MeshLab homepage enjoyed by the public. The cost efficient ARC project in the VIRTEX KNOWHOW booklet
at http://meshlab.sf.net and read the 3D Webservice could change this. So, when included in this series.
documentation. storing new objects, why not record those that

14
 The ARC 3D Webservice

Benefits and notes transportation costs. If the user familiarises means that the reconstruction will only
Many techniques exist for 3D reconstruction herself with the system and tries out a couple contain those parts of the object that are
of existing scenes or objects. However, most of image sequences to understand the limita- visible from that viewpoint, even if you
of them require expensive equipment and/or tions, she will soon find that reconstructing 3D have taken photographs all around the
trained personnel to operate the system. The models can be quite easy. object. The solution for this issue is to
ARC 3D webservice is a very cheap solution create several reconstructions from one
for the acquisition of realistic 3D models. The A tremendous advantage of the webservice set of all-around photographs and merge
only prerequisites are a digital camera and is the fact that all 3D information is retrieved these reconstructions into one model. After
a computer with Internet access. from the images. This means that the correct merging, there can still be holes in the object
texture mapping (i.e. the colour of the object) (especially in creases and cavities of the
As the process only requires photography, the comes for free with the reconstruction. All object, and in object parts without surface
requirements to prepare the object for digi- of the reconstructed 3D points and faces get details). There are several automatic and
talisation are quite few. In most indoor cases, their exact texture from the recorded images manual techniques to fill those holes, the
no transport is needed to a scanning facility, automatically, without any alignment. latter of which does require appropriate
although appropriate lighting may need to skills and tools. Finally, the resulting object
be added on the spot. This not only saves you As we already pointed out, a reconstruction can contain too many polygons to be used
time but also cuts down on the insurance and is made from a certain viewpoint. This in interactive applications, so simplification
needs to be performed.
All of this requires appropriate skills and
tools. A powerful set of tools has been
developed by the ISTI-CNR group (http://vcg.
isti.cnr.it/software_vihap3d/download_
cnr_postprocess.htm) and is available for
the cultural heritage domain. Additionally,
specialised companies can be hired to create
A series of still images from a 3D film of the Arc de Triomphe, Paris. The model is reconstructed
from images with the use of ARC 3D and the film has been produced by VISICS. complete and optimal 3D models.

15
 The ARC 3D Webservice

16
 The ARC 3D Webservice

The ARC 3D Webservice

Project team

VISICS Research Group, University of Leuven:

Luc Van Gool, Maarten Vergauwen

ISTI-CRN, Pisa: Paolo Cignoni

Visual Dimension: Daniel Pletinckx

Copyright © David Nilsson, Daniel Pletinckx,

Luc Van Gool, Maarten Vergauwen
& The Interactive Institute AB, 2007

Publisher: The Interactive Institute AB,

Stockholm, Sweden
Editor: Halina Gottlieb
Graphical design: Rafał Sosin
Print: Drukania SKLENIARZ, Krakow, Poland

ISBN: 978-91-85960-05-7

With support by the European Commission

under the Community’s Sixth Framework
Programme, contract no. IST-2002-507382.

17
The cultural heritage community has always had a the design, development and implementation
keen interest in the creation of 3D models of objects, of ideas and their solutions, and give thoughtful
monuments, scenes, etc. Such models not only help suggestions for alternative applications within
in creating a more vivid picture of the past in the the cultural heritage sector. The KNOWHOW
mind of the general public, but they are also of great booklets aim to support people working in the
value for training and research within the field. area of museums, heritage sites and monuments.
The information covered within the booklets
The ARC 3D Webservice enables its users to transform benefits managers, exhibition producers/curators,
images of scenes or objects into 3D models that enhance pedagogues and professionals working with
the experience of both current and archived collections. digital restoration, as well as those working with
communication and audiences. These booklets
cover projects developed by the partners of
The KNOWHOW booklets are an inspirational EPOCH, and are divided into the following
series cataloguing existing examples of a variety categories: MUSEUMS, HERITAGE SITES and
of projects which use ICT for the recording, display MONUMENTS.
and interpretation of cultural heritage. These
booklets highlight functional information covering www.tii.se/knowhow