Three-dimensional reconstruction of Roman coins from photometric image sets Lindsay MacDonald Vera Moitinho de Almeida Mona Hess Lindsay MacDonald, Vera Moitinho de Almeida, Mona Hess, “Three-dimensional reconstruction of Roman coins from photometric image sets,” J. Electron. Imaging 26(1), 011017 (2017), doi: 10.1117/1.JEI.26.1.011017. JournalofElectronicImaging26(1),011017(Jan∕Feb2017) Three-dimensional reconstruction of Roman coins from photometric image sets Lindsay MacDonald,a,* Vera Moitinho de Almeida,b and Mona Hessa aUniversityCollegeLondon,DepartmentofCivil,EnvironmentalandGeomaticEngineering,ChadwickBuilding,GowerStreet, LondonWC1E6BT,UnitedKingdom bUniversitatAutònomadeBarcelona(UAB),DepartmentofPrehistory,QuantitativeArchaeologyLab(LAQU), EdificiBFacultatdeFilosofiaiLletres,Bellaterra,Barcelona08193,Spain Abstract. Amethodispresentedforincreasingthespatialresolutionofthethree-dimensional(3-D)digitalrep- resentationofcoinsbycombiningfinephotometricdetailderivedfromasetofphotographicimageswithaccu- rategeometricdatafroma3-Dlaserscanner.3-Dreconstructionsweremadeoftheobverseandreversesides oftwoancientRomandenariibyprocessingsetsofimagescapturedunderdirectionallightinginanillumination dome.Surfacenormalvectorswerecalculatedbya“boundedregression”technique,excludingbothshadowand specularcomponentsofreflectionfromthemetallicsurface.Becauseoftheknowndifficultyinachievinggeo- metric accuracy when integrating photometric normals to produce a digital elevation model, the low spatial frequencies were replaced by those derived from the point cloud produced by a 3-D laser scanner. The two datasets were scaled and registered by matching the outlines and correlating the surface gradients. The final result was a realistic rendering of the coins at a spatial resolution of 75 pixels∕mm (13-μm spacing), in which the fine detail modulated the underlying geometric form of the surface relief. The method opens the way to obtain high quality 3-D representations of coins in collections to enable interactive online viewing. © 2017SPIEandIS&T[DOI:10.1117/1.JEI.26.1.011017] Keywords:threedimension;surfacenormals;photometricstereo;three-dimensionallaserscanning;coins;numismatics. Paper16465SSreceivedJun.1,2016;acceptedforpublicationNov.14,2016;publishedonlineFeb.3,2017. 1 Introduction and State-of-the-Art Review games at religious festivals, and a foundation for orphan girls was established in her honor. 1.1 Coins of Diva Faustina Coinsinthenameof“DivaFaustina”wereissuedingreat In the European research network “Color and Space in profusionthroughoutthewholereignofAntoninusPius,and CulturalHeritage”(COSCH),acollaborativestudyhasbeen the coinage of Diva Faustina was the most varied of all conductedfrom2014to2016toapplyvariousthree-dimen- Romanposthumousseries.Basedonevidencefromvarious sional(3-D)techniquestorecordtwoancientRomancoins, hoards, it has been estimated that 30% of all gold coinage to compare their features and properties, and to assess the and 39% of all silver coinage minted during the 20 further metricaccuracyandeffectivenessofthetechniquesforvisu- yearsofthereignofAntoninusPiuswas“female,”i.e.,with alization and conservation assessment. The coins, silver theheadofFaustinaontheobverse.2Onthereverseofthese denariifromtheperiodofAntoninusPius(138to161AD), coins,Faustinawasrepresentedbyaseriesofdivineperson- pose considerable challenges for 3-D recording because of ifications,undertheguiseofgoddesses,suchasJuno,Ceres, their small size and their material and surface properties. Vesta,andAeternitas.3The silverdenariiarenotablydiffer- Thestudyaimstodemonstratethevalueofdigitalrecording ent from the “male denarii” of the same period.4 What also techniques for numismatic collections, which continue to makes the Faustina coinage unique is that her designs rely on traditional documentation, such as two-dimensional evolvedoverthe20-yearperiodseparatelyfromthoseofthe (2-D)photography.Inparticular,itseeksabetterunderstand- emperor and show great typological variety and novelty. ing of the surface characteristics of historic coins and their TwosilverdenariidepictingDivaFaustinawerechosenas measurements,fromwhichcross-sectionalprofilesandshape thetestobjectsfortheCOSCHstudy(Fig.1).Thetwostudy maybedetermined.Theseareofgreatinteresttonumismatists coins are not part of a collection or museum. As Diva as aids to identifying the die type and date of production. Faustinacoinsarestillwidelyavailableforcoincollectorsin On the same day that she died in 140 AD, Faustina was auctions, coin A stems from private ownership (2006) and named a “diva” by the Senate at the behest of her husband, coin B from a coin dealer with a certificate of provenance. the Emperor Antoninus Pius, and her death had an unprec- Theobverseofbothcoinsshowsthebustoftheempress, edented impact on the religious life and physical fabric of facing right, with the raised inscription DIVΛ FΛUSTINΛ, Rome.1 She was consecrated and thereby granted formal buttheyappeartobetheworkofdifferentartistsasthepor- immortality,apriesthoodwasassignedtoherworship,atem- trayaloncoinAisrathermoreflattering.Bothshowherdis- pleandaltarwerededicatedtoher,herimagewascarriedin tinctivehairstyle,withbraidedlockspinnedupatthebackof the procession (pompa circensis) that preceded the official her head. The designs on the reverse are different: coin A shows Aeternitas holding a globe in her right hand, with *Address all correspondence to: Lindsay MacDonald, E-mail: lindsay. [email protected] 1017-9909/2017/$25.00©2017SPIEandIS&T JournalofElectronicImaging 011017-1 Jan∕Feb2017 (cid:129) Vol.26(1) MacDonald,MoitinhodeAlmeida,andHess:Three-dimensionalreconstructionofRomancoins... Fig.1 ObverseandreverseoftwosilverRomandenariidepictingDivaFaustina.CoinAisontheleftand CoinBontheright.TheimagesweretakenbyaNikonD200camerausingasingleflashlighttotheupper leftatanelevationof∼60 deg. palla billowing out around her head, inscription ΛETER- ofillumination is employed, it is difficult to use the images NITΛS; coin B shows Vesta, holding the palladium in her forquantitativeanalysisandcomparison.Forimagingofpre- right hand, scepter in her left, inscription VES-TΛ. Each moderncoins,forexample,numismaticphotographershave coinisbetween17.5and18.0mmindiameterand∼1.5 mm sometimesemployedalightsourceinclinedatananglefrom in thickness. the optical axis of the camera. This has the advantage of reducingthedirectspecularreflectionsfromthemetallicsur- 1.2 Photography of Coins face,butitproducesbrightreflectionsfromgradientsonone Documentationis“essentialinthepracticeofconservation.”5 sideofrelieffeaturesandshadowsontheother(seeFig.1). Codesofethicstypicallycallforapplicationofthe“highest Thisasymmetricalrendering,whichdependsonbothillumi- possible standard” to “written and pictorial records” as part nation and viewing angles, makes it difficult to decide of“thediagnosticexaminationandrecordingofanobject.”6 whether a coin was minted by the same die. In general, the Thisappliestocoincollectionslikeallotherculturalheritage intensity, direction, and number of light sources may vary objects, material culture, and archives. fromonestudiotoanother,leadingtodifferencesintheshad- Coins found on archaeological sites are typically docu- ows cast, the contrast between bright and dark regions, and mented by manual drawings (Fig. 2) and photographs. therenderingofsurfacetexture.Differentlightingdirections This has been considered necessary because the finds gen- may make small patterns on the coin, such as inscriptions erally have signs of wear, dirt, corrosion, and so on. Finds and symbols, look very different, and highlights arising often have to be documented properly, because they cannot fromspecularityofthemetallicsurfacemayaffectthequal- legallybetakenoutsidethecountryhostingtheexcavation.7 ity of the images. The use of polarizing filters to reduce Thestudyofacoin’sfeatureshastraditionallybeencon- specular highlights can cause undesirable birefringence ductedbydirecthands-onexamination,employingmagnifi- effects.8 More fundamentally, the projection of the object cationtoolsandastrongdirectionallightsource.Byholding reliefontotheimageplane meansthatvaluableinformation the coin under the light and tilting it relative to theviewing about the object is discarded, which makes it more difficult direction,thesurfacefeaturesmaybediscerned.Thequality toanalyzedistortionsarisingfromthestrikingofthecoinin of numismatic information that can be gleaned from a coin the die and also of subsequent wear and corrosion.9 decreases further if in-person examination is not possible. Because of the relatively small size, shallow relief, and Traditionalphotographicimagesconveyonlyasubsetofthe metallic specularity of most coins, special techniques are information discernible through direct physical inspection.8 needed for photography. In addition to a copystand and Inthedocumentationofcoins,photographsprovideaddi- macrolens,carefulattentionneedstobepaidtothegeometry tionalinformationbesidesthewrittendescription,conveying oftheincidentillumination.Raisedareasofthesurfaceneed both the visual design and the state of preservation. toberenderedintheimagebytonesdifferentfromthoseof Traditionally, coins have been photographed by numisma- thegroundplaneofthecoin.Thiscannotbeachievedinmost tists as 2-D monochrome images, and these are of great casesbyasinglelightsourceatanobliqueangle,asinFig.1, qualitative value. But unless a standardized configuration becauseforashinysurface,thelightisreflectedawayfrom thecamera,makingitappeartoodark.Moreover,thereflec- tion from the leading edge of each surface feature becomes anoverexposedspecularhighlight,whilethetrailingedgeis castintoshadow.Theresultingimageisharshincontrast,the gradients are strongly directional, and fine surface texture may be exaggerated. Hoberman,writinginthedaysofanalogphotographyrec- ommended axial illumination with an umbrella reflector to reproduce maximum gradation of tone and to record subtle hue variations.10 A traditional technique used by profes- sionalsistoilluminatethecointhroughaperiscopearrange- ment,inwhichthecameraviewsitthroughaplateofglassor a half-silvered mirror.11 Only rays reflected parallel to the Fig.2 NumismaticdrawingofaRomandenariusdepictingFaustina optical axis are captured, resulting in an image that is theElder,fromHoberman(1981),page88.Theradiatingstressmarks onbothsidesofthisspecimenwerecausedbyimperfectstriking. light in horizontal surfaces of the coin and dark in inclined JournalofElectronicImaging 011017-2 Jan∕Feb2017 (cid:129) Vol.26(1) MacDonald,MoitinhodeAlmeida,andHess:Three-dimensionalreconstructionofRomancoins... surfaces,i.e.,theoutlinesofrelieffeatures.Conversely,adif- 1.4 Three-Dimensional Imaging of Coins ferentapparatuswithawhitepaperconeenablesthesloping 3-Dimagingtechnologyallowsnondestructive,noninvasive sides of features to be illuminated without frontal light, recording of the surface to construct a virtual image of the resulting in light outlines and dark surfaces. realobject,i.e.,a3-Dmodel.Researchwithculturalheritage Goodmannotesthatilluminationfromanangleasnearas professionals has confirmed the relevance of 3-D imaging possible to axial improves the relief, helps to prevent dark and established that, by creating accurate geometric repre- spots, makes the details more vibrant, and enhances the sentations, user motivations can be addressed for surface luster.12 Single lights, especially those closer to a point measurement and inspection, deformation monitoring, and source,producehighcontrastintheimage.Hedistinguishes 3-D printing.21,22 Cross-sections, surface heights, and pro- between three classes of coins, according to their degree of filescanbeextractedfromthedataset.73-Ddigitaldocumen- metallicspecularity,correspondingtonormalcoinsincircu- tation may, therefore, be of great value to identify lation(lowcontrast),newcoins(mediumcontrast),andbril- deterioration or damage, for example, before and after liant proof coins (high contrast). For these, he recommends loan.Ineducation,“object-basedlearning”sessionsincreas- illuminationbyone,two,andthreelights,respectively,add- inglyinclude3-Ddigitalrepresentationstostimulatestudent ing a diffused light where needed to soften the shadows in interaction.23 the image. Verylittleresearchhasbeendonetodateon3-Dimaging of coins or their processing. 3-D models for 22 historical coinsdigitizedbya3-Dstructuredlightscanner(SLS)were 1.3 Image Processing of Coins in Two-Dimension compared for diameter and volume against manually mea- sured values, giving coefficients of variation of 1.23% and Because coins are the artifacts most frequently preserved 0.26%,respectively.24Thevolumeisrelevantforcalculation throughout all historical periods, their study and documen- ofthedensityofthecointoidentifydifferencesbetweenthe tation can lead to a new understanding of history. theoretical and actual densities when coins were plated, for Numismatic research can show not only where a particular instance,withsilveroveracoppercore.Interactive3-Dmod- coin was minted, but also how a coin type has developed elshelpnumismatiststorecognizecoinsbecausethemodels, over time. It is of great importance to numismatists to like the originals, can be viewed and analyzed from any havetoolsthatreducetheeffortofanalysisandthecomplex- ity of identifying coins from identical dies.13 viewpoint and in any scale. Another important feature is that 3-D models show the difference between the axes of Thereisasubstantialbodyofliteratureontheprocessing the obverse and reverse faces, which are not necessarily of 2-D digital images of coins for classification and identi- aligned for ancient and medieval coins because they were fication.14 Methods for image-based recognition of modern hammered, not cast.14 coins include artificial neural networks, edge features, gra- Oneofthedisadvantagesoflaser-based3-Dscanningsys- dient directions, eigenspaces and color, shape, and wavelet tems is that they are generally limited in spatial resolution, features.15Additionalapproacheshavebeenusedforclassi- with a typical sampling pitch, or ground sampling distance fying ancient coins using scale-invariant feature transform (GSD), of 100 μm, corresponding to 10 points∕mm on the (SIFT) features.16 Good results have been achieved with a surface of the object. As a general rule, in accordance with generalizedHoughtransformtosegmentthecoinedgesand the Nyquist sampling criterion, the inter-point distance features.17,18 A method called deviation from circular shape should be at most half the size of the smallest feature of was shown to facilitate matching of the irregular shapes of theobjectthatisrequiredtobediscernibleinthedigitalrep- ancient coins by a linear combination of local and global resentation.Thefinestfeaturethatcanberesolved,therefore, shape techniques, in which the local matching was based is0.2mmor200 μm.Coins,however,arerelativelysmallin on the difference of Fourier shape descriptors, whereas size,withdiametersgenerallyintherange15to50mm,and the correlation coefficient between the curves served as a havevery fine surface detail and shallow surface relief. For global measure of shape similarity.19 veridicalrepresentation,allspatialdetaildowntothelimitof Imageprocessingmethodshavealsobeenappliedtoiden- the human visual system of 40 μm should be captured, tify and retrieve coins stolen from excavations due to their requiring a spatial resolution of at least 50 points∕mm.25 value for collectors. The system developed in the project Thus, there is a factor of 5 between what is needed and “Combat On-line Illegal Numismatic Sales (COINS)” was the actual capability of 3-D laser scanners. tested on a dataset provided by the Fitzwilliam Museum, Cambridge, consisting of 2400 images of 240 different coins,recordedinvaryingviewpositionsbymeansofdiffer- 1.5 Reflectance Transform Imaging and Coins ent rotations on both reverse and obverse sides. Independ- Systematic photography with directional illumination ena- ently of shape and local features, the system achieved an bles richer documentation of objects because the ensemble identificationrateof95.2%.15FortheMUSCLECISbench- of images contains implicit information about the surface mark competition, a new coin dataset was developed con- relief.Thiscanbeexploitedthroughthepolynomialtexture taining ∼100;000 coin images. The dataset was divided mapping(PTM)technique,26inwhichabiquadraticfunction into a fixed training set of 20,000 coins, and six fixed test ofincidentilluminationangleisfittedtothevectorofinten- sets of 5,000 coins, with both obverse and reverse of each sity values at each pixel. When the reconstructed image is coin. The training set contained 2270 different coin faces, displayed through the interactive viewer software with a corresponding to 692 coin classes. In the test sets, ∼400 moving virtual light source, the illusion of 3-D produced ofthecoinclassesappeared,and,inaddition,thetestsetcon- by the interplay of light and shadow is compelling, making tained3%to4%ofcoinsnotinthetrainingset,whichshould itparticularlysuitableforshowingthesurfacereliefofcoins. be classified as unknown.20 The disadvantage is that because only six parameters are JournalofElectronicImaging 011017-3 Jan∕Feb2017 (cid:129) Vol.26(1) MacDonald,MoitinhodeAlmeida,andHess:Three-dimensionalreconstructionofRomancoins... used in the biquadratic function, the angular distribution is 2 Processing Coin Images: Digital Elevation Map not much more directional than a simple Lambertian, and from Photometric Plus 3-D Laser Scanner Data cannot properly represent the specular components of The present study combined two datasets representing the shiny metallic surfaces. The PTM image can be analyzed surfacetopographyofthetwoFaustinacoins(Fig.1).First, to find the most advantageous combination of illumination theywerescannedbya3-Dcolorlaserscanner,producinga angle(s)andenhancementeffect(s)thatillustratethecharac- point cloud of the surface shape. Second, they were photo- teristicsunderdiscussion.Theparameterscanthenbesaved graphed in an illumination dome with directional lighting. as part of the descriptive record for the coin, enabling the Thetworepresentationswerecombinedtoproduceadigital visualization to be replicated in the future.8 elevationmap (DEM)ofeachcoinwiththeaccuracyofthe PTMisoneofafamilyofmethodsknownasreflectance scanner and the fine detail of the photography. The same transform imaging (RTI). Two innovations made it a more methodwasappliedtoboththeobverseandreverseofeach versatiletechniqueforavarietyofculturalheritageimaging coin,buttheillustrationsinthefollowingsectionsareallfor applications.27 First, by determining the direction of inci- the obverse of the Faustina A coin. dent illumination from the highlight on a sphere placed within the image frame, it became possible to capture image sets with a movable handheld light source instead 2.1 Coin Outlines from Images of an illumination dome. Second, the implementation of ThedomeimagingsystematUCLenablessetsofimagesof an alternative set of basis functions, known as hemispheri- anobjecttobetakenwithilluminationfromdifferentdirec- cal harmonics, provides a better localization of the direc- tionality of the illumination and, therefore, improves the tions. A hemisphere of 104-cm diameter is fitted with 64 modeling of specular highlights. RTI provides several flash lights and calibrated so that the geometric centroid advantages compared to other close-range 3-D recording coordinates of every light source are known to within techniques for near-flat surfaces: (a) RTI uses inexpensive 3.0 mm.30 A digital camera at the “north pole” captures a and widely available and easily transportable hardware series of 64 color images, each illuminated from a different (SLR camera, tripod, and flash); (b) RTI scales well for direction and all in pixelregister. This enables the object to both small and large object dimensions; and (c) RTI can bevisualizedfromafixedviewingangle,i.e.,verticallyfrom achieve a high sampling density for near-flat surfaces above, for many different angles of incident light. that is only recently achieved by other 3-D imaging tech- For photography, the coins were placed on a black card- niques, such as laser scanners, structured light scanners or board sheet to provide a dark image background free from photogrammetric structure from motion (SfM). It also pro- texture. Each side of each coin was photographed in the duces a highly detailed, colorimetrically correct, visually dome using a Nikon D200 camera fitted with a Nikkor attractive and intuitively understandable digital image of 200 mm macro lens, achieving a resolution of 75.3 pixels∕ the surface of an artifact. For these reasons, RTI techniques mm,sothateachpixelcorrespondsto13 μmonthesurface have become widely adopted in the cultural heritage field of the coin. The dimensions of each coin in the images are for documentation tools and detailed visual analysis,27 par- approximately 1320ðWÞ×1250ðHÞpixels. The images of ticularly in the field of archaeology.28 the coins were captured with the lens aperture set to An experimental system was used to create PTM repre- f∕5.6toachieveagoodexposurewhileminimizingoverex- sentationsofRomanandmedievalcoinsfromthecollection posureandmaximizingdepthoffocus.Theywereconverted at themonastery ofSt. Bernard.8The system consisted of a fromraw(NEF)filesbythesoftwareutilityDCRAWtolin- template for 24 light positions in a hemispherical array sur- ear 16-bit per channel (range: 0 to 65535) and stored as roundingthecoin,afiberopticdirectionallightsource,and TIFF files. a computer-controlled camera. The PTMs were bundled The lightness of the images increases with increasing with a Java-based viewer and displayed, with descriptions angle of elevation of the light source (Fig. 3). By digital inFrench,onthemonasterywebsiteinorderto“allowpeo- processing (in MATLAB throughout this study), it is pos- ple to consult some of the seminal objects of our civiliza- sible to make any weighted sum of the images, and hence tion.”ThestudyconcludedthatPTMsprovideaninteractive to emulate the conventional photographic lighting configu- experience of a more complete dataset than traditional rations described above. For example, the image produced numismatic documentation and that they offer a more by the recommended axial illumination can be formed as informed method for generating images that convey numis- themeanofthefourimagescorrespondingtothefourlights matic ideas. nearestthecamera,withelevationapproximately80deg(last RTIwasalsousedforpresentingacollectionofcoinsfor fourimagesinthebottomrowofFig3).Theimageproduced public display through an interactive kiosk at the National by diffuse (i.e., omnidirectional) illumination is approxi- Museum of San Matteo in Pisa.29 The objectives of the mated as the mean of all 64 images. A weighted sum of project were to enable thevirtual manipulation of the coins the two, with 60% of the specular image plus 40% of the fordetailedinspection,andtherebytorevealtheirfeaturesin overall mean image, retains the shadowed outlines around an easy and understandable way. The collection includes the surface features, softened to reduce contrast (Fig. 4). both gold coins with highly specular surfaces and bronze The outline of the coin is valuable as a feature to deter- coinsthataremorematteandpresentpatinasresultingfrom minethescaleandangle.Also,knowledgeoftheoutlineena- variousdegradationprocesses.Foreachfaceofeachcoin,a blesallsubsequentimageprocessingoperationstobelimited setof260imageswascapturedbyaCCDcamerainanillu- totheinteriorarea,byusingtheoutlineasapixelmask.The minationdomefromtheUniversityofLeuvenequippedwith algorithmforconstructingtheoutline,asillustratedinFig.5, 260 white light-emitting diodes. was as follows: JournalofElectronicImaging 011017-4 Jan∕Feb2017 (cid:129) Vol.26(1) MacDonald,MoitinhodeAlmeida,andHess:Three-dimensionalreconstructionofRomancoins... Fig.3 Montageof64imagesoftheobverseofFaustinacoinA,takenbyaNikon200cameraintheUCL dome,illuminatedbyeachofthe64flashlights.Theimagebrightnessincreaseswithangleofelevationof thelightsource,asshownfromtopofthefigure(lowestangles)tothebottom(highestangles). i. Computethemeanofthe16imagesfromTier4andTier vi. Make a binary mask at the same size as the original 5 illumination. image,with0inthebackground(intensity<threshold) ii. Extract the green channel and apply a 5×5 spatial and 1 in the foreground (intensity>threshold). median filter to the monochrome image. vii. Convert binary mask to floating point and apply a iii. Compute the mean image intensities in outside region 21×21 median filter to mask image to remove noise (corner) and inside region (center). (i.e., spots in the object below threshold level). iv. Calculate intensity threshold as: t¼0.5× viii.Applya3×3smoothingfiltertoantialiastheedgepro- ðmeanoutsideþmeaninside−2×stdevinsideÞ. file around outline mask. v. Plotoneormorecross-sectionalprofiles.Checkthatthe ix. Convertfloatingpointmasktoan8-bitgrayscaleimage threshold level is above the background intensity and in range 0 to 255 and save as file. below the object intensity. x. Inspect the mask image and, if necessary, edit in Photoshop to tidy inside area. xi. Determinecoordinatepointsaroundtheimageoutlineby scanningallrowsfrombothleftandrightandallcolumns from both top and bottom. Filter to remove duplicate pointsandsortintoorderofanglearoundcircumference. xii. Apply seven-point linear boxcar filter to smooth the coordinate points around the outline. xiii.Compute centroid, mean and maximum radius, and length of perimeter. Fig. 4 (a) mean of images illuminated by top four flash lights; Note that processing step (xi) allows, if necessary, for (b) mean of images illuminated by all 64 flash lights in dome; manual editing of the mask. Because the coin is photo- (c) weighted sum of the two, emulating the recommended photo- graphed against a black background, the setting of the graphiclightingconfigurationforcoins. JournalofElectronicImaging 011017-5 Jan∕Feb2017 (cid:129) Vol.26(1) MacDonald,MoitinhodeAlmeida,andHess:Three-dimensionalreconstructionofRomancoins... intensitythresholdinstep(iv)ensuresthattheedgeprofileis to reduce the effect of specular reflectance. The data were welldefined[Fig.5(b)].Theinteriorareaofthemaskiscom- savedasafilein“.asc”format,inwhicheachpointisrepre- pletelyclearifalloftheimagecontentisabovethethreshold. sentedbyone lineofsix numerical fields encoded asASCII Inpractice,darkregionsproducedbyshadowingofthesur- text.Table2showsthefieldsinthefirstfivetextlines,forfive facereliefmayfallbelowthethresholdandleadtounwanted successive points along one vertical laser scan line. intrusions within the mask interior [Fig. 5(c)]. These are ItcanbeseeninTable2thattheY coordinatechangedin easily removed by an erasure brush in Adobe Photoshop. successive points, as the laser beam was swept vertically, Theareawascalculatedbytwomethods,firstbytreating whereas the X coordinate remained almost constant. The the outline as an irregular polygon and triangulating each 8-bit RGB signal values represent the reflected intensity at pairofpointswiththecentroid,31andsecondbysimplysum- thethreelaserwavelengthsof638,532,and450nm.Inorder ming the number of pixels within the mask (Table 1). The to compare and combine the 2-D photographic image set eccentricity is the ratio of maximum to minimum radius with the 3-D point cloud, the latter was converted to a 2-D from the centroid. All values were converted to mm using imagewithapixelateachlocationina2-Dimagearraywith the image resolution of 75.3 pixels∕mm. aresolutionof10×10 pixels∕mm(gridspacingof0.1mm). Thepointcloudwas“flattened”ontotheZplane,discarding the X, Y coordinates from the scanner, andtakingthe mini- 2.2 Coin Outlines from a 3-D Point Cloud mumoftheR,G,Bintensityvaluesineachbin.ForCoinA Thecoinsweredigitizedbya3-Dcolorlaserscanner(Arius obverse, 89,226 points were mapped onto 23,524 pixels, a TechnologyIdentik300L),producingpointcloudsataspa- ratio of 3.79 points per pixel. The frequency distribution tial resolution of 10×10 points∕mm, i.e., a 0.1 mm GSD. and resulting color image are shown in Fig. 6. Measurement uncertainty of the sensor is (cid:2)0.035 mm in Themappingleftsomepixellocationsinthegridunfilled, depth (z-axis) and of the order of (cid:2)0.2 mm in the x- and soasubsequentfillingoperationreplacedeachunfilledpixel y-axes due to planimetric point spacing and laser spot size. by the mean of its nonzero neighbors (Fig. 7). Thetechnologyisanopticalrecordingtechnologycombining A DEM was computed from the 3-D point cloud by the spatialrecordingwithcolorrecordingthroughtheuseofthree same method, giving the height (in mm) at each location in laserwavelengths.Thelaserbeamwasorientatedtoscanthe the2-Dimagearray,takingthemedianofthemultiplescan- surfaceofthecoinsatanangleof30-degoffnormal,inorder nerZ coordinatevalues.TheresultingDEM,afterfilling,is Fig.5 Twocross-sectionsthroughimage:(a)meanimagewithtwocross-sections;(b)profilesofinten- sity of the green channel through cross-sections with threshold; (c) binary mask before editing; and (d)monochromeimageshowingoutline,centroidandvectorofmaximumradius. JournalofElectronicImaging 011017-6 Jan∕Feb2017 (cid:129) Vol.26(1) MacDonald,MoitinhodeAlmeida,andHess:Three-dimensionalreconstructionofRomancoins... Table 1 Dimensions of Faustina coins (mm) from computation of outline. CoinA CoinA CoinB CoinB obverse reverse obverse reverse Perimeterlength 49.26 50.54 48.13 47.88 Areainmm2 226.5 233.4 216.0 213.7 (polygonmethod) Fig.7 (a)DetailofFaustina’shair,showingpixelsunfilledaftermap- Areainmm2 227.3 234.2 216.8 214.5 pingand(b)afterfilling. (maskmethod) 2.3 Scaling and Registration of Images Meanradius 8.49 8.62 8.30 8.25 The photographic images from the camera in the dome and Eccentricity 1.134 1.187 1.100 1.093 thepseudoimagegeneratedbyflatteningthe3-Dpointcloud are of very different sizes and at slightly different orienta- tions.Anefficientwayisneededtodeterminebothscalefac- torandrotationangletobringthemintoalignment.Although Table2 Firstfivelinesfromthe.ascfileforcoinAobverse. this could be done on the 2-D images by a search-and- correlate algorithm, such as SIFT, the method preferred in X Y Z R G B this study uses the two outlines. Their irregularity is suffi- cient to provide a characteristic signature that facilitates −404.830994 −273.773987 −123.266998 162 194 147 the transformation. Normalizing the radius of each point relative to the −404.829010 −273.610992 −123.323997 184 223 159 mean radius and plotting against angle gives the graphs of Fig.10(a),showingrðθÞversusθ.Althoughtheradiusvaries −404.828003 −273.596985 −123.254997 136 161 126 by less than (cid:2)7%, a similar pattern can be seen for both −404.826996 −273.532013 −123.261002 134 160 128 outlines. Thenumberofpointsaround theoutline isamea- sure of the circumference of the shape, hence the ratio of −404.826996 −273.500000 −123.219002 149 178 138 theirlengthsgivestherelativelinearscalefactor,inthiscase, 3709∕493¼7.523. By scaling the two outlines to equal length, then interpo- latingeachtounitsof0.01deg(i.e.,avectorof36,000values) shown in Fig. 8 as a monochrome image (a), scaled from andslidingonevectoralongtheother,thepositionofbestfit black (minimum) towhite(maximum),andasa3-Dterrain canbefound.Toavoidwraparoundat360deg,thereference plot (b). outlineisduplicatedsothatitsvectorof72,000pointsrepre- The outline of the image from the flattened point cloud sentstwofullrevolutions0to720deg.Determinationofthe was computed from theRGB image inthe sameway as for goodness-of-fit is achieved by cross-correlation between the the photographic images, as described in Sec. 2.1. Because two 36,000-element vectors, one fixed (the reference outline thespatialresolutionismuchlower,thescanneroutlinehas fromthe scanner)and the other extractedatsuccessive posi- much less detail (Fig. 9) and the line of maximum radius tionsfromtheduplicatedimageoutline.Themaximumvalue occurs at a different angle (compare Fig. 5). ofthecoefficientofcorrelationindicatestheangleofbestfit Fig.6 (a)Distributionofnumberofpointsmappedontoeachpixeland(b)resultingRGBimage. JournalofElectronicImaging 011017-7 Jan∕Feb2017 (cid:129) Vol.26(1) MacDonald,MoitinhodeAlmeida,andHess:Three-dimensionalreconstructionofRomancoins... Fig.8 Digitalelevationmapfromscanner,representedas(a)monochromeimageand(b)3-Dsurface plot. Fig.9 Flattenedimagefromscanner:(a)Intensityprofiles;(b)outlinemask;and(c)outline,centroid,and vectorofmaximumradius. [Fig.10(b)],withthepeakat354.83deg.Notethatthesecond blue. In Fig. 11(b), there is a two-color composite image positivepeakat170.59degisalmostdiametrically opposite, with the enlarged and rotated height map from the scanner indicating a slightly elliptical outline shape. in the red channel and the Z component of the photometric Bytheabovemethodofoutlinecorrelation,boththescal- normals (see Sec. 2.4) in the green channel. ing factor κ and rotation angle θ can be found to map one outline onto the other. In this case, the scaling factor is 7.523andtherotationangleis5.17deg(anticlockwise).This 2.4 Calculation of Photometric Normals providesameansforeverypointinthereferenceimagesfor Photometricnormalswerecalculatedfromthephotographic the DEM (i.e,. height map), derived from the scanner image sets taken under directional lighting in the dome, point cloud, to be transformed by a 2×2 matrix into close usingthe“shapefromshading”principle.Thedifficultywith registration with a corresponding point in images from the silverandothermetals,andindeedwithallshinyandglossy camera: materials, is that they reflect strongly in the specular direc- (cid:1) (cid:3) (cid:1) (cid:3) EQ-TARGET;temp:intralink-;e001;63;191 xt ¼M xr ; ttihoenre. Saore, ianftehwe vveaclutoersomfu6c4hinlatergnesritythvanaluthese footrhearnsy, cpoixrreel-, y y t r sponding to positions where the surface normal is close to (cid:1) (cid:3) (cid:1) (cid:3) cosdðθÞ sindðθÞ 7.4084 −0.6703 the bisector of the angle between the illumination vector whereM¼κ ¼ : (toward the light) and theview vector (toward the camera). −sindðθÞ cosdðθÞ 0.6703 7.4084 Thisresultsinimageswithhighdynamicrange,whereafew (1) pixelsmaybe10timesgreaterinvaluethanthemajority.The twocoinsinthisstudyhavebeencleaned,andsoarereason- The result of the transformation is shown in Fig. 11(a), in ably bright, but they do not have the mirror-like quality of which the original scanner outline is colored in red, the newlymintedcoins.Closeexaminationshowssignsofwear, scaled scanner outline in green, and the photo outline in pitting, scratches, corrosion, and tarnishing of their surface, JournalofElectronicImaging 011017-8 Jan∕Feb2017 (cid:129) Vol.26(1) MacDonald,MoitinhodeAlmeida,andHess:Three-dimensionalreconstructionofRomancoins... Fig.10 (a)Normalizedoutlineradiusversusangleand(b)outlinecorrelationcoefficientversusangle. Fig.11 (a)Superimposedoutlinesafterscalingandrotationand(b)compositeimageofscannerheight (red)andphotometricnormals(green). allofwhichcontributetothepatinaandcausesomescatter- croppingasectionnearthecenterofthephotometricnormals ing of the reflected rays and diminution of their intensity. and sliding it around within the scaled DEM until the best Figure 12 shows the intensity distribution for a single match was achieved. This was performed on both X and Y pixelonthetipofFaustina’snoseincoinAobverse.There gradient images and the results combined. The gradients p arethreepeaks,wherethepixelintensitiesreachvaluesinthe and q are defined as the partial derivatives of the surface range 3000to4500 butmost othersare <500.The magenta height with respect to the two axes x and y: curve shows what the intensities would be for a perfectly ∂z ∂z matte(Lambertian)surfacewiththesamealbedoandnormal p¼ ; q¼ : (2) angle. It isclearthat thespecular peaks aremuch greaterin EQ-TARGET;temp:intralink-;e002;326;241 ∂x ∂y intensitythanforthecosine,butforotherangles,thedistri- butions are similar. GradientswerederivedfromthescannerDEMdirectlyby Thesurfacenormalswerecalculated,therefore,byusinga taking first-order differences along rows and columns. subset of the intensity distribution, as shown in blue in Gradientswerederivedfromthephotometricnormalsbytak- Fig. 12(b), corresponding to angles of incidence, where the ing ratios of their components and as shown in Figs. 13(b) reflected intensity is similar to that of an equivalent matte and (c): surface. This is dubbed the “bounded regression” tech- nique.32 The resulting normals are shown in conventional p¼Nx; q¼Ny: (3) false color in Fig. 13(a). EQ-TARGET;temp:intralink-;e003;326;143 Nz Nz Following the rotation and scaling of the DEM from the scanner,afurtheroperationwasrequiredtoensurethatitwas Thedisplacementbetweenthephotometricimageandthe spatially aligned as well as possible with the photometric transformedscannerimagewasdeterminedbycross-correlation normals [Fig. 11(a)]. Image cross-correlation was used by betweenthesurroundingregionsoftheimagegradients.One JournalofElectronicImaging 011017-9 Jan∕Feb2017 (cid:129) Vol.26(1)