On my way to the FunGIS LiDAR workshop in Cairns,  I stopped in Adelaide to visit Airborne Research Australia. Flinders University recorded my talk on the capabilities for LiDAR processing with LAStools. In this introduction I use a recent project in the Canary islands, Spain, in Spring of 2012 as the example. We created canopy height maps, forest density grids with LAStools and showed that the LiDAR already owned by the municipal government agency could be used to significantly improve their photogrammetrically derived DTMs.

Often I get asked about the difference between las2dem.exe and blastdem.exe – the latter being part of the BLAST extension of LAStools. In the academic video from 2006 below, I outline the core technology details behind BLAST, which can seamlessly compute gigantic Delaunay Triangulations from billions of LiDAR points – using very little main memory. The blast2dem.exe module does not store the gigantic TINs that BLAST produces, but immediately rasters the finalized triangles streaming out of BLAST into equally massive DEMs. This avoids to ever having to store such huge triangulations – that are only needed temporarily – to disk. Currently blast2dem.exe can “only” process up to 2 billion points per file – a limitation that shall be lifted soon.

After attending Silvilaser 2012 in Vancouver and giving a keynote on “The Story of LAStools”, I had a stop over in Halifax and drove a cute little Fiat 500 convertible on a perfect Indian summer fall day to UNB Fredericton where Patrick Adda had organized a half-day workshop on LiDAR Processing with LAStools. Here is the first 54 minutes of my talk all the way through to LiDAR quality checking with LAStools. Special thanks to Houman for filming and to Patrick for editing the video.

Salzburg is a beautiful city in December. The European LiDAR Mapping Forum coincided with the days when the “Krampus” (= “Christmas monsters”) are roaming the Christmas markets in the old town to scare children and adults alike. One gave me a painful whipping in the legs with its leathery tail when I tried to protect a LAStools user … (-;

More to the point, here is my talk at ELMF 2012 on “LASindex – simple spatial indexing of LiDAR data”. I first give a little update on LASzip, then talk about spatial indexing with LAX, before sneak-previewing PulseWaves – our new and open LiDAR format for storing full waveform data.

Some more detail:
Airborne LiDAR surveys collect large amounts of elevations samples, often resulting in Terabytes of data. The acquired LiDAR points are typically stored and distributed in the LAS format or – its lossless compressed twin – the LAZ format. However, managing a folder of LAS or LAZ files is not a trivial task when a survey consists, for example, of 500 flight strips containing around 200 million points each. Even a simple area-of-interest (AOI) query requires opening all files and loading all those whose bounding box overlaps the queried AOI. One solution is to copy the survey into a dedicated data base such as Oracle Spatial or PostgreSQL. We present a much simpler alternative that works directly on the original LAS or LAZ files.

Our minimal-effort spatial indexing scheme has very small setup costs, avoids creating a second copy of the data, and is already in use in the LAStools software suite. For each LiDAR file we generate a tiny LAX file that resides in the same folder as the *.las or *.laz file and has the same name but with a *.lax extension. The LAX files are generally as small as 0.01 percent (for a LAS file) or 0.1 percent (for a LAZ file) of the file containing the LiDAR data and they can be generated as fast as the points can be read off disk.

The LAX files describe an adaptive quadtree over the x and y coordinates of all points. Each occupied quadtree cell stores a list of point index intervals that together reference all points falling into this cell. By merging all intervals of a cell that are less than 1000 apart in point index space we significantly reduce the number of intervals, the size of the LAX files, and the number of file seek operations.

Although individual cells typically reference too many points this is usually amortized as a typical AOI query will require returning a union of all intervals from many quadtree cells. However, our in-place spatial indexing relies on a certain degree of spatial coherency to be present in the point order. A simple measure of the efficiency of the existing order is to calculate the overhead factor when loading each quadtree cell individually from disk.

The source code for LASindex is part of the open source library LASlib of LAStools. It has been extensively field-tested in the LiDAR delivery pipeline of Open Topography (OT) where it is used to efficiently gather data from folders of LAZ files in accordance to area-of-interest queries that are generated by users via OT’s popular web-based LiDAR download interface. Another important use is on-the-fly point buffering. When batch processing, for example, 2km by 2km LiDAR tiles to create DTMs via rasterization of a temporary TIN, it is beneficial to load a 100 meter point buffer around each tile to avoid tile boundary artifacts. The presence of LAX files allows doing so efficiently on-the-fly.

This will be the part one of a three-part tutorial on how to use LAStools to implement a pipeline that (1) quality checks a newly received set of raw LiDAR flight strips, (2) tiles and prepares the LiDAR for subsequent exploitation, and (3) generates raster and vector deliverables such as DTMs, DSMs, CHMs, iso-contours, and TINs with multi-core batch processing.

To get started, download the latest version of LAStools and these 7 example flight strips: strip1, strip2, strip3, strip4, strip5, strip6, strip7 and put them into a folder called ‘.\lastools\bin\strips_raw’. For simplicity we will work inside the ‘.\lastools\bin’ directory, so open a DOS command line window and change directory to where this folder is on your computer, for example, with the command ‘cd c:\software\lastools\bin’ or with ‘cd d:\lastools\bin’ followed by ‘d:’.

I usually first run ‘lasview’ on newly received LiDAR to see if the data makes any sense. The command below will on-the-fly down-sample the data and display only around 5 million points:

C:\lastools\bin>lasview -i strips_raw\LDR*.laz -points 5000000

I usually inspect a few area-of-interests (AOIs) to get a few close-up looks at the data. If we do this several times for different areas it is best to first create a spatial indexing of the strips with ‘lasindex’ which it significantly speeds-up our spatial queries.

C:\lastools\bin>lasindex -i strips_raw\LDR*.laz -cores 3

Only add the ‘-cores 3′ option if you have a computer with at least 4 cores. If you have an 8 core machine you may even add ‘-cores 7′ and so on. Now start ‘lasview’ in the GUI mode either by double-clicking the executable and using the ‘browse …’ roll-out on the left side to load the seven flight strips or simply by adding ‘-gui’ to the command line.

C:\lastools\bin>lasview -i strips_raw\LDR*.laz -gui

Now you can maneuver up and down the file list to highlight the different strips (close the ‘browse …’ roll-out to see the header contents listed on the bottom left). You can again look at all the data by pressing the ‘VIEW’ button or you can inspect a smaller area by first picking a rectangular region as shown in the screenshot below.

In the GUI above you can see that the files have no projection information. We will add this later. The x, y, and z scaling factors are set to 0.001 which means that points are stored with millimeter resolution. Because airborne LiDAR is far from being that accurate we will later change the scaling factor to a more appropriate centimeter resolution. Finally, the creation day and year is missing, which – if we know when the data was flown – will be an easy fix.

After starting ‘lasview’ you can use the <c> hot-key to toggle through the different ways to color the points. Another way to change the point coloring is by presetting it in the GUI as shown above or via the ‘color by …’ menu that opens when you right click.

The coloring by flight-line which is supposed to be stored in the ‘point_source_ID’ field of each point seems rather odd. This field was probably used for some other purpose and not cleaned up before the data was distributed. Not nice and we will fix that later. There are a few other visualization shown below. Coloring points by return turns single returns yellow, first of many returns red, and last of many returns blue. Usually there are also a few green points corresponding to intermediate (neither first nor last) of many returns but there are none in this data set as the ‘lasinfo’ report will confirm later. Toggle through the colorings with the <c> key until it displays the intensities. You may need to press the <=> key a few times to increase the point size. Next, press the <t> key to triangulate the points and then decrease the point size with the <-> key until they have disappeared and you can see a hillside shading of the TIN. Use the <h> key to toggle through different shadings for the TIN triangles.

Before attempting to improve these LAZ files and prepare them for processing we want to make sure that our work will be worthwhile by running a quick visualization based of how well the flight strips fit together. We do this with the ‘lasoverlap’ tool.

C:\lastools\bin>lasoverlap -i strips_raw\LDR*.laz ^
                           -files_are_flightlines ^
                           -utm 19N ^
                           -step 1.0 -max_diff 1.0 ^
                           -o strip_overlap.png

I happen to know that the missing projection in the LAZ files is the UTM zone 19N. I can simply add this to the command line with ‘-utm 19N’. Then ‘lasoverlap’ will produce in addition to the PNG output also a KML file that allows visualizing the overlap and difference rasters within the geo-spatial context of Google Earth.

Serious troubles in the data would be evidenced by void areas in the overlap raster that are obviously caused by poor flight planning (not water bodies) and by deeply blue or deeply red saturated areas in open (=> non-forested) terrain in the difference raster. If either was the case you should send the data back to the vendor to have him fix it … (-:

Now we compare the LiDAR elevations with a set of 29 exactly known control points that were obtained through a ground survey that are listed below. If you want to follow this exercise along you should copy the text below and store it to a file called ‘strips_raw\cps.csv’.

C:\lastools\bin>more strips_raw\cps.csv
Type,Easting,Northing,Z
Open/Paved,273299.68,4715133.88,74.17
Open/Paved,273477.61,4714979.29,73.85
Open/Paved,274001.2,4714540.29,72.5
Open/Paved,273670.13,4714817.4,73.34
Open/Paved,273677.42,4715018.66,74.26
Open/Paved,273400.1,4714528.98,73.15
Open/Paved,274511.59,4714905.97,95.7
Open/Paved,275074.66,4714841.98,120.18
Open/Paved,275409.65,4714994.76,113.41
Field,273321.18,4714946.83,73.46
Field,273601.49,4715101.78,74.35
Field,273646.76,4714972.94,73.97
Field,273890.5,4714457.59,71.13
Field,274650.24,4714903.44,105.13
Field,274522.36,4714829.74,98.59
Field,275474.47,4714780.03,127.63
Field,275636.39,4714868.85,120.72
Field,274747.37,4714932.57,116.11
Field,272795.36,4714503.86,126.9
Forested,272547.02,4714623.09,127.71
Forested,273205.33,4714900.27,79.36
Forested,272530.52,4715045.46,113.48
Forested,275237.48,4715049.57,120.31
Forested,275268.37,4714543.82,104.99
Forested,274666.09,4714497.49,108.86
Forested,274403.56,4715053.43,93.17
Forested,274901.6,4714493.63,114.64
Forested,274658.37,4715072.74,104.49
Forested,274121.73,4714524.51,72.06

The first entry describes the location of the control point and the following three entries are its x, y, and z coordinate. We now compare the seven LiDAR strips against these 29 control points. You can also do thus via the GUI as shown below but you need to make sure to that the ‘keep ground (2) ‘ and the ‘keep keypoint (8)’ check-buttons are not checked since our LiDAR data is not yet classified.

C:\lastools\bin>lascontrol -i strips_raw\*.laz ^
                           -cp strips_raw\cps.csv ^
                           -parse sxyz
 diff,lidar_z,Type,Easting,Northing,Z
 0.1451,74.3151,Open/Paved,273299.68,4715133.88,74.17
 0.0190857,73.8691,Open/Paved,273477.61,4714979.29,73.85
 0.08872,72.5887,Open/Paved,274001.2,4714540.29,72.5
 -0.00419681,73.3358,Open/Paved,273670.13,4714817.4,73.34
 0.054868,74.3149,Open/Paved,273677.42,4715018.66,74.26
 0.0318439,73.1818,Open/Paved,273400.1,4714528.98,73.15
 -0.0720668,95.6279,Open/Paved,274511.59,4714905.97,95.7
 -0.0670072,120.113,Open/Paved,275074.66,4714841.98,120.18
 0.0756029,113.486,Open/Paved,275409.65,4714994.76,113.41
 0.00132683,73.4613,Field,273321.18,4714946.83,73.46
 0.0750162,74.425,Field,273601.49,4715101.78,74.35
 0.00228472,73.9723,Field,273646.76,4714972.94,73.97
 0.166194,71.2962,Field,273890.5,4714457.59,71.13
 -0.392266,104.738,Field,274650.24,4714903.44,105.13
 -0.0705893,98.5194,Field,274522.36,4714829.74,98.59
 -0.423622,127.206,Field,275474.47,4714780.03,127.63
 0.217528,120.938,Field,275636.39,4714868.85,120.72
 -0.12489,115.985,Field,274747.37,4714932.57,116.11
 0.0195411,126.92,Field,272795.36,4714503.86,126.9
 17.0342,144.744,Forested,272547.02,4714623.09,127.71
 9.20693,88.5669,Forested,273205.33,4714900.27,79.36
 26.355,139.835,Forested,272530.52,4715045.46,113.48
 20.2143,140.524,Forested,275237.48,4715049.57,120.31
 14.5363,119.526,Forested,275268.37,4714543.82,104.99
 0.173861,109.034,Forested,274666.09,4714497.49,108.86
 20.5742,113.744,Forested,274403.56,4715053.43,93.17
 0.00432622,114.644,Forested,274901.6,4714493.63,114.64
 23.8816,128.372,Forested,274658.37,4715072.74,104.49
 0.3645,72.4245,Forested,274121.73,4714524.51,72.06
 sampled TIN at 29 of 29 control points.
 avgabs/rms/stddev/avg of elevation errors are
 4.63438/9.61987/8.62324/4.55475 meter. skew is 1.47593.

The output of ‘lascontrol’ could be directed into a file using the ‘-o control.txt’ option. The tool prefixes two numbers to the beginning of every line: the elevation value that was computed from the LiDAR data at the xy location of the control point and the difference between the computed elevation and the elevation of the control point. At the end a summary reports the average absolute error, the relative mean-square error, the standard deviation, and the average error across all the differences. Of course, for control points of type ‘Forested’ we have terrible results because we use all LiDAR points in this computation and not just those that correspond to ground returns. Therefore we will have to repeat this exercise at a later point once we have ground-classified the LiDAR. Via the GUI of ‘lascontrol’ you can zoom in on different areas-of-interests and visually inspect how well the data matches the control points (see below).

Finally let us run ‘lasinfo’ on one one of the strips and also compute the density:

C:\lastools\bin>lasinfo -i strips_raw\LDR030828_211804_0.laz -cd
reporting all LAS header entries:
 file signature:             'LASF'
 file source ID:             0
 global_encoding:            0
 project ID GUID data 1-4:   0 0 0 ''
 version major.minor:        1.0
 system identifier:          'ALSXX'
 generating software:        'ALSXX_PP V2.56c'
 file creation day/year:     0/0
 header size:                227
 offset to point data:       5587
 number var. length records: 3
 point data format:          0
 point data record length:   20
 number of point records:    2404613
 number of points by return: 2210130 0 194483 0 0
 scale factor x y z:         0.001 0.001 0.001
 offset x y z:               0 4000000 0
 min x y z:                  272254.830 4714389.375 65.050
 max x y z:                  275391.197 4714711.445 169.208
 variable length header record 1 of 3:
 reserved             43707
 user ID              'LeicaGeo'
 record ID            1001
 length after header  5120
 description          ''
 variable length header record 2 of 3:
 reserved             43707
 user ID              'LeicaGeo'
 record ID            1002
 length after header  22
 description          'MissionInfo'
 variable length header record 3 of 3:
 reserved             43707
 user ID              'LeicaGeo'
 record ID            1003
 length after header  54
 description          'UserInputs'
 the header is followed by 2 user-defined bytes
 LASzip compression (version 2.0r2 c2 50000): POINT10 2
 reporting minimum and maximum for all LAS point record entries ...
 X  272254812  275391197
 Y  714389375  714711445
 Z      65050     169208
 intensity 0 255
 edge_of_flight_line 0 1
 scan_direction_flag 0 1
 number_of_returns_of_given_pulse 1 2
 return_number                    1 3
 classification      1     1
 scan_angle_rank   -13    13
 user_data           0     0
 point_source_ID     0   255
 WARNING: 2 points outside of header bounding box
 number of last returns: 2209098
 covered area in square units/kilounits: 728996/0.73
 point density: all returns 3.30 last only 3.03 (per square units)
 spacing: all returns 0.55 last only 0.57 (in units)
 overview over number of returns of given pulse: 2014615 389998 0 0 0 0 0
 histogram of classification of points:
 2404613  Unclassified (1)
 real max x larger than header max x by 0.000422
 real min x smaller than header min x by 0.017715

In recent days more and more users are discovering the LAStools toolbox that adds powerful LiDAR processing capabilities to ArcGIS versions 9.3, 10.0, and 10.1 allowing you to exploit your LiDAR data in the most sophisticated manner through the familiar toolbox interface (see the interface images at the end).

However – as shown above – the most exciting thing I just learned recently myself: You can easily use the ArcGIS model builder to create entire LiDAR workflows in a very intuitive manner by combining individual LAStools from the toolbox into a pipeline of LiDAR processing tasks that can, for example, turn a set of five individual flight strips into DTM, a DSM, and a CHM at a click of a button by connecting up the individual tools with the proper parameters as outlined above. Please let me know if you have created similar model builder LiDAR workflows on your own using the ArcGIS LAStools toolbox for LiDAR processing …

This week rapidlasso GmbH is exhibiting its LAStools LiDAR processing software in tropical Cairns (Australia) at the LiDAR Technologies 2013 conference that is emerging as the premier LiDAR event in Australia. This marks the first time that rapidlasso is acting as a sponsor with its own exhibition booth on the showfloor. During the technical program rapidlasso is providing an update on the recently released PulseWaves open data exchange format for storing full waveform LiDAR data. The PulseWaves DLL is now fully integrated into the RASP software of Airborne Research Australia (ARA) at Flinders University in Adelaide and full waveform data is showcased that was recently acquired by ARA as part of a TERN AUScover campaign.

About rapidlasso GmbH:
Technology start-up rapidlasso GmbH specializes in efficient LiDAR processing tools that are widely known for their high productivity. They combine robust algorithms with efficient I/O and clever memory management to achieve high throughput for data sets containing billions of points. The company’s flagship product – the LAStools software suite – has deep market penetration and is heavily used in industry, government agencies, research labs, and educational institutions. Visit http://rapidlasso.com for more information.

Having a laugh with Toby Clewett (Senior Geospatial Analyst at Sunshine Coast Regional Council) and Anne Henderson (catchement modeler and spatial analyst CSIRO Land and Water) at the rapidlasso booth

We had an ArcGIS LiDAR processing toolbox since April 2012 and now we have one for QGIS as well. It has been tested successfully with QGIS 1.8.0-Lisboa and 2.0.1-Dufour. Download and install QGIS 2.01-Dufour and follow these steps:

1. If you already started QGIS then exit the application.
2. Delete the entire folder “C:\Program Files\QGIS Dufour\apps\qgis\python\plugins\processing\lidar”.
   
3. Put the “.\lidar” folder that is inside this ZIP file into its place (note that users of QGIS 1.8.0-Lisboa need to delete the folder “C:\Users\Martin\.qgis\python\plugins\sextante\lidar” instead and replace it with the “.\lidar” folder from this ZIP file).
4. Get the most recent version of LAStools by downloading the 20 MB “lastools.zip“ file.
5. Drag and drop (or extract) the “.\lastools” folder from inside the ’lastools.zip‘ file such that there is no space in the path (e.g. bad: “C:\Program Files\lastools”, good: “D:\software\lastools”).
6. Start QGIS. If there is a Python script error carefully repeat steps 1 to 3.
7. Turn on the “Processing Toolbox” as shown below.
   
8. Switch the processing toolbox from “Simplified Interface” to ”Advanced Interface” as shown below.
   
9. Open the “Options and configuration” sub menu of “Processing” as shown below.
   
10. Check the “Activate” button for “Providers->Tools for LiDAR data” and fill in the path to your local LAStools folder as shown below.
    
11. Now you should see the “Tools for LiDAR data” in the toolbox and all the LAStools as shown below.
    
12. Start lasground via double click and fill in the settings as shown below.
    
13. Look at the file you just created by running lasview from the toolbox. Yay!
14. Create this with two quick calls to las2dem and las2iso … sooo easy.
    

Kudos to Victor Olaya for creating this whole framework and providing me with example scripts for a simple LAStools toolbox that I just had to modify … (-:

At the code sprint following the FOSS4G 2013 conference, rapidlasso GmbH completed a toolbox that exposes the extensive LiDAR processing capabilities of the LAStools software suite within QGIS. QGIS (previously known as “Quantum GIS”) is a comprehensive desktop geographic information system that is free and open source software (FOSS). QGIS has seen rapid growth in acceptance as a viable and cost-effective alternative to the commerical ArcGIS products of ESRI. The newly integrated “Processing” framework (previously known as “SEXTANTE“) allows to combine the LiDAR functionality of LAStools with native raster and vector operations of QGIS as well as capabilities of other popular GIS packages such as GDAL, GRASS, SAGA, TauDEM, PostGIS and the (geo-)statistics package R. Detailed instructions on how to add the new LAStools toolbox to QGIS are available here.

About rapidlasso GmbH:
Technology start-up rapidlasso GmbH specializes in efficient LiDAR processing tools that are widely known for their high productivity. They combine robust algorithms with efficient I/O and clever memory management to achieve high throughput for data sets containing billions of points. The company’s flagship product – the LAStools software suite – has deep market penetration and is heavily used in industry, government agencies, research labs, and educational institutions. Visit http://rapidlasso.com for more information.

Just in time for INTERGEO 2013, the creators of LAStools and LASzip have partnered with LIDAR Technology Co., Ltd. of Zhubei City, Taiwan as their first exclusive reseller for the Taiwanese market. The two companies decided to strengthen their ties after holding a successful LiDAR processing workshop together with over 60 attendees following the International Symposium on Mobile Mapping Technologies (MMT 2013) in Tainan City last May. “We were looking for a partner with a strong technical mindset who is committed to capacity building,” says Dr. Martin Isenburg, founder of rapidlasso GmbH, “and LIDAR Technology Co., Ltd. is investing in R&D – including full waveform LiDAR – and has strong ties to academia. We couldn’t have found a better collaborator.” Dr. “James” Jin-King Liu, the CEO of LIDAR Technology Co., Ltd. concurs “our companies are a great match. LAStools is already widely popular in Taiwan, yet to employ them to their fullest potential can require some guidance. Our company has the technical expertise to provide this kind of support locally.”

About rapidlasso GmbH:

Technology start-up rapidlasso GmbH specializes in efficient LiDAR processing tools that are widely known for their high productivity. They combine robust algorithms with efficient I/O and clever memory management to achieve high throughput for data sets containing billions of points. The company’s flagship product – the LAStools software suite – has deep market penetration and is heavily used in industry, government agencies, research labs, and educational institutions. Visit http://rapidlasso.com for more information.

About LIDAR Technology Co., Ltd.:

LIDAR Technology Co., Ltd. was founded in 2010 out of the airborne LiDAR team of the Industrial Technology Research Institute (ITRI), a National Laboratory in Taiwan. Their primary business is research, development, and application of earth observation, geoinformatics, and location-based services. LIDAR Technology Co., Ltd. performs acquisition, handling, and analysis of space-borne and airborne data with a focus on LiDAR, including full waveform, for which the company has developed own specialized solutions for their customers. Types of applications include mineral resources, geological mapping, geo-hazards studies, land-use and land-cover mapping, coastal zone mapping, forestry mapping, …  visit http://www.lidar.com.tw for more information.

With CEO “James” Jin-King Liu and General Manager Ian Hsu in front of the office of LIDAR Technology Co., Ltd.

In August and September of 2013, I spent several weeks in the woods of Poland teaching LiDAR processing with LAStools to four groups of forestry students as part of the ForseenPOMERANIA camp. Wanting to be locally relevant, I switched my usual training data for a 500 by 500 meter tile of LiDAR from nearby forest that I randomly clipped from the massive amount of data that was handed to me. I did not do a test run before the workshop because I trust my algorithms. Imagine the shock when - during my live demo on the first day of teaching - the bare-earth points extracted with “lasground.exe” showed unexpected distortions: large, weird-looking bumps appeared when generating a hillshade with “las2dem.exe” for terrain that was completely flat.

Turns out we found remains of  WW II tank positions. We later drove to the site and verified our findings on the ground.

At that time I had generated a larger 6km by 6km bare-earth hillshade of the area which I showed to a local ranger who pointed out the defensive moat that was dug to stop tanks from advancing. It also became clear that the tanks were aiming towards the north-west, hence at Germany, suggesting they were Russian positions. When you follow the direction the line of tanks are pointing at you can find craters and evidence of the German trenches in the hills.

This was in August. On my return in September, I would meet a retired gentleman whose hobby was a variation of geo-caching: finding old German bunkers and pin-pointing their exact GPS coordinates starting from rough locations on old maps available in souvenir shops and historic records. He confirmed that the positions we found were from late WW-II when Russian forces were advancing fast towards the Eastern seaboard of Germany but were digging in whenever encountering pockets of strong resistance. The saga continues here …

This is part two of a three-part tutorial on how to use LAStools to implement a pipeline that (1) quality checks a newly received set of raw LiDAR flight strips, (2) tiles and prepares the LiDAR for subsequent exploitation, and (3) generates raster and vector derivatives such as DTMs, DSMs, building footprints, and iso-contours with multi-core batch processing.

To get started, download the latest version of LAStools and these 7 example flight strips: strip1, strip2, strip3, strip4, strip5, strip6, strip7 and put them into a folder called ‘.\lastools\bin\strips_raw’. For simplicity we will work inside the ‘.\lastools\bin’ directory, so open a DOS command line window and change directory to where this folder is on your computer, for example, with the command ‘cd c:\software\lastools\bin’ or with ‘cd d:\lastools\bin’ followed by ‘d:’. It may be helpful (not mandatory) to follow the tutorial on quality checking first.

Create a new folder called ’.\lastools\bin\tiles_raw’ for storing the LiDAR tiles. Then run ‘lastile.exe’ in GUI mode and load the 7 example flight strips. You can either do this by double-clicking ‘lastile.exe’ and using the ‘browse …’ menu or by entering the command below:

C:\lastools\bin>lastile -i strips_raw\LDR*.laz -gui

Set the GUI options as shown below to create a buffered tiling from the original flight strips. Check the button ‘files are flightlines’ so that points from different flight lines get a unique flight line ID stored in their ‘point source ID’ attribute. This makes it possible to identify later which points of a tile came from the same flight strip. Use the default 1000 to specify the tile size and request a buffer of 50 meters around every tile. This buffer helps to reduce edge artifacts at the tile boundaries in a tile-based processing pipeline.  Shift the tiling grid by 920 in x direction and by 320 in y direction so the flight strips fit in exactly 4 tiles. This is not mandatory but results in fewer tiles and therefore fewer cuts (experimentally discovered). Requests compressed output to overcome the I/O bottleneck. Set the projection to UTM zone 19 north and the vertical datum to NAVD88. When projection information is missing in the original flight strips it is a good idea to add this as early as possible so it is available in subsequent processing steps.

Press ‘RUN’ and the ‘RUN’ window will pop up. If you forgot or need to fix a setting you can directly modify the shown command-line before you press START. Alternatively, if you prefer to work in the command-line, you can use this almost equivalent command here. It has been enhanced by one additional parameter shown in red. It quantizes the input on-the-fly to a more appropriate centimeter [cm] resolution because airborne LiDAR does not have the millimeter [mm] resolution that the raw flight strips are stored with. See the lasinfo report shown at the end of the tutorial on quality checking to see the excessive millimeter resolution that the additional parameter in red is fixing.

C:\lastools\bin>lastile -i strips_raw\LDR*.laz ^
                        -files_are_flightlines ^
                        -rescale 0.01 0.01 0.01 ^
                        -utm 19N -vertical_navd88 ^
                        -tile_size 1000 -buffer 50 ^
                        -tile_ll 920 320 ^
                        -odir tiles_raw -o fitch.laz

Create a new folder called ’.\lastools\bin\tiles_ground’ for storing the ground-classified LiDAR tiles. Then run ‘lasground.exe’ in GUI mode and load the 4 tiles. You can do this by double-clicking ‘lasground.exe’ and using the ‘browse …’ menu or by entering the command below:

C:\lastools\bin>lasground -i tiles_raw\fitch*.laz -gui

Set the GUI options as shown below to classify the bare-earth points in all tiles. Select the ‘metro’ setting (which uses a step size of 50m) because there are very large hangars in the point cloud and the ‘ultra fine’ setting for the initial ground estimate. For more details on these parameters read the corresponding README.txt file. Specify the output directory as ’tiles_ground’ and select compressed LAZ output. Select 4 cores only if you have a computer with that many cores. This will assign tiles to different cores to process multiple tiles in parallel.

Press ‘RUN’ and the ‘RUN’ window will pop up. If you forgot or need to fix a setting you can directly modify the shown command-line before you press START. Alternatively, if you prefer to work in the command-line, you can use this equivalent command here:

C:\lastools\bin>lasground -i tiles_raw\fitch*.laz ^
                          -metro -ultra_fine ^
                          -odir tiles_ground -olaz ^
                          -cores 4

Run ‘lasview.exe’ on ‘fitch_273920_4714320.laz’ in the ’tiles_ground’ folder to inspect the result of your bare-earth classification. Press ‘g’ on the keyboard to show only the ground points. After all points have loaded press ‘t’ on the keyboard to triangulate only the points shown. Press ‘-’ to make the points disappear and press ‘h’ multiple times to iterate over the possible ways to shade the triangulation. Now press ‘=’ to make the points reappear, press ‘u’ to show the unclassified (non-ground) points, and press ‘c’ a few times to iterate over the possible ways to color the points.

There are “dirt” points below the ground and “air” points far above the ground such as the one pointed at by the mouse cursor. We remove them with ‘lasheight.exe’. This tool computes for each point the vertical distance to the triangulation of the ground points and stores it in the ‘user_data’ field. We also need these heights for the following step of finding buildings and vegetation. Create a new folder called ’.\lastools\bin\tiles_height’ for storing the cleaned LiDAR tiles with height above ground information. Run ‘lasheight.exe’ in GUI mode and load the 4 ground-classified tiles by double-clicking ‘lasheight.exe’ and using the ‘browse …’ menu or by entering the command below:

C:\lastools\bin>lasheight -i tiles_ground\fitch*.laz -gui

Configure the GUI with the settings shown below. By default ‘lasheight.exe’ uses the points classified as ground to construct a TIN and then calculates the height of all other points in respect to this ground surface. With ‘drop_above 30′ and  ’drop_below -2′ all points that are 30 meters above or 2 meters below the ground are removed from the compressed LAZ tiles output to the ’tiles_height’ folder. Run the process on multiple cores if possible.

Press ‘RUN’. If you forgot or need to fix a setting you can directly modify the shown command-line before you press START. Or use this in the DOS command-line:

C:\lastools\bin>lasheight -i tiles_ground\fitch*.laz ^
                          -drop_below -2 -drop_above 30 ^
                          -odir tiles_height -olaz ^
                          -cores 4

Next, create a new folder called ’.\lastools\bin\tiles_classified’ for storing classified LiDAR tiles with building and vegetation points. Run ‘lasclassify.exe’ in GUI mode and load the 4 height-cleaned tiles by double-clicking or by entering the command below:

C:\lastools\bin>lasclassify -i tiles_height\fitch*.laz -gui

Set the GUI settings as shown below to identify buildings and trees. Mileage may vary on this step because automatic LiDAR classification is a hard problem. The default settings require a density of at least 2 pulses per square. Setting the step size for computing planarity (or ruggedness) to 3 meters has fewer false positives but misses some smaller buildings. For more details on tuning these parameters see the corresponding README.txt file.

Press ‘RUN’. If you forgot or need to fix a setting you can directly modify the shown command-line before you press START. Or use this command-line:

C:\lastools\bin>lasclassify -i tiles_height\fitch*.laz ^
                            -step 3 ^
                            -odir tiles_classified -olaz ^
                            -cores 4

Run ‘lasview.exe’ on ‘fitch_273920_4714320.laz’ in the ’tiles_classified’ folder and be amazed about the result of your building and vegetation classification. Although not 100 percent correct, ‘lasclassify.exe’ identifies all man-made structures as class 6 and marks all vegetation above two meters as class 5.

Create the last folder called ’.\lastools\bin\tiles_final’ for storing the final LiDAR tiles without the 50 meter buffers that are ready to be shipped to the customer. Run ‘lastile.exe’ in GUI mode and load the 4 classified tiles with:

C:\lastools\bin>lastile -i tiles_classified\fitch*.laz -gui

Set the GUI settings as shown below to remove the buffers that were added in the very beginning and carried through all processing steps to avoid artifacts along the boundaries.

Press ‘RUN’. If you forgot or need to fix a setting you can directly modify the shown command-line before you press START. Or use this command-line that has one additional parameter shown in red. This sets the user data field of each point back to zero that was set by ‘lasheight.exe’ to the height above ground (in decimeters). This number is meaningless to the customer and setting it to zero improves compression.

C:\lastools\bin>lastile -i tiles_classified\fitch*.laz ^
                        -set_user_data 0 ^
                        -remove_buffer ^
                        -odir tiles_final -olaz

Finally let us run ‘lasinfo’ on one of the generated tiles and also compute the density:

C:\lastools\bin>lasinfo -i tiles_final\fitch_271920_4714320.laz -cd
reporting all LAS header entries:
  file signature:             'LASF'
  file source ID:             0
  global_encoding:            0
  project ID GUID data 1-4:   00000000-0000-0000-0000-000000000000
  version major.minor:        1.0
  system identifier:          'LAStools (c) by Martin Isenburg'
  generating software:        'lastile (131011) commercial'
  file creation day/year:     0/0
  header size:                227
  offset to point data:       5689
  number var. length records: 4
  point data format:          0
  point data record length:   20
  number of point records:    2571486
  number of points by return: 2196212 0 375274 0 0
  scale factor x y z:         0.01 0.01 0.01
  offset x y z:               0 4000000 0
  min x y z:                  272044.80 4714466.57 82.34
  max x y z:                  272919.99 4715243.91 169.21
variable length header record 1 of 4:
  reserved             43707
  user ID              'LeicaGeo'
  record ID            1001
  length after header  5120
  description          ''
variable length header record 2 of 4:
  reserved             43707
  user ID              'LeicaGeo'
  record ID            1002
  length after header  22
  description          'MissionInfo'
variable length header record 3 of 4:
  reserved             43707
  user ID              'LeicaGeo'
  record ID            1003
  length after header  54
  description          'UserInputs'
variable length header record 4 of 4:
  reserved             43707
  user ID              'LASF_Projection'
  record ID            34735
  length after header  48
  description          'by LAStools of Martin Isenburg'
    GeoKeyDirectoryTag version 1.1.0 number of keys 5
      key 1024 tiff_tag_location 0 count 1 value_offset 1 - GTModelTypeGeoKey: ModelTypeProjected
      key 3072 tiff_tag_location 0 count 1 value_offset 32619 - ProjectedCSTypeGeoKey: PCS_WGS84_UTM_zone_19N
      key 3076 tiff_tag_location 0 count 1 value_offset 9001 - ProjLinearUnitsGeoKey: Linear_Meter
      key 4099 tiff_tag_location 0 count 1 value_offset 9001 - VerticalUnitsGeoKey: Linear_Meter
      key 4096 tiff_tag_location 0 count 1 value_offset 5103 - VerticalCSTypeGeoKey: VertCS_North_American_Vertical_Datum_1988
the header is followed by 2 user-defined bytes
LASzip compression (version 2.2r0 c2 50000): POINT10 2
LAStiling (idx 8, lvl 2, sub 0, bbox 271920 4.71232e+006 275920 4.71632e+006)
reporting minimum and maximum for all LAS point record entries ...
  X   27204480   27291999
  Y   71446657   71524391
  Z       8234      16921
  intensity 0 255
  edge_of_flight_line 0 1
  scan_direction_flag 0 1
  number_of_returns_of_given_pulse 1 2
  return_number                    1 3
  classification      1     6
  scan_angle_rank   -13    21
  user_data           0     0
  point_source_ID     1     7
number of last returns: 2196301
covered area in square meters/kilometers: 408700/0.41
point density: all returns 6.29 last only 5.37 (per square meter)
      spacing: all returns 0.40 last only 0.43 (in meters)
overview over number of returns of given pulse: 1821027 750459 0 0 0 0 0
histogram of classification of points:
          286793  Unclassified (1)
          589019  Ground (2)
         1586840  High Vegetation (5)
          108834  Building (6)

This is the final part of a three-part tutorial on how to use LAStools to implement a pipeline that (1) quality checks a newly received set of raw LiDAR flight strips, (2) tiles and prepares the LiDAR for subsequent exploitation, and (3) generates raster and vector derivatives from the LiDAR points such as DTMs, DSMs, contour lines and building footprints with multi-core batch processing and Tin streaming with BLAST.

It is maybe helpful to start with the tutorial on quality checking. But it is mandatory to first complete the tutorial on LiDAR preparation because that is where you generate the cleaned and classified LiDAR tiles in folders ’tiles_classified’ and ’tiles_final’ that we are using as input in this tutorial.

Generating matching DTM tiles without edge artifacts

Create a new folder called ’.\lastools\bin\tiles_dtm’ for storing the bare-earth digital terrain models (DTMs). Then run ‘las2dem.exe’ in GUI mode and load the 4 tiles from the folder ’tiles_classified’. You can either do this by double-clicking ‘las2dem.exe’ and using the ‘browse …’ menu or by entering the command below:

C:\lastools\bin>las2dem -i tiles_classified\fitch*.laz -gui

The las2dem tool creates rasters from LiDAR by triangulating the points and then sampling the resulting TIN at the center of each raster pixel. Remember, the tiles in folder ’tiles_classified’ have a buffer of 50 meter that was initially created with ‘lastile’ and that has been carried through ‘lasground’, ‘lasheight’, and ‘lasclassify’. The presence of these buffers allows us now to avoid artifacts along the tile boundaries because our TINs will be 50 meters wider in all directions than the raster that we are sampling. Rasterizing only the extend of the original tile without the surrounding buffer is accomplished with the ’tile_bb_only’ command line parameter.

Set the GUI options as shown below to create one DTM per tile. The DTMs will seamlessly fit together. Check the button ‘use tile bounding box’ for the reason explained above and check the button ‘extra pass’ which makes two passes over the data. In the first pass it only counts the number of points that pass through the filter so that in the second it can optimize the memory used to triangulate them. Choose ‘bil’ as the output format. It is much much much more efficient than the ‘asc’ format many people like to use. Select 4 cores if you have that many and set the output directory to ’tiles_dtm’. Use the ‘filter…’ menu to choose two filters: the first keeps only the ground points with filter ‘keep_classification 2′ and the second keeps only one point every 0.5 by 0.5 meter area with the filter ‘thin_with_grid 0.5′. The order is really important (excercise: explain why!). Because we create DTMs with the default ‘step’ size of 1 meter there is no sense in constructing a temporary TIN that is more detailed than one ground point per 0.5 by 0.5 meter area. Constructing a large TIN requires a lot of main memory. We should only use as many points as necessary to sample the TIN at our target resolution ‘step’. Adding a ‘thin_with_grid’ filter with half the ‘step’ size seems always a good choice for ‘las2dem’.

Press ‘RUN’ and the ‘RUN’ window will pop up. You may have to close the ‘output’ menu before you can actually see the ‘RUN’ button. If you forgot or need to fix a setting you can directly modify the shown command-line before you press START. Alternatively, if you prefer to work in the command-line, you can use this equivalent command here.

C:\lastools\bin>las2dem -i tiles_classified\fitch*.laz ^
                        -keep_class 2 -thin_with_grid 0.5 ^
                        -extra_pass ^
                        -use_tile_bb ^
                        -odir tiles_dtm -obil ^
                        -cores 4

Generating matching DSM tiles without edge artifacts

Create a new folder called ’.\lastools\bin\tiles_dsm’ for storing the first-return digital surface models (DSMs). You could re-use ‘las2dem’ if you have not closed the application yet. Otherwise re-run it in GUI mode and load the 4 tiles from the folder ’tiles_classified’ again. Do this by double-clicking ‘las2dem.exe’ and using the ‘browse …’ menu or by entering the command below:

C:\lastools\bin>las2dem -i tiles_classified\fitch*.laz -gui

Set the GUI options as shown below. The only changes are that the output directory name is now ’tiles_dsm’ and that the filtering is now keeping first returns instead of ground points. If you re-used the GUI from the last step you need to delete the old filters and enter the new ones in the correct order (exercise: why is the order important?). You can delete old filter entries by double clicking them. Select 4 cores only if you have a computer with that many cores. This will assign tiles to different cores and process them in parallel.

Press ‘RUN’ and the ‘RUN’ window will pop up. You may have to close the ‘output’ menu before you can actually see the ‘RUN’ button. If you forgot or need to fix a setting you can directly modify the shown command-line before you press START. Alternatively, if you prefer to work in the command-line, you can use this equivalent command here:

C:\lastools\bin>las2dem -i tiles_classified\fitch*.laz ^
                        -first_only -thin_with_grid 0.5 ^
                        -extra_pass ^
                        -use_tile_bb ^
                        -odir tiles_dsm -obil ^
                        -cores 4

Visualizing and processing DEM rasters as if they were points

Here a little trick. All LAStools can read the BIL format “as if” it was a point format. The other two raster formats that are supported this way are ESRI’s ASC and FUSION’s DTM. But if possible don’t use ASC because it stores the values as ASCII text which is terribly inefficient and much much slower to read than BIL or DTM. Run lasview in GUI mode and load the four BIL rasters from the folder ’tiles_dsm’. You need to check the button labelled ‘.bil’ in the ‘browse …’ menu to be able to select the BIL rasters as input. Alternatively you can use this command-line:

C:\lastools\bin>lasview -i tiles_dsm\fitch*.bil -gui

Choose the option ‘selected file only’ and pick one of the four tiles. Play with the ‘color_by’ or ‘render_only’ options. But these can also be changed once ‘lasview’ is displaying the point cloud.

Visualize the ‘fitch_273920_4714320.bil’ from the ’tiles_dsm’ folder. After all points have loaded press ‘t’ on the keyboard to triangulate them. Press ‘-’ to make the points disappear and press ‘h’ multiple times to iterate over the possible ways to shade the triangulation.

Using lasgrid to merge DEM tiles into one large DEM raster

Run ‘lasgrid.exe’ in GUI mode and load the 4 raster DSM tiles in BIL format from the ’tiles_dsm’ folder by double-clicking ‘lasgrid.exe’ and using the ‘browse …’ menu. Check the button labelled ‘.bil’ to be able to select the BIL rasters as input. Or use the following in the DOS command-line:

C:\lastools\bin>lasgrid -i tiles_dsm\fitch*.bil -gui

Configure the GUI with the settings shown below. Check the ‘merge files into one’ button, specify the output file as ‘dsm.tif’, and specify the northern UTM zone 19 and NAVD88 as the projection information because the BIL files we generated earlier do (currently) not store it.

Press ‘RUN’. If you forgot or need to fix a setting you can directly modify the shown command-line before you press START. Or use this in the DOS command-line:

C:\lastools\bin>lasgrid -i tiles_dsm\fitch*.bil -merged ^
                        -utm 19N -vertical_navd88 ^
                        -o dsm.tif

Load the merged raster ‘dsm.tif’ into a GIS package such as QGIS or ArcGIS and repeat the exercise by creating – in the exact same way – a merged ‘dtm.tif’ from the 4 raster DTM tiles in the ’tiles_dtm’ folder.

Using the BLAST extension for seamless generation of rasters

Run ‘blast2dem.exe’ in GUI mode and load the 4 raster DTM tiles in BIL format by double-clicking ‘blast2dem.exe’ and using the ‘browse …’ menu.  You need to check the button labelled ‘.bil’ to be able to select the BIL rasters as input. Or enter this in the DOS command-line window:

C:\lastools\bin>blast2dem -i tiles_dtm\fitch*.bil -gui

Configure the GUI with the settings shown below. Check the ‘merge files into one’ button, specify the output file as ‘dtm_hillshade_raster.png’, select ‘hillside shading’, and choose ‘png’ as the output format. Use the ‘projection …’ menu to specify the northern UTM zone 19 and NAVD88 as the projection information because the BIL files we generated earlier do (currently) not store it.

Press ‘RUN’. If you forgot or need to fix a setting you can directly modify the shown command-line before you press START. Or use this in the DOS command-line:

C:\lastools\bin>blast2dem -i tiles_dtm\fitch*.bil -merged ^
                          -hillshade ^
                          -utm 19N -vertical_navd88 ^
                          -o dtm_hillshade_raster.png

Whenever ‘blast2dem’, ‘las2dem’, ‘lasgrid’, or ‘lasoverlap’ generate rasters that are standard color or gray-scale PNG, TIF, or JPG that can be loaded as an image overlay into Google Earth, a KML is automatically generated – assuming there is projection information either available in the input or specified in the command-line. If you double-click the KML file that we just created you will see the hillshading overlaid and nicely aligned with Fitchburg airport.

Repeat the exercise by creating – in the exact same way – a merged and hillshaded ‘dsm_hillshade_raster.png’ with KML file from the 4 raster DSM tiles in the ’tiles_dsm’ folder.

Using the BLAST extension for seamless generation of contours

Run ‘blast2iso.exe’ in GUI mode and load the 4 BIL raster tiles from the ’tiles_dtm’ folder by double-clicking and using the ‘browse …’ menu or by entering the command below:

C:\lastools\bin>blast2iso -i tiles_dtm\fitch*.bil -gui

Set the GUI as shown below to compute 1 meter contours from on-the-fly merged bare-earth DTM rasters in BIL format. Check the ‘merge files into one’ button, specify the output file name as ‘dtm_contours_raster_1m.shp’, set the isovalue extraction to every 1 unit (here: meter), and check the buttons for ‘simplify short segments’, ‘simplify small bumps’ and, ‘clean short lines’. Read the corresponding README.txt file if you want more details on what these options exactly do. Add the northern UTM zone 19 as the projection information because the BIL files we generated earlier do (currently) not store it. This is less important when generating SHP output but would be required to produce KML instead.

Press ‘RUN’. If you forgot or need to fix a setting you can directly modify the command-line before you press START. Or use this command-line:

C:\lastools\bin>blast2iso -i tiles_dtm\fitch*.bil -merged ^
                          -iso_every 1 ^
                          -simplify_length 0.5 -simplify_area 1 -clean 5 ^
                          -utm 19N ^
                          -o dtm_contours_raster_1m.shp

Visualize the result with the convenient ShapeViewer.exe program and check if the 1 meter contours that ‘blast2dem’ has produced are what you expected:

Repeat the exercise by creating – in the exact same way – a merged and hillshaded ‘dsm_contours_raster_3m.shp’ with from the 4 raster DSM tiles in the ’tiles_dsm’ folder but with a spacing of 3 meters instead of 1 meter between the contour lines.

Extracting building footprints from on-the-fly merged tiles

Run ‘lasboundary.exe’ in GUI mode and load the 4 classified tiles with:

C:\lastools\bin>lasboundary -i tiles_final\fitch*.laz -gui

Set the GUI as shown below to generate compute polygonal outlines around the buildings points classified in the previous tutorial. Check the ‘merge files into one’ button to create one seamless output file. Add a ‘keep_classification 6′ filter that filters out all points except those classified as building. Check the ‘disjoint’ button – otherwise a funny-looking single polygonal hull will be created connecting all buildings. Set the ‘concavity’ to 1.5 meters to specify the smallest feature that lasboundary can grow into. This value needs to be 2 to 3 times the average point spacing. If you set it too small, lasboundary will “eat” its way into the buildings. If you set it too high nearby buildings will be joined together and the outlines will become coarser. Play with a few different settings such as 1.0, 2.0, 5.0, and 10.0 to get an intuitive idea how this parameter affects the result.

Press ‘RUN’. If you forgot or need to fix a setting you can directly modify the command-line before you press START. Or use the equivalent command-line below directly in the DOS window.

C:\lastools\bin>lasboundary -i tiles_final\fitch*.laz -merged ^
                            -keep_class 6 ^
                            -concavity 1.5 -disjoint ^
                            -o buildings.shp

Visualize the result with the convenient ShapeViewer.exe program and check if the building footprints that lasboundary has produced are what you expected:

By generating KML output instead and overlaying the result on Google Earth imagery you can easily check which and how many buildings were missed and go back to adjust the parameters for ‘lasclassify’ in the previous tutorial to maybe achieve better results.

C:\lastools\bin>lasboundary -i tiles_final\fitch*.laz -merged ^
                            -keep_class 6 ^
                            -concavity 1.5 -disjoint ^
                            -o buildings.kml

After accidentally finding Russian tanks in Polish forests I was curious to see if there was something else hiding under the forest canopy. Remember, I randomly picked a 500 by 500 meter LiDAR tile as example data to introduce a group of forestry students to LiDAR processing with LAStools during the ForseenPOMERANIA camp. After extracting ground points with “lasground.exe“, strange bumps appeared in the bare-earth hillshades generated with “las2dem.exe“ for terrain that was supposed to be completely flat … they turned out to be Russian WW-II positions.

I met Achim when returning to teach the next two groups of students. His hobby is a mix between geo-caching and conflict-archaeology: locating old German bunkers based on approximate coordinates available in historic records and tourist maps and then mapping them precisely with GPS. Achim had a list of longitude/latitude positions as KML files where he was planning to search for known bunkers. I used “lasboundary.exe” to create polygonal outlines in KML format for all areas where we had LiDAR from the forestry project. With Google Earth it was easy to find overlaps between his target areas and our LiDAR coverage.

I extracted the ground points and created bare-earth DTMs of the relevant area with a LAStools batch processing pipeline of “lastile.exe“, “lasground.exe” and “las2dem.exe” and used “blast2dem.exe” to create a seamless hillshading with proper KML referencing (here is a tutorial for such a pipeline). What I found was pretty amazing.

At first glance it looks like a maze of little creeks that are running alongside the ridges of the hillsides but we know that water flows downhill and not “along-hill”. What we see is a network of WW-II trenches that are connecting the bunkers Achim is looking for. A closer look also reveals the likely location of those bunkers.

I placed a pin on each of them and exported their longitude and latitude coordinates for upload into Achim’s GPS device. The next day we set out to verify our LiDAR findings on the ground.

It was a rainy day. Walking through this maze of green and overgrown trenches from one moss-covered bunker ruin to the next felt oddly quiet and peaceful. Achim explained that these bunkers were originally built to defend the border with Poland – long before the Second World War broke out. Only when Russian soldiers were advancing on Germany after the collapse of the Eastern front, the young boys of the Hitler Youth were commanded to dig this network of trenches in order to fortify the bunker and stop enemy lines from gaining ground. After the war the Russians blew up all bunkers that were facing East so that their troops would not ever have to face them again.

GRAFCAN has launched a new product based on LiDAR for the Canary Islands: a digital suface model (DSM) that is literally “on steroids”. It is a synthetic view (not image-derived) that lets the user intuitively understand the territory. The product combines standard hillshading with a height-based color-coding enabling the viewer to “see” where the trees are taller and to grasp height differences between buildings. Darker shades of green indicate high vegetation and lighter shades of green indicate low vegetation. For man-made structures, the shade of red is darker for taller buildings and lightest for one-story houses. The new product is available for all of the Canary Islands at a resolution of 2.5 meters/pixel via the GRAFCAN Web viewer and also as a WMS service.

Example of the new DSM on steroids: The viewer intuitively grasps the height and spatial distribution of trees in vegetation areas and the density and size of buildings in urban areas.

The product is complemented with a layer with the water areas and a layer of the main roads (both extracted from topographic maps). Around the airports the terrain is very flat and mostly empty. Therefore an intensity map was extracted from the LiDAR and fused into the imagery to provide additional details.

Comparison between orthophoto and DSM on steroids: The ridge is practically invisible in the ortho and the dark green colors of the vegetation are quite missleading to the untrained eye.

The LiDAR was flown between 2011 and 2012 with a Leica ALS60 with an average density of 1 pulse per square meter. Most of the LiDAR processing was done with LAStools batch scripts:

• clean-up of header with las2las.exe
• orthometric height correction with lasheight.exe
• tiling with lastile.exe
• bare-earth extraction with lasground.exe
• noise removal (clouds and low points) with lasheight.exe
• DSM rasterization at 2.5 meters per pixel with las2dem.exe
• flattening reservoirs, ponds, swimming pools with las2dem.exe
• normalize LiDAR to height above ground with lasheight.exe
• create height above ground map with lasgrid.exe
• generating intensity rasters for airports using LiDAR with lasgrid.exe

Other processing tasks involved ArcGIS and Photoshop:

• separate height above ground map into vegetation and building height map with ArcGIS
• generating hillshaded rasters from the DSM with ArcGIS
• rasterizing building footprints colored by height with ArcGIS
• rasterizing roads from topographic maps with ArcGIS
• rasterizing reservoirs, ponds, swimming pools with ArcGIS
• extracting water line of Ocean (contour at 1 meter) from DSM with ArcGIS
• some manual editing on complex areas (classification errors, ocean line…) with ArcGIS
• fusing airport intensity rasters into hillshade rasters with Photoshop
• blending all the layers into the final product with Photoshop

Comparison between bare earth DTM and DSM on steroids. The green houses appear as “low planar vegetatation”. They are made out of coarse maze fabric (instead of glass) that lets the laser through and does not deflect it (like glass would).

GRAFCAN is a company that  produces and publishes the geographic information of Canary Islands. The company has done LiDAR flights since 2010. You can explore the Canary LiDAR directly in 3D via their interactive Web viewer of IDECanarias (press the “Lidar” button).

Another beautiful example of a DSM on steroids.

Eric van Rees, the editor of the Geoinformatics magazine, met with rapidlasso GmbH at INTERGEO 2013 in Essen to have a quick chat about LAStools, LASzip, and PulseWaves as well as our LiDAR processing toolboxes for ArcGIS and QGIS.

In the three-page article we talk about the beginnings of rapidlasso’s software, about when and how the company got started, about our various open and closed source products, and about current projects. The interview was published in the most recent October/November edition of the Geoinformatics magazine that was distributed in print at the SPARELMF 2013 conference in Amsterdam this week. If you did not get a copy you can read the online version of the magazine here.

Recently rapidlasso GmbH gave a workshop on LiDAR processing with LAStools at SOPEC in Suva, Fiji. In the exercises we used LiDAR from the Nadi and Ba basins that had been flown in early 2012 by Network Mapping using helicopters with funding from the World Bank.

For checking flight line overlap and alignment ‘lasoverlap.exe‘ needs flight line information present in the LiDAR that was delivered as 500 by 500 meter tiles in the Transverse Mercator 1986 Fiji Map Grid projection. For tiled LiDAR in LAS format the flight line information is meant to be stored in the ‘point source ID’ field for each point.

C:\lastools\bin>lasoverlap -i D:\data_fiji\FMG1986\*.laz -gui

Starting the GUI as shown above loads the 654 tiles (4 GB of LAZ or 30 GB of LAS) and setting the options as shown below creates overlap and a difference images for each of the 654 tiles on 8 cores in parallel. See the tutorial on quality checking for how to run lasoverlap directly on original flightlines.

Alternatively we can merge the 4GB of LAZ input on-the-fly by adding the ‘-merged’ option to the command line and by specifying an explicit output file name (although lasoverlap can only use a single core in this case).

C:\lastools\bin>lasoverlap -i D:\data_fiji\FMG1986\*.laz ^
                           -merged ^
                           -o D:\data_fiji\FMG1986\overlap.png

The resulting overlap image does not entire look like what we expected (note that the image below was downsampled from the original size of 8139 by 7074 pixels).

Although some of the helicopter trajectories are visible in the overlap image, the majority of the flight line information does not seem to be populated correctly in the LAZ files. To verify this we quickly generate a density image where flight lines become visible due to the increase in density in areas of flight line overlap.

C:\lastools\bin>lasgrid -i D:\data_fiji\FMG1986\*.laz ^
                        -merged ^
                        -last_only ^
                        -density ^
                        -false -set_min_max 0 25 ^
                        -o D:\data_fiji\FMG1986\density_25pts.png

Clearly … the flight line information in the point source ID fields of the LAZ files is incomplete. The points from most of the regular back and forth flights seem to have the same point source ID. One could argue that the lack of correct flight line information should be considered a defect in the delivered data because they prevent the recipient from checking the quality in the overlap and alignment of the flight lines and that the vendor should be held resposible to provide a better data set.

This was not the first time we have come across missing flight line information. Therefore we added a new feature called ‘-recover_flightlines’ to the tools ‘lasoverlap.exe‘ and ‘lassplit.exe‘ that uses the GPS time stamps stored with each point in the LAS format to reconstruct missing information about the flight lines. It does, however, rely on the GPS time stamps to be available and correct (e.g. this will not work for the LAS point data format 0 that does not include GPS time stamps).

Above you see the GUI options we use to reconstruct the flight lines from the GPS time stamps and produce a merged overlap and difference image from the 4 GB of LAZ files. We can fix a setting in the ‘RUN’ window that will pop up by modifying the shown command before pressing START. Here is the equivalent command-line:

C:\lastools\bin>lasoverlap -i D:\data_fiji\FMG1986\*.laz ^
                           -merged ^
                           -recover_flightlines ^
                           -o D:\data_fiji\FMG1986\overlap.png

The resulting overlap image shows that the flightlines have been recovered.

And now the corresponding difference image also makes sense. It maps the elevation differences of the lowest point in overlapping flightlines per 2 by 2 grid cell to a color between blue (-0.5 meter), white (0 meter) and red (+0.5 meter) and allows to easily spot potential miss-alignments between flight lines.

After all this work it is also time for rapidlasso to recover. After all … we are in Fiji … (-:

As a sweet foretaste to ILMF 2014, the creators of LAStools, LASzip, and PulseWaves are announcing “LASlayers” already on Valentine’s Day. The new functionality nicely complements their popular and widely-used LiDAR compressor making the compressed LAZ files editable for most practical purposes. LASlayers significantly reduce I/O load for writing modification to LAS or LAZ files, especially when batch-processing LiDAR tiles on many cores in parallel or when sending changes to LiDAR files across bandwidth-limited networks.

LASlayers store modifications and additional  attributes to raw LAS or LAZ files in small LAY files avoiding to replicate data that has not changed.

Most point attributes (e.g. coordinates, intensities, scan angles, GPS times, etc.) are not modified when processing LiDAR. LASlayers avoids re-writing the unchanged portions of LAS or LAZ by storing only the actual differences layer by layer to a new “LAY” file. Changing the point classifications or deleting a few points, for example, can be done with LAY files that are just a tiny fraction of the size of a new LAS or LAZ file. Adding new attributes such as RGB colors or the height-above-ground with LASlayers means only this new information needs to be written.

This also provides simultaneous access to different versions of the data: a LiDAR server or a Web portal can store only a single copy of the raw LiDAR and apply LASlayers as needed on-the-fly, for example, to replace ellipsoidal with orthometric elevations or to add RGB colors.

Even users of other LiDAR processing software can readily take advantage of LASlayers with the new “laslayers.exe” tool that computes the difference between a raw and a modified LAS or LAZ file and expresses it as a LAY file (assuming the point order has not changed). A typical use case is the exchange of modifications to LiDAR files between remote locations such as a vendor in Australia or Canada and a data processing center in China or India. Instead of up- and downloading the entire modified LAS or LAZ files, only the much smaller LAY files need to be send back and forth.

A fully featured prototype of LASlayers is available (10 MB including data) together with three simple exercises that illustrate the concept and allow interested parties to test it already today on their own data.