DRONE IMAGE PROCESSING

TUTORIAL
Miqdad Anwarie
Email : miqdad.anwarie@ui.ac.id

1. Data Management

a. Systematic

Field Data Produced Office Processing

Drone Aerial Photo (Include GPS 1. Rename Photos using Irfanview

Telemetry Log for Photo 2. Generate Photo Coordinate using
Coordinate). Mission Planner

Low Assessment (3 Flight or

Combine and edit OPK from low
more mosaic, generate Omega
assessment using Agisoft Photoscan
Phi Kappa (OPK))

Orthophoto and DEM Generation

Processing using Menci

In whole systematic of drone image data acquisition and processing, there are two part of
works that each other produce data which are linked for producing final drone aerial
imagery. Those are field work which contain drone photo acquisition, omega phi kappa
generation, and low assessment drone orthophoto generation, and office processing work
which contain high assessment drone orthophoto generation, dem layers generation, and
reports.

b. Drone Aerial Photos

Drone photos have been captured from UAV per flight and each flight produced two data
(raw image and telemetry log) that could not be processed directly in the drone image
processing software. There are two step preparation for raw image and telemetry log
that them can be brought to make low assessment in field work and office processing.
Those are batch rename photos using Irfanview and generate photo coordinate from
telemetry log using Mission Planner.

Batch Rename Photos Using Irfanview

Renaming Photos is used to make different name of photos between one flight and
another.
 Step by step of batch renaming photos using Irfanview are described below:

1. Locate the address to raw drone image folder, for example from Sony RX, the name
of raw image folder is 100MSDCF in every flight. This process is repeated every
flight. Before run this batch process is better to create new folder in every flight
folder such as below :
2. Add all raw image in folder for input file using the button as shown in batch
conversion window in number 2.
3. Rename all raw image in name pattern that shown in number 3 based on flight
number and acquisition date, for example the name pattern for PAH Project is
01_PAH_02_Oct2014_####. 01 is flight number, PAH is project name, 02_Oct2014 is
acquisition date, and #### is digit number (usually use 4 digit number).
4. locate the output folder to new folder that described in step number 1 before
named Rename in each flight folder.
5. After all step done, start batch using the button which shown in number 5 and
repeat all step for other flights.

Generate Photo Coordinate From Telemetry Log Using Mission Planner

Photo coordinates from GPS in UAV are logged in Ground Control System (GCS) as
track log, however we need to extract those coordinates from track to point using
Mission Planner which they will be used in generating Omega Phi Kappa in Agisoft
Photoscan.

Step by step process description are describe below :

1. Run Mission Planner from start menu and we use shortcut button CTRL + F to load
temp window.

2. Choose Geo ref images menu in temp window to extract photo coordinate from
UAV telemetry log.

3. Input telemetry log of each flight in Browse Log menu, for example from Flight 05
PAH is 2014-10-01 15-18-14.tlog and locate output photo coordinate in Browse
Directory menu. Output directory must be drone photo directory which have been
renamed before. In short, that is Rename folder of each flight.
4. Estimate offset between GCS time and Camera Time. This step need one photo of
each flight that capture when calibrated before the UAV is taken off. From the
photo, open exif information which contain Date Time Original of the photo. The
information show different between time which has been captured from GCS and
time from exif information, for example in PAH flight 05 that show in the image
below, exif information show the date time original is 2014:10:01 15:15:59 and the
GCS time which has been captured is 3:15:58 or same as 15:15:58. The different
time is 1 second that exif time is faster and that means the log from GCS is minus 1 (-
1) from the camera. So, the offset value that fill in estimate offset form is (-1). In
short, if the different is the GCS time slower than exif time, offset value is minus, and
if the different is the GCS time faster than exif time, offset value is plus.

5. Run Do it! button to generate photo coordinate into photo folder. Repeat all the
step for each Flight. The output file is shown below.
c. Omega Phi Kappa

Omega phi kappa is needed in office processing, actually for Menci processing, although in
photo coordinate that produced from Mission Planner have the content of yaw pitch roll.
Because of the omega phi kappa which was produced from Agisoft Photoscan has been
optimized and besides, when the alignment process, photo coordinate of other photo that not
contain photo coordinate can produced so omega phi kappa data from Agisoft Photoscan is
used in Menci Processing later.

Step by step of omega phi kappa generation using Agisoft Photoscan are described below:

1. Run Agisoft Photoscan and make some chunk as much as flight number of the
project, in fact from field work, after three flight, the data is processed.

2. Add photos in each chunk according to each flight from renamed photo.
3. Import Ground Control that same as photo coordinate from Mission Planner of
each flight or chunk and setting the coordinate system to WGS84 (EPSG:4326).
4. Clearing the unneeded photos that can interrupt the processing such calibration
photo, turn back direction photo, or double photo in overlapping zone of each
flight.

No Overlap
Photo

Flight 15 Flight 14

Unneeded
Photos

5. Merge chunk if all unneeded photos have been removed.

6. Batch process of align photos and optimize alignment of merged chunk. Use
parameter setting of align photos : Accuracy : Low, Pair preselection : Ground
Control, and the other keep in default, for optimize alignment keep all parameters
in default.
7. Export Cameras as omega phi kappa data.
d. Survey Data

Survey data in drone image processing is used for ground control point that make correction for
actual position of the imagery both in horizontal and vertical. Survey data contain ground
control point (x, y, z) and sketch.
2. Managing Block Plan Processing

Block plan processing is created when the whole drone photo and survey data of one project
have been produced. Both of them are needed in block planning because in block plan we have
to measure how much photo of one processing which contain ground control point that
overlapped in each block. For fast processing in Menci, maximum photo of one process is 3000
photos that can be finished the whole process in 2 - 3 days. Block plan is created usually in
horizontal or vertical boundary design. In the example in ArcGIS as shown below, the block
design is vertical.

GCP which located in center of project area that

can be used as boundary for each block

1. Open point coordinate of all photos of each flight and GCP into ArcGIS. From GCP that
spread in the project area, decide block boundary that have no more than 3000 photos for
fast processing but in this example, from total about 9000 photos, there will be two block for
processing, which mean there will be about 4500 photos per process.
 From GCP which are located for boundary of two blocks, select photo coordinate to estimate
how many photo that will be processed. In the example, the first block (left) has 4444 photo
for processing and the second block (right) has 4688 photos for processing.

2. Identify photos of each block from what flight they are. This step is used to pick the OPK
from each flight number in low assessment process that delivered from field. In the
overlapped area, there will be the same flight that will be processed in each block.

Overlapped Area

3. At least, edit the OPK which delivered from field become two new file OPK. Each OPK are
decided from flights in each block processing.
 In the example, the flight which will be processed are flight 38 until 49. Those all flight are in
the two different OPK from the field (OPK flight 36 - 43 & OPK flight 44 - 49) so the two OPK
will be merged first and then split to two file OPK as the flight number from each block
processing.

3. Menci Processing

Menci processing is the main process in the drone aerial ortho-mosaiced production. In this
process, whole data are generated, such as DSM, DTM, and Orthophoto. Before do this process,
some data requirement have to be provided. Those data are renamed photos and OPK in one
folder of one block processing. Step by step of Menci processing are described below.

a. Camera Calibration

First process in Menci Processing is camera calibration. This process is aimed to generate camera
passport file which used in the process. Processing description is shown below.

1. Open New Camera Calibration menu in menci.

2. Pick project folder which will be folder processing for camera calibration then give the
name of project "cam_calibration".

Give the name here

3. Add dataset for new camera passport with RGB type and 90 degree direction.
In setup dataset, add GPS-IMU data which is the OPK data of block processing then define
each column of the data one by one as shown below. Use coordinate format Lat Lon Hgt,
angles format Omega Phi Kappa, and Angles Unit Degree.
4. Do flight plan selection for deciding 4 path with minimum photo is 24 but usually in this
process is used 50 - 60 photos in the homogeneous landscape, bareland is preferable.

Remove unneeded
photos by draw
polygon in propose
removed photos
then right click

Final 4 paths for

camera passport
5. Finish setup dataset then define coordinate system. The default coordinate system is
Universal Transverse Mercator.

6. Define pixel size based on UAV Grid Calculation of camera type then process the camera
calibration.
 In the final process, there will be *.cvcc file which is the camera passport file that can be
copied to other PC for other processing in the same project.

b. Bundle Adjustment

Bundle adjustment is the mosaic processing of all photos in one block process. Processing step is
not different with camera calibration at all. The step is described below.

1. Create new project in main menu.

2. Pick project folder which will be folder processing for block process then give the name of
project based on project name and number of block process, for example
"Project_PAH_Block01".

Give the name here

3. Add dataset and then select camera passport that has been built before for the project with
passport type RGB.

In setup dataset, add GPS-IMU data which is the OPK data of block processing then define
each column of the data one by one as shown below. Use coordinate format Lat Lon Hgt,
angles format Omega Phi Kappa, and Angles Unit Degree.
4. Do flight plan selection and keep all photos then OK.

5. Finish setup dataset and define coordinate system. The default coordinate system is
Universal Transverse Mercator. After that, choose bundle strategy with "Optical Flow
Matching" then run the process.
c. Radiometric Balancing

Radiometric balancing is used for radiometric color balancing of each sensor type in menci, better
used after bundle process. The process is just click the radiometric balancing button in workflow
bar as shown below.
 Before Radiometric Balancing After Radiometric Balancing

The picture above show the different of imagery between before and after radiometric color
balancing.

d. GCP Collection and Correction

GCPs are used to refine the bundle orientation and CPs to introduce a strengthen verify of results.
Their management (import, collimation and use) is the same. Any modify to GCPs should be done
before any next data extraction because their contribution affects the whole project
development. GCPs from survey team are in mapinfo *.tab or esri *.shp file type, so we have to
extract information that needed in menci such GCP's name (Point_No), longitude, latitude, and
GPS height from the database (*.dat or *.dbf) in excel before then save as *.txt (tab delimited)
and give the name as GCP_Project name_Block number.

Menci process for GCP Collection and correction is shown below.

1. Add GCP file in GCPs Manager

Defining each column of the data one by one such as ID, Lon, Lat, and Height then select all GCPs
and import.
2. Collimate point from photos that have the sign of GCP in the field like shown below.

Point Position

Click the GCP

GCP Sign
Correct the point of all image to the true position by clicking the GCP sign. Once you have
click the GCP in one image, the other image will track automatically the same position that
overlapped by the GCP point but they have to check and click again to GCP sign if not in the
true position.

Click to the
GCP Sign for
correction

GCP sign is known by looking to the surveyor sketch of each GCP. After correction, image file
in right position of monoscopic collimation window will change the color from yellow to
green. If there image that not overlapped, the color will still in yellow and if there no GCP sign
in the image, just remove current image from process.
3. After all GCP are collimate, evaluate them.

Have been collimated

After evaluating the GCP, there will be 3 class of correction that define in Legend, green is high
accuracy (0.182 m), yellow is medium accuracy (0.365 m), and red is low accuracy (> 0.365 m).

e. DSM Generation

After collecting GCP and correction, DSM is generated from point cloud which produced from
image overlap based on photogrammetry technique. The step processing is so simple in menci
because just click the button of DSM in workflow bar then define pixel resolution (1m) of DSM
and run the process.
Image below is the result of DSM Generation

f. DTM Generation

DTM is derived from DSM using ground filter or optimization in menci. The step process is similar
to DSM, just click the DTM (Ground) button then choose by filtering or not, commonly use filtering
but when filtering cannot be processed then use no filter. The ground filter parameter have 4 type,
urban, suburban, metropolitan, and country, the suburban usually used when process the project
with threshold 0.1 or 0.3. If any of ground filter cannot be process so that the no filter or
optimization with high type is used.
Image below is the result of DTM Generation using ground filtering.

g. Seamlines Generation and Editing

Seamlines is same as cutline of every photo in the orthomosaic image. It is very important part in
orthophoto generation because when the seamlines have bad geometry in each image, there will
be low quality too in orthophoto, such as broken palm object, broken building, broken road way,
etc. So, after seamlines is generated, there will be seamlines editing for better result. The step
process is so easy because this process is fully automatic without any parameters which just click
the Seamlines button in workflow bar.
Images above show that there are broken object which have to be fixed before do validation. All
image are fixed by expand the line further from the object by click the first starting point in the
photo to expand and right click on mouse for stop editing. The most important object to be
edited first is building because they must not be broken.

After all broken object have been fixed, validate all seamline by clicking Validate All button.

h. Orthophoto Generation

Final step in menci processing is orthophoto generation. This process produce orthophoto with
the resolution based on sensor type (default) or define by user. The step process is shown below.
Defining the orthophoto GSD based on output resolution that defined by user. Usually, target
GSD is checked in Round option that will change the resolution to 0.10 m then run process use
editable seamlines.
4. Orthophoto Mosaic

Project with so many photos cause some block processing in menci and create some
block orthophotos. For building the whole area of project needs mosaic processing. Before do the
mosaic process, all of orthophotos are color balanced using ERDAS image equalizer then the
result from image equalizer are processed in ERDAS mosaic pro for mosaicing. The workflow
below describe whole process for orthophoto mosaic.

ORTHOPHOTOS FROM MENCI

Block 1 IMAGE EQUALIZER
convert to .img
Block 2 1. Build statistic table
Block 3, ... 2. Equalize

MOSAIC PRO
1. Seamline Generation
2. Set overlap function
3. Run Mosaic

5. DEM Generation

Digital Elevation Model (DEM) or terrain has been built in menci processing. Because of the project
split in some processing, DTM mosaic result from each DTM from menci is not have good overlap
height value so that in overlapping zone will be broken like a lineament. In other condition, DTM
result is also illogical from the real terrain in the field that have to be processed more in editing.

Some processing in DEM generation are shown below.

a. DSM Mosaic

DSM mosaic is processed for generating DTM of whole project area. Mosaic will be processed in
PCI OrthoEngine with semi automatic method because first the mosaic is processed in automatic
then the result will be checked in manual mosaic for editing cutline. The workflow of processing is
described below.
DSM FROM MENCI
Block 1
PCI ORTHOENGINE
Block 2
Block 3, ...

1. Create New Project (Mosaic Only) 2. Define Projection System & Resolution

3. Add All of DSM in The Project

4. Define Output

6. Edit Cutline in Manual Mosaic

5. Run Automatic Mosaic

b. Ground Filtering

Ground filter is used to convert DSM to DTM. From DSM which build before in mosaic process,
DTM is generated with ground filter method in TerrainTools that same with filter methods when
DTMs was built in menci. The step processing are shown below.

1. Create new project and add DSM that has been mosaiced before.

2. Do ground filtering to DSM for creating DTM data. In ground filtering, choose method that
match with ground type of the area, for example in plantation area use suburban with
threshold value 0.3.

3. Save the project then the DSM data will update automatically. Because of TerrainTools
process the existing file without produce new file, backup of file that will be processed is a
must.
c. DEM Editing

Sometimes, DTM which has been generated from any software automatically is still wrong or
illogical, for example there still DSM feature effect like tree and building. For cleaning DSM effect,
DEM editing is processed. Some software are used for DEM editing, such as SAGA GIS, PCI
Geomatics (Focus), TerrainTools, and ENVI.

SAGA used for creating DTM data because DTM data from TerrainTools was reduced resolution to
2.5m of result as default, whereas the DTM from SAGA keep the resolution same with DSM
resolution. DSM data that input in SAGA is LAS data (point cloud) which can be extracted from
Menci. After that, DTM from SAGA edited in Focus DEM Editing Layer (PCI). Before DTM in Focus
was edited, better to create polygon layer on broken DTM region to make easy the process.
TerrainTools used to patch some feature that still not good result or close to real terrain and in
the end ENVI used to smooth the whole area until no broken DTM feature. The whole processing
workflow are described below.

convert to DTM
DSM (LAS) SAGA GIS
1. Import LAS
2. Point Cloud to Grid
3. DTM Filter (Slope-based)
4. Multilevel B-Spline Interpolation
5. Multi Direction Lee Filter
6. Convert Grid to GeoTIFF

Still have broken feature between edited

TERRAINTOOLS (DEM EDITING) area and non edited area PCI FOCUS (DEM EDITING)

ENVI (SMOOTHING)