1

An Introduction to IDL
(The Interactive Data Language)
and IDL in Astronomy
Astronomy Seminar Series 11/12/2005
1. IDL Features 2.
Basics of IDL
3. 1D & 2D Display 4.
FITS I/O in IDL
5. Image Processing
cqras@126.com
2
I.IDL Features
http://www.astro.virginia.edu/class/oconnell/astr511/IDLguide.html
3
IDL vs. Mathematica, Matlab,
Maple
http://amath.colorado.edu/computing/mmm/brief.html
4
IDL vs. Traditional Astronomical
Software
http://www.astro.virginia.edu/class/oconnell/astr511/IDLguide.html
5
IDL in Astronomy
http://idlastro.gsfc.nasa.gov/other_url.html
6
Very Good Places and
Packages
IDL Astronomy User's Library
http://idlastro.gsfc.nasa.gov/homepage.html
http://idlastro.gsfc.nasa.gov/ftp/astron.tar.gz
FITS http://idlastro.gsfc.nasa.gov/fitsio.html
Solar Software
http://lmsal.com/solarsoft/sswdoc/index_menu.html
ftp://sohoftp.nascom.nasa.gov/solarsoft/offline/swmaint/tar/
ssw_ssw_gen.tar.Z
Coyotes Guide to IDL Programming
http://www.dfanning.com
ftp://ftp.dfanning.com/pub/dfanning/outgoing/coyote2nd/
IDL Newsgroup (comp.lang.idl-pvwave)
http://groups.google.com/group/comp.lang.idl-pvwave
Markwardt IDL Library (Fitting)
http://cow.physics.wisc.edu/~craigm/idl/idl.html
JHUAPL IDL Library
http://fermi.jhuapl.edu/s1r/idl/s1rlib/local_idl.html
IDL + EMACS http://www.idlwave.org/
http://idlwave.org/download/idlwave-help.tar.bz2
I mportant: Practice, Take notes,
Google idl + keywords
I DL Astro Package
astron.dir.tar.gz
Coyote Program Library
coyote2ndfiles.zip
SSW General Package
ssw_ssw_gen.tar.Z

tar xvfz ssw_ssw_gen.tar.Z
mv gen /usr/local/rsi/idl/lib/sswgen
Others in a similar way; or see Page 8
7
Working with
IDL
Menu
Toolbars
Edit Window
Output Log
Variable Watch
Command Input
Status Bar
compile run
My favorite
Linux/Unix Console
Demo
8
Personal SETUP for IDL
(recommended)
Personal IDL_STARTUP
1.edit ~/.cshrc :
setenv I DL_STARTUP ~/.idl_startup.pro
2.edit ~/.idl_startup.pro (or other file names),
Device, retain =2, decomposed =0
!path = !path + :~/idlpros1:~/idlpros2

#Windows: File Preferences Startup...
you can select any file that is similar to that under Linux.
Flashing Colors in Linux/Unix
#edit ~/.Xresources
idl.gr_visual: TrueColor
idl.gr_depth: 24
idl.retain: 2
idl.colors: -1
Memory Limits in Hubble (Alpha)
# edit ~/.cshrc
limit stacksize unlimited
limit datasize unlimited
Working Directory
IDL> cd, myworkdir
Windows: File Preferences Startup Startup
9
II. Basics of IDL
IDL> PRINT, 3 * 5, [30,5,50]
15 30 5 50

IDL> x = 'Hello! IDL World' & HELP, x
X STRING = 'Hello! IDL World'
IDL> x = indgen(15) & y = sin(2*!dpi*x/15)
X INT = Array[10]
Y DOUBLE = Array[10]

IDL> FOR i = 0, 15-1 DO PRINT, i, x[i], y[i]

IDL> DEVICE, decomposed = 0
IDL> plot, loaddata(1), psym=-4, $
title='plot', xtitle='Month', ytit='Sth ;,$
;/ylog, yrange=[5e-1,40], ystyle=1
Dynamic Datatype
Array Zero-Ordered
Array Operation
Direct Graphics
Parameters
10
Some Symbols, Definitions and
Others
? online help
.run, .compile, .r
; & $ !

http://fermi.jhuapl.edu/s1r/idl/idl_syntx.html
11
IDL Variables: Dynamic
Scalar, Array (18D)
Structure: collection of scalars, arrays, or other structures
System Variables (!)
!dpi (3.1415926)
!p: Display. e.g., !p.font, !p.color
!d: Device. e.g. !d.name

HELP, variable or HELP, variable, /struct
Data type: dynamic
12
Type

Len

Creation

Array

Conversion

Byte

1

A=5B

Bytarr

Byte

Integer

2

B=0;b=0S

Intarr

Fix

Uint

2

C=0U Uintarr

unit

Long

4

D=0L Lonarr

Long

Ulong

4

E=0UL Ulonarr

Ulong

Long64

8

F=0LL Long64arr

Long64

Ulong64

8

G=0ULL Ulon64arr

Ulong64

Float

4

H=0.0

Fltarr

Float

Double

8

I=0.0D

Dblarr

Double

Complex

8

J=complex(1.0,0.0)

Complexarr

Complex

Dcomplex

16

K=dcomplex(1.0,0.0)

Dcomplexarr

Dcomplex

String

?

L=hello Strarr

String

Pointer

4

M=ptr_new() Ptrarr

---

Object

4

N=obj_new()

Objarr

---

IDL Variables: Basic
Datatypes
Indexed Array Creator: e.g., findgen() float index generator
Table c.f. intenet
13
Control Statements
IF: conditional
if exp then statem
if exp then statem1 else statem2
For Loops: for i = init, limit, step do statem
While Loops: while exp do statem
Repeat Loops: repeat statem until exp
Case:
GOTO: goto, label
Blocks:
Begin
statem1

statemx
Endxxx
if x lt 0 then begin print,x & a=2 & endif
e.g.
http://fermi.jhuapl.edu/s1r/idl/idl_syntx.html
if x lt 0 then begin
print,x
a=2
endif
for i=0, 10 do begin
readf, lun, txt
print,txt
endfor
14
Programs: Procedures &
Functions
Batch files: one or more IDL statements or commands. $
@batchfile: interpreted one by one, exactly as if it was from
the keyboard.
Main-level Programs: a series of program statements that are compiled
and executed once an END statement is encountered.
Programs: pro name, param1, param2, ... paramx
ended with END
Functions: function name, param1, param2, ... Paramx
ended with END

Variable Access: Batch and Main (globe), Program & function (local)
IDL: an interactive tool, also a powerful programming language.
15
Parameters Passing
Actual (caller) and Formal (called) Parameters
Correspondence by position or keyword
Keyword Inheritance (_Extra)

Passing Mechanism
Expressions, constants, system variables, and subscripted
variable references are passed by value.
Variables are passed by reference.

Parameters and Keywords Checking
n_params()number of parameters in calling an procedure/ function
n_elements()returns zero for undefined variable.
keyword_set()check a Boolean keyword parameter.
arg_present()defined and reference passing?
16
Working with Arrays! (Essential for
IDL)
A(n,m): Array with
n columns and m rows
A[n,m]: Element
pos = n*i+j
j = pos MOD n
i = (pos - j)/n
A[i
1
, j:*]
17
Array-Oriented Operation
Arrays work the same as Scalars.
e.g. 2*A, A + B, A-B, A/B, SQURT(A),
Try to avoid use of loops (slow!)
Array Creation:
xxxArr, xIndGen, Replicate,

Array Manipulation:
[], ARRAY_INDICES, CONGRID, HISTOGRAM, INVERT,
MAX, MIN, MEDIAN, N_ELEMENTS, REBIN, REFORM,
REVERSE, ROT, ROTATE, SHIFT, SIZE, SORT, TOTAL,
MEAN, TRANSPOSE, UNIQ, WHERE, etc.
18
WHERE much faster than IF
Set all values between 5 to 8 equal = 15.
Loop Way:
For j=0,2 Do Begin
For k=0,3 Do Begin
IF (array(j,k) GE 5) AND (array(j,k) LE 8) THEN array(j,k) = 15
EndFor
EndFor
IDL Way:
index = Where((array GE 5) AND (array LE 8), count)
IF count GT 0 THEN array[index] = 15
http://www.dfanning.com/powerpoint/index.html
19
Where(): where are
they?
file= FILEPATH('galaxy.dat', subdir = ['examples', 'data'])
imagesize = [256, 256]
image = READ_BINARY(file, data_dims =imagesize)
DEVICE, decomposed = 0 & LOADCT, 4
WINDOW, 0, xsize = imagesize[0], ys= 2 * imagesize[1]

indices = Where((image GE 200) AND $
(image LE 230), count)

IF count GT 0 THEN BEGIN
result = Array_Indices(image, indices)
col = Reform(result [0,*])
row = Reform(result [1,*])

TV, image, 0 & TV, image, 1
PlotS, col, row, /Device, $
Color=FSC_Color(white'), psym=1
ENDIF
20
Mask using Where()
file = FILEPATH('worldelv.dat', $
subdir = ['examples', 'data'])
file = FILEPATH('worldtmp.png', $
subdir = ['examples', 'demo', 'demodata'])

TV, elvImage, 0 & TV, tmpImage, 1
ocean = WHERE(elvImage LT 125)
image = tmpImage
image[ocean] = elvImage[ocean]
TV, image, 2

land = WHERE(elvImage GE 125)
image = tmpImage
image[land] = elvImage[land]
TV, image, 3
Temperature Distribution in Land and Ocean
21
img1 = BytScl(Loaddata(4), Top=99)
img2 = BytScl(Loaddata(5), Top=99)+100B
Window, XSize=256, YSize=256*3
LoadCT, 13, NColors=100 & TV, img1, 2
LoadCT, 3, NColors=100, Bottom=100 & TV, img2, 1

LoadCT, 13, NColors=100
LoadCT, 3, NColors=100, Bottom=100
index = $
Where((Indgen(256L*256L) MOD 2) EQ 0)
img1[index] = img2[index]
TV, img1, 0
Simultaneous look at two images.
Index Manipulation
http://www.dfanning.com/powerpoint/index.html
22
Examples: the IDL Way
n = 5
i = REBIN(LINDGEN(n), n, n)
j = REBIN(TRANSPOSE(LINDGEN(n)), n, n)
Print, i Print, j
0 1 2 3 4 0 0 0 0 0
0 1 2 3 4 1 1 1 1 1
0 1 2 3 4 2 2 2 2 2
0 1 2 3 4 3 3 3 3 3
0 1 2 3 4 4 4 4 4 4
mask = (i GE j)
Print, mask
1 1 1 1 1
0 1 1 1 1
0 0 1 1 1
0 0 0 1 1
0 0 0 0 1
e.g., Upper Triangular Matrix
http://www.dfanning.com/idl_way/
23
Direct Graphics
Device oriented; display in a specific device
Set_plot:
X/WIN/MAC, PS, PRINTER, METAFILE, Z, CGM, PCL,
NULL
Device: retain, decomposed,
set_character_size, pseudo_color, index_color, true_color
Window Coordinates: Data, Device, Nomal
24
PS files Creation
curName=!D.name ; X or Win
set_plot,'ps'
device, file=test.ps, posi=[1,1,9.5,9]/10, bits_per_pixel=8
;device, file=test.eps, posi=[1,1,9.5,9]/10 ,/encapsulated
;device, file=test.ps, posi=[1,1,9.5,9]/10,/color
plot, loaddata(1)
device,/close & set_plot,curName
For X/MS windows devices,
write_jpeg, test.jpg', tvrd()
write_jpeg, test.jpg', tvrd(true=1),true=1
PS Layout configurations:
http://www.dfanning/documents/programs.html
http://cow.physics.wisc.edu/~craigm/idl/printing.html
25
Working with Colors
(R,G,B) triple: any Color decomposed to Red,
Green, and Blue components, each with value 0~255

Indexed Color Model and RGB Color Model
Indexed Color Model (also 8 bit color):
Color Lookup Table Color Palette
2^8 = 256 colors
Device, decomposed =0
RGB Color Model (also 24 bit color) :
specify color values explicitly, using an RGB triple
2^8 * 2^8 * 2^8 = 16777216 colors
Device, decomposed =1
Indexed?
Dynamic
HELP, /DEVICE
LOADCT, TVLCT, XLOADCT, XPALETTE
About COLORBAR, FSC_COLOR, etc.
http://www.dfanning/documents/programs.html
26
http://www.dfanning.com/powerpoint/index.html
Comparison: 24-Bit and 8-Bit Color
27
Color Decomposition
Data=loaddata(1)

Plot, data, Color=255, posi=[0,0,1,1]
Help, /Device

Device, Decomposed=1 ; ON
Plot, data, Color=11829830L,$
Background='00ff00'xL, posi=[0,0,1,1]

TVLCT, 70, 130, 180, 240
Device, Decomposed=0; OFF
Plot, data, Color=240,$
Background=255,posi=[0,0,1,1]
28
RGB Image: Ant Nebula
read_jpeg,'Ant.jpg',ant
r=reform(ant[0,*,*])
g=reform(ant[1,*,*])
b=reform(ant[2,*,*])
Help, ant
window, 0, xsize=2*info[2], ysize=2*info[3]
tv, ant, true=1,0
tvscl,r,channel=1 & tvscl,g,channel=2 & tvscl,b,channel=3
tvscl,r,channel=1,1 & tvscl,g,channel=2,2 & tvscl,b,channel=3,3
Google Images:
Ant Nebula
device,decom=1
29
Image Types
Byte: 0~255; otherwise, BYTSCL()
Binary Image: values only with 0 or 1
Gray-scale Image: B-W LUT
Indexed Images: 0~255, LUT
RGB Images: (R,G,B), each 0~255
COLOR_QUAN() : RGB Indexed Image
FILEPATH, QUERY_IMAGE, READ_BINARY, READ_IMAGE
READ_JPEG/WRITE_JPEG, WRITE_IMAGE,
30
III. 1D and 2D Data Display
plot/oplot, plots, axis, xyouts, ploterr/oploterr/errplot/,
ploterror(!!!), vel/velovect/plot_field/flow3

BTLSCL, REBIN, REFORM
tv/tvscl/bytscl, imdisp(!!!), plot_image, tvimage, contour,
image_cont, surface, surf_shade, show3, median,
smooth/convol, reberts/sobel, defroi, profiles, etc.
Histogram, plot_hist

box_cursor, plot_box, rdpix, curval

live_tools(live_plot,...), itools(iplot,icontour,...)
iTools (iplot, )

!p.multi, position, xrange, xstyle, psym,
31
User PSYM
device, decomposed=0
!p.multi=[0,1,2]
data=loaddata(1)
Plot, data, PSym=-2

;filled circle
phi = Findgen(32) * (!PI * 2 / 32.)
phi = [ phi, phi[0] ]
UserSym, Cos(phi), Sin(phi), /Fill

Plot, data, /NoData
;tvlct, 178,34,34,10 & OPlot, data, Color=10
OPlot, data, Color=FSC_Color('firebrick')
Oplot, data, color=FSC_Color('forest green'), $
PSym=8, Symsize=1.5
32
Plotting Error
Bars
xtime = indgen(101) & data = loaddata(1)
xerr = randomN(seed,101)*2
yerr = randomN(seed,101)*4
device,decomposed=0 & loadct, 0
tvlct, r, g, b, /get & tvlct, 255-[[r],[g],[b]] & tvlct, 0,255,0,100
ploterror, xtime, data, xerr, yerr, psym = -2, xstyle=1, $
xtitle='!7b(!6cm!U-2!N !6s!D-1!N)', ytitle='!6H!7a'
oplot,xtime,data,color=100,thick=2
ERRPLOT, X, Y-yerr1, Y+yerr2
33
TV and TVSCL
file = FILEPATH('hurric.dat', subdir = ['examples', 'data'])
hurric = READ_BINARY(file, DATA_DIMS = [440, 340])
34
Pseudo-Color Images in PS
file = FILEPATH('worldelv.dat', subdir = ['examples', 'data'])
image = READ_BINARY(file, DATA_DIMS = [360, 360])
IMDISP better than TV/TVSCL
Also try PLOT_IMAGE
curName=!d.name & set_plot,'PS'
device,/color,file='elev.eps',bits_per_pixel=8,/encap
loadct,13 & imdisp, image ; plot_image,image
device,/close & set_plot,curName
35
True-Color Images in PS
file=filepath('rose.jpg',subdir='examples/data')
read_jpeg,file,rose & tvlct,r,g,b,/get

curName=!d.name & set_plot,'PS
device,/color,file='rose.eps',bits_per=8,/encap
loadct,0 & imdisp,rose
device,/close & set_plot,curName
tvlct,r,g,b
Color table always active
in 24 bits mode, ps device
Google Images: Rosette Nebula
36
VI. FITS I/O in IDL
FITS (Flexible Image Transport System) is a standardized data format
which is widely used in astronomy.

Briefly, a FITS file consists of a sequence of one or more Header and Data
Units (HDUs). A header is composed of ASCII card images that in IDL is
usually read into a string array variable. The header describes the content of
the associated data unit, which might be a spectrum (IDL vector), an image
(IDL array), or tabular data in ASCII or binary format (often read as an IDL
structure). Image and vector data can be present in any HDU, but tabular
data cannot appear in the first HDU. The HDUs following the first (or
primary) HDU are also known as extensions, and thus a FITS file containing
tabular data must contain at least one extension.
The FITS Support Office/NASA http://fits.gsfc.nasa.gov/
Four Classes of Procedures:
MRDFITS()/MWRFITS:
READFITS()/WRITEFITS
FX* Procedures
FITS_* and FTAB_* Procedures
FITS I/O in IDLAstro
http://idlastro.gsfc.nasa.gov/fitsio.html
37
FITS I/O: File Information
IDL> file = WFPC2u5780205r_c0fx.fits
IDL> Fits_Info,
WFPC2u5780205r_c0fx.fits
WFPC2u5780205r_c0fx.fits has 1 extensions
Primary header: 263 records
Image -- Real*4 array ( 200 200 4 )
Extension 1 -- u5780205r_cvt.c0h.tab
Header : 354 records
ASCII Table ( 796 4 )
IDL> file=EUVEngc4151imgx.fits
IDL> fits_help, file
XTENSION EXTNAME EXTVER EXTLEVEL BITPIX GCOUNT PCOUNT NAXIS NAXIS*
0 8 0 0 0
1 IMAGE ds 16 1 0 2 512 x 512
2 IMAGE sw_night 16 1 0 2 2048 x 300
3 IMAGE mw 16 1 0 2 2048 x 300
4 IMAGE lw 16 1 0 2 2048 x 300
5 BINTABLE ds_limits 8 1 0 2 16 x 3
6 BINTABLE sw_night_limits 8 1 0 2 20 x 2
7 BINTABLE mw_limits 8 1 0 2 20 x 2
8 BINTABLE lw_limits 8 1 0 2 20 x 2
IDL> images=mrdfits(file,0,head0)
MRDFITS: Image array (200,200,4) Type=Real*4
IDL> help,images,head0
IMAGES FLOAT = Array[200, 200, 4]
HEAD0 STRING = Array[263]
IDL> table=mrdfits(file,0,head1)
MRDFITS: Image array (200,200,4) Type=Real*4
IDL> help,table,head1
TABLE FLOAT = Array[200, 200, 4]
HEAD1 STRING = Array[263]
http://fits.gsfc.nasa.gov/fits_samples.html
38
Fits Example: Orion Nebula
data=readfits('masterf673.fits.gz',head)
loadct,10 & tvlct,r,g,b,/get
r[0]=34 & g[0]=139 & b[0]=34 & tvlct,r,g,b
plot_image,alog10(data>5e-1)
loadct,10
plot_image,alog10(data[1000:2000,1200:2200]>0.5)
http://casa.colorado.edu/~bally/
HST/HST/master/
39
Solar Map Software
An IDL map is a structure that contains two-dimensional (2-d) image data
with accompanying pixel coordinate and spatial scale information. The latter
parameters are defined as properties of the map and are unique for each image
source. Defined in this manner, an arbitrary image can be manipulated or
transformed in a manner that is independent of the image source.
http://hesperia.gsfc.nasa.gov/~ptg/trace-align/
http://orpheus.nascom.nasa.gov/~zarro/idl/maps.html
40
V. Image Processing
A Short Introduction to Digital Image Processing
http://web.uct.ac.za/depts/physics/laser/hanbury/intro_ip.html
41
BYTSCL (Byte Scale)
READ_DICOM(FILEPATH('mr_brain.dcm', subdir = ['examples', 'data']))
BYTSCL mr_brain
42
Dragon in EIT 304
01/23/2001
19:19:43
http://umbra.nascom.nasa.gov/eit/eit-catalog.html
43
Color Table Highlights,
Contrast
file = FILEPATH('mineral.png', subdir = ['examples', 'data'])
image = READ_PNG(file, r, g, b)
colorLevel = [[0, 0, 0], [255, 0, 0], [255, 255, 0], [0, 255, 0], $
[0, 255, 255], [0, 0, 255], [255, 0, 255], [255, 255, 255]]
numberLevel = CEIL(!D.TABLE_SIZE/8.)
level = INDGEN(!D.TABLE_SIZE)/numberLevel
newR = colorLevel(0, level) & newR[!D.TABLE_SIZE - 1] = 255
newG = colorLevel(1, level) & newG[!D.TABLE_SIZE - 1] = 255
newB = colorLevel(2, level) & newB[!D.TABLE_SIZE - 1] = 255
TVLCT, newR, newG, newB TVLCT, 13
TVLCT, r,g,b
44
Histogram Equalization
Left: BYTSCL(HISTOGRAM(image))
Right: Mineral
also, H_EQ_CT, H_EQ_I NT
Left:
BYTSCL(HISTOGRAM(hq_image))
Right:
hq_image = HI ST_EQUAL(image)
ADAPT_HI ST_EQUAL
Left:
HISTOGRAM(equalizedImage)
Right:
ADAPT_HIST_EQUAL(image)
45
Adjust Histogram
http://web.uct.ac.za/depts/physics/laser/hanbury/intro_ip.html
46
Fast Fourier Transform de-noise
file= FILEPATH('abnorm.dat', subdir = ['examples', 'data'])
powerSpec = ALOG10(SHIFT(FFT(image), zz[0]/2, zz[0]/2))
CONGRID(powerSpec, zz2[0], zz2[1])
mask = FLOAT(scaledSpec) GT 2.6
maskedSpec = scaledSpec * mask + MIN(powerSpec)

inverseTran = ABS(FFT(SHIFT(10.^(maskedSpec), $
zz[0]/2,zz[1]/2), /inverse))
scaledSpec =
powerSpec - Min(powerSpec)
47
FFT: removing corrugated effect
http://web.uct.ac.za/depts/physics/laser/hanbury/intro_ip.html
48
Inverse Laplace Trans
mask = HANNING(imagesize[0], imagesize[1])
maskedSpec = (scaledSpec * mask) + MIN(powerSpec)

inverseTrans = ABS(FFT(SHIFT(10.^(maskedSpec), $
imagesize[0]/2, imagesize[1]/2), /inverse))
maskedSpec inverseTrans
49
Median Smoothing
file= FILEPATH('rbcells.jpg', subdir = ['examples', 'data'])
MEDIAN(rbcells, 5) rbcells
50
Rim Enhancing
croppedSize = [96, 96]
file= FILEPATH('nyny.dat', subdir = ['examples', 'data'])
croppedImage = image[200 : croppedSize[0] - 1 + 200, 180 : croppedSize[0] - 1 + 180]
ROBERTS
SOBEL
croppedImage