rt_idl Spectral Line Data Reduction at Green Bank
The Interactive Data Language is a scientific data environment for data analysis and display. Here we describe IDL procedures for calibration, averaging and display of GBT spectrometer data, that provides an introduction to the capabilities being developed in IDL.This web page is intended for observers at the NRAO Green Bank site, running IDL in the Linux computing environment. The following steps will present GBT data taken towards Orion A at 43 to 46 GHz. It is expected that the observer will modify their data configuration files and write their own IDL procedures for special processing of their observations.
Supported computers
rt_idl was written before the advent of 64 bit CPUs, and only 32 bit CPUs can run the programs which extract observations directly from the GBT Archive. The computers known to run rt_idl successfully are- titania
- ariel
- planck
- euler
- Any other 32 bit cpu
Demonstration
A good package produces quick results. This demo shows how public domain GBT observations may be averaged and displayed in IDL (in a mater of seconds). Any discoveries you make with this data are yours!Within the GB computing environment type the following:
| Step | Type | Note |
| 1 | mkdir ~/idl | Create directory for your IDL work, later many files will be created |
| 2 | cd ~/idl | Go to the new directory |
| 3 | /opt/local/bin/rt_idl | Start up the IDL development environment, windows will appear |
| The latest GBT Spectra will appear! | ||
| 4 | demoOrion | At the IDL prompt, type demoOrion, to start the demo |
| 5 | -ENTER- | Hit Enter at the IDL prompt to step though the data |
Step Details
Steps 1, 2 and 3
The rt_idl procedures assume the observer wants a command line interface to the data. Usually, the observer will perform calibration steps on a small test observation, then the observer will write IDL procedures to perform repetitive calibration steps.The IDL package written by Tom Bania provides the basic functionality of common packages (ie GET,# ; SHOW). This functionality is implemented in IDL procedures and the first steps define the location of the version of these procedures the observer will use. A unix script will search for the latests spectra in the default GBT output data directory.
Step 4
The demo data from an Orion observation is organized as four bands 800 MHz bandwidth, dual polarization spectra. The demo script demoOrion.pro averages the data, then calibrates using Tsys*(Sig-Ref)/Ref for each polarization. An IDL script demoOrionSum.pro performs the summation for a single 800 MHz band. The LCP(white) and RCP (red) polarizations are shown after median filtering the spectral band pass by 20 channels, to remove the continuum contribution and residual band pass ripples. The plot output looks like:
Step 5
The observer is prompted to hit -ENTER- to step through the 4 spectral bands. The development environment windows show messages concerning the data reduction, to allow the observer to assess the performance.Notes on the rt_idl
The rt_idl environment expects three sub-directories:- data - Directory containing the observer pre-selected data
- figs - Directory output postscript figures (from PRINTON and PRINTOFF)
- saves - Directory containing saved versions of the IDL environment.
Exploring Archive Data
The standard GBT reduction system is GB_IDL, which has essentially all the functionality of rt_idl except one important feature, the ability to directly examine all data in the GBT archive. This rt_idl capability is implemented via a C program that is called directly from IDL to import GBT archive data into the internal IDL format.
The observer should think of the GBT archive as a very large data
base, and the observer sets keys
which access the database. At the rt_idl
command prompt the database keys can be listed:
!gbtDirectory : "/home/gbtdata"
!gbtProject : "TRFI_082109_L1"
!gbtFile : "LATEST"
!gbtISampler : 0 (Range 0 to 1)
!gbtStates : " " (Range 0 to 1)
!gbtBank : "A" (Range A to D)
!gbtIntegrations: "" (Range 0 to 11)
!gbtCalibrate : " "
selectspec
- which finds the data and imports the data into the internal format.
For a quick complete processing of position switched data, using the
observer specified signal and reference scan numbers use the
sigRefRef command.
The GBT observations are organized in scans, and "sdfits"
puts the scan data in FITS records
The rt_idl environment works with data records (rec below)
that are loaded into one of 10 special buffers. In addition, the
special buffers may be copied to a storage area NSAVE for
later use. The special 10 buffers have defined functions:
Each IDL procedure is contained in a file in directory
/home/astro-util/idl/gb_macros.
A list of commands is provided below:
One primary development direction for rt_idl is implimentation of
a data browser, for selecting of data to be reduced in IDL.
Keep tuned to this web page for new developments!
rt_idl Command Summary
All of the existing IDL functionality is maintained in the rt_idl
environment, and special functions are implemented to perform
standard astronomical spectral line calibrations.
Command Function
accum Add buffer 0 to accumulation
addem,rec_array Add array of records to Stack
add,rec Add record to Stack
ave Compute weighte averages of accumulation
avgstk Accumulate and average stack records
bbb,nfit Fit Polynomial Baseline of order nfit
bb,nfit Fit Polynomial Baseline of order nfit
bmark Mark baseline region on the plot
boxcar,nch Box car smooth by nch
b Fit polynomial baseline, used previous nfit,regions
bsearch,nmax Baseline fit up to order nmax
cat List the contents of the stack
ccc,xpos,ypos Return xpos,ypos cursor positions
chan Flag plotting channels on X axis
clearset Initials Select Variables to default
clrstk Empty the record stack
comment,msg Write 44 character messag to history
copy,bin,bout Copy buffer bin to buffer bout
curoff Turn off cursor input for scaling
curon Turn on cursor input for scaling
curx Set Plot X region using cursor (forces Cursor on)
curxy Set Plot X-Y region using cursor (forces Cursor on)
cury Set Plot Y region using cursor (forces Cursor on)
dcoff Flag NO subtract mean baseline for display
dcon Flag automatic subtrack mean baseline
dcsub Subtract mean baseline from buffer 0
demoOrion Steps Observer through a demo of Orion A data
demoHe3 Steps Observer through a demo of Helioum 3 data
display,array Display 2-d array
disp,n,m Show buffer n and overplot buffer m
div,n,m Divid buffer n by m, output to buffer 0
fetch,recon Get recon, compute recoff andsubtract
files Print files currently used by rt_idl
fixx,xmin,xmax Set Plot X range with input values
fixxy,xmin,xmax,ymin,ymax Set Plot X-Y ranges with input values
fixy,ymin,ymax Set Plot Y range with input values
flagoff Turn Flag off for plot display option
flagon Turn Flag on for plot display option
flag,ch Set channel for vertical line at x-ch
flg_id,ch,lab,clr Draw vertical flag at x=ch, label=lab, clr=color
freex Flag full X range on next SHOW
freexy Flag full X and Y ranges on next SHOW
freey Flag full Y range on next SHOW
freq Set x axis labeling to frequency
gauss Prompt observer for gaussian fitting parameters
gbtrestore Restore all rt_idl values from previous gbtsave
gbtsave Save all rt_idl values from a session
gbt_win Create plot window taylored to GBT Data reduction
gbt_xwin Create IDL window for data reduction
getns,ns Restore save region ns
getoff,rec Copy rec into buffers 0 and 6
geton,rec Copy rec into buffers 0 and 5
get,rec Copy rec into buffer 0
gg,ngauss Fit ngauus Gaussians to the data
g Tuned gaussian fitting program
hdr,rec Prints data header for rec
hline,val Draw horizontal line at y=val
info Print information about data in buffer 0
keys Display the current GBT realtime data selection Keywords
killwin Kill the rt_idl Plot window
list,start,stop Print summary of records loaded int rt_idl
look Fetch data from records on stack, plot with DC offset subtracted
make_gbt Create and Fill the gbt data structure with records
make_nsave Create NSAVE file for storing the GBT data structures
mask,dsig,index Display regions
mediansub,halfWidth Subtrack median of halfWidth wide from buffer 0
minus,m,n Subtract buffer m from n and put in buffer 0
mk Convert Y axis scale from Kelvins to milli-Kelvins
molecule Show all known molecular lines on current plot
mult,m,n Multiply buffer m by and put in buffer 0
nroff Turn off Flag for baseline fit marks
nron Turn on Flag for baseline fit marks
nrset,n Set n baseline regions with values nreg=[.,.]
nslog Summarize NSAVE data records
nsoff Turn NSAVE write protection off
nson Turn NSAVE write protection on
ortho_fit,xx,yy,nfit,cfit,rms Orthogonal polynomial fit
plthdr Anotate the plot with header values
plus,n,m Add buffer n and m, put output in buffer 0
printoff,prt Turn off postscript command logging, send to printer prt
printon,fname Write postscript print commands to file fname
procs Print list of all compiled procedures
psplot,fname Prints postscript file fname to default printer
ps,sig_rec,ref_rec Power Spectrum TSys*(Sig-Rec)/Rec
putns,ns Save spectrum in buffer 0 to NSAVE area ns
realTimeSetup Initiates real-time data display
rec_info,rec Prints summary of GBT data rec
recombAll Show H, He and C recombination lines on current plot
recombC Show C recombination lines on current plot
recombH Show H recombination lines on current plot
recombHe Show Helium recombination lines on current plot
recombN Show Nitrogen recombination lines on current plot
recombO Show Oxygen recombination lines on current plot
refresh,n Re-plot previous n plots, useful with printon/off
reshow overplots the contents of buffer 0 on plot
rms_mask Calculates the RMS of data in baseline REGIONS
rms,sigma,xmin,xmax Calculates RMS sigma in between xmin,xmax
sampler,iSampler GBT Scan sampler (polarization/frequency) band to RT select
scale,fact Scale buffer 0 by fact
scan,number GBT Scan number in current project to select
sd_hdr,rec Print single dish record header for rec
sd_to_gbt,in,out Convert single dish record to gbt_data record
select Load record number stack with record numbers matching request
set Prompt for scan selection critera
setBandWidth Set maximum bandwidth for all Banks/Samplers
setline Prompt observer for line ID for SELECT search
setplot,device Set plot device/file name
setrec,recmin,recmax Set record range for a SELECT search
setscan,scan Set scan Number for SELECT search
setsource,src Set soruce name for a SELECT search
setsrc,src Set source name for a SELECT search
settype Prompt for scan type of a SELECT search
settyp,typ Sets scan type for SELECT search on data
setx,xmin,xmax Set X min and max for plot
setxy,xmin,ymin,xmax,ymax Set X, Y plot ranges
sety,ymin,ymax Set Y min and max for plot
show Show buffer 0 contents; result of last calculation
showAllInts Show all integrations for the current scan
showLatest Show latest spectra for most recent scan
smooth Gaussian smooth by fixed width
smo,n Smooth buffer 0 with FWHM=n channels
srcvloff Turn off display of source velocity line
srcvlon Flag display of source velocity line
srcvl,ch Draw vertical line at x=ch on plot
state,iStates GBT Scan state numbers to select
syms,no,scale,ifl Create IDL symbol on the plot
tagid Write 32 character line ID to buffer 0
tagpol Write 4 character polarization ID to buffer 0
tagtype Write 32 character anotation string to buffer 0
tellstk Print list of stack recond numbers
track,scans Reduce Track observations of a scanlist
unmk Convert MilliKelvin Y scale to Kevlin scal
velo Set flag for X axis velocity plots
xroll Set X-range to inner 90 % of the data
zline Draw line at Y=0 on the plot
zloff Turn off the zero line
zlon Turn on the zero line Future Developments
This version of rt_idl has minimum capabilites for observers to
test IDLs capabilites for data reduction. Future versions will impliment
more calibration and editing functions.