Frequently Asked Questions about the
Interactive Data Language (IDL).
Additions to previous version:
Changes from previous version:
Added Mike Schienle's site to the following sections.
Please note the new (yet again) IDL FAQ Location. If you provide a pointer to the IDL FAQ on your web site or some other means, please update your link location. I've made the changes so I don't have more than one location on my web site (greatly reducing the chance that I'll forget to update one).
Latest IDL FAQ:
http://www.ivsoftware.com/pub/idl_faq.html
ftp://ftp.ivsoftware.com/pub/idl_faq.html
faq@ivsoftware.com
See Appendix A02. for additional
information
FAQ maintainer: Mike Schienle
Email: mgs@ivsoftware.com
Previous changes may be found in Appendix A04.
This is a list of Frequently Asked Questions about the Interactive Data Language or IDL. These questions pop up fairly regularly in the newsgroup comp.lang.idl-pvwave. This list is an attempt to cut down on net traffic regarding commonly asked questions. Users are encouraged to read through this list before posting a query to the newsgroup.
As of 1997 May 15 Mike Schienle has been maintaining the IDL FAQ. Ray Sterner was handling this previously, and Mike Schienle before him and Patrick Ryan before him; much of the material and many of the comments here were compiled by them.
This list is roughly divided into two categories: general questions and technical questions. General question numbers are prefixed with a G and technical ones with a T.
IDL is the Interactive Data Language. It is a product of Research Systems, Inc. (RSI).
The following is quoted from the README file at rsinc.com:/pub/idl/README. As such, it describes IDL's capabilities in an understandably subjective manner. ;-)
IDL, Interactive Data analysis Language, is a complete package for the interactive reduction, analysis, and visualization of scientific data and images. Optimized for the workstation environment, IDL integrates a responsive array oriented language with numerous data analysis methods and an extensive variety of two and three dimensional displays into a powerful tool for researchers.IDL supports an extensive data import capability, publication quality hard copy output, and user-defined Motif graphical user interfaces.
Users can create complex visualizations in hours instead of weeks with the aid of IDL's high level capabilities and interactive environment.
IDL is useful in physics, astronomy, image and signal processing, mapping, medical imaging, statistics, and other technical disciplines requiring visualization of large amounts of data.
Here is a short history of RSI:
[attributed to ali@rsinc.com (Ali Bahrami)]
IDL is a product of Research Systems, Inc., founded in 1977 by David Stern. The origins of IDL were developed at the Laboratory for Atmospheric and Space Physics (LASP) at the University of Colorado. David was one of the people involved in efforts to make computers easier to use for the physicists at the Lab. The first program in the evolutionary chain to IDL was named Rufus (named after Dave's dog). Rufus was a very simple vector oriented calculator that ran on the PDP-12. It accepted 2 letter codes that specified (1) An arithmetic operation (2) The input registers to serve as operands, and (3) the destination register. The next version was the Mars Mariner Spectrum Editor (MMED) which was a version of Rufus that ran on the PDP-8.The next program in this line was named SOL, and it also ran on the PDP-8. Unlike its predecessors, SOL was a real computer language with a real syntax (no more 2 letter codes). It was an APL influenced array oriented language with some primitive graphics capabilities. The resemblance to IDL was there, but very faintly.
In 1977, Dave left LASP to start Research Systems Inc. (RSI) with the intention of building on the ideas contained in SOL. The initial result of this endeavor was PDP-11 IDL, which was much more capable than SOL. Graphics was usually done on Tektronix terminals and outboard raster graphics displays. I used this version at LASP in 1981 on a PDP11/34 under RSX-11M in 1981 (I worked as a student at LASP from 1981 to 1987). I didn't use it for very long though, because 1981 was the year that Dave released the VAX/VMS version of IDL. This version, which was written in VAX-11 MACRO and FORTRAN, took advantage of the VAX virtual memory and 32-bit address space, and was a huge step beyond the PDP-11 version. It used essentially the same sort of graphics hardware as the PDP-11.
In 1987, Dave decided that Unix workstations were the direction in which IDL should progress, but porting the current VAX IDL to Unix didn't make much sense because of its MACRO and FORTRAN implementation. I had just finished my Masters degree and was looking for work. Dave hired me and together we wrote the current version of IDL for Unix on the Sun 3 taking advantage of the re-write to extend and improve the language. Since then, we've ported it to many Unix machines and moved it back to VMS. RSI has many other employees now, but our focus is still the continued development of IDL. Recently, IDL was ported to PC class systems running Microsoft Windows.
Their address is:
Research Systems, Inc.
2995 Wilderness Place
Boulder, CO 80301
tel: 303-786-9900
fax: 303-786-9909
email: info@rsinc.com or support@rsinc.com
World Wide Web: http://www.rsinc.com/ or http://www.rsinc.com/contactus/index.cfm for contact information and information about RSI's International Distributor Offices.
RSI's distribution scheme is unique in that all of the binaries and IDL code needed are available via anonymous ftp. IDL binaries and code are available at these sites:
The README file describes which files are needed, how to unpack them, and how to install them.
If you install IDL without a valid license, you will get IDL's 7 minute (10 minutes for Mac and PC) demo mode. This mode is designed for users who are considering buying the package.
To actually get IDL running for good, you must pay for a license from RSI and follow their instructions. You will be asked to fill out a form with information unique to your machine. RSI will create a license key which the license manager program (lmgrd) reads to validate your license.
As of release 3.1, upgrades to IDL are no longer free. For details about upgrades and support contracts, contact RSI.
IDL 5.2 is the current version. It was released on November 18th, 1998.
The IDL Student Version began shipping in July, 1997. Information is available at http://www.rsinc.com/sv/index.cfm
The information below is from the 51new.txt file, included as part of the installation procedure. The heading for this information is "Platforms Supported in this Release":
This table describes the computers and operating systems on which IDL version 5.1 runs. In most cases, IDL will run under operating system versions later than those shown.
|
|
|
|
|
|
|
VMS |
DEC |
Alpha AXP |
VMS |
6.2 |
|
Unix |
DEC |
Alpha |
Digital UNIX |
4.0 |
|
HP |
PaRisc |
HP-UX |
10.1 |
|
|
IBM |
RS/6000 |
AIX |
4.1 |
|
|
Intel |
Intel x86 |
Linux |
2.0 |
|
|
SGI |
R4000 and up |
Irix |
5.3 / 6.2 |
|
|
SUN |
Sparc |
Solaris1 (SunOS) |
4.1.3 |
|
|
SUN |
Sparc |
Solaris 2 |
2.5 |
|
|
SUN |
Ultra 1 / 2 |
Solaris 2 |
2.5 |
|
|
SUN |
Intel x86 |
Solaris 2 |
2.5 |
|
|
Windows |
Intel |
Intel x86 |
Windows 95 |
|
|
Intel |
Intel x86 |
Windows NT |
3.51, 4.0 |
|
|
DEC |
Alpha AXP |
Windows NT |
3.51, 4.0 |
|
|
Macintosh |
Apple |
Motorola PowerPC |
MacOS |
7.1.2 |
IDL 5.1 is no longer supported on Macintosh OS on Motorola 680x0 processors, Microsoft Windows version 3.11, or VAX VMS. IDL 5.1 support continues for Macintosh OS on Motorola PowerPC processors, Microsoft Windows 95 and NT, and Alpha VMS.
Around the time that the Unix version of IDL first became available (1988), Precision Visuals Inc. (PVI) entered into an agreement with RSI under which they enhanced and resold IDL under the name PV~WAVE. In September of 1990, they exercised an option in that agreement that resulted in the following:
- They received a copy of the IDL source code as it existed in September 1990 in return for a one-time payment to RSI.
- The connection between RSI and PVI was severed.
IDL and PV~WAVE are now on separate development tracks. Each company enhances, supports, and maintains its own product.
PVI has since merged with IMSL and is now Visual Numerics, Inc. (VNI).
The sites below contain public domain IDL code. See also the list of World Wide Web sites under G13.
Research Systems, Inc. User Contributed libraries
ftp://ftp.rsinc.com/pub/user_contrib/
RSI has excellent telephone and email support. You can contact them at:
Voice: (303) 786-9900
Fax: (303) 786-9909
Email:
(Internet)
info@rsinc.com # general questions
support@rsinc.com # technical support
(SPAN)
ORION::IDL
Keep in mind, however, that RSI's technical support is for their paying customers, i.e. those with current support contracts.
Unfortunately, there are two very different packages with the abbreviation "IDL". The newsgroup comp.lang.idl is for the Interface Definition Language, but has been somewhat superseded by the OMG-related newsgroup comp.object.corba. The newsgroup for discussing issues related to RSI's IDL and VNI's PV~WAVE and IMSL/IDL is comp.lang.idl-pvwave.
The Interface Definition Language is part of the Common Object Request Broker Architecture (CORBA), a distributed computing specification from the Object Management Group (OMG). For more details, see http://www.omg.org/, or the newsgroup comp.object.corba.
[ This question was included at the request of RSI. The answer was provided by Ali Bahrami. -pat ]
Yes, many of us do. We're naturally curious what people think of our product. We make notes about what people like and dislike and this influences our decisions.However, you usually won't get a direct response from us from a posting to this group. There are many reasons for this. Here are a couple of the more important ones:
We believe that this group should belong solely to the user community, and should be free of vendor bias and marketing. It should be noted that both RSI and Visual Numerics (formerly PVI) have shown great restraint in this matter, and that this group is largely left to the actual users. (Long term readers will recall some notable exceptions to this, but in general it is true.)
It could be argued that as long as the topic stays technical, vendor postings are OK. The problem with this is that one persons technical posting is another's blatant product plug, and the line between them is not always obvious.
We provide support for our customers via the phone and email. If you would like an answer from us, you should call us directly. We have no objection to you sharing the information you get in this manner with the newsgroup as long as you quote us accurately and separate fact from conjecture.
In other words, you should view this newsgroup as a way to share questions and information with other users, not as a way to contact the vendor.
See section G04.
[ This question included at the request of RSI. ]
RSI offers a number of IDL training courses for beginning, intermediate, and advanced IDL users. IDL courses are scheduled monthly at RSI's training facility in Boulder. On-site IDL courses are also available. Contact RSI at 303-786-9900 and ask for "training" or send e-mail to training@rsinc.com for complete scheduling and price information.Fanning Software Consulting offers completely customized on-site IDL programming courses for beginning to advanced users. For information, contact David Fanning at 970-221-0438 or at davidf@dfanning.com. Additional information about IDL training courses is available on the Coyote's Guide to IDL Programming web page at http://www.dfanning.com.
See also the list of ftp sites under G07.
One of the most useful tools to find an available routine is Pete Riley's
Searchable List of all IDL Routines.
This list is available from Pete's IDL page at http://nis-www.lanl.gov/~uk2/idl/
If you would like to add your IDL library to Pete's list contact him at uk2@lanl.gov
Additions to this section are welcome
Yes. David Fanning has released his long-awaited book IDL Programming Techniques.
RSI's training manuals are also available for purchase by contacting RSI (see G02).
J.D. Smith gathered responses from an online survey and tabulated the results. Check out IDLSPEC at http://astrosun.tn.cornell.edu/staff/jdsmith/IDLSPEC.html
If you would like the IDL FAQ sent to you via email, just send a message to faq@ivsoftware.com, and an auto-reposnder will email the FAQ to your address. You will need to turn off any spam spoofing for the auto-responder to work properly.
Yes. The anonymous FTP URL is ftp://ftp.ivsoftware.com/pub/idl_faq.html.
If you are unfamiliar with Internet URL's, the following steps ought to work:
For Windows, Macintosh or UNIX with GUI front end to FTP:
For UNIX command line:
There is a mini FAQ for Sun IDL in $IDL_DIR/notes, and the following questions are answered in the file sun.doc:
This problem is described in the POLYCONTOUR manual page.
RESTRICTIONS:This routine will NOT draw open contours. To eliminate open contours in your dataset, surround the original array with a 1-element border on all sides. The border should be set to a value less than or equal to the minimum data array value.
For example, if A is an (N,M) array enter:
B = REPLICATE(MIN(A), N+2, M+2) ;Make background B(1,1) = A ;Insert original data CONTOUR, B, PATH=Filename ... ;Create the contour file.
[ This problem was fixed in IDL 3.1. ]
The following is from Ray Sterner at Johns Hopkins University:
Here is a very simple algorithm that might be a useful addition to
the section of the FAQ on filled contours. It is for evenly spaced
contours only.
Z is an array to be contoured,
CI is the desired contour interval,
C0 is the desired starting color index,
D is the desired step between colors.
T = fix(Z/CI)
M = T - smooth(T,3)
F = (C0 + T*D)*(1-M)
is an array with filled contours with the contours
plotted with color 0. For contours of a different color
simply add M*CC where CC is the desired contour color index.
The system variable !EDIT_INPUT controls command recall. By default, it is set to 1, causing the last 20 commands to be saved. If it is 0, no commands are saved. To save more than 20 commands, just put !EDIT_INPUT=50 (or other large number) in your startup file.
It is important to realize that IDL looks at the value of !EDIT_INPUT the first time it reads anything from the keyboard, and the size of the history buffer is fixed after that. Hence, the command must be in a startup file because entering it at the keyboard is too late.
Jeff Valenti has written a sizable document about calling external FORTRAN routines from IDL. You can find it at eos.crseo.ucsb.edu:/pub/idl/idl-fortran.Z.
Here is the answer from RSI support:
The color applications such as xpalette and xloadct use a common block called "colors" to keep track of the color vectors. When you call tvlct, your vectors are loaded into the colormap, but they are not put into the colors common block.When xpalette starts, it checks to see if the colors have been defined in the common block and uses them if they have. Otherwise it sets them to the standard black and white colormap, in which r,g,b are each linear ramps. (The colors are as you expect because there is only one colormap.)
Admittedly, this is not the most desirable situation. You would like your colors which you loaded with TVLCT to be recognized by xpalette. We modified xpalette (and xloadct) to use the current colormap when they start up by getting the current vectors with TVLCT.
Another alternative would be to use the following lines to define the common block prior to calling your current version of xpalette.
[ assume here that you have a routine called "restore" which reads colors from a file somewhere and creates vectors r, g, and b. -pat ]
IDL> restore, file='ryan.sav', r, g, b ;get the vectors from somewhere IDL> tvlct, r, g, b IDL> common colors,r_orig,g_orig,b_orig,r_curr,g_curr,b_curr IDL> r_orig = r & r_curr = r IDL> g_orig = g & g_curr = g IDL> b_orig = b & b_curr = b IDL> xpalette
Try ? at the IDL prompt.
[Note: This is only a problem under OpenWindows2. -pat ]
Add the following to your .Xdefaults:
Idl*colors: -5
which reserves some colors for the colormap so that IDL does not exhaust all the available colors. (For a nice summary of Sun IDL interactions with OpenWindows, see $IDL_DIR/notes/sun.doc)
Another solution is to put a call to clear_colormap in your .login file to be executed after OpenWindows start up.
Quoting the IDL User's Guide, page 10-8:
IDL users may find that all their variables have seemingly disappeared after an error occurs inside a procedure or function. The misunderstood subtlety is that after the error occurs, IDL's context is inside the called procedure, not in the main level. Typing RETALL or RETURN will make the lost variables reappear.RETALL is best suited for use when an error is detected in a procedure and it is desired to return immediately to the main program level despite nested procedure calls. RETALL issues RETURN commands until the main program level is reached.
The HELP command can be used to see the current call stack (i.e., which program unit IDL is in and which program unit called it).
Yes. Emacs has a major mode for editing IDL code, idl.el, written by Chris Chase.
Get the files: idl.el and idl-shell.el
idl.el is an Emacs major mode for editing IDL and WAVE .pro files.
idl-shell.el is an Emacs major mode for interacting with an inferior IDL process.
Both modes momentarily do not have a maintainer. To preserve their internet accessability, the recent versions are supplied at:
ftp://pivory.mpae.gwdg.de/pub/idl/emacs/
http://pivory.mpae.gwdg.de/idl/emacs/
http://www.mpae.gwdg.de/mpae_RZ/software/idl/emacs/
Two corrections have been made in the IDL mode for Emacs (idl.el).
Fixes in version 1.41:
Other possible sources are:
ftp://eos.crseo.ucsb.edu/pub/idl/ (outdated)
For more information, see http://www.mpae.gwdg.de/mpae_RZ/software/idl/
If you would like to maintain one of these modes or both, please do. (Contact Kevin.Ivory@linmpi.mpg.de )
From John E. Davis, davis@space.mit.edu:
Also available is JED, an extensible programmer's editor that is available for Unix, VMS, OS/2, MSDOS, and MS Windows. The size of the executable is only slightly larger than vi. The extension language resembles C. It provides emacs, EDT, wordstar, and brief editor emulations (Many claim that JED's EDT emulation is the best around). It is the only freely available editor that can perform color syntax highlighting on ordinary character-cell color terminals (e.g., MS-Kermit, etc.) as well as under XWindows (Xjed). Currently supported programming modes include: C, FORTRAN, (La)TeX and BiBTeX, DCL, IDL, NROFF, SH, HTML, and SLANG. Other extensions include mail, rmail, compile, as well as the ability to read GNU info files. It is available from
ftp://space.mit.edu/pub/davis/jed/
See Appendix A02 for details on using URLs.
This subject is discussed on page 53 of the OpenWindows Version 3 User's Guide Release Manual.
OpenWindows 3.0 added 3-D appearance for widgets. In order for the 3-D look to work, it must be enabled and the background color must be a medium tone color such as "grey" or "wheat". Add the following resources to your ~/.Xdefaults file:
For plain IDL: Idl*threeD: TRUE Idl*background: PeachPuff2 For IMSL/IDL: Imslidl*threeD: TRUE Imslidl*background: PeachPuff2
This subject is discussed on page 53 of the OpenWindows Version 3 User's Guide Release Manual.
The OLIT widget toolkit, which is used by IDL, added the concept of "mouseless focus" under version 3.0. The red widget indicates where the current mouseless focus is. Use the arrow keys to traverse the widgets, and the spacebar to make a selection.
The basic routines are not accessible, for obvious reasons. The userlib, statlib and widget procedures are in $IDL_DIR/lib/. The procedure XDL also displays the full pathname. The system variable !path also contains the directory names for all accessible IDL procedures.
Because PostScript has scalable pixels, you must specify the xsize and ysize parameters, as well as the position parameter, in TV or TVSCL. I don't know about PV-Wave, but the following works in IDL:
; Display four images in a 2x2 grid
; Assume data(x,y,4) = array containing the 4 images
set_plot, 'ps' ;request PostScript output
device, ... ;modify page size, orientation, etc. as desired
ximsize = 0.5*!d.x_size ;define output image size
yimsize = 0.5*!d.y_size ;note: 0.5 assumes 2x2 grid
for i=0,3 do begin ;display the 4 images, using i as position index
tv, data(*,*,i), i, xsize=ximsize, ysize=yimsize
endfor
No.
Compiled routines are case insensitive. The only catch is that, on Unix systems, when executing a script via the .RUN command, the file name argument must exactly match the file name as it appears on the disk. Once the routines in the script are compiled, their names can be written in any case. This is not a problem in operating systems such as VMS that do not distinguish case for file names. This is usually not a problem under Unix either since, by convention, most people use lower case file names.
Sometimes.
Case matters when you are doing string comparisons. Comparing "IDL" to "idl" will return a false.
(This answer only applies to PostScript and HP-GL printers -- other printers may differ in having the X and Y offsets measured from the upper left corner of the portrait page instead of the lower left corner.)
IDL uses portrait page orientation as a default. (The x axis is along the shorter dimension of the paper.) In portrait orientation the lower left corner of the page is the origin for the XOFFSET and YOFFSET page offsetting keywords of the DEVICE command that determine the origin (lower left corner) of the output window. (Normally one uses XOFFSET=0 and YOFFSET=0 for portrait orientation.) Size of the output window is determined by the XSIZE and YSIZE keywords of the DEVICE command. The origin for graph positioning variables !P.POSITION and !P.REGION is the output window origin. X and Y coordinates for portrait page orientation are shown on the sketch below as upper case X and Y.
----------
| | ORIGINAL PORTRAIT PAGE (Printer only prints on
Y | | OUTPUT WINDOW this area.)
| |
| X |
o-----------
| y + | OUTPUT WINDOW ROTATED ABOUT LOWER LEFT CORNER
x | + |
| + |
------------
++++++++++ OUTPUT WINDOW RESIZED FOR LANDSCAPE ORIENTATION
^
| DIRECTION OF NEEDED OFFSET
If device,/landscape is specified, then the output window is rotated 90 deg. clockwise about the lower left corner of the page. In this condition nothing will be plotted on the page, since the rotation has carried the output window entirely off the paper as shown in the sketch above. To correct this mismatch, the rotated output window must be offset. XOFFSET AND YOFFSET are still measured in the X and Y coordinates of the portrait page, but now represent the position of the lower left corner of the rotated (and resized) output window (marked by an o above) with respect to the lower left corner of the portrait (actual) page. Hence, one generally uses XOFFSET=0 and YOFFSET=long_dimension_of_page for landscape orientation.
In landscape orientation, the coordinates for graph positioning variables !P.POSITION and !P.REGION are the lower case x and y coordinates shown in the sketch above and having origin marked by the letter o. Position of output window origin o on the page is, of course, affected by the setting of XOFFSET and YOFFSET, as explained before. The XSIZE and YSIZE (output window size) keywords of the DEVICE command are also measured in the x and y directions when in landscape orientation. The resizing of the landscape page generally means interchanging the values of XSIZE and YSIZE appropriate for the portrait page.
Example of settings for a portrait page:
XPAGE=8.5 & YPAGE=11. & XOFFS=0. & YOFFS=0. ;Inches DEVICE,/INCHES,XSIZE=XPAGE,YSIZE=YPAGE,XOFFSET=XOFFS,YOFFSET=YOFFS
Example of settings for a landscape page:
XPAGE=11. & YPAGE=8.5 & XOFFS=0. & YOFFS=XPAGE ;Inches
DEVICE,/LANDSCAPE,/INCHES,XSIZE=XPAGE,YSIZE=YPAGE,XOFFSET=XOFFS, $
YOFFSET=YOFFS
Example of setting position and size of a plot window:
X0=1.374 & Y0=1.283 & XLEN=3.622 & YLEN=6.157 ;Inches !P.POSITION=[X0/XPAGE,Y0/YPAGE,(X0+XLEN)/XPAGE,(Y0+YLEN)/YPAGE]
Example of setting position and orientation of a text string:
x0=.35 & y0=.37 ;Inches xyouts,x0/xpage,y0/ypage,!stime,orient=90,/normal ;Date, time
IDL sets aside a certain amount of memory area for compiling programs. The current code and area sizes can be seen with the HELP command, e.g.
IDL> help
% At $MAIN$ .
Code area used: 0% (0/16384), Symbol area used: 0% (2/4096)
These sizes can be increased with the .SIZE command. Quoting the IDL User's Manual, page 2-11:
These sizes represent a compromise between an unlimited program space and conservation of memory. User procedures and functions are compiled in this large program area. After successful compilation, a new memory are of the required size is allocated to contain the newly compiled program unit.Resizing the code and data areas erases the currently compiled main program and all mail program variables. For example, to extend the code and data areas to 30000 and 5000 bytes respectively:
.SIZE 30000 5000
Getting "Code Area Full" is often an indication that the routine is large, and would benefit by decomposition into sub-procedures/functions. It's better to avoid use of .SIZE because your code will always work on other systems where the users don't use a large .SIZE setting.
% Unable to allocate memory: to make array.
not enough core
RSI support replies:
The circumstances described happen when memory becomes fragmented. Unfortunately, there is nothing you can do except use less memory in your application, or get more for the system to work with.
Ray Sterner of Johns Hopkins University has written a procedure for managing this called crossi. It is in the usr.tar file of the JHU/APL IDL library mentioned in question G07.
Joel Parker has written a procedure called rdplot which manages this using an XOR graphics function and provides additional functionality. It is located in the NASA IDL Astronomy User's Library mentioned in question G07.
newarray=cos(small_array)*sin(large_array)
where I want the data in small_array to be used over and over 300 times in this calculation.
IDL> arr1 = Findgen(8)
IDL> arr2 = Findgen(300, 8)
IDL> newarr = (Replicate(1.0, 300) # Cos(arr1)) *
Sin(arr2)
array2 = array1( Lindgen(n,M) / n )array2 = replicate(1,n) # array1
to convert an array1(M) to array2(M,n) :
array2 = array1( Lindgen(M,n) MOD M )
or array2 = array1 # replicate(1,n)
To make array1(M) to array2(M,n):
array2 = rebin(array1,M,n,/samp)
To make array1(M) to array2(n,M):
array2 = rebin(reform(array1,1,M),M,n,/samp)
Note: use of /overwrite in REFORM avoids copying array1, but changes array1.
REBIN is a little faster than the matrix multiply method. The additional advantage of REBIN is that it will work with higher dimensions, e.g. to make array1(M,N) into array2(M,N,L):
array2 = rebin(array1,M,N,L,/samp)
Using MacX v1.2 and IDL cause the Mac to crash quite often. This happens especially during allocation of color resources or display windows. You can get around the problem by downgrading to MacX v1.1.7 (apparently Apple will supply this if you can prove to them that you rightfully own v1.2). Another solution is to purchase White Pine's eXodus software. Rumors are that eXodus is an overall better product than MacX. White Pine can be contacted at:
White Pine Software 40 Simon St. Suite 201 Nashua, NH 03060-3043 phone: 603-886-9050 Fax: 603-886-9051
Note: MacX 1.5 works correctly with IDL. Also, Tenon Intersystems <http://www.tenon.com> makes a very fast X Server called XTen, and a full UNIX implementation which runs alongside MacOS called MachTen. MachTen includes the functionality and speed of XTen.
It is often useful to determine if an IDL variable is defined. This is easily done using the n_elements function which returns 0 if the given variable is undefined. This is especially useful for setting defaults for keyword parameters. Here are several examples:
if n_elements(start) eq 0 then start=0
if n_elements(dir) eq 0 then cd, current=dir
From Bill Thompson:
The IDL function KEYWORD_SET() is only designed to be used with logical variables, i.e. those which can be either True (usually signalled with the value 1) or False (0). It has the property that if a variable is undefined, then it returns False, so people often make the mistake of using it to test whether a variable is defined or not.
To test whether a variable is defined or not, use N_ELEMENTS() instead. This will return 0 if a variable is undefined, or some positive number otherwise. Only use KEYWORD_SET for truly Boolean (True/False) variables.
From a comp.lang.idl-pvwave post by William Thompson:
The reason that TVRDC is undocumented is because it's not needed any more. It doesn't do anything that CURSOR doesn't do. TVRDC is only retained for compatibility with older programs.
In the old days, before X-windows, CURSOR was used to read coordinates off of line graphics terminals, and TVRDC was used to read coordinates off of image display devices. With the advent of IDL 2.0, the distinctions between different kinds of graphics devices were mostly removed, and both of these functions were merged into CURSOR.
By Jason Mathews, NASA/ Goddard Space Flight Center:
Common Gateway Interface (CGI) Perl scripts are invoked via submiting a HTML form , which execute IDL in batch mode, runs a IDL routine to make a plot of the selected parameters, and writes the results to a GIF. The perl program output is displayed on the WWW browser as a HTML document with an inline GIF image.
The IDL program must use the 'Z' device and write the output results to a GIF file as in the example below:
SET_PLOT, 'Z' ; Select the Z buffer output device PLOT, x, y, ... ; Draw the plot image = TVRD() ; Copy the device contents into an image WRITE_GIF, 'filename.gif', image ; Write image to a GIF file EXIT ; Exit IDL routine
Examples of various perl programs that use IDL on the web and the corresponding HTML forms are available via the following URL: http://coney.gsfc.nasa.gov/Mathews/misc/idl-www.html
Some example WWW-based Data Browsing and Retrieval Systems using IDL:
By Mark Rivers, CARS, Univ. of Chicago (slightly edited)
IDL> plot,x,y,title='This is my plot' ; Make a plot (or display an image). IDL> image = tvrd() ; Read image into an array. IDL> tvlct, r, g, b, /get ; Read color table. IDL> write_gif, file, image, r, g, b ; Save in a GIF image file.
This sequence should work on any windowing display (X, Windows,
Mac). For better speed and more flexibility, first issue the
command
SET_PLOT, 'Z'
to write to the Z-buffer pseudo-device. You can set its resolution,
and its write and readback performance is very fast.
[Additional notes by editor] The screen image may be saved
in other graphics formats by using the correct write routine. Some of
the routines of interest are:
write_gif, tiff_write, write_jpeg, write_bmp, ...
Note the different name pattern for tiff_write, check the
manuals for the calling syntax for each routine.
There are also corresponding routines to read the images back into IDL.
Which format is best? GIF images are compressed so take less space than uncompressed images. JPEG images are also compressed but with a lossy compression, that is, the image when read back into IDL is not identical to the image written. However JPEG images can look very good and also often can be more highly compressed than GIF. Some images may be larger (much larger) using JPEG as compared to GIF, try both and compare. GIF images are widely used on the World Wide Web, JPEG is also supported by some web browsers. TIFF may be needed for publishing purposes.
By Troy Klein, Johns Hopkins Applied Physics Lab
Quoting the IDL version 4.0 online help for the PREVIEW keyword, "Set this keyword to add a 'device independent screen preview' to the PostScript output file, in encapsulated PostScript interchange format (EPSI). Use this keyword only with encapsulated output. Many, but certainly not all, desktop publishing and word processing programs will display this screen preview when the file is imported into a document."
The EPSI format is not compatible with some of the more popular Windows and Macintosh applications. Windows applications expect the preview image to be in TIFF format and Macintosh applications expect the preview image to be in PICT format and in the resource fork of the EPS file. I have found the EPSI file to be compatable with Framemaker for UNIX/X windows.
There are two workarounds available for Mac users and two for Windows users, all are similar. If you are using IDL on a UNIX machine, there is a package available (as of 8/2/95) on the popular University of Michigan Mac shareware/freeware archive (use the mirror site at ftp://mirrors.aol.com) titled "ps2epsmac", which uses Ghostscript and NetPBM on a UNIX machine to interpret the postscript and create a new EPS file in Macbinary format with the PICT preview. There are several Mac programs available (StuffitExpander, Fetch, etc.) to interpret the Macbinary file. I have had great success in using this method to import IDL EPS files into Word, Powerpoint, and MacDraw Pro. There is a similar package at the archive titled "ps2epsplus" which does the same thing, but all on a Mac.
For UNIX IDL users trying to import IDL EPS files into Windows applications, there is a package called "ps2epsf" which can be located using an Archie search. This package essentially does the same thing as the UNIX/Mac package but produces an EPS file with a TIFF preview. The second solution for Windows users is the Windows applicationi GSview (which can also be found with archie). Version 1.2 claims to be capable of adding preview images to EPS files.
In IDL version 4, the sizing and layout of IDL's Motif widgets changed in several ways. This can cause problems. Widget changes are discussed in a file available from RSI at
ftp://boulder.colorado.edu/pub/rsi/idl/notes/widgets.txt.
By Eric Korpela of Berkeley
This is a result of IDL being written in C and using the C library functions (malloc and free) for memory allocation. In most C libraries, memory that is freed is NOT returned to the operating system. The C program retains this memory and will reuse it for future calls to malloc (assuming that the new allocation will fit in the freed block).
Another way of considering it is in terms of how memory allocation is done under UNIX. New memory is allocated using brk() or sbrk() which control the size of the data segment. These routines are called by malloc().
Suppose you allocate 3 1 MB regions of memory under C.
char *p1=(char *)malloc(3*1024*1024);
char *p2=(char *)malloc(3*1024*1024);
char *p3=(char *)malloc(3*1024*1024);
Here's what your data segment would look like assuming malloc had to call
sbrk().
---------------------------------------------------------------
prev stuff | overhead | 3MB | overhead | 3MB | overhead | 3MB |
---------------------------------------------------------------
^ ^ ^ ^
p1 p2 p3 end of segment.
Now we free(p1).
----------------------------------------------------------------
prev stuff | overhead | free | overhead | 3MB | overhead | 3MB |
----------------------------------------------------------------
^ ^ ^
p2 p3 end of segment
Notice that the free memory is still in the data segment. If free had called brk to reduce the size of the segment, the 3MB pointed to my p3 would be outside the data segment! SIGSEGV city! If free had moved the allocated memory to lower addresses so the segment size could be reduced without losing data, then p2 and p3 would point to invalid addresses, and we'd be forced to use handles rather than pointers and call GetPointerFromHandle() every time we wanted to access the memory. Ick! Just like Windows!
By default IDL uses 4 bits per pixel when displaying images in PostScript. This gives only 16 possible gray shades or colors as may be seen by the following example commands:
a = rebin(bindgen(256),256,50) ; Create a color bar.
set_plot,'ps' ; Set PostScript mode.
device,/color ; Specify color PostScript.
loadct,4 ; Load a color table.
tv,a,0,0,xs=12.8,ys=2.5,/centimeters ; Display bar.
device,/close ; Close PostScript file (idl.ps).
$gs idl.ps ; Send to a color PS printer or use
; a PostScript viewer to see results.
Replace first call to device above by:
device,/color,bits_per_pixel=8
This simple change may fix a number of problems with color PostScript. By the way, you will need to use /inches or /centimeters on a tv or tvscl command to get the desired results. This applies to both the position and sizes of the image. By default these are in pixels and PostScript has a lot of pixels as may be seen by doing help,/structure,!d after set_plot,'ps'. Also try help,/device for other useful info when in PS mode.
By Liam Gumley, Space Science and Engineering Center, University of Wisconsin-Madison
The fix I came up with goes as follows. Put the following commands in an IDL startup file (they must be executed immediately after IDL startup):
device,retain=2,pseudo=8 ; 8 bit display with backing store window,/free,/pixmap,colors=-5 ; Create window to allocate colors plot,[0] ; Might not be needed, but won't hurt wdelete,!d.window ; Delete the window device,set_character_size=[6,9] ; Set the vector font size print, 'Number of colors allocated is ', !d.n_colors
If you use this in your startup file, then you should not need any entries in your $HOME/.Xdefaults file. The only command I have in $HOME/.Xdefaults is
Idl*fontlist: screen16
which sets the font used for widgets (unrelated to the graphics vector font).
As RSI technical support explained it to me, the default graphics vector font size is 6x9 at startup. However as soon as you open a graphics window, that font size may be changed (why, I don't know). So the only way to guarantee that you get 6x9 vector fonts is to use the commands above immediately after startup. Please note that you are not guaranteed to get 256 colors allocated to IDL 4.01 if you use the colors=-5 keyword (I usually get about 170 colors on my SGI). On SGI boxes with 24 bit graphics, setting colors=256 usually gets you 256 colors, but this won't work on all Unix boxes. The IDL 5.0 pre-release seems to be a bit smarter (at least on an SGI), and even if you set colors=-5, it allocates 256 colors.
Both RSI and David Fanning have provided answers to this on their respective web pages.
RSI's response is available at http://www.rsinc.com/tipoweek/archive/widgets1.html (link broken as of 11/04/98)
David Fanning's response is available at http://www.dfanning.com/idl5_info/tlb_resize_problem.html
Mike Schienle provided an answer to this on his web site at http://www.ivsoftware.com/IV_Fonts.html.
Several tools exist to do these types of operations. DDREAD, a utility developed by Fred Knight has been recommended. Contributions from Helge Rebhan are available at READTAB, and Mati Meron at RASCII.
From Knight's in-line documentation:
; Purpose: ; This routine reads data in formatted (or unformatted) rows and columns. ; The name stands for Data Dump Read. By default, comments are ; skipped and the number of columns is sensed. Many options ; exist, e.g., selecting rows and columns, reading binary data, ; and selecting non-default data type and delimiters.
Note the following list of other routines needed by DDREAD (available at the same location, or as part of the RSI built-in library):
; - Other routines needed: ; pickfile.pro - to choose file if none is given ; nlines.pro - to count lines in a file ; nbytes.pro - to count bytes in a variable ; replicas.pro - to replicate arrays (not scalars as in replicate.pro) ; typeof.pro - to obtain the type of a variable
Liam Gumley's imagemap.pro is an IDL Satellite Image Mapping Tool, and is made expressly for this type of work.
Heiko Hünnefeld <hhuenne "at symbol" desy.de> has offered an improved CURVEFIT routine. Contact Heiko for information.
Craig B. Markwardt, Ph.D has translated the MINPACK-1 curve-fitting package into IDL, and made the routines available on his site at http://astrog.physics.wisc.edu/~craigm/idl/idl.html. A tutorial for these routines is also available at http://astrog.physics.wisc.edu/~craigm/idl/fittut.html.
Chances are whatever you're curious about with 24-bit color can be answered by David Fanning's Using Colors in IDL.
David Fanning has provided answers to over a dozen PostScript questions at Creating Perfect PostScript Output.
David Fanning has provided instructions for Creating a Run-Time IDL Application for Windows NT.
The following AppleScript provided by Olof Hellman will provide similar capabilities for Macintosh:
set inputFile to path to (choose file)
tell application "IDL"
do script " ignoredResult = anyIDLProcedure(inputFile ) "
end tellThe following portion of a UNIX shell script which would be referenced by the IDL_STARTUP environment variable could be used similarly:
; set a variable to be checked momentarily
status = ''
; check if an environment variable is set for our application (optional)
IF (GetEnv('IDLBIN') EQ '') THEN $
status = Dialog_Message('IDLBIN not set. Aborting IDL', /Error)
; determine if it is OK to start up application
IF (status EQ '') THEN $
myProc
David Fanning provided an explanation that made a lot of sense to me. This was in response to the documentation for HDF_SD_GETDATA, so that context has been left in the explanation.
IDL stores information in a row-order format. That is to say that in memory row elements are contiguous. HDF apparently stores information in a column-order format, so that column elements are contiguous.Does this matter? Most of the time, not a whit. It has nothing to do with how columns or rows are specified in variables, or whether the column notation is first or second. Where it does matter is if you are doing something in a loop. If data is stored row-ordered and you loop over a column index, the loop can be quite slow, since you have to grab successive chunks of memory that are not next to each other.
Shortly afterwards, William B. Clodius provided much greater detail on the subject.
There are four different aspects of arrays, how they are stored in memory, how they are accessed by the language, how matrices are mapped to arrays, and how they are displayed on the screen.
- Memory storage: Many of the aspects of how they are stored in memory are determined by the OS/processor which provides blocks of memory with optimal alignment properties. Any sensible single processor implementation of arrays will try to lay out all elements of the array in one contiguous block for efficient access. (Many languages restrict this block to the heap, but some allow stack allocation). An array oriented language, e.g., APL, IDL, Algol 68, Fortran 90/95, will also create a small descriptor defining the layout.
- Array access: Because contiguous elements are usually more efficiently accessed than non-contiguous elements, and most languages want to make it easy for users to implement efficient code, most languages define a mapping of array elements to memory locations in the block. Historically, it is this mapping that is usually described by the terms column major versus row major. I am not certain how this convention started. Under this definition both IDL and Fortran are column major.
- Matrices are mathematical constructs that can be efficiently implemented as arrays with special operations. The mapping of matrices onto arrays in IDL and Fortran may be different, but I have not looked in detail at this. The interpretation of this mapping depends on whether a vector by default is described as a row or column vector. While using the wrong convention with a given language would be a source of error, using the correct convention should be of comparable efficiency. Note, however, that IDL does not discuss matrices in its discussion of its array naming convention.
- Array display: Many (most) languages (including C and Fortran) are not concerned with the display of arrays. Such a display for these languages is only defined by external library packages, and can follow the library's arbitrary conventions. A sensible package would map display lines to the most efficient access method, I.e., the last index for C, the first index for Fortran. The IDL language defines the same mapping as an efficient library for Fortran, each line mapping to a different value of the second index.
William Thompson adds the following (minor editing/formatting by M. Schienle to keep this from becoming a novel):
In order to make the above statement "IDL stores information in a row-order format", you first have to define what is meant by a row and by a column in terms of IDL matrices. Personally, for matrices, I tend to assume that the first dimension is the row and the second dimension is the column, and would thus agree with that statement. However, the original poster was using the opposite convention, and had a good reason for doing so.From all the mail messages on the subject, I think that the following facts can be derived:
- IDL stores arrays such that the leftmost dimension is the most rapidly changing. This means that in memory, the array element A(0,0) is followed by A(1,0), and then A(2,0), etc. up until A(N-1,0) which is followed by A(0,1), and so on.
- Fortran stores arrays using the same convention as IDL, except that indices start with 1 rather than 0.
- C stores arrays such that the rightmost dimension is the most rapidly changing. In other words, A[0][0] is followed by A[0][1] up to A[0][N-1], which is then followed by A[1][0] and so on.
- Standard software written in C and Fortran both map matrices onto 2D arrays such that the left dimension is the row and the right dimension is the column. However, the storage of these matrices in memory would be different.
- The mapping of matrices onto arrays in IDL appears to depend on what software is used. The original # array multiplication operator assumes that the first dimension is the row dimension and the second dimension is the column dimension. With this convention, IDL treats matrices like Fortran in both notation and storage.
However, with the alternate ## array multiplication operator introduced in IDL/v4.0 the opposite convention is used. I.e. the ## operator assumes that the left dimension is the column dimension and the right dimension is the row dimension. With this convention, the internal storage of matrices matches that of C, but the notation differs from both C and Fortran.- The most recent documentation from RSI appears to encourage the use of the new ## operator over the original # operator. I don't know if this is because there are other elements which have been added to IDL which also assume a (column,row) notation for matrices.
- Data arrays in IDL are usually labelled such that the left dimension is "X" and the second from the left is "Y". This is the convention followed by graphics routines such as TVSCL and CONTOUR, with X increasing to the left and Y increasing up. One could also consider X to be the column and Y to be the row, since that is how the data would be displayed.
Specifically, if A is a float matrix (m x n) and V is a vector with m elements, is there a faster way than ...
(loop operation) FOR i=0,n-1 DO A[*,i]=V
David Fanning provided the following:
(matrix operation) vector = Replicate(1, n) array = V ## vector
Stein Vidar contributed the following addition:
(rebin operation) array = rebin(reform(v,m,1,/overwrite),m,n,/sample)
Timing results from Stein Vidar's UNIX system:
{ alpha OSF unix 5.2 Oct 30 1998}
Time for Loop: 0.27343702
Time for Matrix: 0.093750000
Time for Rebin: 0.067382932Timing results from David Fanning's Windows NT system:
{ x86 Win32 Windows 5.2 Oct 30 1998}
Time for Loop: 0.10000002
Time for Matrix: 0.019999981
Time for Rebin: 0.039999962Stein mentions, "Note that the relative speeds can vary quite a lot on different architectures."
From David Fanning:
Yes. With the possible exception of VMS (where you can set the XDR keyword) all save files are stored in XDR format, which makes them transportable across machine architectures.
The following caveat is from RSI's on-line documentation:
Note also that save files containing routines may not be compatible between different versions of IDL, but that files containing data are always backwards-compatible.
Yes. Mike Schienle has made an IDL Style Guide available at http://www.ivsoftware.com/IDL_Style.html.
Questions and answers in this document are culled from the user community, except where noted otherwise. No warranty, express or implied exists regarding this document. Permission to copy all or part of this work is granted, provided that the copies are not made or distributed for resale.
The current IDL FAQ may be accessed at http://www.ivsoftware.com/pub/idl_faq.html
HTML version: idl_faq.html
The plain text version will be provided at a later time, or use your browser to convert HTML to text.
How to interpret the URL
Using anonymous ftp:
Additions and Corrections
Send additions and corrections to:
Mike Schienle
mgs@ivsoftware.com
John Black Scott Claflin David Edelsohn Gwyn Fireman Joseph B. Gurman JF Pitot de La Beaujardiere Vinay Kashyap Shane Mayor Thomas Oettli Robert M. Moss Philip Sterne William Thompson Jeff Valenti Ray Sterner Joel Parker David Landers Dan Carr Denise Fields Jason Mathews Mark Rivers Troy Klein John Davis Eric Korpela Norbert Hahn Chris Chase Liam Gumley JBob Brown Kevin Ivory Struan Gray
Patrick Ryan: founded the IDL FAQ
Mike Schienle: from 12/01/93 to 10/27/94
Ray Sterner: from 10/28/94 to 5/15/97
Changes in version 4.11:
Changes in version 4.10:
Additions in version 4.10:
Changes in version 4.09:
Additions in version 4.09:
Changes in version 4.08:
Changes in version 4.07:
Changes in version 4.06:
Changes in version 4.05:
Changes in version 4.04:
Changes in version 4.03:
Changes in version 4.02:
Changes in version 4.01:
Changes in version 4.00:
Changes in version 3.21:
Changes in version 3.20:
Changes in version 3.19:
Changes in version 3.18:
Changes in version 3.17:
Changes in version 3.16:
Changes in version 3.15:
Changes in version 3.14:
Changes in version 3.13:
Changes in version 3.12:
Changes in version 3.11:
Changes in version 3.10:
Changes in version 3.9:
Changes in version 3.8:
Changes since version 3.7:
Changes since version 3.6:
Changes since version 3.5:
Changes since version 3.4:
Changes since version 3.3:
Changes since version 3.2:
Changes since version 3.1:
Changes since version 3.0:
Changes since version 2.8:
Changes since version 2.7:
Changes since version 2.6:
Changes since version 2.5:
Changes since version 2.4:
Changes since version 2.3:
Changes since version 2.2:
Changes since version 2.1: