Cornell Theory Center Technical Report CTC92TR101: AN EVALUATION OF INTERACTIVE, X-WINDOW-CAPABLE SOFTWARE FOR SCIENTIFIC VISUALIZATION ----- Steven R. Lantz Cornell Theory Center Cornell University August 1992 TABLE OF CONTENTS ----------------- Abstract Introduction: The CSERG Visualization "Manifesto" List of Software Products Reviewed Brief Summaries of All the Products Evaluation: Advantages and Disadvantages Summary and Recommendations Appendix: Reviewers' Comments ABSTRACT -------- Scientific visualization is on the verge of being revolutionized; but in all the excitement over new techniques, it is important not to lose sight of the basic visualization infrastructure. This is especially true at the CNSF, where major hardware and software changes are presently underway. This Technical Report documents a study led by the author in late 1991 and early 1992 to survey existing graphics packages for their ability to do "bread-and-butter" graphics--i.e. to do the unglamorous kinds of visual data exploration which are the most crucial in day-to-day scientific research. To integrate effectively into current working environments, eligible packages had to run in X Windows, and had to offer the user a significant degree of interactivity. Thirteen different commercial and public-domain packages that were examined in all; about half were actually installed for hands-on testing by CNSF staff, to see if any of them were suitable to support as a general-purpose graphics tool. In this report, after an extended introduction which spells out the basic requirements from the point of view of a computational scientist, summaries of the products are given and their capabilities compared. Then, an analysis of the advantages and disadvantages of each is presented, and recommendations are offered. A list of sources for all reviewed software is included. INTRODUCTION: THE CSERG VISUALIZATION "MANIFESTO" -------------------------------------------------- In mid-1991, the CNSF's Computational Science and Engineering Research Group (CSERG) created a concise statement of what we saw to be the basic requirements for a "bread and butter" graphics package: one which would pull together those tools that would be most useful to us in doing our own day-to- day visualization tasks as computational researchers. This preliminary CSERG report served as an important guideline in a subsequent, wide-ranging search of commercial and academic software, the goal of which was to determine if any of the then-available packages might address these perceived needs. It is the aim of this Technical Report to document the results of our search; so it is fitting to begin with the CSERG visualization "manifesto" that initiated the process. We will go on summarize, in the rest of the present Technical Report, the various options open to us in late 1991 to early 1992. It is important to note the time frame, because at this writing, scientific visualization is a rapidly evolving specialty with new products entering the market at a tremendous rate. * * * * * The visualization of scientific data is a growing field with many exciting avenues for exploration. An unfortunate side effect of this is that the simplest but most often and most widely used visualization tools are neglected. These tools are indispensable for our research. We believe that one important mission of the CNSF and other supercomputer centers is to provide tools that enable scientists to work more efficiently and to integrate the supercomputing environment with their "local" environment, which may be a high-powered workstation, but may also be a dumb terminal. As much as possible, we would like to see the most basic tools be machine indepen- dent and support a large number of output devices. As part of the creation of an integrated "AIX" workstation/supercomputer environment, we would strongly encourage the development or acquisition of more sophisticated graphics tools which run across multiple AIX and Unix systems. In particular, the CNSF should aim to provide a rich visualization environment on RS/6000's. This report is divided into two parts. The first deals with basic capabilities. The second part discusses slightly more advanced or somewhat specialized features. These features would enhance the productivity of either most CNSF scientists, or large and important groups of them, e.g. computational fluid dynamicists or computational chemists. *** BASICS *** The basic data plotting functions listed below should be made available to all CNSF users, ideally, before AIX becomes the default operating system on the CNSF's IBM 3090s. Otherwise most users will experience an intolerable disruption to their research once the switch away from CMS is made. Many users are equipped to view their data only through IBM or Tektronix emulations, and this must taken into consideration, at least for the short term. Therefore we propose the below as short-term goals only: these constitute a bare minimum suite of visualization tools that must be made available to users before more forward-looking types of visualization can be seriously contemplated. We prefer that the basic graphics programs supported by the CNSF be either in the public domain or available inexpensively with site licenses. They should run on a wide range of machines. In particular, we would like to see the tools run on "all" UNIX machines, and make use of X Windows and PostScript. As a general rule, we feel that these tools should not be incorporated into a "grand" application. Such an attempt would surely slow down the development of each element. Rather we would like to see a small set of stand-alone tools to be made available which can perform the most basic visualization needs of scientists. It may be that the features that we describe exist in one or in several public domain or inexpensive programs. If so, we recommend that the CNSF obtain these programs and support them on CNSF machines. If tools are developed at the CNSF, we should make them freely available both to our users and to other members of the scientific community. FEATURES OF ALL BASIC TOOLS * Availability: These tools should be easily available to researchers. Therefore, they should be in the public domain if possible, and if not, they should be available at low or no cost with site licenses. * Machine independence: These tools should run on as wide an array of machines as possible. Machine specific programming should be avoided or localized to allow for easy porting to different architectures. Any tool should initially support all available UNIX architectures. * Output device independence: These tools should be easily adaptable to new output devices. They should initially support at least a core of widely used devices (X Windows, Tektronix, PostScript, etc...). * Data input: These tools should accept ASCII text input of data from files. Almost any other method of storing data is more efficient, but ASCII text is available to everyone and should be the primary method of delivering to the tool the data which will be visualized. Support for other data formats may be included as well, but ASCII text must be fundamental. * Ease of use: Graphics should be easy and intuitive. Most scientists do not want to spend large amounts of time doing graphics programming. LINE-PLOTTING PROGRAM Perhaps the most widely used type of visualization is line plots. There are several programs which run on a wide variety of machines and which perform all of the desired line-plotting functions. An example is "SuperMongo". We would encourage the support of SuperMongo at the CNSF. In general, we feel that the following features are important in any supported line-drawing tool. Control of line styles (thickness, dashing, color, etc...) Control of data-point styles Control of axes (tickmarks, tick-labels, etc...) Axis labels and general labels (including formulas in axis labels) Basic manipulation of data vectors (addition, multiplication, logarithms, exponents, etc...) Ability to use control scripts for making complicated plots A useful feature would be the ability to call many of these functions from a FORTRAN or C code. It is useful to be able to create simple line plots directly from a code without needing to understand a particular display device. All of these features are available in SuperMongo and may exist in other packages, and one (if not several) such packages should be available. CONTOUR-PLOTTING PROGRAM The simplest approach for visualizing 2-dimensional data sets is contour plotting. These are the basic requirements: Contour plotting of uniform, rectangular grids of data. Overlaying of multiple contours on the same plot. Control of line style (thickness, dashing, color, etc...) for each contour. Control over labeling of contour lines. Control over axis and general labels (see Line Plotting). Ability to handle "stretched" grids: Not all contour domains have uniform spacing in a given direction. The ability of the contour plotter to interpolate the data in this case should be included. Ability to handle polar coordinates: Not all contour domains are rectangu- lar. The ability to handle polar coordinates would be helpful. (If possible, a general approach to handling "odd" domains would be helpful.) FALSE-COLOR PROGRAM In some cases, it is easier to extract information from 2-dimensional data sets when "false colors" are used to indicate the value of the quantity instead of the level lines of contour plots. In this vein, a program having the features of the contour plotter described above, but displaying false colors instead, would be very useful. Of course in addition to the features described for the contour plotter, a false-color plotter should allow for control (and construction) of color palettes. SURFACE-PLOTTING PROGRAM The use of surface plots can be very useful in certain situations. We recognize, however, that the 3-dimensional aspects of producing surface plots greatly complicates the task of producing a "general" surface-plotting tool. At minimum, a surface plotter should be able to take uniform Cartesian grids and produce a surface plot with user-defined axes and labels. DATA-FITTING AND DATA MANIPULATION CAPABILITIES Ideally, any plotting program should be able to do more than just to create pictures of data. It should have the capability to: Fit data to various functional forms, perform spline fits, and plot fitted curves. Give various kinds of statistical information, e.g., the value of chi- square, or the coefficient of correlation. Plot curves that are specified by their functional forms, rather than by a set of input data points. Convolve one curve with another. Deconvolute a curve. Undo anything that you plotted during a preceding step. Choose default values for the extent of axes, if not explicitly specified. Plot the integral of a curve. Allow one to choose the viewing point (in elevation plots). Perform Fourier transforms. Write a PostScript file that can be incorporated into a TeX document. Redirect input (as should be possible with any Unix utility). Most but not all of the features described in this section exist in the program PLOT which runs on Suns but has not yet been ported to the RS/6000. We would like to see PLOT ported to the RS/6000's and supported on CNSF workstations. DATA TRANSMISSION It will often be necessary to transmit large amounts of data between the supercomputer and a local workstation, which is running either a general or specialized graphics package. Thus it is important that a fast link exist between the supercomputer and the workstation. Often, the data being transmitted is in an unformatted file. Tools should exist for efficiently transmitting unformatted data between the supercomputer and workstation. *** SLIGHTLY ADVANCED *** Here are some relatively more advanced tools that would either increase a typical researcher's efficiency, or provide him/her with more sophisticated functionality. "MAC-LIKE" INTERACTIVE GRAPHICS Graphics tools would be more useful if they took advantage of Macintosh- like capabilities inherent in UNIX workstations running X-windows. Such tools would allow a scientist to save a great deal of time when preparing plots for publication. Many of the desired features would be MacDraw-like. Other examples of interactive drawing software have long existed for Macs and PC's; in an X environment, analogous features exist in xfix or idraw on Sun SPARCs. We encourage the porting of these or similar codes to the RS/6000 and support by the CNSF, to give researchers the ability to do the following: Position yourself anywhere on a plot with a cursor. Write there with a font size that you can control. Circle any part of curve and move it. Draw simple curves e.g. ellipses, rectangles ... In addition, it should be easy to adjust lengths and fonts, and in general, to tailor your plots for publication. GRID GENERATION For computational fluid dynamicists a tool for viewing and analyzing a grid is an absolute necessity. Starting from a given set of coordinates (x,y,z) with some imposed ordering, one needs to be able to generate a wire-frame mesh, with the possibility of viewing any given planar projection of it. MULTI-DIMENSIONAL AND COMPLEX DATA Tools for visualizing complex and multi-dimensional data would be extremely useful to large numbers of scientists, particularly if the tools were easy to use and readily available. Some examples: (1) For visualizing complex data, make a false color 3-D surface plot that would allow you to use the height of the plot for the magnitude and a color for the phase. (2) Given a set of weighted, randomly positioned points in n-dimensional space, plot a contour of the weighted density of points on any specified 2-dimensional plane. (For this to make sense the points have to be sufficiently dense, so in practice, the dimension of the space "n" will be 2 or 3.) This would be useful for people doing Monte Carlo simulations. ANIMATION Tools for creating simple animations should be made available to the researcher, for use on his/her workstation. REAL-TIME VISUALIZATION From a FORTRAN (or C) program, one ought to be able to open a window and plot out data, so that one can watch some distribution evolve as the program runs. One should also be able to save plots to image files so that one can re-create the plots if desired. The data interface between the FORTRAN program and the graphics package should be simple. APPLICATION SPECIFIC Several disciplines, notably computational chemistry and computational fluid dynamics, have a need for highly sophisticated and specialized visualization software. Some of these needs are indicated below. CFD software: Good CFD software has been developed at the NASA centers. They run on Irises but are not available on RS/6000 or Suns. It would be useful to either port these codes or convince IBM to port them to the RS/6000's. (Examples are PLOT3D, FAST, GAS, SURF, etc.) Computational chemistry graphical user interfaces: In some areas of computational chemistry distributed computing and sophisticated graphics on high-end graphics workstations are rapidly becoming the norm. This is one area where public domain code will probably not have much impact. In the area of molecular modeling, commercial packages like INSIGHT or QUANTA are more and more in use. Graphical user interfaces for quantum chemistry are under intense development. Currently, sophisticated packages either exist or are being developed both by software vendors and by computer companies like STARDENT and CRAY. San Diego Supercomputer Center will be making UNICHEM, the CRAY/Iris package developed by CRAY, available to its users. What the CNSF does in this area will depend to some extent on what IBM does, what gets ported to RS/6000's, and what kinds of licensing can be arranged with vendors. Nevertheless, this is an important and competitive area in high performance computing which will need to be addressed. LIST OF SOFTWARE PRODUCTS REVIEWED ---------------------------------- A variety of sources were consulted for candidate packages that might meet the above criteria. The principal sources of information were: articles or advertising in professional journals (Computers in Physics, plus the Oct. '91 Supercomputing Review); the Supercomputing '91 conference in Albuquerque, NM; and electronic mail "word-of-mouth" from Theory Center staff. The final list is recorded below together with vendor or supplier names, and phone numbers or e-mail addresses. It has been broken down by category: Software Packages Installed and Tested - Commercial Product: Vendor name: Telephone number: IDL Research Systems, Inc. (303) 786-9900 UNIGRAPH+2000, etc. UNIRAS, Inc. (214) 980-1600 PV~WAVE Precision Visuals, Inc. (303) 530-9000 PV~WAVE point & click Precision Visuals, Inc. (303) 530-9000 IMSL/IDL IMSL (800) 222-4675 Software Packages Installed and Tested - Academic or Public Domain Product: Supplier: E-mail address: SciAn SCRI, Florida State Univ. scian-info@scri.fsu.edu xmgr Paul J. Turner, Oregon Grad. Inst. pturner@amb4.ese.ogi.edu anonymous ftp: amb4.ese.ogi.edu Software Packages Researched But Not Tested - Commercial Product: Vendor name: Telephone number: FieldView Intelligent Light (201) 794-7550 4D-SDV IBM-ECSEC, Via Giorgione 159, 00147 Roma, Italy Transform, Dicer, Format Spyglass, Inc. (217) 355-6000 XMath Integrated Systems, Inc. (408) 980-1500 IMSL C graphics IMSL (800) 222-4675 Software Packages Reviewed But Not Tested - Academic or Public Domain Product: Supplier: E-mail address: Khoros Dept. of EECE, Univ. of New Mex. khoros-request@ anonymous ftp: pprg.eece.unm.edu chama.eece.unm.edu user=ftp, your address as password BRIEF SUMMARIES OF ALL THE PRODUCTS ----------------------------------- > Software Packages Installed and Tested - Commercial IDL IDL stands for Interactive Data Language. The user interface is a command- line interpreter, so users can create complicated graphics images step-by-step. The language is scriptable, i.e., commands can be "recorded" and later "played back", to reproduce a given plot. Commands can also be taken from a file. The language is highly programmable; its syntax is rather reminiscent of FORTRAN. The advantage of an interpreted language is that it eliminates the compile step in the programming cycle. This is a real boon, because compiling graphics programs often results in quite a long wait, but quick feedback is what is most helpful when creating graphics. But, the absence of a true compiler also means that numerically intensive tasks are generally slow in IDL. IDL has nearly all of the features a "bread and butter" graphics package ought to have. Here is a complete list of its features. * Overall 1. scriptable command-line interface 2. can create own OSF/Motif widgets 3. output to PostScript & other standards for hardcopy * 1D Data 1. x-y plotting: including log-log, log-linear 2. curve fitting (beyond simple least squares - SVD, e.g.) and splining 3. Fourier filtering 4. customized labeling capability * 2D Data: f(x,y) 1. contouring (NCAR or DI-3000 quality) a) standard & color-fill b) wire-frame surface projection c) shaded surface (simple illumination model) d) "interactive view" with x-y locator 2. color rasters (like NCSA Image) a) customizable palette b) "histogram equalized" and other image processing 3. combinations a) 1 & 2 can be overlaid or made into composites b) 2D Fourier filtering c) animation: adds +1 dimension to all plot types d) map projections of all types * 3D Data: f(x,y,z) 1. slicer (like Spyglass Dicer for Mac) 2. isosurfaces and rendering (recently implemented) * 2 X 2D Data: f(x,y), g(x,y) 1. overlay color rasters on contour-type plots 2. vectors (arrows) They also provide an extensive User's Library with tools to do a number of things, including an X widget to do animation; mapping; and statistics. UNIRAS This is actually a suite of individual applications--UNIGRAPH+2000, UNIEDIT 2000, PICTURE MANAGER, and C subroutines for making widgets (agX/Toolmaster). UNIGRAPH+2000 is a plot generator program and is similar in many ways to IDL, but the user interface is more Macintosh-like, with point-and-click menus (and sub-menus, and sub-sub-menus) plus the usual command line. Also like IDL, it puts you in "its own" environment which isn't just an extension of your Unix shell, so shell commands are available only indirectly. UNIGRAPH+2000 is scriptable as well, but with more difficulty, because one must identify a desired command hierarchically through the menu structure in a form like: Menu-Submenu-Subsubmenu param1 = xxx, param2 = yyy, etc. This makes script writing from scratch (via an editor) somewhat unwieldy; but this limitation is common to all current point & click programs. You can record your menu choices as you go, but if you make a mistake along the way, it's difficult to edit the script that results. As in any full-featured point-and-click program, there are many menu selections and their names do not always give a clear indication of what is comprised, so navigating the menus gets to be a problem. Of course, naming menu items is inherently problematic--UNIRAS probably wanted the names to be intuitive, but one person's intuition is another person's confusion. UNIEDIT is a unique tool. One could call it a "MacDraw for X Windows." It would be useful for "decorating" (adding enhancements to) plots generated in UNIGRAPH+2000, as a final production step. It's probably the most impressive tool of the lot, but even so, mouse operations like rescaling and rotating are counter to "normal" intuition (if your intuition comes from using a Mac). PICTURE MANAGER is a specialty tool for dealing with different types of image data, CGM, PostScript, etc. It could be useful as a conversion tool. agX/Toolmaster allows you to create X widgets from C programs. It's mainly a subroutine library, which is a graphics paradigm that's hopefully becoming obsolete (at least for the purpose of this study). Clearly it could be nice for building user interfaces to simulations (a FORTRAN version does exist) but there's always the question if it really communicates fully with the shell, as would be necessary for true applications steering. The documentation for the full set of the above takes up about 3 feet of shelf space. Curiously, a 3-page, faxed "tip sheet" seems to cover what you need to know pretty well ... there is simply no excuse for burdening a user in 1992 with so much verbiage, whatever the virtues of the software. PV~WAVE This package is more than similar to IDL: it IS IDL, in an earlier form! Precision Visuals, Inc. bought the rights to the IDL software from Research Systems, Inc. a few years ago. They have repackaged it and rearranged chapters in the User's Guide but have done little else to it. It lacks a widget builder (but they are supposed to come out with one soon). IDL's widget builder was added after PVI bought the source from RSI. PVI claims to have better support, since they are a larger company, and they claim to have better connectivity to FORTRAN and C applications. There is a PV~WAVE:GTGRID for generating irregular grids (useful for aerodynamics, e.g.) and PV~WAVE:NAG which gives you access to the NAG subroutine libraries for enhanced mathematical functionality. PV~WAVE POINT & CLICK This gets a separate listing because it is more than just a fancy version of PV~WAVE. Like PV~WAVE, it's based on IDL, but you do not have the option of a command line in the P&C version--you must go through the WIMP (Window- Icon-Mouse-Pointer) interface only. This really is too bad--because the two combined would have been pretty unbeatable. As it is, the two products might as well have different names, because they work entirely differently! It's almost misleading to call PV~WAVE point & click a "version" of PV~WAVE. The point & click interface is very nice, although it suffers from the same navigation problems that UNIGRAPH+2000 does. The scripting seems to be equally difficult. The list of features is the same as for PV~WAVE. The major disappointment is the lack of access to the PV~WAVE (IDL) command line. The documentation is pleasantly compact, plus, a lot of what one needs to know can be deduced by browsing the menus (although this is not always possible). IMSL/IDL IMSL joined forces with Research Systems, Inc. to produce this software, an enhancement to IDL as described above. A sizable fraction of the famous IMSL mathematical function library is now built directly into IDL. This makes plain IDL a more attractive product; it allows you to combine data analysis with math analysis at a sophisticated level. Plus, IDL's rather weak documentation (a too-thin introduction plus a too-thick User's Guide) has been substantially improved by the IMSL people. > Software Packages Installed and Tested - Academic or Public Domain SCIAN Turning to academic software, SciAn was put together at Florida State to render 3D data sets. It is intended as a front end to scientific animation. 3D rendering can be can be done by a volumetric technique, with user-specified colors and translucency (for scalar fields), or 3D arrows (for vector data). It can also do ball-and-stick models and has a 3D slicer. On top of this it has many 2D functions like false-color imaging, surface elevations, and colored surfaces. But it has significant limitations. First, it doesn't always accept data that isn't in the NCSA HDF format. Second, it is very device-dependent. (We only got it to work on z-buffered IBM displays having the GL graphics library; it also works on the Silicon Graphics Iris 4D). But in general, SciAn has a LOT of capabilities, AND it is free for the asking. Among its advertised features are scripting, shading with full-color light sources, perspective and orthographic viewing, annotation and labeling, a color palette editor, fully interactive rotate-pan-zoom, stereo viewing, a flight simulator (!) ... the kitchen sink ... seriously, they have packed in quite a lot here. They've got plans to get SciAn to interact with FORTRAN and C programs via IP sockets for application steering. If you have the right hardware, this has a lot to offer. (There did seem to be some amount of bugginess when we were testing it out.) XMGR/XVGR These are basically x-y plotting tools for X-based workstations, with a nice point-and-click interface for ease of customization. They are, in this respect, reminiscent of popular programs for PC's and Macs--KaleidaGraph or CricketGraph comes to mind. Again, this is academically-produced software which can be freely ftp'ed. Xmgr is for Motif; xvgr is for XView (Suns). A full listing of its capabilities follows. * Motif and XView versions. * Plots up to 10 graphs with 30 data sets per graph. Each data set is limited by the size of available memory. The 10 graphs may be displayed simultaneously with a popup that allows the graphs to be conveniently stacked, in rows and/or columns. * Data read from files and/or pipes. * Graph types XY, log-linear, linear-log, log-log, bar, stacked bar charts. * Sets may be drawn with error bars in both directions equal or unequal on either side of the datum in X and in Y. Support for HI LOW OPEN CLOSE data. Numerous symbol types, both hollow and filled, skyline plots, histograms, impulse. Sets may be drawn as polygons filled in several different ways. * Block data feature for those data sets arranged in equal length columns allowing sets of any type to be formed by any combination of the columns. Presently there is a 30 column limit, the length of columns is limited by available memory. * Continuous display of the location of the pointer depending on current graph scaling. The format of the display can be set to any of the time-date formats, latitude/longitude. The default is to display the XY coordinates of the mouse in user coordinates. As the graph focus changes, so does the format and scaling of the locator. * User-defined scaling, tick marks, labels, symbols, line styles, colors. Tick labels may be drawn at any angle on either or both sides of the graph, staggered (offset for long labels). There are several formats for tick labels, including many time-date formats, latitude-longitude, decimal, exponential, and power-of-10. Tick labels and tick marks may be specified by the user. Grid lines may be drawn in lieu of tickmarks. * Each coordinate direction allows 3 axes: primary, zero, and an alternate axis that allows a scale differing from the graph coordinate scaling. * Graph legends, with an optionally filled frame. * Annotative text with sub/super scripts, lines with/without arrow heads, boxes filled or unfilled. * Operations on data sets, kill, copy, move, reverse, sort, drop, join, coalesce, write, swap. * Region operations, define a region as inside or outside a polygon, or half planes defined by a line. Extract points from a region to a set, delete points, evaluate an expression on points in a region. Area and perimeter calculations from a polygon. * Mouse powered point editing, add points to a set, delete points from a set, move a point in a set (horizontally, vertically, or in both directions). There is a feature called tracker that when active, warps the pointer forward or backward through the points in a set, displaying the set number, the index of the point, and the X, Y location of the point. There is a find point feature that allows information to be displayed about arbitrary points in sets. * Batch mode for unattended plotting. * Read and write parameters used during a session. * Polynomial regression, splines, running averages/minimums/maximums/ standard deviations/medians, DFT/FFT, spectrum, phase, sampling, cross/ auto-correlation, numerical integration, numerical differentiation, histograms, evaluate expressions on sets. * Pan, zoom (mouse powered), shrink and expand the graph scaling, with each graph allowing 20 different views through a mechanism called the world stack. The graph viewport (where it appears on the page) can be set manually or by the mouse. * Command line interpreter with a history list and playback feature. * Autoscaling along each or both coordinate directions, on all sets on a selected set. * Support through device independent graphic drivers for PostScript, HPGL, and FrameMaker .mif format. Frame and xmgr make a powerful combination. > Software Packages Researched But Not Tested - Commercial FIELDVIEW Intelligent Light cooperated with fluid dynamicists at United Technologies to come up with this CFD-oriented package. It is designed to compute a variety of scalar and vector functions of 3D field data and to represent the results as cutting planes, isosurfaces, arrow fields, contour lines, streamlines, and particle paths. The code is supported on IBM RS6000, Silicon Graphics, Sun SPARC, and HP 700 platforms. The interface is a panel with sliders and buttons rather than a command line. It has scripting, shading, annotation, and gridding capabilities. It appears to have a very thorough set of functions for viewing fluid dynamics data. 4D-SDV (IBM-ECSEC) This is billed as "An Advanced Visualization Environment for Three- Dimensional Simulations using Cooperative Processing for Animation." The idea is that visualization is too compute-intensive to run on the same workstation with a simulation, so, run the visualization separately, and the two codes can communicate via IP sockets. This enables one to see simulation results as they are being produced. The visualization is based on graPHIGS in order to run on IBM (of course!) equipment. It allows for a variety of coordinate systems in representing 3D scalar fields, and it can do slicing, coloring, light sources, multiple plots, and scripting. The user interface is a combination of menus and slider widgets. 4D-SDV can do spin fields and ball-and-stick modeling. There is a video server as well. SPYGLASS Here is a suite of tools for the Macintosh by Spyglass, which should be coming out in an X Windows version in the near future. These tools are very much like ones created by NCSA some years ago. The names of the programs are: Transform, Dicer, and Format. Transform is good for functions of 2 variables and does the usual set of plots (contours, false colors, surfaces). It can handle a variety of data formats including image data (PICT, TIFF, etc.). It can also do 2D vectors. You can create templates to save your plot commands. There is a color palette editor. Output is in PostScript or it can be exported to other Mac applications. Dicer adds to this list by including 3D functions in both volumetric and data-slice representation. It has animation capability. Format completes the suite by letting you do annotation. This is about the best you can get for the Mac, currently. XMATH XMath is a newcomer to the field. It is math software, but more and more math software is coming with sophisticated graphics (Matlab and Mathematica are two others in this category). Since it is mostly designed around matrices, it cannot handle irregular data or functions of 3 variables. But it can do simple shading and light sources on surface plots. Its MathScript feature is meant for numerical programming. But, IMSL/IDL might be superior in its combined math and graphics programming. XMath, however, is relatively inexpensive. IMSL C GRAPHICS IMSL has introduced (or will introduce) a C-callable subroutine library for doing graphics in conjunction with their C/MATH/LIBRARY. This will supposedly be more powerful than, say, NCAR graphics, because one can click in a graphics window to change a plot's characteristics on the fly. However, since this is a subroutine library it is not interactive in the truest sense. At last report, IMSL C graphics are in the beta-test stage. > Software Packages Reviewed But Not Tested - Academic or Public Domain KHOROS As a guess, its name is derived from the Greek word for "chorus"(?). It is designed to enhance communication between researchers. The programming model is a data flow graph (or as they say, a visual programming language), called "cantata". You specify a program by connecting glyphs in a net. There is also a command line interface. It is based on Athena widgets, note; but it does run on a variety of platforms. It primarily handles image data, but it can also do transforms, and it has some (unspecified) capabilities for visualizing 2D and 3D data sets. EVALUATION: ADVANTAGES AND DISADVANTAGES ----------------------------------------- The suitability of each of the above packages for the CNSF was analyzed from the standpoint of (1) the criteria of the CSERG report and (2) whether or not the software could be run on available equipment. Of primary interest was software that could be run on the CNSF's cluster of RS-6000's. What follows next is thus a brief summary of the strengths and weaknesses of each of the above packages, when considered specifically as a potential general-purpose visualization tool for the CNSF. Detailed comments by CNSF staff on software that underwent testing are included in an appendix to this report. The organization of the material below is identical to that of the previous section. > Software Packages Installed and Tested - Commercial IDL (evaluation) This was the first package to undergo evaluation, and it made a wonderful first impression. A closer look, though, revealed potential trouble spots. Of particular concern were: the idiosyncrasies in its syntax, which would make it a difficult language to learn and teach; inadequate documentation; and some bugs that turned up in the trial period. Also, the issue of getting reliable support from a small company was raised. Otherwise, it did seem to meet most of the stated criteria. More and more recommendations of IDL kept piling up as other, rival packages were being evaluated. These included very positive statements from scientists (really, trusted friends) at: * JILA, Univ. of Colo.-Boulder ("Don't buy anything which isn't IDL. IDL does pretty much everything you could possibly want with style, elegance and grace with only a few inbuilt quirks.") * Cornell's Astronomy Department (a professor was using IDL over the network, by running it remotely on the workstation of a colleague!) * Air Force-Phillips Laboratory * NASA-Goddard Space Flight Center * Cornell's Laboratory of Plasma Studies (a supporting agency actually gave LPS the money to buy IDL when they found out the lab didn't yet have it!) No other package received any outside recommendations at all--let alone ones as favorable as these. UNIRAS (evaluation) The UNIRAS tools received much less testing than IDL, for a good reason: the trial period was almost over before it was working properly. There was a great deal of trouble in getting it to work with IBM Xstations, and the fixes that were eventually applied were just jury-rigs. For instance, a substitute font had to be used, which was on occasion too big for the allotted space in the window; and the color map had to be dithered. This is not what one hopes for from software "straight from the box"! The UNIRAS support was relatively slow in getting back with the fixes for such complaints, too. This, plus the comparatively awkward interface of UNIGRAPH+2000 advised against obtaining the UNIRAS products. PV~WAVE (evaluation) The PVI salesman argued that, despite the fact that PV~WAVE is merely IDL resold under another name, his company could give better support and training, and that enhancements would come faster. None of this was borne out in the trials, which made it clear that essentially nothing had been done to IDL since PVI picked it up. In fact, IDL from RSI, the smaller company, had clearly been enhanced substantially since PVI had acquired the earlier version. Those of us who develop software for a living know too the advantage of sticking with a code's original authors; PVI seems here to have the role of a less-than- knowledgeable reseller of someone else's old software. PV~WAVE POINT & CLICK (evaluation) Even if this were a great product, it would be unsuitable for the CNSF as it currently runs only on Sun workstations. But even so it has significant limitations; of these, its lack of communication with the PV~WAVE's command line is proabably the most serious. As with any menu-driven software, having to do everything "their way" with no shortcuts can sometimes drive you crazy. But the real clincher in deciding between PVI and RSI can be found in the next paragraph. IMSL/IDL (evaluation) This product did not even exist when the search began, but fortunately it was announced in early 1992, in the thick of the hunt. Many of the perceived weaknesses of the original IDL package were overcome by its combination with IMSL. First of all, the alliance of RSI with IMSL, a big company with possibly the longest track record in the software business, allayed worries about the consistency of support and training. In fact, documentation for this package was already noticeably better than that which had come with IDL the first time around. Secondly, the addition of a lot of interesting math functions makes IMSL/IDL almost the equivalent of a Mathematica (minus the symbolic algebra capacity, but plus more potent graphics--and rumor has it that Maple will soon be incorporated into IMSL/IDL, so algebra is on the way.) The Theory Center's longstanding cooperation with IMSL enhanced this package still further, so it was eventually chosen to be the supported graphics package on the RS/6000 cluster. > Software Packages Installed and Tested - Academic or Public Domain SCIAN (evaluation) The functions were not analyzed in detail; the fact that this package only ran on certain X displays (with some reported bugginess) made it unworkable for present purposes. Nevertheless it remains an attractive package on nearly all other counts. XMGR/XVGR (evaluation) Since xmgr offers only x-y plotting, it is insufficient for use as a stand- alone, general graphics package. But what it does it appears to do well, and like SciAn, it is free of charge. > Software Packages Researched But Not Tested - Commercial FIELDVIEW (evaluation) For computational fluid dynamicists, this would possibly be a top choice (if one could afford the cost). But for non-CFD users its functions are too specialized. 4D-SDV (evaluation) Again, this software is designed to fill a specialized niche. It focuses on a few kinds of advance graphics but has little basic functionality. It is a good choice for IBM'ers doing 3D simulations with application steering. SPYGLASS (evaluation) When this comes out in a Motif version it should be interesting. Hopefully all the functions will be put in a unified package at that time (why not?). For now, a Macintosh product is not what is sought. XMATH (evaluation) This package is perhaps a bit short on the graphics end. The restriction to but a single data type (matrices) is quite limiting. IMSL C GRAPHICS (evaluation) IMSL is not yet offering this beyond the beta test version. Even so, it does not seem to represent a new direction in graphics to the same extent that some of the other reviewed software does. It is a subroutine library; and even if it does have some built-in interactivity, it is questionable if its level of interactivity is enough. It seems, at this point, that it is more desirable to remove the compilation phase from graphics programming altogether. However, it may be that this way of doing things offers better accessibility to/from the Unix shell. > Software Researched But Not Tested - Academic or Public Domain KHOROS (evaluation) The emphasis of this package is on science collaboration via software, rather than just graphics. It may be going to far to require users to get into a whole new working environment when they just want to do the occasional plot. The graphical programming language looks nice, but probably has nothing more to offer than does Data Explorer on the Power Visualization System, which the CNSF already has. There are many interesting image processing capabilities. Their use of the Athena widget set vs. Motif is a slight drawback. SUMMARY AND RECOMMENDATIONS --------------------------- This search was seeking to fill a particular niche: a need for a set of workaday graphics tools that would assist the computational scientist with his or her routine data visualization and analysis tasks. It was not expected that the selected tools (if any) would necessarily have the ability to make, say, highly complex and beautiful 3D-rendered images, for instance--because tasks like these are still relatively difficult and specialized at this date. It was seen as more important to provide researchers with tools that were simple and flexible enough so that he or she could take a "quick-and-dirty first cut" at a data set. The detailed requirements for this software were spelled out in a CSERG report, which has been reproduced in the Introduction to this Technical Report. The main points or criteria of the CSERG report are contained in the following list of attributes which "bread and butter" graphics software should have: * Availability at low cost * False-color imaging * Machine independence (Unix) * Control of color palettes * Display device independence (X) * Elementary surface plotting * A variety of data formats * Data fitting and data analysis * Customizable line plotting * Truly interactive speed * Fairly general contour plotting * Relatively ease of use In addition, CSERG identified a list of features that were desirable but not essential. They were: * A point-and-click interface * Animation * Drawing tools (a la MacDraw) * Application steering (real-time) * Grid generation (CFD) * Other discipline-specific needs Many of the reviewed packages hit most of the points on this list, and that being the case, it was clearly advantageous to see if a single package could fulfill all the requirements (to simplify training and support). Of the ones researched, some did not adequately cover the "low" (x-y plotting) end of the scale: these would be FieldView, 4D-SDV, and Spyglass. Others, like xmgr and Xmath, did not adequatedly cover the high end. In general, academic software was often more tailored to specific applications or specific hardware. Hardware-constrained packages included SciAn, 4D-SDV, and Spyglass. Finally, some packages just had different emphases that didn't match overall with the CSERG plan, like IMSL C graphics (a subroutine library) or Khoros (really a user environment, of sorts). This left the bigger commercial packages. None of these was perfect, but IMSL/IDL had the edge because of the IMSL alliance and because the original developer still had control of the product. Its programmability surpassed the point-and-click interfaces, even though the latter tended to be friendlier. Adding to these factors the strength of the recommendations that were received from scientific users elsewhere, IMSL/IDL came out a clear favorite. But were its capabilities sufficient to justify buying it? Checking against the above list, IMSL/IDL really does satisfy every one of the basic points and adds a couple of the advanced ones (animation, user-definable point-and-click widgets, some others). A recommendation was made to purchase this package as the supported general-purpose graphics tool at the CNSF. This does not mean that IMSL/IDL is the last word in scientific graphics. It has shortcomings; early experience with undergraduates who have been using IMSL/IDL in the summer of 1992 in the SPUR program has borne out some of the worries about the difficulty (quirks) of the language, errors in documentation, etc. But it is also clear that IMSL/IDL is a powerful package with lots of potential. In other situations, other recommendations could be made. For instance, the free software that was investigated turned out to be surprisingly well- constructed and for some people could fill an important niche: as a simple x-y plotter xmgr is quite attractive; on the right hardware SciAn would be pretty impressive; and Khoros is really interesting in its own right. FieldView would be hard to beat (though expensive) for CFD, Spyglass might be right for a Mac user, and so on. Nothing that was reviewed here could be called a "turkey". And, in all likelihood, things will keep getting better--which is good news for computational scientists, who have been waiting a long while for just this kind of development in software. Now, if there were only something with a truly intuitive point-and-click interface to do all of this ... APPENDIX: REVIEWERS' COMMENTS ------------------------------ Before launching into this compilation of the reviewers' comments, it would be appropriate to thank both the reviewers, and everyone else at the CNSF who made a contribution to this report, by name: CSERG: Evi Goldfield, Cyrus Umrigar, Greg Cook, Andrew Abrahams, Yoram Yadlin, Jianguo Wu, and the author (Steve Lantz). Consulting Services: Ray Kujawski, Blaise Barney, and Mark Scannapieco. Many others gave their feedback, but the above contibuted written material and reviews that have been incorporated into this report. A note of thanks also goes to Edward Nicolescu (CNSF) and Roger Jagoda (LPS, Cornell) for helping to obtain access to IMSL/IDL and PV~WAVE, respectively. This report now concludes with minimally-edited comments from reviewers who participated in the graphics software trials. IDL Blaise Barney: 1. FEATURES TESTED The following are IDL procedures that I created to test various features of the package. These are all located on titan in "/u/lost+found/idl", in the bbarney subdirectory, in case anybody wants to review them later. gauss.pro Plots a bell curve in 3D wire mesh shaded.pro Plots a shaded bell curve in 3D with color bar.pro Plots a bar graph flash.pro Reads input file (idlinput.dat) which was generated from C program (idlinput.c). Plots bar chart and loops forever through the standard color maps 3flash.pro Splits window into 3 parts Reads input file (idlinput.dat) which was generated from C program (idlinput.c). Plots 3 bar charts and loops forever through the standard color maps 3in1.pro Divides window into 3 parts and plots a sine wave, a noisy sine wave, and a surface plot. sine.pro Makes sure window is set to one; plots sine wave maps.pro Orthographic globe projection that accepts mouse input to rotate globe to desired location. postscriptmap.pro Creates postscript file of map projection. consplt.pro XY plot of consulting data from input file cons9091.dat Selects colors and creates labels bmb.pro 3-D shaded (color) surface plot, values read from input file bmb.dat. anim1.pro Reads input file (anim.input) and creates a 20 frame output file (anim.output) of a changing surface plot that can be used by anim2.pro to run a "movie". Window size is 500 x 500 pixels = 250,000 bytes. Output file size is 20 frames x 250,000 bytes = 5Meg. anim2.pro Reads the 20 frames output file (anim.output) from the anim1.pro program, and runs a movie of a changing surface plot. Other things (more general) tested: Command Line interface Xsurface routine Xinteranimate widget-based tool Access to operating system from within IDL by SPAWN command 2. COMMENTS On the "plus" side: 1. X Window based help screens - nice 2. Time to create my first (very simple) IDL main program, compile and run took about 15 minutes. This isn't too bad. 3. It is full featured and powerful. 4. Many complex things are easy to do (but then, some simple things are hard to do!). 5. It's better than anything else that I've had a chance to work with and certainly better than nothing at all. On the "minus" side: 1. It is yet another programming language to learn. 2. The documentation is inadequate, especially in the "helpful examples" department. 3. Warnings of potential performance pitfalls from the IDL manuals give me hesitation. Quote from chapter 9: "Due to the ability to specify complicated computations at a high level inefficient IDL programs SEVERELY penalize the author - perhaps MUCH more so than with most other languages." (emphasis added). 4. I would never want to see Consulting have to support this product because it is a programming language/tool - with syntax that is is non-intuitive and poorly documented. 5. Came across some obvious bugs. 6. Not really portable - because of license - which means you have to do your Viz. work on one machine (or whatever your license permits). 3. BUGS 1. IDL will terminate due to an X error if the display window is moved during an animation sequence (sometimes) 2. During animation, the Xwindow display will "get messed up". "Shadows" of other windows will overlay part of the animation display window, or, "black space" will overlay part of the animation display window. This seems to happen (sometimes) when windows are raised, lowered or moved during the animation display sequence. Mark Scannapieco: I would not recommend it as a tool to generate interfaces. Although it does more than xgen, certainly procedurally and with the shell, it is not portable or in the public domain. Something that generated X or C code would be preferable. The package's strength appears to be its facilities to easily generate fairly complex 2 and 3 D, contour and shaded plots. Also, IDL can generate a variety of postscript forms for inclusion into documentation such as LaTeX or for direct printing. The documentation is adequate for the programming and ploting aspects and adequate to poor for the user-interface / widget builder. For example, there are references to nonexistent appendices. The whole package would benefit from more examples. Steve Lantz: Syntax is not easy to learn - comma/parenthesis placement and command names are sometimes "surprising" Building interfaces is not much different from C programming in difficulty - IDL syntax looks more like Fortran, but that's a matter of taste Things like widgets and animation may be done better by other tools (e.g., swb, or flipbook) Specific colors aren't very easy to choose It's easy to make a default contour plot--much harder to customize it May be better to focus on something to make contour plots and surface plots. IDL may be trying to do too much Some bugs were found UNIRAS Steve Lantz: Unigraph+2000 from Uniras was made available for testing, actually it was up and running on titan for nearly 2 weeks, but there was a big problem in getting it to work on xstations. It only ran on titan's console or non-IBM X workstations. Uniras was not forthcoming with a fix, which argues against this package. One nice feature of the software was uniedit, a MacDraw-type drawing program that lets you add fancy doodles to your data plots. It also came with agX, a graphics subroutine library; but I would reject agX in favor of the nicely-interfaced unigraph, uniedit, unietcetera. PV~WAVE Greg Cook: (Questions with answers from D. Ditch, the PVI salesman -SRL) Isn't PV~WAVE a close cousin of IDL? Yes, but Research Systems Inc. refused to develop PV~WAVE along the lines that PVI wanted--so PVI bought the source code for a big lump sum. What does PV~WAVE have that IDL doesn't? Essentially nothing. Better support, possibly better future developments. PV~WAVE has a point'n'click "version" that's actually a completely separate and incompatible piece of software. Why did IMSL choose IDL over PV~WAVE? PVI approached IMSL several years ago with a proposal for providing graphics for IMSL software, but were rebuffed. PVI then went to NAG instead, and put together PV~WAVE:NAG(!); IMSL/IDL came later, after IMSL "woke up." Steve Lantz: PV~WAVE is very similar to IDL, in fact, they started life as one and the same product, but the two companies involved had differences of opinion over future development, so evolution over the past 1-2 years has moved the products farther and farther apart. I would say the main differences are: - PV~WAVE has a true point & click version (not yet available for AIX/Motif). This is a totally separate entity from the command line version I installed on titan. IDL lets you build X widgets, but has little in the way of built-in p&c features. - PV~WAVE has no widget builder currently, but will soon. They scrapped their first attempt as unworkable. IDL has widgets that are OK. - PVI is the bigger company by at least a factor of two if not more. I'd expect faster service and future developments from PVI as compared to Research Systems, Inc., but perhaps less cooperation in development. - They have joined forces with different math/statistics software houses. IDL has IMSL/IDL; PV~WAVE has PV~WAVE:NAG (not installed here). - I have determined that the IDL and PV~WAVE manuals are EXACTLY THE SAME. The chapters have been reordered, is the biggest difference. There a few other slight differences. I agree with what Blaise lists below as the deciding factors. I think we also need to assess whether it would be more useful to have point'n'click (Wave), or X-widgets/mapping/statistics (IDL)--these being the most recent developments in the two packages. Blaise Barney: My observations to this point is that there's a lot of similarity between IDL and PV~WAVE. I didn't get a chance to evaluate the reference manual for PV~WAVE, but Steve and Ray did that. My opinion at this point is that either PV~WAVE or IDL could work for us. The other issues (cost, DX, TC-directions, etc) will probably be the deciding factors.