New features in Ferret and PyFerret v7.2
1. Updated Memory Management
For version 7.2, significant enhancements were made to how Ferret manages memory. Instead of allocating a chunk of memory on startup, memory is allocated only as needed. SET MEMORY now simply specifies an upper limit on memory which will be allocated. See the new section in the Users Guide entitled Understanding Ferret Memory Management Concepts and Tools. Here are the new tools that enable users to interact with memory management
- MODE FRUGAL is added.
- SHOW MEMORY now reveals the current cache size and the peak "essential" memory that has been in use.
- SHOW MEMORY/DIAGNOSTIC reveals the memory management choices that were taken during the previous command that read or computed data.
- The current cache is no longer cleared when SET MEMORY/SIZE=<size> is issued.
- MODE DESPERATE is deprecated. Use of the command no longer has any effect.
- The new symbol ($PEAK_MEMORY) is always set; it contains the peak amount of “essential” memory that has been required since the last time this symbol was zeroed by SET MEMORY/SIZE=<size> or CANCEL MEMORY
2. More computations are broken into pieces.
Ferret has always done calculations in chunks where possible for improved performance. For instance, to compute an area average from data on an XYT grid to return a time series, it may read in subsets in the time direction and computes the area average for each set of timesteps before reading and computing the next set of timesteps. Now, transformations such as AVERAGE or MAXIMUM, which compress the result along an axis, can flexibly do the computation in chunks along the transformed axis if that is most efficient. For instance, if our operation is an average in time of our XYT grid, the entire time series need not be loaded at once.
3. Scaleability enhancements
Defining aggregations on the fly means we may need to open many, many datasets, which define many grids. Data structures for handling internal information about datasets and grids are streamlined in v7.2, and these limits are raised:
- The number of datasets that can be open at a time is increased from 400 to 5000
- The number of different grids defined at one time from 500 to 5000
4. New symbol ($SPAWN_STATUS) is always defined.
It records the return flag of the last SPAWN command. 0 = Success = 0; nonzero = Failure. It is particularly useful in scripting, for example, to test for the existence of a file. On startup, its value is 0.
SPAWN ls myFile.dat IF `($SPAWN_STATUS) EQ 0` THEN ! stuff to do if myFile.dat exists ELSE ! stuff to do if the file does not exist ENDIFn
3. New function DOT performs a dot product (matrix multiplication).
yes? show function dot DOT(VAR1,VAR2,IDIM) Compute the inner product of two variables along given dimension VAR1: Variable 1 VAR2: Variable 2 IDIM: Dimension (e.g.1,2,...)
DOT is one of a new class of functions: direction-dependent functions. One or more arguments are designated to determine the direction of the function's operation. In fact, there are functions DOT_X, DOT_Y, ..., and DOT calls one of those based on the value of argument 3.
4. New function TRANSPOSE transposes data in specified directions
yes? show function transpose TRANSPOSE(VAR,DIM1,DIM2) Compute the transpose along two given dimensions VAR: Variable to transpose DIM1: Dimension 1 for transpose (1,2,...) DIM2: Dimension 2 for transpose (1,2,...)
This direction-changing function calls one of the many functions TRANSPOSE_XY, TRANSPOSE_ZT, etc. It is written mainly as a demonstration of the concept of direction-dependent functions. Further development will give us shorthand methods for calling many of the sets of functions, e.g. in the future we will be able to to call XCAT, YCAT, ... using a single CONCATENATE function call.
5. PLOT/vs now draws time-formatted axis labels.
For a command such as "PLOT/VS xvar,yvar", if xvar and/or yvar have units of longitude or latitude, the plot axes are drawn with formatted Longitude or Latitude labeling. Now, if one of them has units of time, a formatted time axis is drawn:
yes? set var/units="DAYS since 1-JAN-1970" ttvar yes? plot/vs ttvar, myvar
Bug fixes in Ferret and PyFerret
1. For PLOT/VS, if the units are degrees but we do not want a labeled longitude or latitude axis, for instance if the degrees are a ship heading, then MODE LONG_LABEL or MODE LATIT_LABEL should be used to turn off the longitude/latitude formatting. PLOT/VS previously did not apply these modes; inV7.2 these do apply.
2. A bug fixed on the command "show attributes myvar", showing all attributes of a user-defined variable.
3. When accessing information using `var,RETURN=`, previously the whole variable was evaluated to access the information. For many RETURN keywords, the data need not be loaded. Now, commands such as `var,RETURN=title` no longer do this.
4. Adjustments are made when drawing a line plot whose value is constant, making the vertical range more reasonable.
5. A bug is fixed in the formatting of numbers returned with list/precision= reported on the Ferret Users List.
6. A bug is fixed when doing "striding" on a user-defined variable, also reported on the Ferret Users List.
7. A bug using the SAVE/RIGID command was fixed, where the attributes on the time axis were incorrect.
8. SET REDIRECT is used to send messages and output to a file. This failed if the file specified had a directory name that includes a dash. This is legal and is now fixed.
9. A bug is fixed with `var,RETURN=isready` which in some instances reported that an undefined variable was available to evaluate.
10. "Delimited" reading had a bug when some variables are requested to be skipped. This is fixed.
11. The command "ppl show" caused Ferret to stop. This is in fact erroneous PPLUS syntax. It is now handled as an error.
12. A bug is fixed when listing results of operations on string variables. This was reported on the Ferret Users List.
13. A bug caused the set of color levels computed by Ferret for FILL or SHADE plots leave uncolored some data at one end of the range of values. In adition the automatically-set symbols LEV_MIN, LEV_MAX, and LEV_DEL were not set correctly when the data magnitudes are very small. This is fixed. This bug was also reported on the Ferret Users List.
14. A parsing error is fixed for REPEAT loops. An extra parentheses appearing after a MESSAGE or PAUSE statement was not reported as an error, which resulted in overall incorrect parsing of the command and wrong results. Now this would be processed as an error. This bug was also reported on the Ferret Users List.