This page was created by the IDL library routine `mk_html_help2`.

Last modified: Sat Jul 26 18:17:38 2025.

# Directory Listing of Routines

# general/tools/misc

- AVERAGE
  Returns the average value of an array.
- AVERAGE_HIST
- DGEN
- DYNAMICARRAY__DEFINE
- ENCLOSED
  Returns the indices of a set of x,y points that are inside a contour.
- EXEC
  Widget tool that executes a user specified routine. This tool runs in the background.
- GETXY
  Routine that uses the cursor to select points.
- PRINT_STRUCT
  prints data in an array of structures.
- READ_ASC
  Reads data from an ascii file and puts data in an array of structures.
- RECORDER
  Widget tool that opens a socket and records streaming data from a server (host) and can save it to a file
- SPLINE_SMOOTH
  Uses spline smoothing to create a smooth curve through noisy data.
- SPLINECOEFF
- TMEAN
  Calculate the mean, median, and standard deviation of a 1-D or 2-D
- TOPSMOOTH
  Calculate a smooth curve through the upper envelope of a tplot variable.
- XY_EDIT
  Interactively select data points
- YYMMDD_TO_TIME
  Returns time (seconds since 1970) given date in format: YYMMDD HHMM

# Routine Descriptions

## AVERAGE

[Next Routine] [List of Routines]

    FUNCTION average(array,d [,STDEV=stdev] [,/NAN])
    PURPOSE:
       Returns the average value of an array.
       The input array can be an array of structures
       Similar to TOTAL, but returns the average over the given dimension.
       Also returns standard deviation via an optional keyword argument.
       Works with structures only if d eq 0

    Notes:

     $LastChangedBy: ali $
     $LastChangedDate: 2017-05-01 12:37:30 -0700 (Mon, 01 May 2017) $
     $LastChangedRevision: 23254 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/trunk/general/tools/misc/average.pro $

(See general/tools/misc/average.pro)

## AVERAGE_HIST

[Previous Routine] [Next Routine] [List of Routines]

    FUNCTION average_hist(d,x [,STDEV=stdev])
    returns the average of d binned according to x
    See "HISTBINS" for a description of keywords
    USAGE:
    assuming:
      x = randomu(seed,1000)*10
      y = 10-.1*x^2 + randomn(seed,1000)
      d = y
       avg = average_hist(d,x,xbins=xc)
       avg = average_hist(d,x,xbins=xc,range=[2,8],binsize=.25)
      plot,x,y,psym=3
      oplot,xc,avg,psym=-4
    NOTE:  d can be an array of structures:
      d=replicate({x:0.,y:0.},1000)
      d.x = x
      d.y = y
      plot,d.x,d.y,psym=3
      avg = average_hist(d,d.x)
      oplot,avg.x,avg.y,psym=-4

(See general/tools/misc/average_hist.pro)

## DGEN

[Previous Routine] [Next Routine] [List of Routines]

    FUNCTION:   dgen(n)
    PURPOSE:  returns an array of n doubles that are scaled between two limits.
    INPUT:   n:  number of data points.   (uses 100 if no value is passed)
    KEYWORDS:  one of the next 3 keywords should be set:
       XRANGE:  uses !x.crange (current x limits) for the scaling.
       YRANGE:  uses !y.crange (current y limits) for the scaling.
       RANGE:   user selectable range.
       LOG:     user selectable log scale (Used with RANGE)
    EXAMPLES:
      x = dgen(/x)  ; Returns 100 element array of points evenly distributed along
                    ; the x-axis.

(See general/tools/misc/dgen.pro)

## DYNAMICARRAY__DEFINE

[Previous Routine] [Next Routine] [List of Routines]

     Written by Davin Larson - August 2016
     $LastChangedBy: rjolitz $
     $LastChangedDate: 2025-03-04 10:57:07 -0800 (Tue, 04 Mar 2025) $
     $LastChangedRevision: 33161 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/trunk/general/tools/misc/dynamicarray__define.pro $

     Purpose: Object that provides an efficient means of concatenating arrays
     da= DynamicArray([InitialArray][,name='name1')
     Purpose:  Returns a "dynamic array" object.  This dynamic array can have any number of elements and can be efficiently
     appended to.

     This routine is particularly useful when appending to large arrays on numerous occassions.  This is especially useful
     when the final size of the array is not known when first initialized.

     It is functionally equivalent to:
     a= findgen(10)
     b = 1.
     a = [a,b]
     but considerably more effiecient because the array size does not need to be increased at every append operation.

     Because one can produce arrays of objects, it is a conveniant way of constructing arrays of arrays
     containing different things.

     Works with multidimensional arrays too.

     USAGE:
       da = dynamicarray(findgen(1000000), name='Test1')
          Or
       da.array = findgen(1000000)    ; equivalent
          Or
       da = dynamicarray()     &   da.append, findgen(1000000)
       da.append,  findgen(1000)  ;  append some data
       da.append, !values.f_nan      ; add a NAN at the end.
       da.name = 'NewName'   ; change name

       a = da.array   ; retrieve a copy of the array
       print,da.size   ; print the number of elements in the array  (first dimension of multidimensional arrays)
       help,da.name   ; display the optional user name

       object_destroy, da   ; cleanup when done.

(See general/tools/misc/dynamicarray__define.pro)

## ENCLOSED

[Previous Routine] [Next Routine] [List of Routines]

    FUNCTION:
      w = enclosed(x,y [cx,cy],NCIRCS=NCIRCS,COUNT=COUNT)
    PURPOSE:
     Returns the indices of a set of x,y points that are inside a contour.
    INPUT:  x,y:    data set of points.  (x and y must be the same dimension)
            cx,cy:  vector of x,y pairs that describe a closed contour.
            if cx,cy are not provided then the cursor is used to obtain it.
    OUTPUT:
        W:  Array of indices of x (& y) that are within the contour cx,cy.
        NCIRCS: Same dimension as x (& y); integer array giving the number
            of times each point is encircled.
        COUNT:  Size of array W

(See general/tools/misc/enclosed.pro)

## EXEC

[Previous Routine] [Next Routine] [List of Routines]

    WIDGET Procedure:
      EXEC
    PURPOSE:
     Widget tool that executes a user specified routine. This tool runs in the background.
     Author:
        Davin Larson - Feb 2012

     $LastChangedBy: $
     $LastChangedDate: $
     $LastChangedRevision: $
     $URL: $

(See general/tools/misc/exec.pro)

## GETXY

[Previous Routine] [Next Routine] [List of Routines]

    NAME:
      getxy
    PURPOSE:
      Routine that uses the cursor to select points.

(See general/tools/misc/getxy.pro)

## PRINT_STRUCT

[Previous Routine] [Next Routine] [List of Routines]

    PROCEDURE:  print_struct, data, tags=tags
    PURPOSE:
       prints data in an array of structures.
    CALLING PROCEDURE:
       print_struct, data
    KEYWORDS:
       TAGS:  tagnames of structure to print

    CREATED BY: Davin Larson, 1997

(See general/tools/misc/print_struct.pro)

## READ_ASC

[Previous Routine] [Next Routine] [List of Routines]

    FUNCTION:  data=read_asc(filename)
    PURPOSE:
       Reads data from an ascii file and puts data in an array of structures.
       Columns of data should be delimited by spaces.
       Data is returned as an array of structures. The elements of the structure
       correspond to the columns of the file.
    CALLING PROCEDURE:
       read_ascii,data,'file.dat'
    KEYWORDS:
       TAGS:  If set then the labels in the text line
          preceeding the data will be used for the default struct tag names.
          (There should be one label per column of data)
       FORMAT:  a structure that specifies the output format
          of the data.  For example if the input file has the
          following data:
          Year Day  secs    Vx     Vy     Vz      N
          1996 123  13.45  512.3  -10.3   10.5   5.3
          the format could be specified as:
       FORMAT={year:0,day:0,sec:0.d,v:fltarr(3),n:0.}
       if this keyword is not specified then a default structure will be created.
       NHEADER:  set to number of header lines
    CREATED BY: Davin Larson
     $LastChangedBy: davin-mac $
     $LastChangedDate: 2021-11-08 16:33:53 -0800 (Mon, 08 Nov 2021) $
     $LastChangedRevision: 30411 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/trunk/general/tools/misc/read_asc.pro $

(See general/tools/misc/read_asc.pro)

## RECORDER

[Previous Routine] [Next Routine] [List of Routines]

    WIDGET Procedure:
      RECORDER
    PURPOSE:
     Widget tool that opens a socket and records streaming data from a server (host) and can save it to a file
     or send to a user specified routine. This tool runs in the background.
     Keywords:
       SET_FILE_TIMERES : defines how often the current output file will be closed and a new one will be opened
       DIRECTORY:  string prepended to fileformat when opening an output file.
     Author:
        Davin Larson - April 2011

     $LastChangedBy: ali $
     $LastChangedDate: 2021-09-24 16:07:57 -0700 (Fri, 24 Sep 2021) $
     $LastChangedRevision: 30319 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/trunk/general/tools/misc/recorder.pro $

(See general/tools/misc/recorder.pro)

## SPLINECOEFF

[Previous Routine] [Next Routine] [List of Routines]

     NAME:
           SPLINECOEFF

     PURPOSE:

           This procedure computes coefficients of cubic splines for a
           given observational set and smoothing parameter lambda. The
           method is coded after Pollock D.S.G. (1999), "A Handbook of
           Time-Series Analysis, Signal Processing and Dynamics, Academic
           Press", San Diego.

     AUTHOR:

           Nikola Vitas
           Instituto de Astrofísica de Canarias (IAC)
           C/ Vía Láctea, s/n
           E38205 - La Laguna (Tenerife), España
           Email: n.vitas@iac.es
           Homepage: nikolavitas.blogspot.com

     CATEGORY:

           Numerics, statistics.

     CALLING SEQUENCE:

           coeffs = SPLINECOEFF([x,] y, [sigma=sigma,] lambda=lambda)

     OPTIONAL INPUTS:

     OUTPUTS:

     INPUT KEYWORDS:

           LAMBDA = Scalar, float. Smoothing parameter (It can be determined
                    empirically, by the LS method or by cross- validation, e.g. see
                    the book of Pollock.) Lambda equals 0 results in a cubic spline
                    interpolation. In the other extreme, for a very large lambda,
                    the result is smoothing by a linear function.

           SIGMA  = Array. Weight of each data point. If not specified, all
                    measurements have same weight.

     OUTPUT KEYWORDS:

           X     = Array. Independent variable.
           Y     = Array. Function to be smoothed or interpolated.

      EXAMPLE:

           x = .....
           y = .....
           coeffs = SPLINECOEFF(x, y, lambda = 1.d5)
           y1 = FLTARR(N_ELEMENTS(y) - 1)
           x1 = x[0:N_ELEMENTS(y)-2]
           FOR i = 0, N_ELEMENTS(y)-2 DO y1[i] = coeff.d[I] + $
                                                 coeff.c[I] * (x[I+1]-x[I]) + $
                                                 coeff.b[I] * (x[I+1]-x[I])^2 + $
                                                 coeff.a[I] * (x[I+1]-x[I])^3
       PLOT, x, y, psym = 3
       OPLOT, x1, y1

     DEPENDENCIES:

     MODIFICATION HISTORY:

           Written by Nikola Vitas, December 2005.
           - Small modifications, () replaced with [], NV, October, 2012

(See general/tools/misc/spline_smooth.pro)

## SPLINE_SMOOTH

[Previous Routine] [Next Routine] [List of Routines]

    FUNCTION:   spline_smooth
    PURPOSE:
      Uses spline smoothing to create a smooth curve through noisy data.
      This routine uses splinecoeff.pro, written by Nikola Vitas.  See
      detailed documentation below (and https://github.com/nikolavitas).

    USAGE:
      ys = spline_smooth(x, y, weight=w, labmda=lambda)

    INPUTS:
           x:         Independent variable.

           y:         Dependent variable.  Same number of elements as x.

    KEYWORDS:
           WEIGHT:    Weights for y.  Default = replicate(1.,n_elements(y))

           LAMBDA:    Smoothing factor.  A value of 0 corresponds to cubic
                      splines.  Larger values increase the smoothing.

     $LastChangedBy: dmitchell $
     $LastChangedDate: 2023-06-23 12:31:58 -0700 (Fri, 23 Jun 2023) $
     $LastChangedRevision: 31907 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/trunk/general/tools/misc/spline_smooth.pro $

    CREATED BY:	David L. Mitchell  2021-10-18

(See general/tools/misc/spline_smooth.pro)

## TMEAN

[Previous Routine] [Next Routine] [List of Routines]

    PROCEDURE:   tmean
    PURPOSE:
      Calculate the mean, median, and standard deviation of a 1-D or 2-D
      tplot variable over a specified time range.  The variable and time range
      are selected with the cursor or via keyword.  Skew and kurtosis are also
      calculated:

        skewness: = 0 -> distribution is symmetric about the maximum
                  < 0 -> distribution is skewed to the left
                  > 0 -> distribution is skewed to the right

        kurtosis: = 0 -> distribution is peaked like a Gaussian
                  < 0 -> distribution is less peaked than a Gaussian
                  > 0 -> distribution is more peaked than a Gaussian

      This routine can optionally perform cluster analysis to divide the data
      into two groups (Jenks natural breaks optimization).  Statistics are given
      for each group separately.

    USAGE:
      tmean, var

    INPUTS:
           var:     Tplot variable name or number.  If not specified, determine
                    based on which panel the mouse is in when clicked.

                    If the variable has two dimensions in y (time and some other
                    parameter), then you must specify which indices of the second
                    dimension to calculate statistics for.  Data can be either
                    summed or averaged over the second dimension.  See keywords
                    IND and AVG.

                    The variable cannot be compound (list of variables to plot
                    in the same panel).  You must specify which variable in the
                    list you are interested in.

    KEYWORDS:
           TRANGE:  Use this time range instead of getting it interactively
                    with the cursor.  In this case, you must specify var.

           IND:     If y has two dimensions (time and some other parameter), this
                    keyword specifies the indices of the second dimension to
                    calculate statistics for.  No default.

           AVG:     If IND is set and this keyword is also set, then average over
                    the second dimension.  Otherwise, sum over the second dimension.
                    NaN's are treated as missing data (see MEAN and TOTAL).

           ZERO:    Treat NaN's as zeroes.  This affects the mean but not the sum.

           OFFSET:  Value to subtract from the data before calculating statistics.
                    Default = 0.

           OUTLIER: Ignore values more than OUTLIER sigma from the mean.
                    Default = infinity.

           MINPTS:  If OUTLIER is set, this specifies the minimum number of
                    points remaining after discarding outliers.  Default = 3.

           CLUSTER: Perform 1-D cluster analysis to separate the data into two
                    groups.  Statistics are calculated for each cluster separately.
                    Diagnostics of the quality of the cluster separation are also
                    provided (see keyword DIAG).  Disables OUTLIER.

           MAXDZ:   Use largest break between clusters near minimum variance
                    to divide the clusters.  Default = 1.

           RESULT:  Named variable to hold the result.

           HIST:    Plot a histogram of the distribution in a separate window.

           KEEP:    Keep the last histogram window open on exit.

           RANGE:   Range of values for the histogram.

           NBINS:   If HIST is set, number of bins in the histogram.

           NPTS:    If this is set, then statistics are calculated for NPTS centered
                    on the time nearest the cursor when clicked (as opposed to
                    selecting a time range with two clicks).

           DST:     Retain the distribution.  Does not allow compiling multiple
                    results.

           T0:      Times in cluster 0.

           T1:      Times in cluster 1.

           DIAG:    Return outlier and cluster analysis diagnostics:
                      minvar : minimum total variance for both clusters
                      maxvar : maximum total variance for both clusters
                      maxsep : separation between the clusters
                      sepval : value of optimal separation between the clusters
                      edge   : distance between sepval and the edge of the distribution
                      ngud   : number of points in the main distribution
                      nbad   : number of outliers
                      delta  : separation* between the core and outliers in SDEV units
                                                 -or-
                               separation# between the clusters in SDEV units
                      frac   : fraction of "bad" points (nbad/(nbad+ngud))

                      * separation is the distance between the 2-sigma level of the core
                        distribution and the closest outlier

                      # separation is the closest distance between the 2-sigma levels of the
                        clusters

           SILENT:  Shhh.

     $LastChangedBy: dmitchell $
     $LastChangedDate: 2024-04-19 08:29:46 -0700 (Fri, 19 Apr 2024) $
     $LastChangedRevision: 32527 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/trunk/general/tools/misc/tmean.pro $

    CREATED BY:    David L. Mitchell

(See general/tools/misc/tmean.pro)

## TOPSMOOTH

[Previous Routine] [Next Routine] [List of Routines]

    PROCEDURE:   topsmooth
    PURPOSE:
      Calculate a smooth curve through the upper envelope of a tplot variable.

    USAGE:
      topsmooth, var

    INPUTS:
           var:     Variable name or number.

    KEYWORDS:
           NPTS:    Half-width of boxcar.  Default = 16.

           FRAC:    Fraction of points to ignore in average or interpolates.
                    Default = 0.5.

           INTERP:  Interpolate using data that pass through topfilter.

           SIGMA:   Spline sigma.  Default = 1.

           LAMBDA:  Spline smooth lambda.  Default = 1.

           SILENT:  Shh.

     $LastChangedBy: dmitchell $
     $LastChangedDate: 2023-06-23 12:31:58 -0700 (Fri, 23 Jun 2023) $
     $LastChangedRevision: 31907 $
     $URL: svn+ssh://thmsvn@ambrosia.ssl.berkeley.edu/repos/spdsoft/trunk/general/tools/misc/topsmooth.pro $

    CREATED BY:    David L. Mitchell

(See general/tools/misc/topsmooth.pro)

## XY_EDIT

[Previous Routine] [Next Routine] [List of Routines]

    PROCEDURE: xy_edit,x,y,bins
    PURPOSE:
       Interactively select data points

(See general/tools/misc/xy_edit.pro)

## YYMMDD_TO_TIME

[Previous Routine] [List of Routines]

    FUNCTION:  yymmdd_to_time
    PURPOSE:
        Returns time (seconds since 1970) given date in format:  YYMMDD  HHMM
    USAGE:
      t = yymmdd_to_time(yymmdd [,hhmm])
      (yymmdd can be either a long or a string)
    Examples:
      t = yymmdd_to_time(990421,1422)
      print,t,' ',time_string(t)
    Created by: Davin Larson, April 1999

(See general/tools/misc/yymmdd_to_time.pro)