FEEDGNUPLOT(1) User Contributed Perl Documentation FEEDGNUPLOT(1)

feedgnuplot - General purpose pipe-oriented plotting tool

Simple plotting of piped data:

 $ seq 5 | awk '{print 2*$1, $1*$1}'
 2 1
 4 4
 6 9
 8 16
 10 25
 $ seq 5 | awk '{print 2*$1, $1*$1}' |
   feedgnuplot --lines --points --legend 0 "data 0" --title "Test plot" --y2 1
               --unset grid --terminal 'dumb 80,40' --exit
                                  Test plot
  10 +-----------------------------------------------------------------+ 25
     |       +        +       +       +       +        +       +    *##|
     |                                                  data 0 ***A*#* |
     |                                                          ** #   |
   9 |-+                                                      ** ##    |
     |                                                      **  #      |
     |                                                    **   #       |
     |                                                  **   ##      +-| 20
   8 |-+                                               A    #          |
     |                                               **    #           |
     |                                             **    ##            |
     |                                           **     #              |
     |                                         **      B               |
   7 |-+                                     **      ##                |
     |                                     **      ##                +-| 15
     |                                   **       #                    |
     |                                 **       ##                     |
   6 |-+                             *A       ##                       |
     |                             **       ##                         |
     |                           **        #                           |
     |                         **        ##                          +-| 10
   5 |-+                     **        ##                              |
     |                     **        #B                                |
     |                   **        ##                                  |
     |                 **        ##                                    |
   4 |-+              A       ###                                      |
     |              **      ##                                         |
     |            **      ##                                         +-| 5
     |          **      ##                                             |
     |        **    ##B#                                               |
   3 |-+    **  ####                                                   |
     |    **####                                                       |
     |  ####                                                           |
     |##     +        +       +       +       +        +       +       |
   2 +-----------------------------------------------------------------+ 0
     1      1.5       2      2.5      3      3.5       4      4.5      5

Here we asked for ASCII plotting, which is useful for documentation.

Simple real-time plotting example: plot how much data is received on the wlan0 network interface in bytes/second (uses bash, awk and Linux):

 $ while true; do sleep 1; cat /proc/net/dev; done |
   gawk '/wlan0/ {if(b) {print $2-b; fflush()} b=$2}' |
   feedgnuplot --lines --stream --xlen 10 --ylabel 'Bytes/sec' --xlabel seconds

This is a flexible, command-line-oriented frontend to Gnuplot. It creates plots from data coming in on STDIN or given in a filename passed on the commandline. Various data representations are supported, as is hardcopy output and streaming display of live data. For a tutorial and a gallery please see the guide at <https://github.com/dkogan/feedgnuplot/blob/master/guide/guide.org>

A simple example:

 $ seq 5 | awk '{print 2*$1, $1*$1}' | feedgnuplot

You should see a plot with two curves. The "awk" command generates some data to plot and the "feedgnuplot" reads it in from STDIN and generates the plot. The "awk" invocation is just an example; more interesting things would be plotted in normal usage. No commandline-options are required for the most basic plotting. Input parsing is flexible; every line need not have the same number of points. New curves will be created as needed.

The most commonly used functionality of gnuplot is supported directly by the script. Anything not directly supported can still be done with options such as "--set", "--cmds" "--style", etc. Arbitrary gnuplot commands can be passed in with "--cmds". For example, to turn off the grid, you can pass in --cmds 'unset grid'. Commands "--set" and "--unset" exists to provide nicer syntax, so this is equivalent to passing "--unset grid". As many of these options as needed can be passed in. To add arbitrary curve styles, use "--style curveID extrastyle". Pass these more than once to affect more than one curve.

To apply an extra style to all the curves that lack an explicit "--style", pass in "--styleall extrastyle". In the most common case, the extra style is "with something". To support this more simply, you can pass in "--with something" instead of "--styleall 'with something'". "--styleall" and "--with" are mutually exclusive. Furthermore any curve-specific "--style" overrides the global "--styleall" or "--with" setting.

By default, each value present in the incoming data represents a distinct data point, as demonstrated in the original example above (we had 10 numbers in the input and 10 points in the plot). If requested, the script supports more sophisticated interpretation of input data

Domain selection

If "--domain" is passed in, the first value on each line of input is interpreted as the X-value for the rest of the data on that line. Without "--domain" the X-value is the line number, and the first value on a line is a plain data point like the others. Default is "--nodomain". Thus the original example above produces 2 curves, with 1,2,3,4,5 as the X-values. If we run the same command with "--domain":

 $ seq 5 | awk '{print 2*$1, $1*$1}' | feedgnuplot --domain

we get only 1 curve, with 2,4,6,8,10 as the X-values. As many points as desired can appear on a single line, but all points on a line are associated with the X-value at the start of that line.

Curve indexing

We index the curves in one of 3 ways: sequentially, explicitly with a "--dataid" or by "--vnlog" headers.

By default, each column represents a separate curve. The first column (after any domain) is curve 0. The next one is curve 1 and so on. This is fine unless sparse data is to be plotted. With the "--dataid" option, each point is represented by 2 values: a string identifying the curve, and the value itself. If we add "--dataid" to the original example:

 $ seq 5 | awk '{print 2*$1, $1*$1}' | feedgnuplot --dataid --autolegend

we get 5 different curves with one point in each. The first column, as produced by "awk", is 2,4,6,8,10. These are interpreted as the IDs of the curves to be plotted.

If we're plotting "vnlog" data (<https://www.github.com/dkogan/vnlog>) then we can get the curve IDs from the vnlog header. Vnlog is a trivial data format where lines starting with "#" are comments and the first comment contains column labels. If we have such data, "feedgnuplot --vnlog" can interpret these column labels if the "vnlog" perl modules are available.

The "--autolegend" option adds a legend using the given IDs to label the curves. The IDs need not be numbers; generic strings are accepted. As many points as desired can appear on a single line. "--domain" can be used in conjunction with "--dataid" or "--vnlog".

Multi-value style support

Depending on how gnuplot is plotting the data, more than one value may be needed to represent the range of a single point. Basic 2D plots have 2 numbers representing each point: 1 domain and 1 range. But if plotting with "--circles", for instance, then there's an extra range value: the radius. Many other gnuplot styles require more data: errorbars, variable colors ("with points palette"), variable sizes ("with points ps variable"), labels and so on. The feedgnuplot tool itself does not know about all these intricacies, but they can still be used, by specifying the specific style with "--style", and specifying how many values are needed for each point with any of "--rangesizeall", "--tuplesizeall", "--rangesize", "--tuplesize". These options are required only for styles not explicitly supported by feedgnuplot; supported styles do the right thing automatically.

Specific example: if making a 2d plot of y error bars, the exact format can be queried by running "gnuplot" and invoking "help yerrorbars". This tells us that there's a 3-column form: "x y ydelta" and a 4-column form: "x y ylow yhigh". With 2d plots feedgnuplot will always output the 1-value domain "x", so the rangesize is 2 and 3 respectively. Thus the following are equivalent:

 $ echo '1 2 0.3
         2 3 0.4
         3 4 0.5' | feedgnuplot --domain --rangesizeall 2 --with 'yerrorbars'
 $ echo '1 2 0.3
         2 3 0.4
         3 4 0.5' | feedgnuplot --domain --tuplesizeall 3 --with 'yerrorbars'
 $ echo '1 2 1.7 2.3
         2 3 2.6 3.4
         3 4 3.5 4.5' | feedgnuplot --domain --rangesizeall 3 --with 'yerrorbars'

3D data

To plot 3D data, pass in "--3d". "--domain" MUST be given when plotting 3D data to avoid domain ambiguity. If 3D data is being plotted, there are by definition 2 domain values instead of one (Z as a function of X and Y instead of Y as a function of X). Thus the first 2 values on each line are interpreted as the domain instead of just 1. The rest of the processing happens the same way as before.

Time/date data

If the input data domain is a time/date, this can be interpreted with "--timefmt". This option takes a single argument: the format to use to parse the data. The format is documented in 'set timefmt' in gnuplot, although the common flags that "strftime" understands are generally supported. The backslash sequences in the format are not supported, so if you want a tab, put in a tab instead of \t. Whitespace in the format is supported. When this flag is given, some other options act a little bit differently:

  • "--xlen" and "--binwidth" are integers in seconds
  • "--xmin" and "--xmax" must use the format passed in to "--timefmt"

Using this option changes both the way the input is parsed and the way the x-axis tics are labelled. Gnuplot tries to be intelligent in this labelling, but it doesn't always do what the user wants. The labelling can be controlled with the gnuplot "set format" command, which takes the same type of format string as "--timefmt". Example:

 $ sar 1 -1 |
   awk '$1 ~ /..:..:../ && $8 ~/^[0-9\.]*$/ {print $1,$8; fflush()}' |
   feedgnuplot --stream --domain
                --lines --timefmt '%H:%M:%S'
                --set 'format x "%H:%M:%S"'

This plots the 'idle' CPU consumption against time.

Note that while gnuplot supports the time/date on any axis, feedgnuplot currently supports it only as the x-axis domain. This may change in the future.

'using' expressions

We just described how feedgnuplot parses its input data. When passing this data to gnuplot, each curve is sent independently. The domain appears in the leading columns followed by "--rangesize" columns to complete each row. Without "--domain", feedgnuplot explicitly writes out sequential integers. gnuplot then knows how many values it has for each point, and it knows which style we're using, so it's able to interpret the data appropriately, and to make the correct plot.

As an example, if gnuplot is passed 2 columns of data, and it is plotting "with points", it will use column 1 for the x coordinate and column 2 for the y coordinate. This is the default behavior, but the meaning of each column can be controlled via a "using" expression in gnuplot (not feedgnuplot; keep reading). The default is sequential integers, so this example uses "using 1:2" by default. We can flip the meaning of the columns by passing "using 2:1". Arbitrary expressions may be specified by enclosing each field in "()", and using "$" to denote each data column. So to use the 2nd column as the x coordinate and the sum of the two columns as the y coordinate, using 2:($1+$2) is passed. Furthermore, the number of columns can vary. For instance gnuplot can read the same two columns of data, but produce a plot with the extra column encoding the sum as the color: "using 1:2:($1+$2) with points palette". Please see the gnuplot documentation for lots of detail.

That's how gnuplot works. Most of the time, feedgnuplot doesn't pass any "using" expressions at all, and gnuplot does the default thing. But if we want to do something fancy, feedgnuplot supports "--using curveID expression" and "--usingall expression". So we can plot a parabola:

  seq 100 | feedgnuplot --lines --usingall '1:($2*$2)'

This is powerful, but there are some things to keep in mind:

  • "--using" overrides whatever "using" expression feedgnuplot was going to pass. feedgnuplot passes a "using" expression only if "--histogram" or "--timefmt" or "--xticlabels" are given. So if "--using" is given together with any of these, the user must take care to do the right thing (whatever that means at that time).
  • The "--tuplesize" controls the data passed to feedgnuplot and the data then passed to gnuplot. It does not directly control how gnuplot eventually interprets the data: "--using" does that. So for instance we can plot color-coded points:

      seq 10 | feedgnuplot --with 'points pt 7 palette' --usingall '1:2:2'
        

    Here feedgnuplot read 1 column of data. It defauled to "--tuplesize 2", so it passed 2 columns of data to gnuplot. gnuplot then produced 3 values for each point, and plotted them as indicated with the "points palette" style.

  • You always need a column of data to generate a curve. You might want to use a "using" expression to plot a time series and its cumulative integral. The "using" expression can compute the integral, but you must pass in the data twice; once for each curve to plot:

      seq 100 |                           \
        awk '{print $1,$1}' |             \
        feedgnuplot                       \
          --cmds 'sum=0'                  \
          --cmds 'accum(x) = (sum=sum+x)' \
          --using 1 '1:(accum($2))'       \
          --lines --y2 1
        

To plot real-time data, pass in the "--stream [refreshperiod]" option. Data will then be plotted as it is received. The plot will be updated every "refreshperiod" seconds. If the period isn't specified, a 1Hz refresh rate is used. To refresh at specific intervals indicated by the data, set the refreshperiod to 0 or to 'trigger'. The plot will then only be refreshed when a data line 'replot' is received. This 'replot' command works in both triggered and timed modes, but in triggered mode, it's the only way to replot. Look in "Special data commands" for more information.

To plot only the most recent data (instead of all the data), "--xlen windowsize" can be given. This will create an constantly-updating, scrolling view of the recent past. "windowsize" should be replaced by the desired length of the domain window to plot, in domain units (passed-in values if "--domain" or line numbers otherwise). If the domain is a time/date via "--timefmt", then "windowsize" is and integer in seconds. If we're plotting a histogram, then "--xlen" causes a histogram over a moving window to be computed. The subtlely here is that with a histogram you don't actually see the domain since only the range is analyzed. But the domain is still there, and can be utilized with "--xlen". With "--xlen" we can plot only histograms or only non-histograms.

Special data commands

If we are reading streaming data, the input stream can contain special commands in addition to the raw data. Feedgnuplot looks for these at the start of every input line. If a command is detected, the rest of the line is discarded. These commands are

"replot"
This command refreshes the plot right now, instead of waiting for the next refresh time indicated by the timer. This command works in addition to the timed refresh, as indicated by "--stream [refreshperiod]".
"clear"
This command clears out the current data in the plot. The plotting process continues, however, to any data following the "clear".
"exit"
This command causes feedgnuplot to exit.

The script is able to produce hardcopy output with "--hardcopy outputfile". The output type can be inferred from the filename, if .ps, .eps, .pdf, .svg, .png or .gp is requested. If any other file type is requested, "--terminal" must be passed in to tell gnuplot how to make the plot. If "--terminal" is passed in, then the "--hardcopy" argument only provides the output filename.

The .gp output is special. Instead of asking gnuplot to plot to a particular terminal, writing to a .gp simply dumps a self-executable gnuplot script into the given file. This is similar to what "--dump" does, but writes to a file, and makes sure that the file can be self-executing.

This script can be used to enable self-plotting data files. There are several ways of doing this: with a shebang (#!) or with inline perl data.

Self-plotting data with a #!

A self-plotting, executable data file "data" is formatted as

 $ cat data
 #!/usr/bin/feedgnuplot --lines --points
 2 1
 4 4
 6 9
 8 16
 10 25
 12 36
 14 49
 16 64
 18 81
 20 100
 22 121
 24 144
 26 169
 28 196
 30 225

This is the shebang (#!) line followed by the data, formatted as before. The data file can be plotted simply with

 $ ./data

The caveats here are that on Linux the whole #! line is limited to 127 characters and that the full path to feedgnuplot must be given. The 127 character limit is a serious limitation, but this can likely be resolved with a kernel patch. I have only tried on Linux 2.6.

Self-plotting data with gnuplot

Running "feedgnuplot --hardcopy plotdata.gp ...." will create a self-executable gnuplot script in "plotdata.gp"

Self-plotting data with perl inline data

Perl supports storing data and code in the same file. This can also be used to create self-plotting files:

 $ cat plotdata.pl
 #!/usr/bin/perl
 use strict;
 use warnings;
 open PLOT, "| feedgnuplot --lines --points" or die "Couldn't open plotting pipe";
 while( <DATA> )
 {
   my @xy = split;
   print PLOT "@xy\n";
 }
 __DATA__
 2 1
 4 4
 6 9
 8 16
 10 25
 12 36
 14 49
 16 64
 18 81
 20 100
 22 121
 24 144
 26 169
 28 196
 30 225

This is especially useful if the logged data is not in a format directly supported by feedgnuplot. Raw data can be stored after the __DATA__ directive, with a small perl script to manipulate the data into a useable format and send it to the plotter.

  • "--nodomain --nodataid" would parse the 4 numbers as points in 4 different curves at x=3
  • "--domain --nodataid" would parse the 4 numbers as points in 3 different curves at x=0. Here, 0 is the x-variable and 9,1,20 are the data values
  • "--nodomain --dataid" would parse the 4 numbers as points in 2 different curves at x=3. Here 0 and 1 are the data IDs and 9 and 20 are the data values
  • "--domain --dataid" would parse the 4 numbers as a single point at x=0. Here 9 is the data ID and 1 is the data value. 20 is an extra value, so it is ignored. If another value followed 20, we'd get another point in curve ID 20

The possibilities are:

Alive. Need to Ctrl-C to get back into the shell
"--exit"
Half-alive. Non-interactive prompt up, and the shell accepts new commands. Without "--stream" the goal is to show a plot, so a Dead state would not be useful.
"--stream", all data read in or the "feedgnuplot" process terminated
Alive. Need to Ctrl-C to get back into the shell. This means that when making live plots, the first Ctrl-C kills the data feeding process, but leaves the final plot up for inspection. A second Ctrl-C kills feedgnuplot as well.
"--exit"
Dead. No plot is shown, and the shell accepts new commands. With "--stream" the goal is to show a plot as the data comes in, which we have been doing. Now that we're done, we can clean up everything.

Note that one usually invokes "feedgnuplot" as a part of a shell pipeline:

 $ write_data | feedgnuplot

If the user terminates this pipeline with ^C, then all the processes in the pipeline receive SIGINT. This normally kills "feedgnuplot" and all its "gnuplot" children, and we let this happen unless "--stream" and no "--exit". If "--stream" and no "--exit", then we ignore the first ^C. The data feeder dies, and we behave as if the input data was exhausted. A second ^C kills us also.

For a tutorial and a gallery please see the guide at <https://github.com/dkogan/feedgnuplot/blob/master/guide/guide.org>

 $ seq 5 | awk '{print 2*$1, $1*$1}'
 2 1
 4 4
 6 9
 8 16
 10 25
 $ seq 5 | awk '{print 2*$1, $1*$1}' |
   feedgnuplot --lines --points --legend 0 "data 0" --title "Test plot" --y2 1

Looks at wlan0 on Linux.

 $ while true; do sleep 1; cat /proc/net/dev; done |
   gawk '/wlan0/ {if(b) {print $2-b; fflush()} b=$2}' |
   feedgnuplot --lines --stream --xlen 10 --ylabel 'Bytes/sec' --xlabel seconds

Uses the result of the "acpi" command.

 $ while true; do acpi; sleep 15; done |
   perl -nE 'BEGIN{ $| = 1; } /([0-9]*)%/; say join(" ", time(), $1);' |
   feedgnuplot --stream --ymin 0 --ymax 100 --lines --domain --xlabel 'Time' --timefmt '%s' --ylabel "Battery charge (%)"

Uses "/proc/acpi/ibm/thermal", which reports temperatures at various locations in a Thinkpad.

 $ while true; do cat /proc/acpi/ibm/thermal | awk '{$1=""; print}' ; sleep 1; done |
   feedgnuplot --stream --xlen 100 --lines --autolegend --ymax 100 --ymin 20 --ylabel 'Temperature (deg C)'

 $ ls -l | awk '{print $5/1e6}' |
   feedgnuplot --histogram 0
     --binwidth 10
     --ymin 0 --xlabel 'File size (MB)' --ylabel Frequency

 $ ping -D 8.8.8.8 |
   perl -anE 'BEGIN { $| = 1; }
              $F[0] =~ s/[\[\]]//g or next;
              $F[7] =~ s/.*=//g    or next;
              say "$F[0] $F[7]"' |
   feedgnuplot --stream --domain --histogram 0 --binwidth 10 \
               --xlabel 'Ping round-trip time (s)'  \
               --ylabel Frequency --xlen 20

This can be done with "--image":

 $ < features_xy.data
   feedgnuplot --points --domain --image "image.png"

or with "--equation":

 $ < features_xy.data
   feedgnuplot --points --domain
     --equation '"image.png" binary filetype=auto flipy with rgbimage'
     --set 'yrange [:] reverse'

The "--image" invocation is a convenience wrapper for the "--equation" version. Finer control is available with "--equation".

Here an existing image is given to gnuplot verbatim, and data to plot on top of it is interpreted by feedgnuplot as usual. "flipy" is useful here because usually the y axis points up, but when looking at images, this is usually reversed: the origin is the top-left pixel.

This program is originally based on the driveGnuPlots.pl script from Thanassis Tsiodras. It is available from his site at <http://users.softlab.ece.ntua.gr/~ttsiod/gnuplotStreaming.html>

<https://github.com/dkogan/feedgnuplot>

Dima Kogan, "<dima@secretsauce.net>"

Copyright 2011-2021 Dima Kogan.

This program is free software; you can redistribute it and/or modify it under the terms of either: the GNU General Public License as published by the Free Software Foundation; or the Artistic License.

See http://dev.perl.org/licenses/ for more information.

2024-01-28 perl v5.38.2