Overview
========

Binary data
-----------

The data files that are generated by RDI ADCPs are in a binary
format. The detailed format is described in various manuals, and may
differ between different types of instruments. This software is
capable of processing most blocks that are common to all formats. Data
blocks that are present in the file, but for which no reader method
has been implemented will be ignored, silently or not, depending on
the configuration of the logger.

The general structure of the binary files is that for each acoustic
ping a number of data blocks are written. Some examples are:

* fixed leader
* variable leader
* velocity
* echo
* bottom track

The **fixed leader** block contains information regarding the instrument,
and its configuration. During a deployment this information is
static. Nevertheless, this block is processed and output for each ping
to allow the processing of multiple files from different deployments.

The **variable leader** block contains information that can vary from ping
to ping. Typical examples are time, attitude, temperature.

The **velocity** block contains the velocity arrays. Their
representation depend on the coordinate system set.

The **echo** block contains the arrays of the recorded echo
intensities, for each beam.

The **bottom track** block contains information on bottom track
velocities, expressed in the coordinate system set.

Pipe line processing
--------------------

For each file or list of files, the binary data are processed ping by
ping. The blocks present in the pings are decoded. The **rdi_reader**
module provides a generator that generates decoded ping data. This
generator will typically be the starting point of a pipe line of
operations. The final operation in this pipe line is most likely an
operation that saves the data in some form of data format, such as
netCDF. The operations in between can be:

* coordinate transformations,
* speed of sound corrections,
* quality control of data.
  
Making a Pipe line
------------------

Each operation is a coroutine. Coroutines can be easily joined
together, making the output of one being the input of the next. If we
have defined two operations :code:`op1` and :code:`op2` we can chain
them together as ::

  op1.send_to(op2)

Now, all data sent to :code:`op1`, will also be sent to
:code:`op2`. Chaining many operations together results in considerable
visual pollution. Instead, we can use the pipe symbol :code:`|` to
chain operations together ::
   
  pipeline = op1 | op2


which is equivalent to ::
  
  op1.send_to(op2)
  pipeline = op1