[Acoustics] SUM: Request for software to analyze water-level data

[Kurt P Kowalski]

Thanks to the 9 people who responded.  I received a number of suggestions 
and will begin exploring each one.

Most of the messages I received were cc:'d to the list, but I thought it 
might be helpful for me to provide a slightly edited version of them here. 


Thanks again for the input.

-Kurt

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Kurt P. Kowalski
U.S. Geological Survey                   Phone:  (734)214-9308
Great Lakes Science Center           FAX:    (734)214-7230
1451 Green Road                           email:  kkowalski at usgs.gov 
Ann Arbor, MI  48105-2807             http://www.glsc.usgs.gov

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Messages received:
1)  We use WISKI http://www.kisters.de/ and I believe The Environment of 
UK also.

2)  Hey Kurt, could you clarify are you asking about software to help QAQC 
of data or doing statistical analyzes of processed data?  If you mean 
plotting, applying corrections, and removing erroneous data you should be 
loading all of your data into ADAPS for this. If you are not working with 
ADAPS for this step you will most likely be spending a lot of time trying 
to reinvent the wheel. 

3)  I use MATLAB on a regular basis, but if you do not have a license it's 
probably out of the question b/c of $$.  Another option is "GR" which is a 
tool that was developed specifically for veiwing time-series data.  It was 
developed by John Donovan here in the CA District and you can get more 
information and download the most recent version from: 

http://ca.water.usgs.gov/program/sfbay/gr/ 

4)  MATLAB cannot be beat for long data sets; we commonly analyze at least 
35000 points on a regular basis with no troubles, I have worked with data 
sets 10 times longer than that effortlessly as well. The only drawback is 
that the learning curve is steeper than excel, but if you have any 
experience with Fortran, or other codes it's not as difficult. In this 
office we look at time-series at 15 min intervals of depth, salinity, and 
sediment concentrations, and each one of us uses MATLAB constantly. You 
can get a trial version easily enough, full version may be discounted for 
government ($1000 regular price), student version is about $100.

5)  We are both archiving and making data available over the internet as 
well as feeding it into various three dimensional models.  We are using 
data from about 10 different sources, including instruments from various 
manufcaterers and say NOAA buoys, national weather service, etc. 

We convert all data from all sources into netCDF.  netCDF allows a lot of 
latitute in what you can do, so we follow EPIC conventions. 
NetCDF allows partial extraction of data, is platform independent and 
handles files up to 2GB.  It can handle multidimensional data sets.  The 
best we've found for large time series. 
http://my.unidata.ucar.edu/content/software/netcdf/index.html 
http://www.epic.noaa.gov/epic/ 

netCDF is open source. There are many free packages to manipulate the data 
depending on the conventions being followed. 

We have written conversion tools for various instruments into netCDF in 
MATLAB.  We settled on MATLAB because it is also platform independent, 
also a tool our modelers use, is widespread in oceanography, allows us to 
avoid compiled software, and has lots of plotting and display options.  It 
is not, however, free.  But there is a huge user and support community 
besides the Mathworks, which is a pretty responsive vendor.  MATLAB also 
has nice tools for importing and exporting excel spreadsheets.  We think 
it is weel worth the price. 

We convert and process data from RDI ADCPs, Sontek Hydra (ADV & PCADP), 
Seabird Seacats and Microcats and more 
You can see our archive at http://stellwagen.er.usgs.gov/index.html 
Our ADCP conversion toolbox for MATLAB is described at 
http://woodshole.er.usgs.gov/operations/stg/pubs/ADCPtools/index.html 
We are going to publish a similar toolbox for Sontek ADV/PCADP data, we 
are beta testing it now internally. 
Our west coast office has developed a toolbox for viewing time series 
data, it can combine netCDF data from various sources, do spectral 
analysis, etc.  Very nice, all point and click. 

netCDF allows us to store metadata with the time series, eliminating lots 
of metadata headaches. 

There is a learning curve with this stuff, but the payback has been 
enormous.   

We don't have toolboxes for the Sontek ADPs, but others have brought up 
this issue.  A lot of the pieces to doing this for commonly used 
instruments in WRD are around. 

6)  S-Plus should be available to anyone in the USGS and is the 
recommended Stats and graphics package. It has a learning curve but is 
very powerful. It also allows plotting of data from within Excel.

7)  I would say that Matlab is the standard for this type of data analysis

8)  Aquatic Informatics Inc. is a Vancouver, BC based company which has 
developed software specifically for processing large time series data 
sets.  You might want to check their website  (www.aquaticinformatics.com) 
for more information on the software.

9) Another possible solution for viewing time series is a software package 
developed by the California District office however, the data sets must be 
converted to the appropriate format.  See this url for information.

http://wwwdcascr.wr.usgs.gov/projects/gr/