[Acoustics] RE:Delay in GPS processing rant (Aaron Blake, USGS)

[Aaron Blake]

Hello all,
I wanted to share a few observations about GPS/ADCP data co-registration 
issues with WinRiver; this was supposed to be a short and to-the-point 
email, but it seems to have turned into a bit of a rant.  So, I will 
repeat the punch line upfront to save time.

Take Home Message: 
I believe that the problems people are experiencing with their serial 
cards/converters is really just a manifestation/exacerbation of WinRiver?s 
data acquisition/synchronization issues. 

Disclaimer: 
First, I should say that I have not used WinRiver lately, so if a 
dramatically new version has come out that has addressed these issues, I 
apologize. 

And now the rest: 
I have also experienced issues with windows XP using a serial to USB 
converter (I think connect tech) that worked flawlessly on windows 2000 
machines.  Using windows XP, my converter ?miss-translates? several 
samples of data to the USB (?fake serial?) buffer before it gets it right, 
so that I am reading good data packets several (1-8) samples after 
acquisition begins.  If I was using a data acquisition program that waited 
for a new data event of a different buffer (such as the ADCP buffer?), and 
then scanned the converter?s buffer (perhaps used for a GPS?) for an end 
of communication character, and then time stamped the communication 
preceding that character with the data from the other (ADCP) buffer, then 
I would wind up with a data stream with two pieces of data time stamped 
together but consistently lagged. 

I believe that some version of the above scenario is the cause of the 
problems people are reporting in using WinRiver with the new converters. I 
havn?t played around with the newest version of WinRiver and a converter 
to verify this, but I feel moderately comfortable making this assumption, 
because every version of WinRiver I know of uses that same basic data 
acquisition algorithm that samples all instrument buffers at the same rate 
as the ADCP buffer, and then gives them the same time stamp as the ADCP 
data.  In addition, the program allocates fairly small buffers and doesn?t 
appear to multithread robustly, so if the user generates enough high 
priority mouse or keyboard events, data is lost due to overruns, possibly 
resulting in increased synchronization errors.  If you look closely at 
WinRiver data (or at least the versions of WinRiver I have used), you will 
see that the GPS boat track and the ADCP data are never correctly 
synchronized at high data rates, but most people don?t notice/care because 
they are averaging enough pings/ensembles that the associated GPS fix 
usually ends up falling within the temporal window of the reported ADCP 
sample. 

Randal Dinehart at the USGS California Water Science Center, Sacramento, 
is an expert on the WinRiver ADCP ? GPS synchronization problem, and on 
methods of manually post processing the data to fix boat tracks.  If 
anyone wants to talk to an expert on dealing with this problem, I would 
suggest they contact him.  However, be forewarned that, by its very 
nature, the process of post-processing large data sets to develop new boat 
tracks can be very time consuming and fairly complicated. 

I think that the long term solution to this problem is a mature rewrite of 
(or replacement for) WinRiver, so that each instrument?s buffer is sampled 
and times-stamped independently (not to mention robustly and correctly), 
and then written to a data base separately.  (Also, GPS data should be 
time stamped based on the time reported in its data string, not the time 
it is read from a buffer.)  This asynchronous data could then be entered 
into a state-space model/filter to produce truly robust estimates of the 
system?s state at the time (sub second) of the ADCP sample.  This would 
allow the combination of boat track and bottom track data, allow the user 
to integrate multiple ADCPs into one acquisition package, and perhaps add 
a HPR sensor that doesn?t have as high a time lag as the ADCPs 
electrolytic sensor (or at the very least, try to cleverly filter the 
ADCPs HPR sensor).  In short, I am ranting in favor of a data acquisition 
package that allows the user to take full advantage of the capabilities of 
all types instrumentation relevant to hydrodynamic/hydraulic research, 
rather than a data acquisition program developed around the capabilities 
and requirements of one specific instrument manufactured by one company, 
and origionaly written for computers with capabilities equivalent to 
today?s high end PDAs. 

I believe that on the cutting edge of environmental science, the quality 
of our science is constrained by the quality of our data, and the quality 
of our data is determined in part by the instrumentation we use, and the 
software we use to communicate with that instrumentation.  At this point 
in time, the capabilities of our scientific hardware, computers, and our 
researchers have outpaced WinRiver, so that researchers interested in 
higher order processes must either tiptoe around WinRiver so as not to 
upset the delicate balance between the timing of buffer reads (ie, not 
touching the computer during acquisition, not upgrading operating systems, 
eliminating background programs, not using certain serial cards, etc), or, 
write custom software to replace WinRiver or extensively post-process 
WinRiver data to back out errors. 

I could go on, but this email already takes up enough inbox space, so I 
will summarize by saying: I believe that the problems people are 
experiencing with their serial cards/converters is really just a 
manifestation/exacerbation of WinRiver?s data acquisition/synchronization 
issues. 

Thanks for your time, 
Aaron Blake
San Francisco Bay/Delta Hydrodynamics
USGS, California Water Science Center, Sacramento 
ablake at usgs.gov 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://simon.er.usgs.gov/pipermail/acoustics/attachments/20060404/1f165177/attachment.html