View source for GPS Data Logger

This page was created to address the use cases mentioned in [[Neo FreeRunner GPS]], including

* I want to log GPS data for later analysis.
* I want to collect GPS data for scientific field work (forestry, biology, etc)

The page was started on 17 August 2008 mostly as a thought exercise.
Please jump in and add to it.

== Overview ==

The typical commercial approach to managing GPS data is to create one big application that does everything well. Unfortunately everyone has a different idea of what "doing everything well" means. One person might want to use GPS to log and analyze bicycle rides while another does weed mapping and another might be doing asset management. One application cannot do all this well.

A better approach is to break it up into a set of components. Each component can be a standalone application. This relies on some kind of multiplexer such as gpsd (does gypsy support multiplexing?) so that each app has access to the GPS data stream.

Initially the main functions could be broken out these components.

# A '''GPS multiplexer''' that talks to the GPS receiver and feeds data to many applications.
# A '''GPS Controller''' to that can talk to the mulitplexer (and hence the GPS receiver) to display status and adjust settings.
# A '''Data Logger''' that can run all the time and record GPS data.
# A '''Field Collection Tool''' that allows doing specific operations such as averaging GPS data while waiting at a point for increased accuracy and entering attributes for a point.

'''Mapping''' We should consider how to support mapping as a part of the field collection tool; it is not strictly necessary but it can be very nice to have it incorporated. Likewise if you are logging a breadcrumb trail it can be very nice to be able to see it on the map. A mapping component will need to be able to update its display with data collected from the logger and the field collection tool.

== Main components ==

=== Very simple data logger ===

A very simple way to capture all GPS data is
 echo rw | netcat localhost gpsd > a.dump
 gpsbabel -i nmea -f a.dump -o gpx -F a.gpx

=== GPS multiplexer ===

Look at gpsd and gypsy and either use one of them or use them for inspiration. 
See the Gypsy author's take on GPSd here: http://folks.o-hand.com/iain/gypsy/why-not-gpsd.html

So far Gypsy looks pretty good to me.

Maybe the control program should be separated out from the multiplexer.

Partial list of requirements for the multiplexer
* Control internal GPS receiver
* Control external GPS connected via serial port (USB or bluetooth)
* Feed data from the selected GPS receiver to multiple applications.
* Handle a RTCM stream to allow realtime DGPS postprocessing.
* Supply all attributes to applications, including such things as bearing, PDOP and satellites in use. (Not just time and position).

Angle / inclination -- it might be possible to use the accelerometers in the smartphone to determine current inclination and calculate percent grade. You'd have to be able to calibrate this: mount the smartphone in a bracket and then read the acceleratometers to set it to 0 when on level ground.

==== External GPS ====

There is no reason we are limited to using the built-in GPS receiver. For applications requiring more accuracy (for example sub-meter), the GPS multiplexer should be able to accept data via USB or bluetooth and feed it to our applications. The controller program should be able to handle this situation.

Bearing - some GPS receivers include an electronic compass that provides azimuth even when not moving. The multiplexer should know if bearing values are valid or not and indicate this. The value should indicate whether it is set for magnetic or true north.

=== Support for postprocessing? ===

Real-time DGPS correction possibilities include receiving an RTCM stream from an external device, receiving RTCM over an IP connection, and WAAS. Does the u-blox receiver support RTCM? Most receivers that support it just accept a stream of data from a serial line. You don't have to reverse engineer it, you just accept data from some source such as a DGPS Beacon Receiver and feed the output to the GPS receiver and you get something approaching 1 meter accuracy.

Postprocessed DGPS correction requires recording pseudorange data and running an application later. In the open source world this means [http://en.wikipedia.org/wiki/RINEX RINEX]. Anyone with more knowledge of RINEX should jump in and edit this section.

=== GPS controller ===

''I think OM gps test program (what's it called) does much of this.''

Some GPS receivers include a feature to stop logging or to issue warnings if data quality goes too low for any reason. Where should this be handled? In the multiplexer or in the applications? Maybe the settings should be applied to the multiplexer and it should issue warnings to the applications which would then decide what to do.

=== Data Logger ===

The basic concept of the logger is to create a bread crumb trail as you move about. 
It has its own preference settings to determine if points are plotted based on change in position or on a time basis or both. 

It should also be possible to set what is logged. Possibilities include time, position, bearing, velocity, elevation, pseudo-range data, HDOP, PDOP, satellites used, type of GPS, type of antenna, DGPS on/off, type of DGPS.

==== Speedometer, Pedomater -- Usage of logger by itself ====

The data logger would be one crucial component in an application of GPS for bicyclists (and outdoor enthusiasts such as hikers and runners.) Other requirements would be a good display (to substitute for a speedometer) and a good mapping program. 

Should we expand on that on this page? It would be neat to have something that interoperates with the GPS multiplexer and the data logger so maybe we should?

Speedometer functions would include tracking minimums, maximums, averages for speed, distance, and elevation (stopped, moving, combined). Bicyclists (and runners? I am not a runner...) want to know total climb.

''I ride a bicycle and I use GPS as a speedometer and to do field work and I want to be able to do both with my FreeRunner.'' [[User:Brian H Wilson|Brian H Wilson]] 19:38, 18 August 2008 (UTC)

=== Field Collection Tool ===

'''Attributes''' Mission planning should include setting up a file to be used for data collection, and that file should define attributes to be included. The data collector should be configurable to facilitate easy entry of the attribute data. For example, if you define a field to have a finite list of values, the values could be available as a drop down list.

'''Point averaging''' Stand in one place and collect points until you are happy. The display should show current PDOP/HDOP and expected accuracy and number of readings used to create tehe point. Preferences should include storing all data used to create the averaged point.

'''Points, lines and polygons.''' You should be able to collect all three. A typical use for polygons is to walk around a space while collecting GPS points to create a polygon showing the location of some physical object such as a wetland.

==== Other related tools ====

Other specialized field collection tools could be used for 

* Asset management, example: urban forestry, agriculture, public works
* Map making, for example to collect data for the Open Street Map project
* Geocaching
* ??? Add yours here ???

== Data format and exchange ==

The data should be stored internally in a compact form that is open or at least well documented (for example ESRI Shapefiles). Storing in XML based formats is bulky and should probably be avoided. 

There should be a desktop tool to manage transferring the data that has been collected from the smartphone to more portable exchange formats such as GPX and GML and KML. "GPS Office"

=== Native u-blox format ===

One suggestion is to simply store and process the data in the u-blox binary format.
This might be a starting point but leaves out the possibility of the software interoperating
with other receivers or even running on the Neo 1973.

=== RINEX ===

Would RINEX be an appropriate general format for internal storage? No.
It's an exchange format but it's intended to include data required to perform post-processing.
It stores direct satellite observations, not things needed for navigation like calculated position.

This guy wrote a program that takes u-blox Antaris 4 data and writes it as RINEX data.
http://home.comcast.net/~dmilbert/softs/ant2rin.htm

GPX would accommodate everything we need but it's ASCII XML and that means 1/2 the file is just XML formatting. It might be possible to used tokenized GPX to make it more compact.

== More features ==

=== Audio clips ===

It would be useful to be able to capture short audio comments, especially when collecting data for Openstreetmap. This can be easily done by starting recording when AUX is pressed and stopping it when it is released. However, AUX is not easy to press when the phone is inside water-proof case and a large button on touchscreen is more suitable in that case. Here is a really simple proof-of-concept of such an UI:

 #!/usr/bin/python
 import gobject
 import time
 import os
 import sys
 import pygtk
 import gtk
 import pango
 
 def dbg(s):
     print("%s %s" % (time.strftime("%H:%M:%S"), s))
 
 class UI:
     def __init__(self, loop):
         self.loop = loop
 
         w = gtk.Window(gtk.WINDOW_TOPLEVEL)
         w.set_default_size(480, 300)
         w.connect("delete_event", (lambda w,e: False))
         w.connect("destroy", (lambda w: self.loop.quit()))
         w.set_border_width(0)
         w.show()
 
         vbox = gtk.VBox(False, 0)
         w.add(vbox)
         vbox.show()
 
         b = gtk.Button("Record")
         b.connect("pressed", self.cbRecordStart)
         b.connect("released", self.cbRecordStop)
         b.set_focus_on_click(False)
         b.set_size_request(480, 400)
         self.record_button = b
         self.record_button.modify_bg(gtk.STATE_ACTIVE, gtk.gdk.color_parse('red'))
         vbox.pack_start(b)
         b.show()
 
     def cbRecordStart(self, w):
         dbg("cbRecordStart")
         os.system("start-arecord")
 
     def cbRecordStop(self, w):
         dbg("cbRecordStop")
         os.system("stop-arecord")
 
 if __name__ == "__main__":
     loop = gobject.MainLoop()
     UI(loop)
     loop.run()
 
=== Laser rangefinder ===

There is another common situtaion where people use a laser rangefinder in conjunction with GPS, to get the location of an object that is not reachable (for example a boat) or a tree trunk (under heavy canopy).

==== Communication with camera ====

Since Neo doesn't have internal camera and because any internal camera could not be so quality as hi-end reflex cameras, it would be great if there will be an application wich access camera connected to UBS port as mass storage device, reads list of photos with their acquiring times from EXIF information, compare this times with GPS log and sets photo acquiring position back to EXIF information. All of this on field, with a "single click" and without any other device than Neo. ''Is there any other communication protocol between cameras and GPS devices?''

== External links ==

GPSd http://gpsd.berlios.de/

GPX http://www.topografix.com/gpx.asp http://en.wikipedia.org/wiki/GPX_(data_transfer)

gypsy http://folks.o-hand.com/iain/gypsy/ GPS data multiplexer

RINEX specs ftp://ftp.unibe.ch/aiub/rinex/

u-blox Antaris product page http://www.u-blox.com/products/a4products.html

=== Commercial software ===

ESRI ArcPad http://www.tdsway.com/products/solo_field

TDS Solo Field http://www.tdsway.com/products/solo_field

Trimble Terrasync http://www.trimble.com/terrasync.shtml

== People working on this project ==

[[User:Brian H Wilson]] is blabbing a lot here and wishing he had more time to work on this.

[[Category: GPS]]