• Autonomous data acquisition and recording (currently up to 8 channels @ 500Hz)
  • Based on the (Raspberry PI microcomputer) (low power 3W, running high-availability Linux)
  • Online data in real-time
  • ssh command line access even from browser
  • www lightweight web server with diagnostic tools running on the Raspberry
  • Advanced remote administration tools like remote-syslog over a websocket.
  • ftp access for data transfer, remote synchronization etc.
  • Standard components (Raspberry PI + Analog to Digital converter e.g. ADS1115 + some enclosure)
  • Creative Commons License open source software written in C (community-driven, and still under maintenance and development) and efficient system configuration.
  • Powerful visualization: FlotCharts
  • Highly efficient binary output (NetCDF > compressed with HDF5, JSON, postgreSQL etc.)

It is an adaptable system, even from one configuration file you can define the number of channels used, the voltage range (if not dynamic tracking chosen) or the output format. There are countless opportunities to adapt or extend any part of the software/hardware. You are invited to use, give feedback or even contribute!

Examples

data logger with differential inputs front face of a data logger with display 4 differential channels with isolated BNC connectors power over ethernet (and the alternative power supply input 5V) sensor supply, converting 5V to regulated dual voltage with LM2587 a raspberry equipped with a GPS module to record accurate time power supply 5V 3A data loggers in robust metalic casing

The Operating System (OS) (real time Linux)

The Raspberry PI hosts an RT-PREEMPT kernel Linux which is a real-time OS. All the graphical and disk consuming packages are dropped, and those responsible for e.g network-security (fail2ban) and stable operation (e.g. watchdog) are installed and configured. Only the absolute minimum of the kernel modules are loaded. There is no sound support. This is necessary to have low-latency sampling intervals (in order of microseconds and less). The current logger application runs in the background (starting automatically, writing to log files, checking elapsed time since the last file write, checking disk usage, error handling...), but currently with a bit of overhead (threaded interrupts softirq. However the logger running with 4 channels each 500Hz sampling, utilizes 30% CPU on a Raspberry Pi B (800MHz) only. (The new version based on this article

preemptive OS (www.embeddedlinux.org)

preemptive OS (www.embeddedlinux.org)

Analog-to-Digital converter (ADC)

The universal ADC ADS1115 is a very versatile and cheap IC. The key features

  • One single IC has 2 differential channels (or 4 single ended) where 1 differential (or 2 single ended) are converted at a time. (They are switchable with a multiplexer (MUX) controlled by software)
  • sampling rates from 1Hz to 860Hz possible
  • although the resolution is 16 bit only there is a programmable gain amplifier (PGA) which can be set by software in 5 steps. By tracking the signal's amplitude and setting the PGA dynamically it converges to a 20 bit ADC. (auto-pga feature)
  • at maximum 4 IC-s can be connected simultaneously to the I2C bus of the Raspberry PI (4 differential channels sampled at a time, altogether 8 in two steps).
  • excellent linearity, simple commands, low consumption, high input impedance, internal voltage reference, no thermal drift...
  • fast data transfer using the i2c-dev kernel module, I2C-bus clock speed (SCK) of 3MHz verified!
ADS1115 analog to digital converter

ADS115 analog to digital converter (www.adafruit.com)

File formats

binary files (fast sampling rates)

After trying out many ways for large data storage (4 channels á 500 times a second sampled á 4 bytes (float)), I ended up at the high density (HDF5) compressed binary file format the Network Common Data Form or net-cdf (nc) which is fully implemented. It is as efficient that the 2 channel recording of 24 hours (sampling rate 500Hz each) which sums up to 860MB of ASCII data (as txt or csv files) will be compressed to only 55MB. There is excellent support on the command line (e.g. ncdump) to convert between ascii and binary but also python/matlab/octave deals with it very easily.

databases (lower sampling rates)

There are several C interfaces to put recordings into databases. After an excessive research (I am not a specialist on databases) postgreSQL turned out to be lightweight enough to work smoothly on the raspberry while carrying out performant INSERT INTO-tasks. However, IMHO for sampling rates above 10Hz (assuming 24/7 logging) it slows down just after 1 month with no special precautions (10*60*1440*31 > 26 mil. rows). For temperature logging or whatever slow dynamics measurement it's tool number one for me.

cloud storage of sensor data

Examples: nimbits, thingspeak, orchestra, sensorcloud, relayr, xively or tempoiq. I have not excessively used these, but there are surely no limitations on the side of the logger software, if this is one's preference to store data. The source code of the logger is kept as modular as possible.

There is a chance to implement JSON and some other timeseries data formats soon. However, contributors are welcome!

Further reading

References

Hungarian National Report on IUGG 2011-2014

universal raspberry pi based data logger developed for the NCK geophysical observatory - IAGA division 5. observatory, instruments, surveys and analyses



Comments

comments powered by Disqus

Further articles


The Linux embedded data-logger

  • Autonomous data acquisition and recording (currently up to 8 channels @ 500Hz)
  • Based on the (Raspberry PI microcomputer) (low power 3W, running high-availability Linux)
  • Online data in real-time
  • ssh command line access even from browser
  • www lightweight web server with diagnostic tools running on the Raspberry
  • Advanced remote administration tools like ...
more ...