A2. Loading sensornet filesΒΆ
This example loads sensornet files. Both single-ended and double-ended measurements are supported.
[1]:
import os
import glob
from dtscalibration import read_sensornet_files
The example data files are located in ./python-dts-calibration/tests/data
.
[2]:
filepath = os.path.join("..", "..", "tests", "data", "sensornet_oryx_v3.7")
print(filepath)
../../tests/data/sensornet_oryx_v3.7
[3]:
filepathlist = sorted(glob.glob(os.path.join(filepath, "*.ddf")))
filenamelist = [os.path.basename(path) for path in filepathlist]
for fn in filenamelist:
print(fn)
channel 1 20180107 202119 00001.ddf
channel 1 20180107 202149 00001.ddf
channel 1 20180107 202219 00001.ddf
channel 1 20180107 202249 00001.ddf
channel 1 20180107 202319 00001.ddf
channel 1 20180107 202349 00001.ddf
channel 1 20180107 202418 00001.ddf
We will simply load in the sensornet files. As the sensornet files are of low spatial and temporal resolution, reading the data lazily into dask is not supported.
[4]:
ds = read_sensornet_files(directory=filepath)
7 files were found, each representing a single timestep
Recorded at 2068 points along the cable
The measurement is single ended
/home/docs/checkouts/readthedocs.org/user_builds/python-dts-calibration/envs/latest/lib/python3.9/site-packages/dtscalibration/datastore.py:81: UserWarning: Converting non-nanosecond precision timedelta values to nanosecond precision. This behavior can eventually be relaxed in xarray, as it is an artifact from pandas which is now beginning to support non-nanosecond precision values. This warning is caused by passing non-nanosecond np.datetime64 or np.timedelta64 values to the DataArray or Variable constructor; it can be silenced by converting the values to nanosecond precision ahead of time.
super().__init__(*args, **kwargs)
/home/docs/checkouts/readthedocs.org/user_builds/python-dts-calibration/envs/latest/lib/python3.9/site-packages/dtscalibration/datastore.py:81: UserWarning: Converting non-nanosecond precision timedelta values to nanosecond precision. This behavior can eventually be relaxed in xarray, as it is an artifact from pandas which is now beginning to support non-nanosecond precision values. This warning is caused by passing non-nanosecond np.datetime64 or np.timedelta64 values to the DataArray or Variable constructor; it can be silenced by converting the values to nanosecond precision ahead of time.
super().__init__(*args, **kwargs)
The object tries to gather as much metadata from the measurement files as possible (temporal and spatial coordinates, filenames, temperature probes measurements). All other configuration settings are loaded from the first files and stored as attributes of the DataStore
.
[5]:
print(ds)
<dtscalibration.DataStore>
Sections: ()
Dimensions: (x: 1380, time: 7, trans_att: 0)
Coordinates:
* x (x) float64 -49.97 -48.96 ... 1.348e+03 1.349e+03
filename (time) <U35 'channel 1 20180107 202119 00001.ddf' ...
timestart (time) datetime64[ns] 2018-01-07T20:20:49 ... 2018...
timeend (time) datetime64[ns] 2018-01-07T20:21:19 ... 2018...
* time (time) datetime64[ns] 2018-01-07T20:21:04 ... 2018...
acquisitiontimeFW (time) timedelta64[ns] 00:00:30 00:00:30 ... 00:00:30
* trans_att (trans_att) float64
Data variables:
st (x, time) float64 1.482e+03 1.482e+03 ... -0.324
ast (x, time) float64 956.4 956.4 954.7 ... -0.121 0.458
tmp (x, time) float64 17.5 17.51 17.22 ... 700.0 312.9
probe1Temperature (time) float64 3.12 3.09 3.09 3.07 3.07 3.12 3.07
probe2Temperature (time) float64 1.259e+03 1.259e+03 ... 1.259e+03
referenceTemperature (time) float64 15.34 15.37 15.34 ... 15.34 15.31
gamma_ddf (time) float64 498.8 498.8 498.8 ... 498.8 498.8
k_internal (time) float64 0.2786 0.2787 0.2786 ... 0.2785 0.2785
k_external (time) float64 0.2786 0.2787 0.2786 ... 0.2785 0.2785
userAcquisitionTimeFW (time) float64 30.0 30.0 30.0 30.0 30.0 30.0 30.0
userAcquisitionTimeBW (time) float64 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Attributes: (12/21)
DTS Sentinel unit serial number: SN409017
Multiplexer serial number: ORYX
Hardware model number: OX4
Software version number: ORYX F/W v1,02 Oryx Data Collector v3.7...
data status: ok
installation: speulderbos2017nov21
... ...
fibre end: 0.00
default loss term dB per km: 0.3730
.. and many more attributes. See: ds.attrs
Double ended sensornet files are also supported. Note the REV-ST and REV-AST data variables.
[6]:
filepath = os.path.join("..", "..", "tests", "data", "sensornet_halo_v1.0")
ds = read_sensornet_files(directory=filepath)
print(ds)
5 files were found, each representing a single timestep
Recorded at 978 points along the cable
The measurement is double ended
<dtscalibration.DataStore>
Sections: ()
Dimensions: (x: 712, time: 5, trans_att: 0)
Coordinates: (12/14)
* x (x) float64 -49.28 -47.25 ... 1.391e+03 1.393e+03
filename (time) <U32 'channel 1 20030111 002 00004.ddf' ......
timeFWstart (time) datetime64[ns] 2003-01-11T03:17:09 ... 2003...
timeFWend (time) datetime64[ns] 2003-01-11T03:18:09 ... 2003...
timeFW (time) datetime64[ns] 2003-01-11T03:17:39 ... 2003...
timeBWstart (time) datetime64[ns] 2003-01-11T03:18:09 ... 2003...
... ...
timestart (time) datetime64[ns] 2003-01-11T03:17:09 ... 2003...
timeend (time) datetime64[ns] 2003-01-11T03:19:09 ... 2003...
* time (time) datetime64[ns] 2003-01-11T03:18:09 ... 2003...
acquisitiontimeFW (time) timedelta64[ns] 00:01:00 00:01:00 ... 00:01:00
acquisitiontimeBW (time) timedelta64[ns] 00:01:00 00:01:00 ... 00:01:00
* trans_att (trans_att) float64
Data variables: (12/13)
st (x, time) float64 1.876e+03 1.877e+03 ... -0.513
ast (x, time) float64 2.138e+03 2.139e+03 ... -0.603
tmp (x, time) float64 60.57 81.6 84.19 ... -200.0 -108.4
probe1Temperature (time) float64 nan nan nan nan nan
probe2Temperature (time) float64 nan nan nan nan nan
referenceTemperature (time) float64 34.25 34.25 34.42 34.25 34.31
... ...
k_internal (time) float64 0.1898 0.1898 0.1902 0.1898 0.1898
k_external (time) float64 0.1898 0.1898 0.1902 0.1898 0.1898
userAcquisitionTimeFW (time) float64 60.05 60.05 60.05 60.05 60.05
userAcquisitionTimeBW (time) float64 60.05 60.05 60.08 60.05 60.06
rst (x, time) float64 -0.459 -0.504 ... 1.764e+03
rast (x, time) float64 -0.663 -0.622 ... 2.249e+03
Attributes: (12/21)
DTS Sentinel unit serial number: SN307009
Multiplexer serial number: multiplexer serial number
Hardware model number: HL4
Software version number: Halo DTS v1.0
data status: ok
installation: NYAN30AUG2019
... ...
fibre end: 1298.10
default loss term dB per km: 0.3938
.. and many more attributes. See: ds.attrs
/home/docs/checkouts/readthedocs.org/user_builds/python-dts-calibration/envs/latest/lib/python3.9/site-packages/dtscalibration/datastore.py:81: UserWarning: Converting non-nanosecond precision timedelta values to nanosecond precision. This behavior can eventually be relaxed in xarray, as it is an artifact from pandas which is now beginning to support non-nanosecond precision values. This warning is caused by passing non-nanosecond np.datetime64 or np.timedelta64 values to the DataArray or Variable constructor; it can be silenced by converting the values to nanosecond precision ahead of time.
super().__init__(*args, **kwargs)
/home/docs/checkouts/readthedocs.org/user_builds/python-dts-calibration/envs/latest/lib/python3.9/site-packages/dtscalibration/datastore.py:81: UserWarning: Converting non-nanosecond precision timedelta values to nanosecond precision. This behavior can eventually be relaxed in xarray, as it is an artifact from pandas which is now beginning to support non-nanosecond precision values. This warning is caused by passing non-nanosecond np.datetime64 or np.timedelta64 values to the DataArray or Variable constructor; it can be silenced by converting the values to nanosecond precision ahead of time.
super().__init__(*args, **kwargs)
/home/docs/checkouts/readthedocs.org/user_builds/python-dts-calibration/envs/latest/lib/python3.9/site-packages/dtscalibration/datastore.py:81: UserWarning: Converting non-nanosecond precision timedelta values to nanosecond precision. This behavior can eventually be relaxed in xarray, as it is an artifact from pandas which is now beginning to support non-nanosecond precision values. This warning is caused by passing non-nanosecond np.datetime64 or np.timedelta64 values to the DataArray or Variable constructor; it can be silenced by converting the values to nanosecond precision ahead of time.
super().__init__(*args, **kwargs)
/home/docs/checkouts/readthedocs.org/user_builds/python-dts-calibration/envs/latest/lib/python3.9/site-packages/dtscalibration/datastore.py:81: UserWarning: Converting non-nanosecond precision timedelta values to nanosecond precision. This behavior can eventually be relaxed in xarray, as it is an artifact from pandas which is now beginning to support non-nanosecond precision values. This warning is caused by passing non-nanosecond np.datetime64 or np.timedelta64 values to the DataArray or Variable constructor; it can be silenced by converting the values to nanosecond precision ahead of time.
super().__init__(*args, **kwargs)