1. Load your first measurement files

The goal of this notebook is to show the different options of loading measurements from raw DTS files. These files are loaded into a xarray.Dataset object. This object has various methods for calibration, plotting. Both single-ended and double-ended measurements are supported. The current supported devices are: - Silixa - Sensornet - AP Sensing - Sensortran

See notebooks A2, A3, and A4.

[1]:
import os
import glob

from dtscalibration import read_silixa_files

The example data files are located in ./python-dts-calibration/tests/data.

[2]:
filepath = os.path.join("..", "..", "tests", "data", "double_ended2")
print(filepath)
../../tests/data/double_ended2
[3]:
# Bonus: Just to show which files are in the folder
filepathlist = sorted(glob.glob(os.path.join(filepath, "*.xml")))
filenamelist = [os.path.basename(path) for path in filepathlist]

for fn in filenamelist:
    print(fn)
channel 1_20180328014052498.xml
channel 1_20180328014057119.xml
channel 1_20180328014101652.xml
channel 1_20180328014106243.xml
channel 1_20180328014110917.xml
channel 1_20180328014115480.xml

Define in which timezone the measurements are taken. In this case it is the timezone of the Silixa Ultima computer was set to ‘Europe/Amsterdam’. The default timezone of netCDF files is UTC. All the steps after loading the raw files are performed in this timezone. Please see www..com for a full list of supported timezones. We also explicitely define the file extension (.xml) because the folder is polluted with files other than measurement files.

[4]:
ds = read_silixa_files(directory=filepath, timezone_netcdf="UTC", file_ext="*.xml")
6 files were found, each representing a single timestep
6 recorded vars were found: LAF, ST, AST, REV-ST, REV-AST, TMP
Recorded at 1693 points along the cable
The measurement is double ended
Reading the data from disk
/home/docs/checkouts/readthedocs.org/user_builds/python-dts-calibration/envs/latest/lib/python3.9/site-packages/dtscalibration/io/silixa.py:100: UserWarning: Converting non-nanosecond precision timedelta values to nanosecond precision. This behavior can eventually be relaxed in xarray, as it is an artifact from pandas which is now beginning to support non-nanosecond precision values. This warning is caused by passing non-nanosecond np.datetime64 or np.timedelta64 values to the DataArray or Variable constructor; it can be silenced by converting the values to nanosecond precision ahead of time.
  ds = xr.Dataset(data_vars=data_vars, coords=coords, attrs=attrs, **kwargs)
/home/docs/checkouts/readthedocs.org/user_builds/python-dts-calibration/envs/latest/lib/python3.9/site-packages/dtscalibration/io/silixa.py:100: UserWarning: Converting non-nanosecond precision timedelta values to nanosecond precision. This behavior can eventually be relaxed in xarray, as it is an artifact from pandas which is now beginning to support non-nanosecond precision values. This warning is caused by passing non-nanosecond np.datetime64 or np.timedelta64 values to the DataArray or Variable constructor; it can be silenced by converting the values to nanosecond precision ahead of time.
  ds = xr.Dataset(data_vars=data_vars, coords=coords, attrs=attrs, **kwargs)

The object tries to gather as much metadata from the measurement files as possible (temporal and spatial coordinates, filenames, temperature probes measurements). All other configuration settings are loaded from the first files and stored as attributes of the DataStore.

[5]:
print(ds)
<xarray.Dataset>
Dimensions:                (x: 1693, time: 6)
Coordinates: (12/14)
  * x                      (x) float64 -80.5 -80.38 -80.25 ... 134.3 134.4 134.5
    filename               (time) <U31 'channel 1_20180328014052498.xml' ... ...
    filename_tstamp        (time) int64 20180328014052498 ... 20180328014115480
    timeFWstart            (time) datetime64[ns] 2018-03-28T00:40:52.097000 ....
    timeFWend              (time) datetime64[ns] 2018-03-28T00:40:54.097000 ....
    timeFW                 (time) datetime64[ns] 2018-03-28T00:40:53.097000 ....
    ...                     ...
    timeBW                 (time) datetime64[ns] 2018-03-28T00:40:55.097000 ....
    timestart              (time) datetime64[ns] 2018-03-28T00:40:52.097000 ....
    timeend                (time) datetime64[ns] 2018-03-28T00:40:56.097000 ....
  * time                   (time) datetime64[ns] 2018-03-28T00:40:54.097000 ....
    acquisitiontimeFW      (time) timedelta64[ns] 00:00:02 00:00:02 ... 00:00:02
    acquisitiontimeBW      (time) timedelta64[ns] 00:00:02 00:00:02 ... 00:00:02
Data variables: (12/14)
    st                     (x, time) float64 1.281 -0.5321 ... -43.44 -41.08
    ast                    (x, time) float64 0.4917 1.243 ... -30.14 -32.09
    rst                    (x, time) float64 0.4086 -0.568 ... 4.822e+03
    rast                   (x, time) float64 2.569 -1.603 ... 4.224e+03
    tmp                    (x, time) float64 196.1 639.1 218.7 ... 8.442 18.47
    acquisitionTime        (time) float32 2.098 2.075 2.076 2.133 2.085 2.062
    ...                     ...
    probe2Temperature      (time) float32 18.58 18.58 18.58 18.58 18.58 18.57
    referenceProbeVoltage  (time) float32 0.1217 0.1217 0.1217 ... 0.1217 0.1217
    probe1Voltage          (time) float32 0.114 0.114 0.114 0.114 0.114 0.114
    probe2Voltage          (time) float32 0.121 0.121 0.121 0.121 0.121 0.121
    userAcquisitionTimeFW  (time) float32 2.0 2.0 2.0 2.0 2.0 2.0
    userAcquisitionTimeBW  (time) float32 2.0 2.0 2.0 2.0 2.0 2.0
Attributes: (12/350)
    uid:                                                                     ...
    nameWell:                                                                ...
    nameWellbore:                                                            ...
    name:                                                                    ...
    indexType:                                                               ...
    startIndex:uom:                                                          ...
    ...                                                                                                                                                    ...
    customData:UserConfiguration:ChannelConfiguration_3:FibreCheckConfigurati...
    customData:UserConfiguration:ChannelConfiguration_3:FibreCorrectionConfig...
    customData:UserConfiguration:ChannelConfiguration_3:FibreCorrectionConfig...
    isDoubleEnded:                                                           ...
    forwardMeasurementChannel:                                               ...
    backwardMeasurementChannel:                                              ...