Tutorial 5: Saving data to files

Once a robot backend is started, for example by calling in a terminal:

o80_mujoco

it starts dumping at each iteration instances of Observation. Observation is a class encapsulating information about the robot, such as:

  • current pressure for each muscle

  • desired pressure for each muscle

  • current backend iteration

  • current backend frequency

  • time stamp of the Observation

  • position of each joint (in radian)

  • velocity of each joint (in radian per seconds)

Dumping data

It is possible to dump the Observation instances continuously generated by the backend into files for later analysis, either by starting an executable or via the python API.

Via executable

Start in a terminal:

o80_logger

A dialog will allow you to configure the logger. Type ‘h’ for more information. Note that the proposed file path does not correspond to an existing file, so using it will not result into deleting an existing file.

The process will start dumping observations in the selected file, until ctrl+c is pressed (or the duration is passed, if you configured the logger this way).

Via Python API

import time
import o80_pam

segment_id = "o80_pam_robot" # should correspond to the segment_id of the backend robot
file_path = "/tmp/mysave" # warning: will be overwritten if it already exists
frequency = 500 # frequency at which new observations will be dumped 

logger = o80_pam.Logger(segment_id,file_path,frequency=frequency)

logger.start() # spawn a process which starts dumping observations into the file
time.sleep(10)
logger.stop() # stops the process

alternatively, a context manager can be used:

print("\nsaving data for 5 seconds ...")
with o80_pam.Logger("o80_pam_robot",path):
    time_start = time.time()
    while time.time()-time_start < 5:
        time.sleep(0.01)
print("... done")

The file manager proposes path to files in which the data can be saved:

fm = o80_pam.FileManager()
path = fm.next()
print("\ndata will be saved in {}".format(path))

print("\nsaving data for 5 seconds ...")
with o80_pam.Logger("o80_pam_robot",path):
    time_start = time.time()
    while time.time()-time_start < 5:
        time.sleep(0.01)
print("... done")

# latest will be the same as path
print("data was saved in: {}",fm.latest())

Warning

The file manager returns path that are in the tmp folder, so the files will be deleted automatically by the OS at some point. Be careful to backup the important files

Reading the data

Via executable

This executable will provide some basic info about a data file:

o80_log_file_stats

The dialog will allow you to select the file path. Note that if the data was collected via the executable o80_logger, the last dumped file is proposed as default file path.

Via Python API

import o80_pam

for observation in o80_pam.read_file("/path/to/my/file"):
    print(observation)

Converting to Pandas dataframe files

You can convert a log file to a pickled pandas dataframe:

from pathlib import Path
import o80_pam

log_file = Path("/path/to/log/file")
dataframe_file = Path("/path/to/new/file")

# reading log_file and writing dataframe_file
o80_pam.observation_convertors.convert_native_file_to_pandas(
    log_file, dataframe_file
)

# reading dataframe_file to panda dataframe
df = o80_pam.observation_convertors.read_pandas(dataframe_file)

# you can also directly read a log file into a pandas dataframe
df = o80_pam.observation_convertors.native_file_to_pandas(log_file)

More information on convertions between Pandas dataframe and instances of Observation can be found in Tutorial 7.