# Tutorial 5: Saving data to files

Once a robot backend is started, for example by calling in a terminal:

```bash
o80_mujoco
```

it starts dumping at each iteration instances of Observation. Observation is a class encapsulating information about the robot, such as:
- current pressure for each muscle
- desired pressure for each muscle 
- current backend iteration
- current backend frequency
- time stamp of the Observation
- position of each joint (in radian)
- velocity of each joint (in radian per seconds)

## Dumping data

It is possible to dump the Observation instances continuously generated by the backend into files for later analysis, either by starting an executable or via the python API.

### Via executable

Start in a terminal:

```bash
o80_logger
```

A dialog will allow you to configure the logger. Type 'h' for more information. Note that the proposed file path does not correspond to an existing file, so using it will not result into deleting an existing file.

The process will start dumping observations in the selected file, until ctrl+c is pressed (or the duration is passed, if you configured the logger this way).

### Via Python API

```python
import time
import o80_pam

segment_id = "o80_pam_robot" # should correspond to the segment_id of the backend robot
file_path = "/tmp/mysave" # warning: will be overwritten if it already exists
frequency = 500 # frequency at which new observations will be dumped 

logger = o80_pam.Logger(segment_id,file_path,frequency=frequency)

logger.start() # spawn a process which starts dumping observations into the file
time.sleep(10)
logger.stop() # stops the process
```
alternatively, a context manager can be used:

```python
print("\nsaving data for 5 seconds ...")
with o80_pam.Logger("o80_pam_robot",path):
    time_start = time.time()
    while time.time()-time_start < 5:
        time.sleep(0.01)
print("... done")
```
The file manager proposes path to files in which the data can be saved:

```python
fm = o80_pam.FileManager()
path = fm.next()
print("\ndata will be saved in {}".format(path))

print("\nsaving data for 5 seconds ...")
with o80_pam.Logger("o80_pam_robot",path):
    time_start = time.time()
    while time.time()-time_start < 5:
        time.sleep(0.01)
print("... done")

# latest will be the same as path
print("data was saved in: {}",fm.latest())
```

```{Warning} 
The file manager returns path that are in the tmp folder, so the files will be deleted automatically by the OS at some point. Be careful to backup the important files
```

## Reading the data

### Via executable

This executable will provide some basic info about a data file:

```bash
o80_log_file_stats
```

The dialog will allow you to select the file path. Note that if the data was collected via the executable o80_logger, the last dumped file is proposed as default file path.

### Via Python API

```python
import o80_pam

for observation in o80_pam.read_file("/path/to/my/file"):
    print(observation)
```

## Converting to Pandas dataframe files

You can convert a log file to a pickled pandas dataframe:

```python
from pathlib import Path
import o80_pam

log_file = Path("/path/to/log/file")
dataframe_file = Path("/path/to/new/file")

# reading log_file and writing dataframe_file
o80_pam.observation_convertors.convert_native_file_to_pandas(
    log_file, dataframe_file
)

# reading dataframe_file to panda dataframe
df = o80_pam.observation_convertors.read_pandas(dataframe_file)

# you can also directly read a log file into a pandas dataframe
df = o80_pam.observation_convertors.native_file_to_pandas(log_file)

```

More information on convertions between Pandas dataframe and instances of Observation
can be found in [Tutorial 7](B8_tutorial7).