# MSK-Bench Weights and Experiment Artifacts

The `data/` directory contains policy weights for MyoSuite/MSK-Bench tasks, together with their configuration files, training logs, and evaluation results. The top-level folders identify algorithms or experiment series. Weights from different folders are not interchangeable, even when their task names look similar.

## Where to find weights

| Directory | Contents | Start with |
| --- | --- | --- |
| [`deprl/`](./deprl/) | DEPRL walking, running, and stair-climbing baselines; the configurations use DEP + MPO | `walk/`, `run/`, or `stair/` → `checkpoints/best.pt` |
| [`msgym/`](./msgym/) | `SAC_DynSyn` policies organized by MyoSuite environment and experiment run | `<environment>/<run>/checkpoint/best_model.zip` |
| [`ppo/`](./ppo/) | PPO weights, normalization state, evaluation reports, and training logs | `ppo_<task>_checkpoints/best_model.zip` |
| [`residualrl/`](./residualrl/) | ResidualRL walking, running, and stair-climbing experiments; the configurations use DEP + MPO | `<task>/checkpoints/best.pt` |
| [`sac/`](./sac/) | SAC weights, normalization state, evaluation reports, and training logs | `sac_<task>_checkpoints/best_model.zip` |

The word `best` is part of the original filenames; this README does not infer which metric was used to select those checkpoints. To reproduce an evaluation, also check the configuration, environment version, and any associated preprocessing or normalization state.

## `deprl/`: DEPRL baselines

This directory has three task folders. Each numbered subdirectory contains one experiment run. The environment names below come from the corresponding `config.yaml` files.

| Subdirectory | Environment in the configuration | Purpose |
| --- | --- | --- |
| `walk/260521.131030/` | `myoLegWalk-v0` | Leg-walking policy and evaluation results |
| `run/260502.131631/` | `MyoRun-v0` | Running policy and evaluation results |
| `stair/260202.174137/` | `myoLegStair-v0` | Stair-climbing policy and evaluation results |

A run typically contains:

- `config.yaml`: environment, DEP, MPO, training, and checkpoint settings.
- `script.py`: the saved training script, useful when inspecting or reproducing the experiment.
- `checkpoints/best.pt`: the checkpoint labeled best; `step_*.pt` files are checkpoints saved at specific training steps.
- `checkpoints/logger.pt` and `checkpoints/time.pt`: saved logging or training-progress state, not standalone policy weights.
- `log.csv`: training and test metrics.
- `eval_*.png` and `*_compliance_report.csv`: plots and numerical reports for energy, jitter, muscle actions, and physical joints.
- `EMG_*`: electromyography-related results, where present.

## `msgym/`: Policies grouped by environment and run

The first level is a MyoSuite environment, such as `myoLegWalk-v0/` or `MyoRun-v0/`. The next level, such as `0625-005737_0/`, contains the artifacts from one run. Some environments have several runs; keep a model with the configuration and environment state from the same run. All current run configuration JSON files identify the agent as `SAC_DynSyn`.

Environment folders:

```text
MyoBunnyHop-v0       MyoCatch-v0            MyoChinUp-v0
MyoCrawl-v0          MyoDoorOpen-v0         MyoHurdle-v0
myoLegBalance-v0     myoLegSquat-v0         myoLegStair-v0
myoLegStandRandom-v0 myoLegWalk-v0          MyoMazeWalk-v0
MyoPoleWalk-v0       myoPowerlift-v0        MyoRun-v0
MyoSidestep-v0       MyoSingleLegStance-v0  MyoSteppingStones-v0
myoWalkAndSit-v0
```

Common files in a run:

| Path or file | Purpose |
| --- | --- |
| `<task>.json` | Algorithm, environment, network, and training parameters; the filename varies by task. |
| `checkpoint/best_model.zip` and `final_model.zip` | Stable-Baselines3 policy archives labeled best and final by the original experiment. Some runs may contain only one of them. |
| `checkpoint/best_env.zip` and `final_env.zip` | Environment state saved alongside the models. **Despite their `.zip` extensions, these are Python pickle streams, not ordinary ZIP archives.** Load them using the original experiment's code. |
| `eval/evaluations.npz` | Periodic evaluation data in NumPy format. |
| `monitor/*.monitor.csv` | Episode records, including reward `r`, length `l`, and time `t`. |
| `events.out.tfevents.*` | TensorBoard event logs. |

## `ppo/` and `sac/`: Task-specific checkpoints and logs

Both directories pair a `<algorithm>_<task>_checkpoints/` folder (models and evaluation results) with a `<algorithm>_<task>_logs/` folder (evaluation and training logs). For example, `ppo_balance_checkpoints/` pairs with `ppo_balance_logs/`. SAC uses the `sac_` prefix.

Current task suffixes in `ppo/`:

```text
balance, bunny, catch, chinup, crawl, door, hurdle, maze, pole_walk,
powerlift, reach, run, sidestep, sit, slidehard, squat, stair, stance,
stepping_stones, walkandsit
```

Current task suffixes in `sac/`:

```text
balance, bunnyhop, catch, chinup, crawl, door, hurdle, maze, pole_walk,
powerlift, reach, run, sidestep, singlelegstance, sit, slidehard, squat,
stair, stepping_stones, walkandsit
```

These suffixes preserve the original directory names. The two algorithms do not have exactly the same task set or spelling. Key files in each `*_checkpoints/` folder are:

- `best_model.zip`: a Stable-Baselines3 policy archive.
- `vec_normalize.pkl`: vector-environment normalization state associated with that task's model; check and use the matching file when evaluating.
- `eval_*_energy_plot.png` and `eval_*_jitter_report.png`: energy and jitter evaluation plots.
- `Muscle_Actions_compliance_report.csv` and `Physical_Joints_compliance_report.csv`: per-dimension muscle-action and joint statistics, including derivative-energy and jerk fields.

The matching `*_logs/` folder typically contains `eval_logs/evaluations.npz` (evaluation data) and `events.out.tfevents.*` under `PPO_1/` or `SAC_1/` (TensorBoard training logs).

## `residualrl/`: ResidualRL experiments

The `run/`, `stair/`, and `walk/` folders correspond to `MyoRunResidual-v0`, `MyoStairResidual-v0`, and `MyoWalkResidual-v0` in their configurations. Each task folder contains:

- `config.yaml`: environment, DEP, MPO, and training parameters.
- `script.py`: the saved training script.
- `checkpoints/best.pt`: the checkpoint labeled best.
- `checkpoints/step_*.pt`: checkpoints saved at specific training steps; `walk/` retains more intermediate steps.
- `checkpoints/logger.pt` and `checkpoints/time.pt`: logging or training-progress state.
- `log.csv`: training and test metrics.

## Keeping files together

1. Choose the **algorithm and environment/task** first, then select a weight file inside that directory. Similar task names do not make weights from different algorithms interchangeable.
2. For `ppo/` and `sac/`, check `best_model.zip` together with the task's `vec_normalize.pkl`. For `msgym/`, keep the model with the JSON configuration and environment state from the same run.
3. Loading `.pt`, `.pkl`, or `msgym`'s `*_env.zip` files may require the corresponding Python classes and library versions. These are experiment artifacts; the weight files alone are not guaranteed to provide a working inference setup.
4. `log.csv`, `*.npz`, `*.monitor.csv`, `events.out.tfevents.*`, and the evaluation plots are for inspecting training or comparing experiments. They are not policy weights.