Update Space (evaluate main: 1c421923)

README.md

@@ -1,12 +1,115 @@
 ---
-title: Rl_reliability
-emoji: 👀
-colorFrom: indigo
-colorTo: yellow
+title: RL Reliability
+datasets:
+-  
+tags:
+- evaluate
+- metric
 sdk: gradio
-sdk_version: 3.0.9
+sdk_version: 3.0.2
 app_file: app.py
 pinned: false
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces#reference
+# Metric Card for RL Reliability
+
+## Metric Description
+The RL Reliability Metrics library provides a set of metrics for measuring the reliability of reinforcement learning (RL) algorithms. 
+
+## How to Use
+
+```python
+import evaluate
+import numpy as np
+
+rl_reliability = evaluate.load("rl_reliability", "online")
+results = rl_reliability.compute(
+    timesteps=[np.linspace(0, 2000000, 1000)],
+    rewards=[np.linspace(0, 100, 1000)]
+    )
+
+rl_reliability = evaluate.load("rl_reliability", "offline")
+results = rl_reliability.compute(
+    timesteps=[np.linspace(0, 2000000, 1000)],
+    rewards=[np.linspace(0, 100, 1000)]
+    )
+```
+
+
+### Inputs
+- **timesteps** *(List[int]): For each run a an list/array with its timesteps.*
+- **rewards** *(List[float]): For each run a an list/array with its rewards.*
+
+KWARGS:
+- **baseline="default"** *(Union[str, float]) Normalization used for curves. When `"default"` is passed the curves are normalized by their range in the online setting and by the median performance across runs in the offline case. When a float is passed the curves are divided by that value.*
+- **eval_points=[50000, 150000, ..., 2000000]** *(List[int]) Statistics will be computed at these points*
+- **freq_thresh=0.01** *(float) Frequency threshold for low-pass filtering.*
+- **window_size=100000** *(int) Defines a window centered at each eval point.*
+- **window_size_trimmed=99000** *(int) To handle shortened curves due to differencing*
+- **alpha=0.05** *(float)The "value at risk" (VaR) cutoff point, a float in the range [0,1].*
+
+### Output Values
+
+In `"online"` mode:
+- HighFreqEnergyWithinRuns: High Frequency across Time (DT)
+- IqrWithinRuns: IQR across Time (DT)
+- MadWithinRuns: 'MAD across Time (DT)
+- StddevWithinRuns: Stddev across Time (DT)
+- LowerCVaROnDiffs: Lower CVaR on Differences (SRT)
+- UpperCVaROnDiffs: Upper CVaR on Differences (SRT)
+- MaxDrawdown: Max Drawdown (LRT)
+- LowerCVaROnDrawdown: Lower CVaR on Drawdown (LRT)
+- UpperCVaROnDrawdown: Upper CVaR on Drawdown (LRT)
+- LowerCVaROnRaw: Lower CVaR on Raw
+- UpperCVaROnRaw: Upper CVaR on Raw
+- IqrAcrossRuns: IQR across Runs (DR)
+- MadAcrossRuns: MAD across Runs (DR)
+- StddevAcrossRuns: Stddev across Runs (DR)
+- LowerCVaROnAcross: Lower CVaR across Runs (RR)
+- UpperCVaROnAcross: Upper CVaR across Runs (RR)
+- MedianPerfDuringTraining: Median Performance across Runs
+
+In `"offline"` mode:
+- MadAcrossRollouts: MAD across rollouts (DF)
+- IqrAcrossRollouts: IQR across rollouts (DF)
+- LowerCVaRAcrossRollouts: Lower CVaR across rollouts (RF)
+- UpperCVaRAcrossRollouts: Upper CVaR across rollouts (RF)
+- MedianPerfAcrossRollouts: Median Performance across rollouts
+
+
+### Examples
+First get the sample data from the repository:
+
+```bash
+wget https://storage.googleapis.com/rl-reliability-metrics/data/tf_agents_example_csv_dataset.tgz
+tar -xvzf tf_agents_example_csv_dataset.tgz
+```
+
+Load the sample data:
+```python
+dfs = [pd.read_csv(f"./csv_data/sac_humanoid_{i}_train.csv") for i in range(1, 4)]
+```
+
+Compute the metrics:
+```python
+rl_reliability = evaluate.load("rl_reliability", "online")
+rl_reliability.compute(timesteps=[df["Metrics/EnvironmentSteps"] for df in dfs],
+                       rewards=[df["Metrics/AverageReturn"] for df in dfs])
+```
+
+## Limitations and Bias
+This implementation of RL reliability metrics does not compute permutation tests to determine whether algorithms are statistically different in their metric values and also does not compute bootstrap confidence intervals on the rankings of the algorithms. See the [original library](https://github.com/google-research/rl-reliability-metrics/) for more resources.
+
+## Citation
+
+```bibtex
+@conference{rl_reliability_metrics,
+  title = {Measuring the Reliability of Reinforcement Learning Algorithms},
+  author = {Stephanie CY Chan, Sam Fishman, John Canny, Anoop Korattikara, and Sergio Guadarrama},
+  booktitle = {International Conference on Learning Representations, Addis Ababa, Ethiopia},
+  year = 2020,
+}
+```
+
+## Further References
+- Homepage: https://github.com/google-research/rl-reliability-metrics

app.py

@@ -0,0 +1,6 @@
+import evaluate
+from evaluate.utils import launch_gradio_widget
+
+
+module = evaluate.load("rl_reliability", "online")
+launch_gradio_widget(module)

requirements.txt

@@ -0,0 +1,7 @@
+# TODO: fix github to release
+git+https://github.com/huggingface/evaluate.git@main
+datasets~=2.0
+git+https://github.com/google-research/rl-reliability-metrics
+scipy
+tensorflow
+gin-config

rl_reliability.py

@@ -0,0 +1,186 @@
+# Copyright 2020 The HuggingFace Datasets Authors and the current dataset script contributor.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Computes the RL Reliability Metrics."""
+
+import datasets
+import numpy as np
+from rl_reliability_metrics.evaluation import eval_metrics
+from rl_reliability_metrics.metrics import metrics_offline, metrics_online
+
+import evaluate
+
+
+logger = evaluate.logging.get_logger(__name__)
+
+DEFAULT_EVAL_POINTS = [
+    50000,
+    150000,
+    250000,
+    350000,
+    450000,
+    550000,
+    650000,
+    750000,
+    850000,
+    950000,
+    1050000,
+    1150000,
+    1250000,
+    1350000,
+    1450000,
+    1550000,
+    1650000,
+    1750000,
+    1850000,
+    1950000,
+]
+
+N_RUNS_RECOMMENDED = 10
+
+_CITATION = """\
+@conference{rl_reliability_metrics,
+  title = {Measuring the Reliability of Reinforcement Learning Algorithms},
+  author = {Stephanie CY Chan, Sam Fishman, John Canny, Anoop Korattikara, and Sergio Guadarrama},
+  booktitle = {International Conference on Learning Representations, Addis Ababa, Ethiopia},
+  year = 2020,
+}
+"""
+
+_DESCRIPTION = """\
+This new module is designed to solve this great NLP task and is crafted with a lot of care.
+"""
+
+
+_KWARGS_DESCRIPTION = """
+Computes the RL reliability metrics from a set of experiments. There is an `"online"` and `"offline"` configuration for evaluation.
+Args:
+    timestamps: list of timestep lists/arrays that serve as index.
+    rewards: list of reward lists/arrays of each experiment.
+Returns:
+    dictionary: a set of reliability metrics
+Examples:
+    >>> import numpy as np
+    >>> rl_reliability = evaluate.load("rl_reliability", "online")
+    >>> results = rl_reliability.compute(
+    ...     timesteps=[np.linspace(0, 2000000, 1000)],
+    ...     rewards=[np.linspace(0, 100, 1000)]
+    ...     )
+    >>> print(results["LowerCVaROnRaw"].round(4))
+    [0.0258]
+"""
+
+
+@evaluate.utils.file_utils.add_start_docstrings(_DESCRIPTION, _KWARGS_DESCRIPTION)
+class RLReliability(evaluate.EvaluationModule):
+    """Computes the RL Reliability Metrics."""
+
+    def _info(self):
+        if self.config_name not in ["online", "offline"]:
+            raise KeyError("""You should supply a configuration name selected in '["online", "offline"]'""")
+
+        return evaluate.EvaluationModuleInfo(
+            module_type="metric",
+            description=_DESCRIPTION,
+            citation=_CITATION,
+            inputs_description=_KWARGS_DESCRIPTION,
+            features=datasets.Features(
+                {
+                    "timesteps": datasets.Sequence(datasets.Value("int64")),
+                    "rewards": datasets.Sequence(datasets.Value("float")),
+                }
+            ),
+            homepage="https://github.com/google-research/rl-reliability-metrics",
+        )
+
+    def _compute(
+        self,
+        timesteps,
+        rewards,
+        baseline="default",
+        freq_thresh=0.01,
+        window_size=100000,
+        window_size_trimmed=99000,
+        alpha=0.05,
+        eval_points=None,
+    ):
+        if len(timesteps) < N_RUNS_RECOMMENDED:
+            logger.warning(
+                f"For robust statistics it is recommended to use at least {N_RUNS_RECOMMENDED} runs whereas you provided {len(timesteps)}."
+            )
+
+        curves = []
+        for timestep, reward in zip(timesteps, rewards):
+            curves.append(np.stack([timestep, reward]))
+
+        if self.config_name == "online":
+            if baseline == "default":
+                baseline = "curve_range"
+            if eval_points is None:
+                eval_points = DEFAULT_EVAL_POINTS
+
+            metrics = [
+                metrics_online.HighFreqEnergyWithinRuns(thresh=freq_thresh),
+                metrics_online.IqrWithinRuns(
+                    window_size=window_size_trimmed, eval_points=eval_points, baseline=baseline
+                ),
+                metrics_online.IqrAcrossRuns(
+                    lowpass_thresh=freq_thresh, eval_points=eval_points, window_size=window_size, baseline=baseline
+                ),
+                metrics_online.LowerCVaROnDiffs(baseline=baseline),
+                metrics_online.LowerCVaROnDrawdown(baseline=baseline),
+                metrics_online.LowerCVaROnAcross(
+                    lowpass_thresh=freq_thresh, eval_points=eval_points, window_size=window_size, baseline=baseline
+                ),
+                metrics_online.LowerCVaROnRaw(alpha=alpha, baseline=baseline),
+                metrics_online.MadAcrossRuns(
+                    lowpass_thresh=freq_thresh, eval_points=eval_points, window_size=window_size, baseline=baseline
+                ),
+                metrics_online.MadWithinRuns(
+                    eval_points=eval_points, window_size=window_size_trimmed, baseline=baseline
+                ),
+                metrics_online.MaxDrawdown(),
+                metrics_online.StddevAcrossRuns(
+                    lowpass_thresh=freq_thresh, eval_points=eval_points, window_size=window_size, baseline=baseline
+                ),
+                metrics_online.StddevWithinRuns(
+                    eval_points=eval_points, window_size=window_size_trimmed, baseline=baseline
+                ),
+                metrics_online.UpperCVaROnAcross(
+                    alpha=alpha,
+                    lowpass_thresh=freq_thresh,
+                    eval_points=eval_points,
+                    window_size=window_size,
+                    baseline=baseline,
+                ),
+                metrics_online.UpperCVaROnDiffs(alpha=alpha, baseline=baseline),
+                metrics_online.UpperCVaROnDrawdown(alpha=alpha, baseline=baseline),
+                metrics_online.UpperCVaROnRaw(alpha=alpha, baseline=baseline),
+                metrics_online.MedianPerfDuringTraining(window_size=window_size, eval_points=eval_points),
+            ]
+        else:
+            if baseline == "default":
+                baseline = "median_perf"
+
+            metrics = [
+                metrics_offline.MadAcrossRollouts(baseline=baseline),
+                metrics_offline.IqrAcrossRollouts(baseline=baseline),
+                metrics_offline.StddevAcrossRollouts(baseline=baseline),
+                metrics_offline.LowerCVaRAcrossRollouts(alpha=alpha, baseline=baseline),
+                metrics_offline.UpperCVaRAcrossRollouts(alpha=alpha, baseline=baseline),
+                metrics_offline.MedianPerfAcrossRollouts(baseline=None),
+            ]
+
+        evaluator = eval_metrics.Evaluator(metrics=metrics)
+        result = evaluator.compute_metrics(curves)
+        return result