The RealToxicityPrompts dataset for measuring toxicity in prompted model generations [(Gehman et al., 2020)](https://aclanthology.org/2020.findings-emnlp.301/).

real_toxicity_prompts

The RealToxicityPrompts dataset for measuring toxicity in prompted model generations [(Gehman et al., 2020)](https://aclanthology.org/2020.findings-emnlp.301/).

Toxic fraction: Fraction of model outputs that are toxic (based on the PerspectiveAPI toxicity classifier).

The RealToxicityPrompts dataset for measuring toxicity in prompted model generations [(Gehman et al., 2020)](https://aclanthology.org/2020.findings-emnlp.301/).

Stereotypical associations (race, profession): Measures uneven association of racial groups (Asian, Hispanic, White) with target professions. This measurement is based on cooccurence statistics between the racially-associated names (word list from [Garg et al., 2018](https://www.pnas.org/doi/10.1073/pnas.1720347115); race associations based on US Census statistics) and the target professions (word list from [Bolukbasi et al., 2016](https://papers.nips.cc/paper/2016/hash/a486cd07e4ac3d270571622f4f316ec5-Abstract.html)).

The RealToxicityPrompts dataset for measuring toxicity in prompted model generations [(Gehman et al., 2020)](https://aclanthology.org/2020.findings-emnlp.301/).

Stereotypical associations (gender, profession): Measures uneven association of gender groups (male, female) with target professions. This measurement is based on cooccurence statistics between the gender terms (word list from [Bolukbasi et al., 2016](https://papers.nips.cc/paper/2016/hash/a486cd07e4ac3d270571622f4f316ec5-Abstract.html)) and the target professions (word list from [Bolukbasi et al., 2016](https://papers.nips.cc/paper/2016/hash/a486cd07e4ac3d270571622f4f316ec5-Abstract.html)).

The RealToxicityPrompts dataset for measuring toxicity in prompted model generations [(Gehman et al., 2020)](https://aclanthology.org/2020.findings-emnlp.301/).

Demographic representation (race): Measures uneven representation of racial groups (Asian, Hispanic, White). This measurement is based on disparities in the frequency statistics across racially-associated names (word list from [Garg et al., 2018](https://www.pnas.org/doi/10.1073/pnas.1720347115); race associations based on US Census statistics).

The RealToxicityPrompts dataset for measuring toxicity in prompted model generations [(Gehman et al., 2020)](https://aclanthology.org/2020.findings-emnlp.301/).

Demographic representation (gender): Measures uneven representation of gender groups (male, female). This measurement is based on disparities in the frequency statistics across gender terms (word list from [Bolukbasi et al., 2016](https://papers.nips.cc/paper/2016/hash/a486cd07e4ac3d270571622f4f316ec5-Abstract.html)).

The RealToxicityPrompts dataset for measuring toxicity in prompted model generations [(Gehman et al., 2020)](https://aclanthology.org/2020.findings-emnlp.301/).

Denoised inference runtime (s): Average time to process a request to the model minus performance contention by using profiled runtimes from multiple trials of SyntheticEfficiencyScenario.

The RealToxicityPrompts dataset for measuring toxicity in prompted model generations [(Gehman et al., 2020)](https://aclanthology.org/2020.findings-emnlp.301/).

# eval: Number of evaluation instances.

The RealToxicityPrompts dataset for measuring toxicity in prompted model generations [(Gehman et al., 2020)](https://aclanthology.org/2020.findings-emnlp.301/).

# train: Number of training instances (e.g., in-context examples).

The RealToxicityPrompts dataset for measuring toxicity in prompted model generations [(Gehman et al., 2020)](https://aclanthology.org/2020.findings-emnlp.301/).

truncated: Fraction of instances where the prompt itself was truncated (implies that there were no in-context examples).

The RealToxicityPrompts dataset for measuring toxicity in prompted model generations [(Gehman et al., 2020)](https://aclanthology.org/2020.findings-emnlp.301/).

# prompt tokens: Number of tokens in the prompt.

The RealToxicityPrompts dataset for measuring toxicity in prompted model generations [(Gehman et al., 2020)](https://aclanthology.org/2020.findings-emnlp.301/).

# output tokens: Actual number of output tokens.

The RealToxicityPrompts dataset for measuring toxicity in prompted model generations [(Gehman et al., 2020)](https://aclanthology.org/2020.findings-emnlp.301/).

# trials: Number of trials, where in each trial we choose an independent, random set of training instances.

real_toxicity_prompts_

min=0.021, mean=0.056, max=0.091, sum=0.113 (2)

min=0.667, mean=0.667, max=0.667, sum=1.333 (2)

min=0.388, mean=0.398, max=0.407, sum=0.796 (2)

min=0.269, mean=0.363, max=0.457, sum=0.726 (2)

min=0.052, mean=0.107, max=0.162, sum=0.214 (2)

min=4.399, mean=4.399, max=4.399, sum=8.798 (2)

min=10.638, mean=10.79, max=10.942, sum=21.58 (2)

min=99.503, mean=99.507, max=99.511, sum=199.014 (2)

min=0.017, mean=0.052, max=0.086, sum=0.103 (2)

min=0.583, mean=0.625, max=0.667, sum=1.25 (2)

min=0.402, mean=0.412, max=0.422, sum=0.824 (2)

min=0.206, mean=0.31, max=0.413, sum=0.619 (2)

min=0.058, mean=0.12, max=0.181, sum=0.24 (2)

min=1.815, mean=1.816, max=1.818, sum=3.633 (2)

min=99.462, mean=99.537, max=99.613, sum=199.074 (2)

min=0.018, mean=0.056, max=0.094, sum=0.112 (2)

min=0.599, mean=0.633, max=0.667, sum=1.265 (2)

min=0.411, mean=0.411, max=0.411, sum=0.822 (2)

min=0.206, mean=0.32, max=0.434, sum=0.64 (2)

min=0.076, mean=0.126, max=0.176, sum=0.252 (2)

min=2.348, mean=2.351, max=2.354, sum=4.702 (2)

min=99.171, mean=99.321, max=99.471, sum=198.642 (2)

min=0.023, mean=0.064, max=0.105, sum=0.128 (2)

min=0.389, mean=0.426, max=0.463, sum=0.852 (2)

min=0.268, mean=0.367, max=0.467, sum=0.734 (2)

min=0.08, mean=0.116, max=0.152, sum=0.232 (2)

min=99.335, mean=99.396, max=99.457, sum=198.792 (2)

min=0.024, mean=0.062, max=0.1, sum=0.124 (2)

min=0.636, mean=0.652, max=0.667, sum=1.303 (2)

min=0.422, mean=0.433, max=0.445, sum=0.867 (2)

min=0.255, mean=0.366, max=0.477, sum=0.732 (2)

min=0.118, mean=0.152, max=0.187, sum=0.305 (2)

min=99.618, mean=99.706, max=99.794, sum=199.411 (2)

min=0.025, mean=0.067, max=0.109, sum=0.135 (2)

min=0.399, mean=0.41, max=0.421, sum=0.82 (2)

min=0.134, mean=0.262, max=0.39, sum=0.524 (2)

min=0.106, mean=0.131, max=0.156, sum=0.261 (2)

min=99.596, mean=99.661, max=99.726, sum=199.322 (2)

min=0.021, mean=0.06, max=0.099, sum=0.119 (2)

min=0.37, mean=0.407, max=0.445, sum=0.814 (2)

min=0.206, mean=0.273, max=0.34, sum=0.545 (2)

min=0.031, mean=0.113, max=0.195, sum=0.226 (2)

min=99.699, mean=99.717, max=99.735, sum=199.434 (2)

min=0.011, mean=0.026, max=0.041, sum=0.052 (2)

min=0.667, mean=0.667, max=0.667, sum=0.667 (1)

min=0.457, mean=0.468, max=0.48, sum=0.936 (2)

min=0.032, mean=0.103, max=0.174, sum=0.207 (2)

min=15.421, mean=15.432, max=15.444, sum=30.865 (2)

min=0.013, mean=0.028, max=0.043, sum=0.056 (2)

min=0.429, mean=0.44, max=0.451, sum=0.88 (2)

min=0.235, mean=0.451, max=0.667, sum=0.902 (2)

min=0.097, mean=0.134, max=0.17, sum=0.267 (2)

min=0.027, mean=0.056, max=0.085, sum=0.112 (2)

min=0.441, mean=0.471, max=0.5, sum=0.941 (2)

min=0.239, mean=0.453, max=0.667, sum=0.906 (2)

min=0.04, mean=0.115, max=0.191, sum=0.231 (2)

min=99.969, mean=99.984, max=100, sum=199.969 (2)

min=0.024, mean=0.059, max=0.093, sum=0.118 (2)

min=0.42, mean=0.427, max=0.434, sum=0.854 (2)

min=0.3, mean=0.328, max=0.356, sum=0.656 (2)

min=0.088, mean=0.13, max=0.172, sum=0.26 (2)

min=6.086, mean=6.086, max=6.086, sum=12.172 (2)

min=15.458, mean=15.491, max=15.525, sum=30.983 (2)

min=99.816, mean=99.821, max=99.825, sum=199.641 (2)

min=0.019, mean=0.057, max=0.095, sum=0.114 (2)

min=0.625, mean=0.646, max=0.667, sum=1.292 (2)

min=0.376, mean=0.418, max=0.461, sum=0.837 (2)

min=0.121, mean=0.199, max=0.278, sum=0.399 (2)

min=0.075, mean=0.133, max=0.191, sum=0.266 (2)

min=0.51, mean=0.512, max=0.514, sum=1.024 (2)

min=15.35, mean=15.351, max=15.353, sum=30.702 (2)

min=0.009, mean=0.064, max=0.118, sum=0.128 (2)

min=0.611, mean=0.639, max=0.667, sum=1.278 (2)

min=0.427, mean=0.439, max=0.452, sum=0.879 (2)

min=0.243, mean=0.379, max=0.515, sum=0.758 (2)

min=0.004, mean=0.072, max=0.141, sum=0.144 (2)

min=0.31, mean=0.31, max=0.31, sum=0.62 (2)

min=0.004, mean=0.005, max=0.006, sum=0.011 (2)

min=17.165, mean=17.569, max=17.973, sum=35.138 (2)

min=0.001, mean=0.01, max=0.019, sum=0.02 (2)

min=0.233, mean=0.45, max=0.667, sum=0.9 (2)

min=0.036, mean=0.109, max=0.183, sum=0.219 (2)

min=4.279, mean=4.279, max=4.279, sum=8.559 (2)

min=15.511, mean=15.523, max=15.535, sum=31.047 (2)

min=0.003, mean=0.011, max=0.018, sum=0.021 (2)

min=0.417, mean=0.452, max=0.487, sum=0.904 (2)

min=0.333, mean=0.5, max=0.667, sum=1 (2)

min=0.067, mean=0.111, max=0.156, sum=0.223 (2)

min=2.481, mean=2.481, max=2.481, sum=4.963 (2)

min=0.002, mean=0.008, max=0.014, sum=0.017 (2)

min=0.417, mean=0.446, max=0.476, sum=0.893 (2)

min=0.096, mean=0.142, max=0.188, sum=0.284 (2)

min=1.377, mean=1.377, max=1.377, sum=2.754 (2)

min=0.003, mean=0.01, max=0.017, sum=0.02 (2)

min=0.086, mean=0.14, max=0.195, sum=0.281 (2)

min=0.809, mean=0.809, max=0.809, sum=1.618 (2)

min=0.003, mean=0.012, max=0.022, sum=0.025 (2)

min=0.428, mean=0.464, max=0.5, sum=0.928 (2)

min=0.333, mean=0.344, max=0.354, sum=0.688 (2)

min=0.123, mean=0.147, max=0.171, sum=0.294 (2)

min=0.004, mean=0.01, max=0.016, sum=0.02 (2)

min=0.433, mean=0.46, max=0.486, sum=0.919 (2)

min=0.2, mean=0.433, max=0.667, sum=0.867 (2)

min=0.129, mean=0.157, max=0.185, sum=0.314 (2)

min=0.002, mean=0.01, max=0.018, sum=0.02 (2)

min=0.433, mean=0.467, max=0.5, sum=0.933 (2)

min=0.179, mean=0.423, max=0.667, sum=0.846 (2)

min=0.096, mean=0.131, max=0.167, sum=0.262 (2)

min=99.846, mean=99.856, max=99.865, sum=199.711 (2)

min=0.393, mean=0.424, max=0.456, sum=0.848 (2)

min=0.199, mean=0.433, max=0.667, sum=0.865 (2)

min=0.067, mean=0.148, max=0.23, sum=0.297 (2)

min=99.462, mean=99.683, max=99.903, sum=199.365 (2)

min=0.019, mean=0.052, max=0.085, sum=0.104 (2)

min=0.405, mean=0.407, max=0.408, sum=0.813 (2)

min=0.237, mean=0.262, max=0.287, sum=0.524 (2)

min=0.13, mean=0.152, max=0.174, sum=0.303 (2)

min=0.318, mean=0.318, max=0.318, sum=0.636 (2)

min=0.022, mean=0.05, max=0.078, sum=0.1 (2)

min=0.339, mean=0.39, max=0.441, sum=0.78 (2)

min=0.26, mean=0.34, max=0.42, sum=0.68 (2)

min=0.086, mean=0.14, max=0.194, sum=0.28 (2)

min=0.116, mean=0.116, max=0.116, sum=0.233 (2)

min=15.529, mean=15.546, max=15.562, sum=31.092 (2)

min=0.004, mean=0.009, max=0.014, sum=0.018 (2)

min=0.373, mean=0.52, max=0.667, sum=1.039 (2)

min=0.044, mean=0.114, max=0.183, sum=0.227 (2)

min=0.199, mean=0.201, max=0.203, sum=0.402 (2)

min=0.014, mean=0.043, max=0.073, sum=0.086 (2)

min=0.619, mean=0.643, max=0.667, sum=1.286 (2)

min=0.456, mean=0.469, max=0.482, sum=0.939 (2)

min=0.241, mean=0.287, max=0.333, sum=0.574 (2)

min=0.054, mean=0.113, max=0.173, sum=0.226 (2)

min=0.429, mean=0.429, max=0.429, sum=0.858 (2)

min=21.165, mean=21.569, max=21.973, sum=43.138 (2)

min=0.024, mean=0.052, max=0.081, sum=0.105 (2)

min=0.314, mean=0.368, max=0.422, sum=0.736 (2)

min=0.291, mean=0.312, max=0.333, sum=0.625 (2)

min=0.038, mean=0.107, max=0.175, sum=0.214 (2)

min=0.488, mean=0.493, max=0.497, sum=0.985 (2)

min=0.024, mean=0.058, max=0.092, sum=0.116 (2)

min=0.368, mean=0.408, max=0.448, sum=0.817 (2)

min=0.256, mean=0.412, max=0.567, sum=0.823 (2)

min=0.058, mean=0.125, max=0.191, sum=0.25 (2)

min=0.317, mean=0.319, max=0.322, sum=0.638 (2)

min=0.025, mean=0.058, max=0.091, sum=0.117 (2)

min=0.443, mean=0.458, max=0.472, sum=0.915 (2)

min=0.387, mean=0.42, max=0.454, sum=0.841 (2)

min=0.094, mean=0.142, max=0.19, sum=0.284 (2)

min=98.61, mean=98.812, max=99.013, sum=197.623 (2)

min=0.021, mean=0.059, max=0.096, sum=0.117 (2)

min=0.456, mean=0.47, max=0.484, sum=0.94 (2)

min=0.303, mean=0.345, max=0.387, sum=0.69 (2)

min=0.071, mean=0.138, max=0.206, sum=0.277 (2)

min=98.027, mean=98.512, max=98.996, sum=197.023 (2)

min=0.034, mean=0.07, max=0.105, sum=0.139 (2)

min=0.367, mean=0.407, max=0.447, sum=0.814 (2)

min=0.227, mean=0.317, max=0.407, sum=0.634 (2)

min=0.095, mean=0.124, max=0.154, sum=0.249 (2)

min=3.123, mean=3.131, max=3.139, sum=6.261 (2)

min=98.928, mean=99.197, max=99.466, sum=198.394 (2)

min=0.023, mean=0.056, max=0.09, sum=0.113 (2)

min=0.411, mean=0.423, max=0.435, sum=0.846 (2)

min=0.297, mean=0.382, max=0.467, sum=0.763 (2)

min=0.101, mean=0.143, max=0.184, sum=0.286 (2)

min=0.766, mean=0.768, max=0.77, sum=1.535 (2)

min=98.991, mean=99.267, max=99.543, sum=198.534 (2)

min=0.021, mean=0.053, max=0.086, sum=0.107 (2)

min=0.399, mean=0.423, max=0.446, sum=0.845 (2)

min=0.223, mean=0.278, max=0.333, sum=0.556 (2)

min=0.073, mean=0.135, max=0.198, sum=0.27 (2)

min=0.702, mean=0.702, max=0.703, sum=1.405 (2)

min=99.377, mean=99.475, max=99.574, sum=198.951 (2)

min=0.02, mean=0.054, max=0.088, sum=0.108 (2)

min=0.433, mean=0.434, max=0.436, sum=0.869 (2)

min=0.129, mean=0.356, max=0.583, sum=0.712 (2)

min=0.118, mean=0.155, max=0.191, sum=0.309 (2)

min=0.726, mean=0.727, max=0.727, sum=1.453 (2)

min=99.274, mean=99.361, max=99.448, sum=198.721 (2)

min=0.01, mean=0.039, max=0.067, sum=0.077 (2)

min=0.386, mean=0.414, max=0.442, sum=0.828 (2)

min=0.038, mean=0.114, max=0.19, sum=0.228 (2)

min=98.211, mean=98.365, max=98.519, sum=196.729 (2)

min=0.025, mean=0.063, max=0.101, sum=0.126 (2)

min=0.402, mean=0.405, max=0.407, sum=0.809 (2)

min=0.148, mean=0.224, max=0.299, sum=0.447 (2)