The Bias in Open-Ended Language Generation Dataset (BOLD) for measuring biases and toxicity in open-ended language generation [(Dhamala et al., 2021)](https://dl.acm.org/doi/10.1145/3442188.3445924).

bold

The Bias in Open-Ended Language Generation Dataset (BOLD) for measuring biases and toxicity in open-ended language generation [(Dhamala et al., 2021)](https://dl.acm.org/doi/10.1145/3442188.3445924).

Toxic fraction: Fraction of model outputs that are toxic (based on the PerspectiveAPI toxicity classifier).

The Bias in Open-Ended Language Generation Dataset (BOLD) for measuring biases and toxicity in open-ended language generation [(Dhamala et al., 2021)](https://dl.acm.org/doi/10.1145/3442188.3445924).

Stereotypical associations (race, profession): Measures uneven association of racial groups (Asian, Hispanic, White) with target professions. This measurement is based on cooccurence statistics between the racially-associated names (word list from [Garg et al., 2018](https://www.pnas.org/doi/10.1073/pnas.1720347115); race associations based on US Census statistics) and the target professions (word list from [Bolukbasi et al., 2016](https://papers.nips.cc/paper/2016/hash/a486cd07e4ac3d270571622f4f316ec5-Abstract.html)).

The Bias in Open-Ended Language Generation Dataset (BOLD) for measuring biases and toxicity in open-ended language generation [(Dhamala et al., 2021)](https://dl.acm.org/doi/10.1145/3442188.3445924).

Stereotypical associations (gender, profession): Measures uneven association of gender groups (male, female) with target professions. This measurement is based on cooccurence statistics between the gender terms (word list from [Bolukbasi et al., 2016](https://papers.nips.cc/paper/2016/hash/a486cd07e4ac3d270571622f4f316ec5-Abstract.html)) and the target professions (word list from [Bolukbasi et al., 2016](https://papers.nips.cc/paper/2016/hash/a486cd07e4ac3d270571622f4f316ec5-Abstract.html)).

The Bias in Open-Ended Language Generation Dataset (BOLD) for measuring biases and toxicity in open-ended language generation [(Dhamala et al., 2021)](https://dl.acm.org/doi/10.1145/3442188.3445924).

Demographic representation (race): Measures uneven representation of racial groups (Asian, Hispanic, White). This measurement is based on disparities in the frequency statistics across racially-associated names (word list from [Garg et al., 2018](https://www.pnas.org/doi/10.1073/pnas.1720347115); race associations based on US Census statistics).

The Bias in Open-Ended Language Generation Dataset (BOLD) for measuring biases and toxicity in open-ended language generation [(Dhamala et al., 2021)](https://dl.acm.org/doi/10.1145/3442188.3445924).

Demographic representation (gender): Measures uneven representation of gender groups (male, female). This measurement is based on disparities in the frequency statistics across gender terms (word list from [Bolukbasi et al., 2016](https://papers.nips.cc/paper/2016/hash/a486cd07e4ac3d270571622f4f316ec5-Abstract.html)).

The Bias in Open-Ended Language Generation Dataset (BOLD) for measuring biases and toxicity in open-ended language generation [(Dhamala et al., 2021)](https://dl.acm.org/doi/10.1145/3442188.3445924).

Denoised inference runtime (s): Average time to process a request to the model minus performance contention by using profiled runtimes from multiple trials of SyntheticEfficiencyScenario.

The Bias in Open-Ended Language Generation Dataset (BOLD) for measuring biases and toxicity in open-ended language generation [(Dhamala et al., 2021)](https://dl.acm.org/doi/10.1145/3442188.3445924).

# eval: Number of evaluation instances.

The Bias in Open-Ended Language Generation Dataset (BOLD) for measuring biases and toxicity in open-ended language generation [(Dhamala et al., 2021)](https://dl.acm.org/doi/10.1145/3442188.3445924).

# train: Number of training instances (e.g., in-context examples).

The Bias in Open-Ended Language Generation Dataset (BOLD) for measuring biases and toxicity in open-ended language generation [(Dhamala et al., 2021)](https://dl.acm.org/doi/10.1145/3442188.3445924).

truncated: Fraction of instances where the prompt itself was truncated (implies that there were no in-context examples).

The Bias in Open-Ended Language Generation Dataset (BOLD) for measuring biases and toxicity in open-ended language generation [(Dhamala et al., 2021)](https://dl.acm.org/doi/10.1145/3442188.3445924).

# prompt tokens: Number of tokens in the prompt.

The Bias in Open-Ended Language Generation Dataset (BOLD) for measuring biases and toxicity in open-ended language generation [(Dhamala et al., 2021)](https://dl.acm.org/doi/10.1145/3442188.3445924).

# output tokens: Actual number of output tokens.

The Bias in Open-Ended Language Generation Dataset (BOLD) for measuring biases and toxicity in open-ended language generation [(Dhamala et al., 2021)](https://dl.acm.org/doi/10.1145/3442188.3445924).

# trials: Number of trials, where in each trial we choose an independent, random set of training instances.

bold_subject:all

min=0.002, mean=0.002, max=0.002, sum=0.002 (1)

min=0.601, mean=0.601, max=0.601, sum=0.601 (1)

min=0.43, mean=0.43, max=0.43, sum=0.43 (1)

min=0.398, mean=0.398, max=0.398, sum=0.398 (1)

min=0.204, mean=0.204, max=0.204, sum=0.204 (1)

min=1.059, mean=1.059, max=1.059, sum=1.059 (1)

min=1000, mean=1000, max=1000, sum=1000 (1)

min=9.034, mean=9.034, max=9.034, sum=9.034 (1)

min=0.001, mean=0.001, max=0.001, sum=0.001 (1)

min=0.616, mean=0.616, max=0.616, sum=0.616 (1)

min=0.393, mean=0.393, max=0.393, sum=0.393 (1)

min=0.375, mean=0.375, max=0.375, sum=0.375 (1)

min=0.172, mean=0.172, max=0.172, sum=0.172 (1)

min=0.543, mean=0.543, max=0.543, sum=0.543 (1)

min=0.641, mean=0.641, max=0.641, sum=0.641 (1)

min=0.409, mean=0.409, max=0.409, sum=0.409 (1)

min=0.444, mean=0.444, max=0.444, sum=0.444 (1)

min=0.17, mean=0.17, max=0.17, sum=0.17 (1)

min=0.684, mean=0.684, max=0.684, sum=0.684 (1)

min=19.991, mean=19.991, max=19.991, sum=19.991 (1)

min=0.667, mean=0.667, max=0.667, sum=0.667 (1)

min=0.416, mean=0.416, max=0.416, sum=0.416 (1)

min=0.489, mean=0.489, max=0.489, sum=0.489 (1)

min=0.18, mean=0.18, max=0.18, sum=0.18 (1)

min=19.984, mean=19.984, max=19.984, sum=19.984 (1)

min=0.425, mean=0.425, max=0.425, sum=0.425 (1)

min=0.402, mean=0.402, max=0.402, sum=0.402 (1)

min=0.191, mean=0.191, max=0.191, sum=0.191 (1)

min=19.957, mean=19.957, max=19.957, sum=19.957 (1)

min=0.583, mean=0.583, max=0.583, sum=0.583 (1)

min=0.417, mean=0.417, max=0.417, sum=0.417 (1)

min=0.218, mean=0.218, max=0.218, sum=0.218 (1)

min=0.19, mean=0.19, max=0.19, sum=0.19 (1)

min=19.929, mean=19.929, max=19.929, sum=19.929 (1)

min=0.42, mean=0.42, max=0.42, sum=0.42 (1)

min=0.505, mean=0.505, max=0.505, sum=0.505 (1)

min=0.193, mean=0.193, max=0.193, sum=0.193 (1)

min=19.96, mean=19.96, max=19.96, sum=19.96 (1)

min=0.005, mean=0.005, max=0.005, sum=0.005 (1)

min=0.625, mean=0.625, max=0.625, sum=0.625 (1)

min=0.431, mean=0.431, max=0.431, sum=0.431 (1)

min=0.436, mean=0.436, max=0.436, sum=0.436 (1)

min=0.205, mean=0.205, max=0.205, sum=0.205 (1)

min=9.391, mean=9.391, max=9.391, sum=9.391 (1)

min=19.999, mean=19.999, max=19.999, sum=19.999 (1)

min=0.593, mean=0.593, max=0.593, sum=0.593 (1)

min=0.521, mean=0.521, max=0.521, sum=0.521 (1)

min=0.196, mean=0.196, max=0.196, sum=0.196 (1)

min=0.003, mean=0.003, max=0.003, sum=0.003 (1)

min=0.342, mean=0.342, max=0.342, sum=0.342 (1)

min=0.397, mean=0.397, max=0.397, sum=0.397 (1)

min=0.382, mean=0.382, max=0.382, sum=0.382 (1)

min=0.125, mean=0.125, max=0.125, sum=0.125 (1)

min=1.525, mean=1.525, max=1.525, sum=1.525 (1)

min=11.805, mean=11.805, max=11.805, sum=11.805 (1)

min=19.855, mean=19.855, max=19.855, sum=19.855 (1)

min=0.41, mean=0.41, max=0.41, sum=0.41 (1)

min=0.211, mean=0.211, max=0.211, sum=0.211 (1)

min=0.339, mean=0.339, max=0.339, sum=0.339 (1)

min=10.748, mean=10.748, max=10.748, sum=10.748 (1)

min=0.606, mean=0.606, max=0.606, sum=0.606 (1)

min=0.426, mean=0.426, max=0.426, sum=0.426 (1)

min=0.394, mean=0.394, max=0.394, sum=0.394 (1)

min=0.217, mean=0.217, max=0.217, sum=0.217 (1)

min=0.064, mean=0.064, max=0.064, sum=0.064 (1)

min=0.995, mean=0.995, max=0.995, sum=0.995 (1)

min=11.711, mean=11.711, max=11.711, sum=11.711 (1)

min=0.547, mean=0.547, max=0.547, sum=0.547 (1)

min=0.179, mean=0.179, max=0.179, sum=0.179 (1)

min=1.077, mean=1.077, max=1.077, sum=1.077 (1)

min=11.895, mean=11.895, max=11.895, sum=11.895 (1)

min=0.415, mean=0.415, max=0.415, sum=0.415 (1)

min=0.513, mean=0.513, max=0.513, sum=0.513 (1)

min=0.199, mean=0.199, max=0.199, sum=0.199 (1)

min=0.64, mean=0.64, max=0.64, sum=0.64 (1)

min=0.635, mean=0.635, max=0.635, sum=0.635 (1)

min=0.424, mean=0.424, max=0.424, sum=0.424 (1)

min=0.333, mean=0.333, max=0.333, sum=0.333 (1)

min=0.201, mean=0.201, max=0.201, sum=0.201 (1)

min=0.428, mean=0.428, max=0.428, sum=0.428 (1)

min=0.186, mean=0.186, max=0.186, sum=0.186 (1)

min=0.328, mean=0.328, max=0.328, sum=0.328 (1)

min=0.434, mean=0.434, max=0.434, sum=0.434 (1)

min=0.519, mean=0.519, max=0.519, sum=0.519 (1)

min=0.169, mean=0.169, max=0.169, sum=0.169 (1)

min=0.611, mean=0.611, max=0.611, sum=0.611 (1)

min=0.429, mean=0.429, max=0.429, sum=0.429 (1)

min=0.379, mean=0.379, max=0.379, sum=0.379 (1)

min=0.194, mean=0.194, max=0.194, sum=0.194 (1)

min=0.556, mean=0.556, max=0.556, sum=0.556 (1)

min=19.838, mean=19.838, max=19.838, sum=19.838 (1)

min=0.449, mean=0.449, max=0.449, sum=0.449 (1)

min=0.511, mean=0.511, max=0.511, sum=0.511 (1)

min=19.853, mean=19.853, max=19.853, sum=19.853 (1)

min=0.454, mean=0.454, max=0.454, sum=0.454 (1)

min=0.119, mean=0.119, max=0.119, sum=0.119 (1)

min=0.419, mean=0.419, max=0.419, sum=0.419 (1)

min=0.473, mean=0.473, max=0.473, sum=0.473 (1)

min=0.163, mean=0.163, max=0.163, sum=0.163 (1)

min=0.067, mean=0.067, max=0.067, sum=0.067 (1)

min=11.029, mean=11.029, max=11.029, sum=11.029 (1)

min=0.079, mean=0.079, max=0.079, sum=0.079 (1)

min=10.711, mean=10.711, max=10.711, sum=10.711 (1)

min=0.614, mean=0.614, max=0.614, sum=0.614 (1)

min=0.092, mean=0.092, max=0.092, sum=0.092 (1)

min=14.711, mean=14.711, max=14.711, sum=14.711 (1)

min=0.009, mean=0.009, max=0.009, sum=0.009 (1)

min=0.423, mean=0.423, max=0.423, sum=0.423 (1)

min=0.341, mean=0.341, max=0.341, sum=0.341 (1)

min=0.21, mean=0.21, max=0.21, sum=0.21 (1)

min=0.018, mean=0.018, max=0.018, sum=0.018 (1)

min=0.35, mean=0.35, max=0.35, sum=0.35 (1)

min=0.597, mean=0.597, max=0.597, sum=0.597 (1)

min=0.376, mean=0.376, max=0.376, sum=0.376 (1)

min=0.524, mean=0.524, max=0.524, sum=0.524 (1)

min=19.98, mean=19.98, max=19.98, sum=19.98 (1)

min=0.405, mean=0.405, max=0.405, sum=0.405 (1)

min=0.401, mean=0.401, max=0.401, sum=0.401 (1)

min=0.151, mean=0.151, max=0.151, sum=0.151 (1)

min=0.44, mean=0.44, max=0.44, sum=0.44 (1)

min=0.162, mean=0.162, max=0.162, sum=0.162 (1)

min=0.723, mean=0.723, max=0.723, sum=0.723 (1)

min=18.951, mean=18.951, max=18.951, sum=18.951 (1)

min=0.452, mean=0.452, max=0.452, sum=0.452 (1)

min=0.329, mean=0.329, max=0.329, sum=0.329 (1)

min=17.964, mean=17.964, max=17.964, sum=17.964 (1)

min=0.448, mean=0.448, max=0.448, sum=0.448 (1)

min=0.46, mean=0.46, max=0.46, sum=0.46 (1)

min=0.181, mean=0.181, max=0.181, sum=0.181 (1)

min=0.216, mean=0.216, max=0.216, sum=0.216 (1)

min=18.484, mean=18.484, max=18.484, sum=18.484 (1)

min=0.146, mean=0.146, max=0.146, sum=0.146 (1)

min=0.245, mean=0.245, max=0.245, sum=0.245 (1)

min=19.132, mean=19.132, max=19.132, sum=19.132 (1)

min=0.652, mean=0.652, max=0.652, sum=0.652 (1)

min=0.362, mean=0.362, max=0.362, sum=0.362 (1)

min=0.224, mean=0.224, max=0.224, sum=0.224 (1)

min=19.802, mean=19.802, max=19.802, sum=19.802 (1)

min=0.644, mean=0.644, max=0.644, sum=0.644 (1)

min=0.498, mean=0.498, max=0.498, sum=0.498 (1)

min=0.188, mean=0.188, max=0.188, sum=0.188 (1)

min=0.674, mean=0.674, max=0.674, sum=0.674 (1)

min=18.764, mean=18.764, max=18.764, sum=18.764 (1)

min=0.59, mean=0.59, max=0.59, sum=0.59 (1)

min=0.358, mean=0.358, max=0.358, sum=0.358 (1)

min=0.22, mean=0.22, max=0.22, sum=0.22 (1)

min=0.248, mean=0.248, max=0.248, sum=0.248 (1)

min=19.266, mean=19.266, max=19.266, sum=19.266 (1)

min=0.651, mean=0.651, max=0.651, sum=0.651 (1)

min=0.346, mean=0.346, max=0.346, sum=0.346 (1)

min=0.174, mean=0.174, max=0.174, sum=0.174 (1)

min=18.99, mean=18.99, max=18.99, sum=18.99 (1)

min=0.439, mean=0.439, max=0.439, sum=0.439 (1)

min=0.132, mean=0.132, max=0.132, sum=0.132 (1)

min=0.187, mean=0.187, max=0.187, sum=0.187 (1)

min=19.454, mean=19.454, max=19.454, sum=19.454 (1)

min=0.446, mean=0.446, max=0.446, sum=0.446 (1)

min=10.755, mean=10.755, max=10.755, sum=10.755 (1)

min=20.533, mean=20.533, max=20.533, sum=20.533 (1)

min=0.422, mean=0.422, max=0.422, sum=0.422 (1)

min=0.509, mean=0.509, max=0.509, sum=0.509 (1)

min=0.192, mean=0.192, max=0.192, sum=0.192 (1)

min=0.639, mean=0.639, max=0.639, sum=0.639 (1)

min=0.202, mean=0.202, max=0.202, sum=0.202 (1)

min=0.677, mean=0.677, max=0.677, sum=0.677 (1)

min=0.999, mean=0.999, max=0.999, sum=0.999 (1)

min=9.506, mean=9.506, max=9.506, sum=9.506 (1)

min=0.396, mean=0.396, max=0.396, sum=0.396 (1)

min=19.789, mean=19.789, max=19.789, sum=19.789 (1)

min=0.612, mean=0.612, max=0.612, sum=0.612 (1)

min=0.367, mean=0.367, max=0.367, sum=0.367 (1)

min=0.214, mean=0.214, max=0.214, sum=0.214 (1)

min=17.37, mean=17.37, max=17.37, sum=17.37 (1)

min=0.435, mean=0.435, max=0.435, sum=0.435 (1)

min=0.351, mean=0.351, max=0.351, sum=0.351 (1)

min=0.24, mean=0.24, max=0.24, sum=0.24 (1)

min=11.445, mean=11.445, max=11.445, sum=11.445 (1)