The TwitterAAE corpus of [Blodgett et al. (2016)](https://aclanthology.org/D16-1120/) for measuring language model performance in tweets as a function of speaker dialect.

twitter_aae

The TwitterAAE corpus of [Blodgett et al. (2016)](https://aclanthology.org/D16-1120/) for measuring language model performance in tweets as a function of speaker dialect.

Bits/byte: Average number of bits per byte according to model probabilities.

The TwitterAAE corpus of [Blodgett et al. (2016)](https://aclanthology.org/D16-1120/) for measuring language model performance in tweets as a function of speaker dialect.

Denoised inference runtime (s): Average time to process a request to the model minus performance contention by using profiled runtimes from multiple trials of SyntheticEfficiencyScenario.

The TwitterAAE corpus of [Blodgett et al. (2016)](https://aclanthology.org/D16-1120/) for measuring language model performance in tweets as a function of speaker dialect.

# eval: Number of evaluation instances.

The TwitterAAE corpus of [Blodgett et al. (2016)](https://aclanthology.org/D16-1120/) for measuring language model performance in tweets as a function of speaker dialect.

# train: Number of training instances (e.g., in-context examples).

The TwitterAAE corpus of [Blodgett et al. (2016)](https://aclanthology.org/D16-1120/) for measuring language model performance in tweets as a function of speaker dialect.

truncated: Fraction of instances where the prompt itself was truncated (implies that there were no in-context examples).

The TwitterAAE corpus of [Blodgett et al. (2016)](https://aclanthology.org/D16-1120/) for measuring language model performance in tweets as a function of speaker dialect.

# prompt tokens: Number of tokens in the prompt.

The TwitterAAE corpus of [Blodgett et al. (2016)](https://aclanthology.org/D16-1120/) for measuring language model performance in tweets as a function of speaker dialect.

# output tokens: Actual number of output tokens.

The TwitterAAE corpus of [Blodgett et al. (2016)](https://aclanthology.org/D16-1120/) for measuring language model performance in tweets as a function of speaker dialect.

# trials: Number of trials, where in each trial we choose an independent, random set of training instances.

min=1.829, mean=2.18, max=2.53, sum=4.359 (2)

min=0.263, mean=0.263, max=0.263, sum=0.525 (2)

min=1000, mean=1000, max=1000, sum=2000 (2)

min=11.764, mean=12.325, max=12.885, sum=24.649 (2)

min=1.888, mean=2.245, max=2.603, sum=4.491 (2)

min=0.24, mean=0.24, max=0.24, sum=0.481 (2)

min=1.852, mean=2.205, max=2.559, sum=4.411 (2)

min=0.287, mean=0.288, max=0.288, sum=0.575 (2)

min=1.824, mean=2.156, max=2.489, sum=4.312 (2)

min=1.798, mean=2.121, max=2.443, sum=4.242 (2)

min=1.813, mean=2.137, max=2.462, sum=4.275 (2)

min=1.841, mean=2.179, max=2.518, sum=4.359 (2)

min=1.86, mean=2.109, max=2.359, sum=4.219 (2)

min=0.731, mean=0.736, max=0.742, sum=1.473 (2)

min=14.861, mean=16.422, max=17.983, sum=32.844 (2)

min=5.874, mean=5.959, max=6.044, sum=11.918 (2)

min=1.644, mean=1.986, max=2.328, sum=3.972 (2)

min=0.035, mean=0.036, max=0.036, sum=0.071 (2)

min=13.993, mean=15.618, max=17.243, sum=31.236 (2)

min=1.941, mean=2.305, max=2.669, sum=4.609 (2)

min=0.319, mean=0.32, max=0.321, sum=0.64 (2)

min=14.845, mean=16.471, max=18.096, sum=32.941 (2)

min=1.954, mean=2.324, max=2.693, sum=4.647 (2)

min=0.204, mean=0.205, max=0.205, sum=0.409 (2)

min=1.968, mean=2.341, max=2.715, sum=4.683 (2)

min=0.189, mean=0.189, max=0.19, sum=0.379 (2)

min=2.136, mean=2.525, max=2.913, sum=5.049 (2)

min=0.215, mean=0.215, max=0.215, sum=0.43 (2)

min=1.955, mean=2.325, max=2.694, sum=4.649 (2)

min=2.039, mean=2.428, max=2.817, sum=4.856 (2)

min=2.201, mean=2.61, max=3.019, sum=5.22 (2)

min=1.001, mean=1.002, max=1.003, sum=2.004 (2)

min=2.308, mean=2.73, max=3.151, sum=5.46 (2)

min=1.675, mean=1.989, max=2.302, sum=3.977 (2)

min=0.437, mean=0.438, max=0.438, sum=0.875 (2)

min=1.632, mean=1.943, max=2.254, sum=3.886 (2)

min=0.824, mean=0.824, max=0.824, sum=1.648 (2)

min=14.966, mean=16.553, max=18.141, sum=33.107 (2)

min=1.506, mean=1.81, max=2.114, sum=3.62 (2)

min=0.134, mean=0.184, max=0.233, sum=0.368 (2)

min=1.57, mean=1.85, max=2.13, sum=3.7 (2)

min=1.262, mean=1.274, max=1.285, sum=2.548 (2)

min=1.903, mean=2.245, max=2.587, sum=4.49 (2)

min=1.972, mean=2.323, max=2.673, sum=4.645 (2)

min=1.806, mean=2.141, max=2.476, sum=4.282 (2)

min=0.185, mean=0.185, max=0.185, sum=0.37 (2)

min=1.864, mean=2.21, max=2.556, sum=4.42 (2)

min=0.08, mean=0.08, max=0.08, sum=0.16 (2)

min=1.936, mean=2.291, max=2.646, sum=4.582 (2)

min=0.111, mean=0.111, max=0.111, sum=0.223 (2)

min=2.025, mean=2.395, max=2.764, sum=4.789 (2)

min=0.136, mean=0.136, max=0.136, sum=0.273 (2)

min=1.862, mean=2.192, max=2.522, sum=4.384 (2)

min=1.815, mean=2.147, max=2.478, sum=4.293 (2)

min=0.159, mean=0.16, max=0.16, sum=0.319 (2)

min=2.221, mean=2.628, max=3.034, sum=5.255 (2)

min=0.12, mean=0.12, max=0.12, sum=0.24 (2)

min=2.506, mean=2.946, max=3.385, sum=5.892 (2)

min=0.122, mean=0.123, max=0.123, sum=0.245 (2)

min=3.048, mean=3.505, max=3.963, sum=7.011 (2)

min=0.077, mean=0.077, max=0.077, sum=0.153 (2)

TwitterAAE

twitter_aae_demographic:aa

min=2.53, mean=2.53, max=2.53, sum=2.53 (1)

min=0.263, mean=0.263, max=0.263, sum=0.263 (1)

min=1000, mean=1000, max=1000, sum=1000 (1)

min=11.764, mean=11.764, max=11.764, sum=11.764 (1)

min=2.603, mean=2.603, max=2.603, sum=2.603 (1)

min=0.24, mean=0.24, max=0.24, sum=0.24 (1)

min=2.559, mean=2.559, max=2.559, sum=2.559 (1)

min=0.287, mean=0.287, max=0.287, sum=0.287 (1)

min=2.489, mean=2.489, max=2.489, sum=2.489 (1)

min=2.443, mean=2.443, max=2.443, sum=2.443 (1)

min=2.462, mean=2.462, max=2.462, sum=2.462 (1)

min=2.518, mean=2.518, max=2.518, sum=2.518 (1)

min=2.359, mean=2.359, max=2.359, sum=2.359 (1)

min=0.731, mean=0.731, max=0.731, sum=0.731 (1)

min=14.861, mean=14.861, max=14.861, sum=14.861 (1)

min=5.874, mean=5.874, max=5.874, sum=5.874 (1)

min=2.328, mean=2.328, max=2.328, sum=2.328 (1)

min=0.035, mean=0.035, max=0.035, sum=0.035 (1)

min=13.993, mean=13.993, max=13.993, sum=13.993 (1)

min=2.669, mean=2.669, max=2.669, sum=2.669 (1)

min=0.319, mean=0.319, max=0.319, sum=0.319 (1)

min=14.845, mean=14.845, max=14.845, sum=14.845 (1)

min=2.693, mean=2.693, max=2.693, sum=2.693 (1)

min=0.204, mean=0.204, max=0.204, sum=0.204 (1)

min=2.715, mean=2.715, max=2.715, sum=2.715 (1)

min=0.189, mean=0.189, max=0.189, sum=0.189 (1)

min=2.913, mean=2.913, max=2.913, sum=2.913 (1)

min=0.215, mean=0.215, max=0.215, sum=0.215 (1)

min=2.694, mean=2.694, max=2.694, sum=2.694 (1)

min=2.817, mean=2.817, max=2.817, sum=2.817 (1)

min=3.019, mean=3.019, max=3.019, sum=3.019 (1)

min=1.003, mean=1.003, max=1.003, sum=1.003 (1)

min=3.151, mean=3.151, max=3.151, sum=3.151 (1)

min=2.302, mean=2.302, max=2.302, sum=2.302 (1)

min=0.438, mean=0.438, max=0.438, sum=0.438 (1)

min=2.254, mean=2.254, max=2.254, sum=2.254 (1)

min=0.824, mean=0.824, max=0.824, sum=0.824 (1)

min=14.966, mean=14.966, max=14.966, sum=14.966 (1)

min=2.114, mean=2.114, max=2.114, sum=2.114 (1)

min=0.134, mean=0.134, max=0.134, sum=0.134 (1)

min=2.13, mean=2.13, max=2.13, sum=2.13 (1)

min=1.285, mean=1.285, max=1.285, sum=1.285 (1)

min=2.587, mean=2.587, max=2.587, sum=2.587 (1)

min=2.673, mean=2.673, max=2.673, sum=2.673 (1)

min=2.476, mean=2.476, max=2.476, sum=2.476 (1)

min=0.185, mean=0.185, max=0.185, sum=0.185 (1)

min=2.556, mean=2.556, max=2.556, sum=2.556 (1)

min=0.08, mean=0.08, max=0.08, sum=0.08 (1)

min=2.646, mean=2.646, max=2.646, sum=2.646 (1)

min=0.111, mean=0.111, max=0.111, sum=0.111 (1)

min=2.764, mean=2.764, max=2.764, sum=2.764 (1)

min=0.136, mean=0.136, max=0.136, sum=0.136 (1)

min=2.522, mean=2.522, max=2.522, sum=2.522 (1)

min=2.478, mean=2.478, max=2.478, sum=2.478 (1)

min=0.159, mean=0.159, max=0.159, sum=0.159 (1)

min=3.034, mean=3.034, max=3.034, sum=3.034 (1)

min=0.12, mean=0.12, max=0.12, sum=0.12 (1)

min=3.385, mean=3.385, max=3.385, sum=3.385 (1)

min=0.122, mean=0.122, max=0.122, sum=0.122 (1)

min=3.963, mean=3.963, max=3.963, sum=3.963 (1)

min=0.077, mean=0.077, max=0.077, sum=0.077 (1)

demographic: aa

twitter_aae_demographic:white

min=1.829, mean=1.829, max=1.829, sum=1.829 (1)

min=12.885, mean=12.885, max=12.885, sum=12.885 (1)

min=1.888, mean=1.888, max=1.888, sum=1.888 (1)

min=1.852, mean=1.852, max=1.852, sum=1.852 (1)

min=0.288, mean=0.288, max=0.288, sum=0.288 (1)

min=1.824, mean=1.824, max=1.824, sum=1.824 (1)

min=1.798, mean=1.798, max=1.798, sum=1.798 (1)

min=1.813, mean=1.813, max=1.813, sum=1.813 (1)

min=1.841, mean=1.841, max=1.841, sum=1.841 (1)

min=1.86, mean=1.86, max=1.86, sum=1.86 (1)

min=0.742, mean=0.742, max=0.742, sum=0.742 (1)

min=17.983, mean=17.983, max=17.983, sum=17.983 (1)

min=6.044, mean=6.044, max=6.044, sum=6.044 (1)

min=1.644, mean=1.644, max=1.644, sum=1.644 (1)

min=0.036, mean=0.036, max=0.036, sum=0.036 (1)

min=17.243, mean=17.243, max=17.243, sum=17.243 (1)

min=1.941, mean=1.941, max=1.941, sum=1.941 (1)

min=0.321, mean=0.321, max=0.321, sum=0.321 (1)

min=18.096, mean=18.096, max=18.096, sum=18.096 (1)

min=1.954, mean=1.954, max=1.954, sum=1.954 (1)

min=0.205, mean=0.205, max=0.205, sum=0.205 (1)

min=1.968, mean=1.968, max=1.968, sum=1.968 (1)

min=0.19, mean=0.19, max=0.19, sum=0.19 (1)

min=2.136, mean=2.136, max=2.136, sum=2.136 (1)

min=1.955, mean=1.955, max=1.955, sum=1.955 (1)

min=2.039, mean=2.039, max=2.039, sum=2.039 (1)

min=2.201, mean=2.201, max=2.201, sum=2.201 (1)

min=1.001, mean=1.001, max=1.001, sum=1.001 (1)

min=2.308, mean=2.308, max=2.308, sum=2.308 (1)

min=1.675, mean=1.675, max=1.675, sum=1.675 (1)

min=0.437, mean=0.437, max=0.437, sum=0.437 (1)

min=1.632, mean=1.632, max=1.632, sum=1.632 (1)

min=18.141, mean=18.141, max=18.141, sum=18.141 (1)

min=1.506, mean=1.506, max=1.506, sum=1.506 (1)

min=0.233, mean=0.233, max=0.233, sum=0.233 (1)

min=1.57, mean=1.57, max=1.57, sum=1.57 (1)

min=1.262, mean=1.262, max=1.262, sum=1.262 (1)

min=1.903, mean=1.903, max=1.903, sum=1.903 (1)

min=1.972, mean=1.972, max=1.972, sum=1.972 (1)

min=1.806, mean=1.806, max=1.806, sum=1.806 (1)

min=1.864, mean=1.864, max=1.864, sum=1.864 (1)

min=1.936, mean=1.936, max=1.936, sum=1.936 (1)

min=2.025, mean=2.025, max=2.025, sum=2.025 (1)

min=1.862, mean=1.862, max=1.862, sum=1.862 (1)

min=1.815, mean=1.815, max=1.815, sum=1.815 (1)

min=0.16, mean=0.16, max=0.16, sum=0.16 (1)

min=2.221, mean=2.221, max=2.221, sum=2.221 (1)

min=2.506, mean=2.506, max=2.506, sum=2.506 (1)

min=0.123, mean=0.123, max=0.123, sum=0.123 (1)

min=3.048, mean=3.048, max=3.048, sum=3.048 (1)