Scenario from Magellan [(Konda et al., 2016)](https://dl.acm.org/doi/10.14778/3007263.3007314) that tests the ability to determine if two entities match.

entity_matching

Scenario from Magellan [(Konda et al., 2016)](https://dl.acm.org/doi/10.14778/3007263.3007314) that tests the ability to determine if two entities match.

Quasi-exact match: Fraction of instances that the predicted output matches a correct reference up to light processing.

Scenario from Magellan [(Konda et al., 2016)](https://dl.acm.org/doi/10.14778/3007263.3007314) that tests the ability to determine if two entities match.

Denoised inference runtime (s): Average time to process a request to the model minus performance contention by using profiled runtimes from multiple trials of SyntheticEfficiencyScenario.

Scenario from Magellan [(Konda et al., 2016)](https://dl.acm.org/doi/10.14778/3007263.3007314) that tests the ability to determine if two entities match.

# eval: Number of evaluation instances.

Scenario from Magellan [(Konda et al., 2016)](https://dl.acm.org/doi/10.14778/3007263.3007314) that tests the ability to determine if two entities match.

# train: Number of training instances (e.g., in-context examples).

Scenario from Magellan [(Konda et al., 2016)](https://dl.acm.org/doi/10.14778/3007263.3007314) that tests the ability to determine if two entities match.

truncated: Fraction of instances where the prompt itself was truncated (implies that there were no in-context examples).

Scenario from Magellan [(Konda et al., 2016)](https://dl.acm.org/doi/10.14778/3007263.3007314) that tests the ability to determine if two entities match.

# prompt tokens: Number of tokens in the prompt.

Scenario from Magellan [(Konda et al., 2016)](https://dl.acm.org/doi/10.14778/3007263.3007314) that tests the ability to determine if two entities match.

# output tokens: Actual number of output tokens.

Scenario from Magellan [(Konda et al., 2016)](https://dl.acm.org/doi/10.14778/3007263.3007314) that tests the ability to determine if two entities match.

# trials: Number of trials, where in each trial we choose an independent, random set of training instances.

min=0.752, mean=0.841, max=0.934, sum=7.573 (9)

min=0.544, mean=0.693, max=0.9, sum=6.233 (9)

min=542.813, mean=895.507, max=1409.431, sum=8059.561 (9)

min=0.734, mean=0.827, max=0.89, sum=7.446 (9)

min=0.433, mean=0.533, max=0.665, sum=4.796 (9)

min=0.752, mean=0.831, max=0.88, sum=7.477 (9)

min=0.484, mean=0.603, max=0.774, sum=5.425 (9)

min=0.761, mean=0.844, max=0.901, sum=7.599 (9)

min=0.367, mean=0.79, max=0.923, sum=7.114 (9)

min=0.752, mean=0.849, max=0.945, sum=7.64 (9)

min=0.752, mean=0.836, max=0.88, sum=7.521 (9)

min=0.303, mean=0.543, max=0.726, sum=4.888 (9)

min=591.725, mean=889.812, max=1345.211, sum=8008.306 (9)

min=0.248, mean=0.635, max=0.912, sum=5.711 (9)

min=0.523, mean=0.707, max=0.912, sum=6.363 (9)

min=0.237, mean=0.71, max=0.923, sum=6.388 (9)

min=0.549, mean=0.633, max=0.774, sum=5.698 (9)

min=572.055, mean=890.493, max=1343.394, sum=8014.436 (9)

min=0.752, mean=0.852, max=0.897, sum=7.669 (9)

min=0.224, mean=0.568, max=1.151, sum=5.114 (9)

min=549.978, mean=850.495, max=1308.009, sum=7654.457 (9)

min=0.166, mean=0.168, max=0.17, sum=1.509 (9)

min=2.422, mean=4.138, max=5, sum=37.242 (9)

min=670.308, mean=848.446, max=959.85, sum=7636.012 (9)

min=0.679, mean=0.823, max=0.874, sum=7.404 (9)

min=0.493, mean=0.587, max=0.727, sum=5.28 (9)

min=586.319, mean=887.65, max=1340.89, sum=7988.851 (9)

min=0.385, mean=0.713, max=0.872, sum=6.416 (9)

min=0.339, mean=0.412, max=0.522, sum=3.707 (9)

min=0.128, mean=0.482, max=0.846, sum=4.334 (9)

min=0.293, mean=0.343, max=0.414, sum=3.086 (9)

min=0.126, mean=0.176, max=0.248, sum=1.582 (9)

min=0.306, mean=0.361, max=0.449, sum=3.246 (9)

min=0.486, mean=0.812, max=0.934, sum=7.307 (9)

min=0.174, mean=0.535, max=0.846, sum=4.812 (9)

min=0.176, mean=0.532, max=0.925, sum=4.792 (9)

min=0.275, mean=0.569, max=0.923, sum=5.121 (9)

min=0.126, mean=0.194, max=0.367, sum=1.745 (9)

min=0.183, mean=0.197, max=0.273, sum=1.777 (9)

min=0.752, mean=0.82, max=0.874, sum=7.383 (9)

min=0.17, mean=0.247, max=0.371, sum=2.226 (9)

min=578.978, mean=906.402, max=1388.477, sum=8157.617 (9)

min=0.257, mean=0.652, max=0.864, sum=5.868 (9)

min=0.23, mean=0.236, max=0.25, sum=2.123 (9)

min=0.138, mean=1.66, max=3, sum=14.942 (9)

min=0, mean=0.132, max=0.651, sum=1.185 (9)

min=288, mean=421.841, max=491.736, sum=3796.569 (9)

min=0.037, mean=0.672, max=0.874, sum=6.048 (9)

min=0.215, mean=0.222, max=0.228, sum=1.997 (9)

min=0.11, mean=1.631, max=3, sum=14.676 (9)

min=0, mean=0.143, max=0.679, sum=1.288 (9)

min=283.798, mean=419.916, max=489.308, sum=3779.24 (9)

min=0.126, mean=0.374, max=0.819, sum=3.363 (9)

min=0.449, mean=0.63, max=0.745, sum=5.671 (9)

min=0.128, mean=0.466, max=0.846, sum=4.196 (9)

min=0.063, mean=0.554, max=1.342, sum=4.99 (9)

min=0.817, mean=0.881, max=0.923, sum=7.929 (9)

min=0.257, mean=0.694, max=0.874, sum=6.246 (9)

min=0.154, mean=0.603, max=0.923, sum=5.425 (9)

min=0.201, mean=0.21, max=0.22, sum=1.887 (9)

min=0.126, mean=0.453, max=0.857, sum=4.081 (9)

min=0.094, mean=0.099, max=0.106, sum=0.895 (9)

min=0.126, mean=0.632, max=0.874, sum=5.685 (9)

min=0.118, mean=0.121, max=0.126, sum=1.091 (9)

min=0.154, mean=0.349, max=0.67, sum=3.137 (9)

min=0.139, mean=0.14, max=0.141, sum=1.264 (9)

min=0.895, mean=0.93, max=1, sum=8.374 (9)

min=0.84, mean=0.931, max=0.972, sum=8.377 (9)

min=0.173, mean=0.19, max=0.221, sum=1.711 (9)

min=0.716, mean=0.852, max=0.901, sum=7.668 (9)

min=0.138, mean=0.142, max=0.149, sum=1.282 (9)

min=0.418, mean=0.734, max=0.874, sum=6.603 (9)

min=0.134, mean=0.142, max=0.153, sum=1.275 (9)

min=0.734, mean=0.808, max=0.874, sum=7.276 (9)

min=0.088, mean=0.095, max=0.105, sum=0.856 (9)

min=0.89, mean=0.933, max=0.974, sum=8.395 (9)

min=541.692, mean=873.751, max=1347.092, sum=7863.758 (9)

min=0.134, mean=0.533, max=0.752, sum=1.6 (3)

min=587.978, mean=942.402, max=1388.477, sum=2827.206 (3)

min=0.245, mean=0.472, max=0.802, sum=4.25 (9)

min=0.484, mean=0.657, max=0.992, sum=5.917 (9)

min=613.044, mean=1041.057, max=1642.339, sum=9369.515 (9)

min=0.431, mean=0.77, max=0.901, sum=6.931 (9)

min=0.998, mean=1.0, max=1, sum=8.998 (9)

min=0.945, mean=0.966, max=0.991, sum=8.693 (9)

min=0.269, mean=0.516, max=0.798, sum=4.646 (9)

min=561.659, mean=848.586, max=1301.431, sum=7637.274 (9)

Entity matching

entity_matching_dataset:Abt_Buy

min=0.876, mean=0.879, max=0.88, sum=2.637 (3)

min=0.66, mean=0.669, max=0.687, sum=2.008 (3)

min=799.276, mean=818.943, max=857.276, sum=2456.828 (3)

min=0.822, mean=0.855, max=0.872, sum=2.566 (3)

min=0.518, mean=0.525, max=0.538, sum=1.575 (3)

min=0.872, mean=0.876, max=0.88, sum=2.627 (3)

min=0.572, mean=0.579, max=0.592, sum=1.736 (3)

min=0.87, mean=0.872, max=0.874, sum=2.617 (3)

min=0.649, mean=0.819, max=0.913, sum=2.456 (3)

min=0.872, mean=0.876, max=0.882, sum=2.629 (3)

min=0.854, mean=0.869, max=0.88, sum=2.606 (3)

min=0.54, mean=0.629, max=0.726, sum=1.886 (3)

min=810.499, mean=840.166, max=873.499, sum=2520.497 (3)

min=0.473, mean=0.663, max=0.872, sum=1.988 (3)

min=0.523, mean=0.566, max=0.637, sum=1.698 (3)

min=0.237, mean=0.573, max=0.876, sum=1.72 (3)

min=0.598, mean=0.602, max=0.608, sum=1.807 (3)

min=838.696, mean=860.029, max=891.696, sum=2580.087 (3)

min=0.876, mean=0.885, max=0.897, sum=2.655 (3)

min=0.373, mean=0.416, max=0.442, sum=1.247 (3)

min=770.832, mean=792.165, max=819.832, sum=2376.495 (3)

min=0.166, mean=0.166, max=0.166, sum=0.499 (3)

min=4.422, mean=4.622, max=4.84, sum=13.866 (3)

min=946.241, mean=951.1, max=959.85, sum=2853.3 (3)

min=0.828, mean=0.859, max=0.874, sum=2.576 (3)

min=0.562, mean=0.571, max=0.582, sum=1.714 (3)

min=807.075, mean=838.742, max=874.075, sum=2516.225 (3)

min=0.385, mean=0.71, max=0.872, sum=2.13 (3)

min=0.392, mean=0.399, max=0.407, sum=1.197 (3)

min=0.128, mean=0.383, max=0.779, sum=1.15 (3)

min=0.333, mean=0.339, max=0.346, sum=1.018 (3)

min=0.126, mean=0.126, max=0.126, sum=0.377 (3)

min=0.341, mean=0.346, max=0.351, sum=1.037 (3)

min=0.86, mean=0.87, max=0.878, sum=2.611 (3)

min=0.174, mean=0.429, max=0.726, sum=1.288 (3)

min=0.744, mean=0.861, max=0.925, sum=2.584 (3)

min=0.312, mean=0.517, max=0.763, sum=1.55 (3)

min=0.183, mean=0.213, max=0.273, sum=0.64 (3)

min=0.872, mean=0.874, max=0.874, sum=2.621 (3)

min=0.17, mean=0.175, max=0.183, sum=0.524 (3)

min=821.751, mean=855.417, max=893.751, sum=2566.252 (3)

min=0.643, mean=0.785, max=0.864, sum=2.355 (3)

min=0.231, mean=0.234, max=0.235, sum=0.701 (3)

min=1.081, mean=1.508, max=1.738, sum=4.523 (3)

min=0, mean=0.007, max=0.02, sum=0.02 (3)

min=406.391, mean=431.516, max=445.201, sum=1294.548 (3)

min=0.86, mean=0.87, max=0.874, sum=2.609 (3)

min=0.215, mean=0.22, max=0.228, sum=0.661 (3)

min=1.059, mean=1.483, max=1.708, sum=4.45 (3)

min=0, mean=0.007, max=0.02, sum=0.022 (3)

min=406.621, mean=431.677, max=444.641, sum=1295.032 (3)

min=0.126, mean=0.358, max=0.819, sum=1.075 (3)

min=0.691, mean=0.693, max=0.695, sum=2.08 (3)

min=0.128, mean=0.32, max=0.704, sum=0.959 (3)

min=0.263, mean=0.269, max=0.273, sum=0.808 (3)

min=0.89, mean=0.892, max=0.897, sum=2.677 (3)

min=0.574, mean=0.771, max=0.874, sum=2.312 (3)

min=0.154, mean=0.551, max=0.765, sum=1.653 (3)

min=0.21, mean=0.211, max=0.212, sum=0.632 (3)

min=0.126, mean=0.34, max=0.767, sum=1.02 (3)

min=0.1, mean=0.101, max=0.102, sum=0.303 (3)

min=0.126, mean=0.376, max=0.874, sum=1.128 (3)

min=0.121, mean=0.121, max=0.121, sum=0.363 (3)

min=0.201, mean=0.383, max=0.619, sum=1.148 (3)

min=0.141, mean=0.141, max=0.141, sum=0.423 (3)

min=0.895, mean=0.92, max=0.957, sum=2.761 (3)

min=0.84, mean=0.899, max=0.957, sum=2.698 (3)

min=0.181, mean=0.181, max=0.182, sum=0.544 (3)

min=0.856, mean=0.877, max=0.89, sum=2.631 (3)

min=0.147, mean=0.148, max=0.149, sum=0.443 (3)

min=0.852, mean=0.867, max=0.874, sum=2.6 (3)

min=0.14, mean=0.14, max=0.141, sum=0.421 (3)

min=0.734, mean=0.827, max=0.874, sum=2.481 (3)

min=0.094, mean=0.094, max=0.095, sum=0.283 (3)

min=0.963, mean=0.968, max=0.974, sum=2.905 (3)

min=803.469, mean=836.135, max=877.469, sum=2508.406 (3)

min=0.134, mean=0.134, max=0.134, sum=0.134 (1)

min=850.751, mean=850.751, max=850.751, sum=850.751 (1)

min=0.245, mean=0.335, max=0.481, sum=1.004 (3)

min=0.631, mean=0.632, max=0.633, sum=1.896 (3)

min=952.122, mean=988.455, max=1024.122, sum=2965.365 (3)

min=0.7, mean=0.815, max=0.89, sum=2.444 (3)

min=0.998, mean=0.999, max=1, sum=2.998 (3)

min=0.957, mean=0.964, max=0.97, sum=2.892 (3)

min=0.466, mean=0.502, max=0.57, sum=1.507 (3)

min=771.667, mean=795.001, max=823.667, sum=2385.002 (3)

dataset: Abt_Buy

entity_matching_dataset:Beer

min=0.857, mean=0.89, max=0.934, sum=2.67 (3)

min=0.544, mean=0.546, max=0.548, sum=1.638 (3)

min=542.813, mean=546.147, max=550.813, sum=1638.44 (3)

min=0.857, mean=0.868, max=0.89, sum=2.604 (3)

min=0.433, mean=0.434, max=0.436, sum=1.303 (3)

min=0.857, mean=0.864, max=0.879, sum=2.593 (3)

min=0.484, mean=0.485, max=0.487, sum=1.456 (3)

min=0.879, mean=0.89, max=0.901, sum=2.67 (3)

min=0.846, mean=0.883, max=0.923, sum=2.648 (3)

min=0.89, mean=0.912, max=0.945, sum=2.736 (3)

min=0.857, mean=0.864, max=0.868, sum=2.593 (3)

min=0.659, mean=0.689, max=0.714, sum=2.066 (3)

min=591.725, mean=598.725, max=604.725, sum=1796.176 (3)

min=0.901, mean=0.905, max=0.912, sum=2.714 (3)