4.37.2 vs 4.38.0 tokenizer diff

Created Diff never expires
3 removals
Lines
Total
Removed
Words
Total
Removed
To continue using this feature, upgrade to
Diffchecker logo
Diffchecker Pro
10 lines
3 additions
Lines
Total
Added
Words
Total
Added
To continue using this feature, upgrade to
Diffchecker logo
Diffchecker Pro
10 lines
QUESTION: Janet’s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers' market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers' market?
QUESTION: Janet’s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers' market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers' market?
ANSWER: Janet sells 16 - 3 - 4 = <<16-3-4=9>>9 duck eggs a day.
ANSWER: Janet sells 16 - 3 - 4 = <<16-3-4=9>>9 duck eggs a day.
She makes 9 * 2 = $<<9*2=18>>18 every day at the farmer’s market.
She makes 9 * 2 = $<<9*2=18>>18 every day at the farmer’s market.
#### 18
#### 18
LABELS: [1, 1186, 28747, 2997, 299, 28809, 28713, 281, 18352, 4897, 28705, 28740, 28784, 14636, 660, 1370, 28723, 985, 317, 1449, 1712, 354, 11814, 1012, 3970, 304, 287, 1593, 290, 1292, 1126, 354, 559, 3282, 1012, 1370, 395, 2308, 28723, 985, 6112, 28713, 272, 23317, 438, 272, 17130, 28742, 2668, 6790, 354, 429, 28750, 660, 6138, 27103, 9119, 28723, 1602, 1188, 297, 9407, 1235, 630, 1038, 1012, 1370, 438, 272, 17130, 28742, 2668, 28804, 13, 28741, 28747, 28705, 28740, 28783, 13, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2]
LABELS: [1, 28824, 28747, 2997, 299, 28809, 28713, 281, 18352, 4897, 28705, 28740, 28784, 14636, 660, 1370, 28723, 985, 317, 1449, 1712, 354, 11814, 1012, 3970, 304, 287, 1593, 290, 1292, 1126, 354, 559, 3282, 1012, 1370, 395, 2308, 28723, 985, 6112, 28713, 272, 23317, 438, 272, 17130, 28742, 2668, 6790, 354, 429, 28750, 660, 6138, 27103, 9119, 28723, 1602, 1188, 297, 9407, 1235, 630, 1038, 1012, 1370, 438, 272, 17130, 28742, 2668, 28804, 13, 28741, 28747, 28705, 28740, 28783, 13, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2]
LABELS (DETOKENIZED): <s> Q: Janet’s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers' market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers' market?
LABELS (DETOKENIZED): <s>Q: Janet’s ducks lay 16 eggs per day. She eats three for breakfast every morning and bakes muffins for her friends every day with four. She sells the remainder at the farmers' market daily for $2 per fresh duck egg. How much in dollars does she make every day at the farmers' market?
A: 18
A: 18


SLICED: ['1: <s>', '330: A', '28747: :', '28705: ', '28740: 1', '28783: 8', '13: <0x0A>']
SLICED: ['1: <s>', '28747: :', '28705: ', '28740: 1', '28783: 8', '13: <0x0A>']
[28740, 28750, 28770, 28781, 28782, 28784, 28787, 28783, 28774, 28734, 13] [330, 365, 334, 384, 413, 13]
[28740, 28750, 28770, 28781, 28782, 28784, 28787, 28783, 28774, 28734, 13] [330, 365, 334, 384, 413, 13]