Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 19 additions & 19 deletions chapters/en/chapter12/2.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -142,15 +142,15 @@ In the next module, we'll get our hands dirty and dive into the DeepSeek R1 pape

<Question
choices={[
{
text: "Model, Data, Loss Function, and Optimizer",
explain: "These are components more commonly associated with supervised learning."
},
{
text: "Agent, Environment, Action, Reward, and Policy",
explain: "Correct! These are the fundamental components that make up a reinforcement learning system.",
correct: true
},
{
text: "Model, Data, Loss Function, and Optimizer",
explain: "These are components more commonly associated with supervised learning."
},
{
text: "Input, Output, and Hidden Layers",
explain: "These are components of neural network architecture, not specifically RL components."
Expand All @@ -162,18 +162,18 @@ In the next module, we'll get our hands dirty and dive into the DeepSeek R1 pape

<Question
choices={[
{
text: "It helps align models with human preferences and values",
explain: "Correct! RLHF uses human feedback to guide models toward more helpful, harmless, and aligned behavior.",
correct: true
},
{
text: "It makes models generate text faster",
explain: "RLHF isn't primarily about improving generation speed."
},
{
text: "It reduces the model's memory usage",
explain: "RLHF doesn't focus on model efficiency or memory optimization."
},
{
text: "It helps align models with human preferences and values",
explain: "Correct! RLHF uses human feedback to guide models toward more helpful, harmless, and aligned behavior.",
correct: true
}
]}
/>
Expand Down Expand Up @@ -202,18 +202,18 @@ In the next module, we'll get our hands dirty and dive into the DeepSeek R1 pape

<Question
choices={[
{
text: "To provide feedback on how well the model's responses align with desired behavior",
explain: "Correct! Rewards guide the model toward generating more helpful, truthful, and appropriate responses.",
correct: true
},
{
text: "To measure the model's vocabulary size",
explain: "Rewards aren't used to evaluate vocabulary knowledge."
},
{
text: "To determine the model's training speed",
explain: "Rewards provide feedback on response quality, not training efficiency."
},
{
text: "To provide feedback on how well the model's responses align with desired behavior",
explain: "Correct! Rewards guide the model toward generating more helpful, truthful, and appropriate responses.",
correct: true
}
]}
/>
Expand All @@ -222,18 +222,18 @@ In the next module, we'll get our hands dirty and dive into the DeepSeek R1 pape

<Question
choices={[
{
text: "A numerical score that measures the quality of a response",
explain: "Correct! Rewards provide feedback on response quality, guiding the model toward desired behavior.",
correct: true
},
{
text: "A function that generates responses",
explain: "Rewards are feedback on response quality, not the generation process itself."
},
{
text: "A model that evaluates the quality of responses",
explain: "Rewards are feedback on response quality, not an evaluation model."
},
{
text: "A numerical score that measures the quality of a response",
explain: "Correct! Rewards provide feedback on response quality, guiding the model toward desired behavior.",
correct: true
}
]}
/>
Expand Down
28 changes: 14 additions & 14 deletions chapters/en/chapter12/3.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -231,18 +231,18 @@ In the next section, we'll explore practical implementations of these concepts,

<Question
choices={[
{
text: "The GRPO algorithm that enables learning from preferences with and without a reward model",
explain: "Correct! GRPO's key innovation is its ability to directly optimize for preference rectification, making it more efficient than traditional RL methods.",
correct: true
},
{
text: "Using more GPUs for training than any previous model",
explain: "The paper's innovation is in its algorithmic approach (GRPO) rather than computational resources used."
},
{
text: "Creating a larger language model than existing ones",
explain: "The innovation lies in the training methodology and GRPO algorithm, not in model size."
},
{
text: "The GRPO algorithm that enables learning from preferences with and without a reward model",
explain: "Correct! GRPO's key innovation is its ability to directly optimize for preference rectification, making it more efficient than traditional RL methods.",
correct: true
}
]}
/>
Expand Down Expand Up @@ -271,18 +271,18 @@ In the next section, we'll explore practical implementations of these concepts,

<Question
choices={[
{
text: "A process where the model recognizes errors, self-corrects, and explains its corrections",
explain: "Correct! The paper describes this as a four-step process: initial attempt, recognition of errors, self-correction, and explanation of the improvement.",
correct: true
},
{
text: "The point where the model reaches human-level performance",
explain: "The 'Aha Moment' specifically refers to the model's ability to recognize and correct its own mistakes, similar to human problem-solving realizations."
},
{
text: "When the model completes its training process",
explain: "The 'Aha Moment' is about the model's emergent ability to recognize errors and self-correct during problem-solving, not about training completion."
},
{
text: "A process where the model recognizes errors, self-corrects, and explains its corrections",
explain: "Correct! The paper describes this as a four-step process: initial attempt, recognition of errors, self-correction, and explanation of the improvement.",
correct: true
}
]}
/>
Expand Down Expand Up @@ -311,15 +311,15 @@ In the next section, we'll explore practical implementations of these concepts,

<Question
choices={[
{
text: "R1-Zero is smaller than R1",
explain: "The difference is in their training approaches (pure RL vs. multi-phase), not their model sizes."
},
{
text: "R1-Zero uses pure RL while R1 combines RL with supervised fine-tuning",
explain: "Correct! As shown in the comparison table, R1-Zero uses pure RL training while R1 uses a multi-phase approach combining supervised fine-tuning with RL, resulting in better language consistency.",
correct: true
},
{
text: "R1-Zero is smaller than R1",
explain: "The difference is in their training approaches (pure RL vs. multi-phase), not their model sizes."
},
{
text: "R1-Zero was trained on less data",
explain: "The key distinction is their training methodology: pure RL for R1-Zero versus a combined SFT and RL approach for R1."
Expand Down
Loading