Skip to content

Update dataset sources #1273

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lhoestq wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Update dataset sources #1273

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e2cc310
Update dataset sources
lhoestq Jul 6, 2026
47ac5dd
Update dataset loading to use nyu-mll/glue
lhoestq Jul 6, 2026
fd683e7
more changes
lhoestq Jul 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion chapters/de/chapter3/2.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ Die Bibliothek 🤗 Datasets bietet einen leichten Befehl zum Herunterladen und
```py
from datasets import load_dataset

raw_datasets = load_dataset("glue", "mrpc")
raw_datasets = load_dataset("nyu-mll/glue", "mrpc")
raw_datasets
```

Expand Down
2 changes: 1 addition & 1 deletion chapters/de/chapter3/3.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ In den folgenden Code-Beispielen wird davon ausgegangen, dass du die Beispiele a
from datasets import load_dataset
from transformers import AutoTokenizer, DataCollatorWithPadding

raw_datasets = load_dataset("glue", "mrpc")
raw_datasets = load_dataset("nyu-mll/glue", "mrpc")
checkpoint = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)

Expand Down
2 changes: 1 addition & 1 deletion chapters/de/chapter3/3_tf.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ from datasets import load_dataset
from transformers import AutoTokenizer, DataCollatorWithPadding
import numpy as np

raw_datasets = load_dataset("glue", "mrpc")
raw_datasets = load_dataset("nyu-mll/glue", "mrpc")
checkpoint = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)

Expand Down
2 changes: 1 addition & 1 deletion chapters/de/chapter3/4.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ In diesem Abschnitt befassen wir uns damit, wie wir die gleichen Ergebnisse wie
from datasets import load_dataset
from transformers import AutoTokenizer, DataCollatorWithPadding

raw_datasets = load_dataset("glue", "mrpc")
raw_datasets = load_dataset("nyu-mll/glue", "mrpc")
checkpoint = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)

Expand Down
2 changes: 1 addition & 1 deletion chapters/en/chapter3/2.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ The 🤗 Datasets library provides a very simple command to download and cache a
```py
from datasets import load_dataset

raw_datasets = load_dataset("glue", "mrpc")
raw_datasets = load_dataset("nyu-mll/glue", "mrpc")
raw_datasets
```

Expand Down
2 changes: 1 addition & 1 deletion chapters/en/chapter3/3.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ The code examples below assume you have already executed the examples in the pre
from datasets import load_dataset
from transformers import AutoTokenizer, DataCollatorWithPadding

raw_datasets = load_dataset("glue", "mrpc")
raw_datasets = load_dataset("nyu-mll/glue", "mrpc")
checkpoint = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)

Expand Down
2 changes: 1 addition & 1 deletion chapters/en/chapter3/4.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Now we'll see how to achieve the same results as we did in the last section with
from datasets import load_dataset
from transformers import AutoTokenizer, DataCollatorWithPadding

raw_datasets = load_dataset("glue", "mrpc")
raw_datasets = load_dataset("nyu-mll/glue", "mrpc")
checkpoint = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)

Expand Down
2 changes: 1 addition & 1 deletion chapters/en/chapter5/6.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ df["comments"][0].tolist()
```

```python out
['the bug code locate in ：\r\n if data_args.task_name is not None:\r\n # Downloading and loading a dataset from the hub.\r\n datasets = load_dataset("glue", data_args.task_name, cache_dir=model_args.cache_dir)',
['the bug code locate in ：\r\n if data_args.task_name is not None:\r\n # Downloading and loading a dataset from the hub.\r\n datasets = load_dataset("nyu-mll/glue", data_args.task_name, cache_dir=model_args.cache_dir)',
'Hi @jinec,\r\n\r\nFrom time to time we get this kind of `ConnectionError` coming from the github.com website: https://raw.githubusercontent.com\r\n\r\nNormally, it should work if you wait a little and then retry.\r\n\r\nCould you please confirm if the problem persists?',
'cannot connect，even by Web browser，please check that there is some problems。',
'I can access https://raw.githubusercontent.com/huggingface/datasets/1.7.0/datasets/glue/glue.py without problem...']
Expand Down
4 changes: 2 additions & 2 deletions chapters/en/chapter5/8.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Before moving on, though, let's test what you learned in this chapter.
```py
from datasets import load_dataset

dataset = load_dataset("glue", "mrpc", split="train")
dataset = load_dataset("nyu-mll/glue", "mrpc", split="train")
```

Which of the following commands will produce a random sample of 50 elements from `dataset`?
Expand Down Expand Up @@ -128,7 +128,7 @@ Which of the following commands will produce a random sample of 50 elements from
```py
from datasets import load_dataset

dataset = load_dataset("allocine", streaming=True, split="train")
dataset = load_dataset("tblard/allocine", streaming=True, split="train")
dataset[0]
```

Expand Down
2 changes: 1 addition & 1 deletion chapters/en/chapter6/2.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ The [🤗 Datasets](https://github.com/huggingface/datasets) library can help us
from datasets import load_dataset

# This can take a few minutes to load, so grab a coffee or tea while you wait!
raw_datasets = load_dataset("code_search_net", "python")
raw_datasets = load_dataset("code-search-net/code_search_net", "python")
```

We can have a look at the training split to see which columns we have access to:
Expand Down
2 changes: 1 addition & 1 deletion chapters/en/chapter6/8.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ To train our new tokenizer, we will use a small corpus of text (so the examples
```python
from datasets import load_dataset

dataset = load_dataset("wikitext", name="wikitext-2-raw-v1", split="train")
dataset = load_dataset("Salesforce/wikitext", name="wikitext-2-raw-v1", split="train")


def get_training_corpus():
Expand Down
2 changes: 1 addition & 1 deletion chapters/en/chapter7/7.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ As usual, we can download and cache the dataset in just one step thanks to `load
```py
from datasets import load_dataset

raw_datasets = load_dataset("squad")
raw_datasets = load_dataset("rajpurkar/squad")
```

We can then have a look at this object to learn more about the SQuAD dataset:
Expand Down
10 changes: 5 additions & 5 deletions chapters/en/chapter8/4.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ from transformers import (
Trainer,
)

raw_datasets = load_dataset("glue", "mnli")
raw_datasets = load_dataset("nyu-mll/glue", "mnli")

model_checkpoint = "distilbert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
Expand Down Expand Up @@ -108,7 +108,7 @@ from transformers import (
Trainer,
)

raw_datasets = load_dataset("glue", "mnli")
raw_datasets = load_dataset("nyu-mll/glue", "mnli")

model_checkpoint = "distilbert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
Expand Down Expand Up @@ -300,7 +300,7 @@ from transformers import (
Trainer,
)

raw_datasets = load_dataset("glue", "mnli")
raw_datasets = load_dataset("nyu-mll/glue", "mnli")

model_checkpoint = "distilbert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
Expand Down Expand Up @@ -426,7 +426,7 @@ from transformers import (
Trainer,
)

raw_datasets = load_dataset("glue", "mnli")
raw_datasets = load_dataset("nyu-mll/glue", "mnli")

model_checkpoint = "distilbert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
Expand Down Expand Up @@ -631,7 +631,7 @@ from transformers import (
Trainer,
)

raw_datasets = load_dataset("glue", "mnli")
raw_datasets = load_dataset("nyu-mll/glue", "mnli")

model_checkpoint = "distilbert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
Expand Down
2 changes: 1 addition & 1 deletion chapters/en/chapter8/4_tf.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ from transformers import (
TFAutoModelForSequenceClassification,
)

raw_datasets = load_dataset("glue", "mnli")
raw_datasets = load_dataset("nyu-mll/glue", "mnli")

model_checkpoint = "distilbert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
Expand Down
2 changes: 1 addition & 1 deletion chapters/es/chapter3/2.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ La librería 🤗 Datasets provee un comando muy simple para descargar y memoriz
```py
from datasets import load_dataset

raw_datasets = load_dataset("glue", "mrpc")
raw_datasets = load_dataset("nyu-mll/glue", "mrpc")
raw_datasets
```

Expand Down
2 changes: 1 addition & 1 deletion chapters/es/chapter3/3.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Los siguientes ejemplos de código suponen que ya has ejecutado los ejemplos de
from datasets import load_dataset
from transformers import AutoTokenizer, DataCollatorWithPadding

raw_datasets = load_dataset("glue", "mrpc")
raw_datasets = load_dataset("nyu-mll/glue", "mrpc")
checkpoint = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)

Expand Down
2 changes: 1 addition & 1 deletion chapters/es/chapter3/3_tf.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ from datasets import load_dataset
from transformers import AutoTokenizer, DataCollatorWithPadding
import numpy as np

raw_datasets = load_dataset("glue", "mrpc")
raw_datasets = load_dataset("nyu-mll/glue", "mrpc")
checkpoint = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)

Expand Down
2 changes: 1 addition & 1 deletion chapters/es/chapter3/4.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Ahora veremos como obtener los mismos resultados de la última sección sin hace
from datasets import load_dataset
from transformers import AutoTokenizer, DataCollatorWithPadding

raw_datasets = load_dataset("glue", "mrpc")
raw_datasets = load_dataset("nyu-mll/glue", "mrpc")
checkpoint = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)

Expand Down
2 changes: 1 addition & 1 deletion chapters/es/chapter5/6.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ df["comments"][0].tolist()
```

```python out
['the bug code locate in ：\r\n if data_args.task_name is not None:\r\n # Downloading and loading a dataset from the hub.\r\n datasets = load_dataset("glue", data_args.task_name, cache_dir=model_args.cache_dir)',
['the bug code locate in ：\r\n if data_args.task_name is not None:\r\n # Downloading and loading a dataset from the hub.\r\n datasets = load_dataset("nyu-mll/glue", data_args.task_name, cache_dir=model_args.cache_dir)',
'Hi @jinec,\r\n\r\nFrom time to time we get this kind of `ConnectionError` coming from the github.com website: https://raw.githubusercontent.com\r\n\r\nNormally, it should work if you wait a little and then retry.\r\n\r\nCould you please confirm if the problem persists?',
'cannot connect，even by Web browser，please check that there is some problems。',
'I can access https://raw.githubusercontent.com/huggingface/datasets/1.7.0/datasets/glue/glue.py without problem...']
Expand Down
4 changes: 2 additions & 2 deletions chapters/es/chapter5/8.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Antes de seguir, probemos lo que aprendiste en este capítulo:
```py
from datasets import load_dataset

dataset = load_dataset("glue", "mrpc", split="train")
dataset = load_dataset("nyu-mll/glue", "mrpc", split="train")
```

¿Cuál de los siguientes comandos a a producir una muestra aleatoria de 50 elementos de `dataset`?
Expand Down Expand Up @@ -128,7 +128,7 @@ dataset = load_dataset("glue", "mrpc", split="train")
```py
from datasets import load_dataset

dataset = load_dataset("allocine", streaming=True, split="train")
dataset = load_dataset("tblard/allocine", streaming=True, split="train")
dataset[0]
```

Expand Down
2 changes: 1 addition & 1 deletion chapters/es/chapter6/2.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ La librería [🤗 Datasets](https://github.com/huggingface/datasets) nos puede
from datasets import load_dataset

# Esto puede tomar varios minutos para cargarse, así que ¡Agarra un té o un café mientras esperas!
raw_datasets = load_dataset("code_search_net", "python")
raw_datasets = load_dataset("code-search-net/code_search_net", "python")
```
Podemos echar un vistazo a la porción de entrenamiento para ver a qué columnas tenemos acceso:

Expand Down
2 changes: 1 addition & 1 deletion chapters/es/chapter6/8.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ Para entrenar nuestro nuevo tokenizador, usaremos un pequeño corpus de texto (p
```python
from datasets import load_dataset

dataset = load_dataset("wikitext", name="wikitext-2-raw-v1", split="train")
dataset = load_dataset("Salesforce/wikitext", name="wikitext-2-raw-v1", split="train")


def get_training_corpus():
Expand Down
2 changes: 1 addition & 1 deletion chapters/fa/chapter3/2.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ model.train_on_batch(batch, labels)
```py
from datasets import load_dataset

raw_datasets = load_dataset("glue", "mrpc")
raw_datasets = load_dataset("nyu-mll/glue", "mrpc")
raw_datasets
```

Expand Down
2 changes: 1 addition & 1 deletion chapters/fa/chapter3/3.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
from datasets import load_dataset
from transformers import AutoTokenizer, DataCollatorWithPadding

raw_datasets = load_dataset("glue", "mrpc")
raw_datasets = load_dataset("nyu-mll/glue", "mrpc")
checkpoint = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)

Expand Down
2 changes: 1 addition & 1 deletion chapters/fa/chapter3/3_tf.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ from datasets import load_dataset
from transformers import AutoTokenizer, DataCollatorWithPadding
import numpy as np

raw_datasets = load_dataset("glue", "mrpc")
raw_datasets = load_dataset("nyu-mll/glue", "mrpc")
checkpoint = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)

Expand Down
2 changes: 1 addition & 1 deletion chapters/fr/chapter3/2.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ La bibliothèque 🤗 *Datasets* propose une commande très simple pour téléch
```py
from datasets import load_dataset

raw_datasets = load_dataset("glue", "mrpc")
raw_datasets = load_dataset("nyu-mll/glue", "mrpc")
raw_datasets
```

Expand Down
2 changes: 1 addition & 1 deletion chapters/fr/chapter3/3.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Les exemples de code ci-dessous supposent que vous avez déjà exécuté les exe
from datasets import load_dataset
from transformers import AutoTokenizer, DataCollatorWithPadding

raw_datasets = load_dataset("glue", "mrpc")
raw_datasets = load_dataset("nyu-mll/glue", "mrpc")
checkpoint = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)

Expand Down
2 changes: 1 addition & 1 deletion chapters/fr/chapter3/3_tf.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ from datasets import load_dataset
from transformers import AutoTokenizer, DataCollatorWithPadding
import numpy as np

raw_datasets = load_dataset("glue", "mrpc")
raw_datasets = load_dataset("nyu-mll/glue", "mrpc")
checkpoint = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)

Expand Down
2 changes: 1 addition & 1 deletion chapters/fr/chapter3/4.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Maintenant nous allons voir comment obtenir les mêmes résultats que dans la de
from datasets import load_dataset
from transformers import AutoTokenizer, DataCollatorWithPadding

raw_datasets = load_dataset("glue", "mrpc")
raw_datasets = load_dataset("nyu-mll/glue", "mrpc")
checkpoint = "bert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)

Expand Down
2 changes: 1 addition & 1 deletion chapters/fr/chapter5/6.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ df["comments"][0].tolist()
```

```python out
['the bug code locate in ：\r\n if data_args.task_name is not None:\r\n # Downloading and loading a dataset from the hub.\r\n datasets = load_dataset("glue", data_args.task_name, cache_dir=model_args.cache_dir)',
['the bug code locate in ：\r\n if data_args.task_name is not None:\r\n # Downloading and loading a dataset from the hub.\r\n datasets = load_dataset("nyu-mll/glue", data_args.task_name, cache_dir=model_args.cache_dir)',
'Hi @jinec,\r\n\r\nFrom time to time we get this kind of `ConnectionError` coming from the github.com website: https://raw.githubusercontent.com\r\n\r\nNormally, it should work if you wait a little and then retry.\r\n\r\nCould you please confirm if the problem persists?',
'cannot connect，even by Web browser，please check that there is some problems。',
'I can access https://raw.githubusercontent.com/huggingface/datasets/1.7.0/datasets/glue/glue.py without problem...']
Expand Down
4 changes: 2 additions & 2 deletions chapters/fr/chapter5/8.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Avant de poursuivre, testons ce que vous avez appris dans ce chapitre.
```py
from datasets import load_dataset

dataset = load_dataset("glue", "mrpc", split="train")
dataset = load_dataset("nyu-mll/glue", "mrpc", split="train")
```

Laquelle des commandes suivantes produira un échantillon aléatoire de 50 éléments à partir de `dataset` ?
Expand Down Expand Up @@ -128,7 +128,7 @@ Laquelle des commandes suivantes produira un échantillon aléatoire de 50 élé
```py
from datasets import load_dataset

dataset = load_dataset("allocine", streaming=True, split="train")
dataset = load_dataset("tblard/allocine", streaming=True, split="train")
dataset[0]
```

Expand Down
2 changes: 1 addition & 1 deletion chapters/fr/chapter6/2.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ La bibliothèque [🤗 *Datasets*](https://github.com/huggingface/datasets) peut
from datasets import load_dataset

# Cela peut prendre quelques minutes alors prenez un thé ou un café pendant que vous patientez !
raw_datasets = load_dataset("code_search_net", "python")
raw_datasets = load_dataset("code-search-net/code_search_net", "python")
```

Nous pouvons jeter un coup d'œil au jeu d'entraînement pour voir quelles sont les colonnes auxquelles nous avons accès :
Expand Down
2 changes: 1 addition & 1 deletion chapters/fr/chapter6/8.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Pour entraîner notre nouveau *tokenizer*, nous utiliserons un petit corpus de t
```python
from datasets import load_dataset

dataset = load_dataset("wikitext", name="wikitext-2-raw-v1", split="train")
dataset = load_dataset("Salesforce/wikitext", name="wikitext-2-raw-v1", split="train")


def get_training_corpus():
Expand Down
2 changes: 1 addition & 1 deletion chapters/fr/chapter7/7.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Comme d'habitude, nous pouvons télécharger et mettre en cache le jeu de donné
```py
from datasets import load_dataset

raw_datasets = load_dataset("squad")
raw_datasets = load_dataset("rajpurkar/squad")
```

Nous pouvons jeter un coup d'œil à cet objet pour en savoir plus sur le jeu de données SQuAD :
Expand Down
Loading
Loading