ASSIN2 dataset checksum bug

ASSIN2 dataset checksum bug
 ## Describe the bug

Checksum error after trying to load the [ASSIN 2 dataset](https://huggingface.co/datasets/assin2).

`NonMatchingChecksumError` triggered by calling `load_dataset("assin2")`.

Similar to #3952 , #3942 , #3941 , etc.

```
---------------------------------------------------------------------------
NonMatchingChecksumError                  Traceback (most recent call last)
[<ipython-input-13-c664a92ad5e7>](https://localhost:8080/#) in <module>()
----> 1 load_dataset('assin2')

4 frames
[/usr/local/lib/python3.7/dist-packages/datasets/utils/info_utils.py](https://localhost:8080/#) in verify_checksums(expected_checksums, recorded_checksums, verification_name)
     38     if len(bad_urls) > 0:
     39         error_msg = "Checksums didn't match" + for_verification_name + ":\n"
---> 40         raise NonMatchingChecksumError(error_msg + str(bad_urls))
     41     logger.info("All the checksums matched successfully" + for_verification_name)
     42

NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://drive.google.com/u/0/uc?id=1Q9j1a83CuKzsHCGaNulSkNxBm7Dkn7Ln&export=download']
```

## Steps to reproduce the bug
```python
from datasets import load_dataset
load_dataset("assin2")
```

## Expected results
Load the dataset.

## Actual results
The dataset won't load.

## Environment info
- `datasets` version: 2.0.1.dev0
- Platform: Google Colab
- Python version:  3.7.12
- PyArrow version: 6.0.1

 Using latest code, I am still facing the issue.

```python
(base) vimos@vimosmu ➜  ~ ipython
Python 3.6.7 | packaged by conda-forge | (default, Nov  6 2019, 16:19:42)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.11.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from datasets import load_dataset

In [2]: load_dataset("assin2")
Downloading builder script: 4.24kB [00:00, 244kB/s]
Downloading metadata: 2.58kB [00:00, 2.19MB/s]
Using custom data configuration default
Downloading and preparing dataset assin2/default (download: 2.02 MiB, generated: 1.21 MiB, post-processed: Unknown size, total: 3.23 MiB) to /home/vimos/.cache/huggingface/datasets/assin2/default/1.0.0/8467f7acbda82f62ab960ca869dc1e96350e0e103a1ef7eaa43bbee530b80061...
Downloading data: 1.51MB [00:00, 102MB/s]
Downloading data: 116kB [00:00, 63.6MB/s]
Downloading data: 493kB [00:00, 95.8MB/s]
Downloading data files: 100%|██████████████████████████████████████████| 3/3 [00:00<00:00,  8.27it/s]
---------------------------------------------------------------------------
ExpectedMoreDownloadedFiles               Traceback (most recent call last)
<ipython-input-2-b367d1ffd68e> in <module>
----> 1 load_dataset("assin2")

~/anaconda3/lib/python3.6/site-packages/datasets/load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, keep_in_memory, save_infos, revision, use_auth_token, task, streaming, **config_kwargs)
   1694         ignore_verifications=ignore_verifications,
   1695         try_from_hf_gcs=try_from_hf_gcs,
-> 1696         use_auth_token=use_auth_token,
   1697     )
   1698

~/anaconda3/lib/python3.6/site-packages/datasets/builder.py in download_and_prepare(self, download_config, download_mode, ignore_verifications, try_from_hf_gcs, dl_manager, base_path, use_auth_token, **download_and_prepare_kwargs)
    604                     if not downloaded_from_gcs:
    605                         self._download_and_prepare(
--> 606                             dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
    607                         )
    608                     # Sync info

~/anaconda3/lib/python3.6/site-packages/datasets/builder.py in _download_and_prepare(self, dl_manager, verify_infos)
   1102
   1103     def _download_and_prepare(self, dl_manager, verify_infos):
-> 1104         super()._download_and_prepare(dl_manager, verify_infos, check_duplicate_keys=verify_infos)
   1105
   1106     def _get_examples_iterable_for_split(self, split_generator: SplitGenerator) -> ExamplesIterable:

~/anaconda3/lib/python3.6/site-packages/datasets/builder.py in _download_and_prepare(self, dl_manager, verify_infos, **prepare_split_kwargs)
    675         if verify_infos:
    676             verify_checksums(
--> 677                 self.info.download_checksums, dl_manager.get_recorded_sizes_checksums(), "dataset source files"
    678             )
    679

~/anaconda3/lib/python3.6/site-packages/datasets/utils/info_utils.py in verify_checksums(expected_checksums, recorded_checksums, verification_name)
     31         return
     32     if len(set(expected_checksums) - set(recorded_checksums)) > 0:
---> 33         raise ExpectedMoreDownloadedFiles(str(set(expected_checksums) - set(recorded_checksums)))
     34     if len(set(recorded_checksums) - set(expected_checksums)) > 0:
     35         raise UnexpectedDownloadedFile(str(set(recorded_checksums) - set(expected_checksums)))

ExpectedMoreDownloadedFiles: {'https://drive.google.com/u/0/uc?id=1kb7xq6Mb3eaqe9cOAo70BaG9ypwkIqEU&export=download', 'https://drive.google.com/u/0/uc?id=1J3FpQaHxpM-FDfBUyooh-sZF-B-bM_lU&export=download', 'https://drive.google.com/u/0/uc?id=1Q9j1a83CuKzsHCGaNulSkNxBm7Dkn7Ln&export=download'}
```


ASSIN2 dataset checksum bug

ASSIN2 dataset checksum bug
 ## Describe the bug

Checksum error after trying to load the [ASSIN 2 dataset](https://huggingface.co/datasets/assin2).

`NonMatchingChecksumError` triggered by calling `load_dataset("assin2")`.

Similar to #3952 , #3942 , #3941 , etc.

```
---------------------------------------------------------------------------
NonMatchingChecksumError                  Traceback (most recent call last)
[<ipython-input-13-c664a92ad5e7>](https://localhost:8080/#) in <module>()
----> 1 load_dataset('assin2')

4 frames
[/usr/local/lib/python3.7/dist-packages/datasets/utils/info_utils.py](https://localhost:8080/#) in verify_checksums(expected_checksums, recorded_checksums, verification_name)
     38     if len(bad_urls) > 0:
     39         error_msg = "Checksums didn't match" + for_verification_name + ":\n"
---> 40         raise NonMatchingChecksumError(error_msg + str(bad_urls))
     41     logger.info("All the checksums matched successfully" + for_verification_name)
     42

NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://drive.google.com/u/0/uc?id=1Q9j1a83CuKzsHCGaNulSkNxBm7Dkn7Ln&export=download']
```

## Steps to reproduce the bug
```python
from datasets import load_dataset
load_dataset("assin2")
```

## Expected results
Load the dataset.

## Actual results
The dataset won't load.

## Environment info
- `datasets` version: 2.0.1.dev0
- Platform: Google Colab
- Python version:  3.7.12
- PyArrow version: 6.0.1

 That's true. Steps to reproduce the bug on Google Colab:

```
git clone https://github.com/huggingface/datasets.git
cd datasets
pip install -e .
python -c "from datasets import load_dataset; print(load_dataset('assin2')['train'][0])"
```

However the dataset will load without any problems if you just install version 2.0.0:

 ```
pip install datasets
python -c "from datasets import load_dataset; print(load_dataset('assin2')['train'][0])"
```

Any thoughts @lhoestq ?


ASSIN2 dataset checksum bug

ASSIN2 dataset checksum bug
 ## Describe the bug

Checksum error after trying to load the [ASSIN 2 dataset](https://huggingface.co/datasets/assin2).

`NonMatchingChecksumError` triggered by calling `load_dataset("assin2")`.

Similar to #3952 , #3942 , #3941 , etc.

```
---------------------------------------------------------------------------
NonMatchingChecksumError                  Traceback (most recent call last)
[<ipython-input-13-c664a92ad5e7>](https://localhost:8080/#) in <module>()
----> 1 load_dataset('assin2')

4 frames
[/usr/local/lib/python3.7/dist-packages/datasets/utils/info_utils.py](https://localhost:8080/#) in verify_checksums(expected_checksums, recorded_checksums, verification_name)
     38     if len(bad_urls) > 0:
     39         error_msg = "Checksums didn't match" + for_verification_name + ":\n"
---> 40         raise NonMatchingChecksumError(error_msg + str(bad_urls))
     41     logger.info("All the checksums matched successfully" + for_verification_name)
     42

NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://drive.google.com/u/0/uc?id=1Q9j1a83CuKzsHCGaNulSkNxBm7Dkn7Ln&export=download']
```

## Steps to reproduce the bug
```python
from datasets import load_dataset
load_dataset("assin2")
```

## Expected results
Load the dataset.

## Actual results
The dataset won't load.

## Environment info
- `datasets` version: 2.0.1.dev0
- Platform: Google Colab
- Python version:  3.7.12
- PyArrow version: 6.0.1

 Right indeed ! Let me open a PR to fix this.
The dataset_infos.json file that stores some metadata about the dataset to download (and is used to verify it was correctly downloaded) hasn't been updated correctly


ASSIN2 dataset checksum bug

ASSIN2 dataset checksum bug
 ## Describe the bug

Checksum error after trying to load the [ASSIN 2 dataset](https://huggingface.co/datasets/assin2).

`NonMatchingChecksumError` triggered by calling `load_dataset("assin2")`.

Similar to #3952 , #3942 , #3941 , etc.

```
---------------------------------------------------------------------------
NonMatchingChecksumError                  Traceback (most recent call last)
[<ipython-input-13-c664a92ad5e7>](https://localhost:8080/#) in <module>()
----> 1 load_dataset('assin2')

4 frames
[/usr/local/lib/python3.7/dist-packages/datasets/utils/info_utils.py](https://localhost:8080/#) in verify_checksums(expected_checksums, recorded_checksums, verification_name)
     38     if len(bad_urls) > 0:
     39         error_msg = "Checksums didn't match" + for_verification_name + ":\n"
---> 40         raise NonMatchingChecksumError(error_msg + str(bad_urls))
     41     logger.info("All the checksums matched successfully" + for_verification_name)
     42

NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://drive.google.com/u/0/uc?id=1Q9j1a83CuKzsHCGaNulSkNxBm7Dkn7Ln&export=download']
```

## Steps to reproduce the bug
```python
from datasets import load_dataset
load_dataset("assin2")
```

## Expected results
Load the dataset.

## Actual results
The dataset won't load.

## Environment info
- `datasets` version: 2.0.1.dev0
- Platform: Google Colab
- Python version:  3.7.12
- PyArrow version: 6.0.1

 Not sure what the status of this is, but personally I am still getting this error, with glue.


How to use generate this multitask dataset for SQUAD? I am getting a value error.
 ## Dataset viewer issue for 'squad_multitask*'

**Link:** https://huggingface.co/datasets/vershasaxena91/squad_multitask

*short description of the issue*

I am trying to generate the multitask dataset for squad dataset. However, gives the error in dataset explorer as well as my local machine.
I tried the command: dataset = load_dataset("vershasaxena91/squad_multitask", 'highlight_qg_format')
Error:
Status code:   400
Exception:     TypeError
Message:       argument of type 'Value' is not iterable

Kindly advice.

 Hi! Replacing `nlp.<obj>` with `datasets.<obj>` in the script should fix the problem. `nlp` has been renamed to `datasets` more than a year ago, so please use `datasets` instead to avoid weird issues.


How to use generate this multitask dataset for SQUAD? I am getting a value error.
 ## Dataset viewer issue for 'squad_multitask*'

**Link:** https://huggingface.co/datasets/vershasaxena91/squad_multitask

*short description of the issue*

I am trying to generate the multitask dataset for squad dataset. However, gives the error in dataset explorer as well as my local machine.
I tried the command: dataset = load_dataset("vershasaxena91/squad_multitask", 'highlight_qg_format')
Error:
Status code:   400
Exception:     TypeError
Message:       argument of type 'Value' is not iterable

Kindly advice.

 But I request you to please fix the same in the dataset hub explorer as well...


load_dataset error: sndfile library not found
 ## Describe the bug
Can't load ami dataset

## Steps to reproduce the bug
```
python3 -c "from datasets import load_dataset; print(load_dataset('ami', 'headset-single', split='validation')[0])"
```

## Expected results

## Actual results
Downloading and preparing dataset ami/headset-single (download: 10.71 GiB, generated: 49.99 MiB, post-processed: Unknown size, total: 10.76 GiB) to /home/neo/.cache/huggingface/datasets/ami/headset-single/1.6.2/2accdf810f7c0585f78f4bcfa47684fbb980e35d29ecf126e6906dbecb872d9e...
AMI corpus cannot be downloaded using multi-processing. Setting number of downloaded processes `num_proc` to 1.
100%|██████████████████████████████████████████████████████| 136/136 [00:00<00:00, 36004.88it/s]
100%|█████████████████████████████████████████████████████████| 136/136 [00:01<00:00, 79.10it/s]
100%|████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 25343.23it/s]
100%|█████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 2874.78it/s]
100%|████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 27950.38it/s]
100%|█████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 2892.25it/s]
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/load.py", line 1707, in load_dataset
    use_auth_token=use_auth_token,
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/builder.py", line 595, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/builder.py", line 690, in _download_and_prepare
    ) from None
OSError: Cannot find data file.
Original error:
sndfile library not found

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.18.3
- Platform: Linux-4.19.0-18-cloud-amd64-x86_64-with-debian-10.11
- Python version: 3.7.3
- PyArrow version: 7.0.0

 Hi @i-am-neo,

The audio support is an extra feature of `datasets` and therefore it must be installed as an additional optional dependency:
```shell
pip install datasets[audio]
```
Additionally, for specific MP3 support (which is not the case for AMI dataset, that contains WAV audio files), there is another third-party dependency on `torchaudio`.

You have all the information in our docs: https://huggingface.co/docs/datasets/audio_process#installation


load_dataset error: sndfile library not found
 ## Describe the bug
Can't load ami dataset

## Steps to reproduce the bug
```
python3 -c "from datasets import load_dataset; print(load_dataset('ami', 'headset-single', split='validation')[0])"
```

## Expected results

## Actual results
Downloading and preparing dataset ami/headset-single (download: 10.71 GiB, generated: 49.99 MiB, post-processed: Unknown size, total: 10.76 GiB) to /home/neo/.cache/huggingface/datasets/ami/headset-single/1.6.2/2accdf810f7c0585f78f4bcfa47684fbb980e35d29ecf126e6906dbecb872d9e...
AMI corpus cannot be downloaded using multi-processing. Setting number of downloaded processes `num_proc` to 1.
100%|██████████████████████████████████████████████████████| 136/136 [00:00<00:00, 36004.88it/s]
100%|█████████████████████████████████████████████████████████| 136/136 [00:01<00:00, 79.10it/s]
100%|████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 25343.23it/s]
100%|█████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 2874.78it/s]
100%|████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 27950.38it/s]
100%|█████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 2892.25it/s]
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/load.py", line 1707, in load_dataset
    use_auth_token=use_auth_token,
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/builder.py", line 595, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/builder.py", line 690, in _download_and_prepare
    ) from None
OSError: Cannot find data file.
Original error:
sndfile library not found

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.18.3
- Platform: Linux-4.19.0-18-cloud-amd64-x86_64-with-debian-10.11
- Python version: 3.7.3
- PyArrow version: 7.0.0

 Thanks @albertvillanova . Unfortunately the error persists after installing ```datasets[audio]```. Can you direct towards a solution?

```
pip3 install datasets[audio]
```
### log
Requirement already satisfied: datasets[audio] in ./.virtualenvs/hubert/lib/python3.7/site-packages (1.18.3)
Requirement already satisfied: numpy>=1.17 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from datasets[audio]) (1.21.5)
Requirement already satisfied: xxhash in ./.virtualenvs/hubert/lib/python3.7/site-packages (from datasets[audio]) (3.0.0)
Requirement already satisfied: fsspec[http]>=2021.05.0 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from datasets[audio]) (2022.2.0)
Requirement already satisfied: dill in ./.virtualenvs/hubert/lib/python3.7/site-packages (from datasets[audio]) (0.3.4)
Requirement already satisfied: pandas in ./.virtualenvs/hubert/lib/python3.7/site-packages (from datasets[audio]) (1.3.5)
Requirement already satisfied: huggingface-hub<1.0.0,>=0.1.0 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from datasets[audio]) (0.4.0)
Requirement already satisfied: packaging in ./.virtualenvs/hubert/lib/python3.7/site-packages (from datasets[audio]) (21.3)
Requirement already satisfied: multiprocess in ./.virtualenvs/hubert/lib/python3.7/site-packages (from datasets[audio]) (0.70.12.2)
Requirement already satisfied: pyarrow!=4.0.0,>=3.0.0 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from datasets[audio]) (7.0.0)
Requirement already satisfied: tqdm>=4.62.1 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from datasets[audio]) (4.63.1)
Requirement already satisfied: aiohttp in ./.virtualenvs/hubert/lib/python3.7/site-packages (from datasets[audio]) (3.8.1)
Requirement already satisfied: importlib-metadata in ./.virtualenvs/hubert/lib/python3.7/site-packages (from datasets[audio]) (4.11.3)
Requirement already satisfied: requests>=2.19.0 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from datasets[audio]) (2.27.1)
Requirement already satisfied: librosa in ./.virtualenvs/hubert/lib/python3.7/site-packages (from datasets[audio]) (0.9.1)
Requirement already satisfied: pyyaml in ./.virtualenvs/hubert/lib/python3.7/site-packages (from huggingface-hub<1.0.0,>=0.1.0->datasets[audio]) (6.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from huggingface-hub<1.0.0,>=0.1.0->datasets[audio]) (4.1.1)
Requirement already satisfied: filelock in ./.virtualenvs/hubert/lib/python3.7/site-packages (from huggingface-hub<1.0.0,>=0.1.0->datasets[audio]) (3.6.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from packaging->datasets[audio]) (3.0.7)
Requirement already satisfied: idna<4,>=2.5 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from requests>=2.19.0->datasets[audio]) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from requests>=2.19.0->datasets[audio]) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from requests>=2.19.0->datasets[audio]) (2.0.12)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from requests>=2.19.0->datasets[audio]) (1.26.9)
Requirement already satisfied: attrs>=17.3.0 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from aiohttp->datasets[audio]) (21.4.0)
Requirement already satisfied: frozenlist>=1.1.1 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from aiohttp->datasets[audio]) (1.3.0)
Requirement already satisfied: aiosignal>=1.1.2 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from aiohttp->datasets[audio]) (1.2.0)
Requirement already satisfied: yarl<2.0,>=1.0 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from aiohttp->datasets[audio]) (1.7.2)
Requirement already satisfied: asynctest==0.13.0 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from aiohttp->datasets[audio]) (0.13.0)
Requirement already satisfied: multidict<7.0,>=4.5 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from aiohttp->datasets[audio]) (6.0.2)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from aiohttp->datasets[audio]) (4.0.2)
Requirement already satisfied: zipp>=0.5 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from importlib-metadata->datasets[audio]) (3.7.0)
Requirement already satisfied: decorator>=4.0.10 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from librosa->datasets[audio]) (5.1.1)
Requirement already satisfied: soundfile>=0.10.2 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from librosa->datasets[audio]) (0.10.3.post1)
Requirement already satisfied: numba>=0.45.1 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from librosa->datasets[audio]) (0.55.1)
Requirement already satisfied: pooch>=1.0 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from librosa->datasets[audio]) (1.6.0)
Requirement already satisfied: resampy>=0.2.2 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from librosa->datasets[audio]) (0.2.2)
Requirement already satisfied: audioread>=2.1.5 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from librosa->datasets[audio]) (2.1.9)
Requirement already satisfied: joblib>=0.14 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from librosa->datasets[audio]) (1.1.0)
Requirement already satisfied: scipy>=1.2.0 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from librosa->datasets[audio]) (1.7.3)
Requirement already satisfied: scikit-learn>=0.19.1 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from librosa->datasets[audio]) (1.0.2)
Requirement already satisfied: python-dateutil>=2.7.3 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from pandas->datasets[audio]) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from pandas->datasets[audio]) (2022.1)
Requirement already satisfied: setuptools in ./.virtualenvs/hubert/lib/python3.7/site-packages (from numba>=0.45.1->librosa->datasets[audio]) (60.10.0)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from numba>=0.45.1->librosa->datasets[audio]) (0.38.0)
Requirement already satisfied: appdirs>=1.3.0 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from pooch>=1.0->librosa->datasets[audio]) (1.4.4)
Requirement already satisfied: six>=1.5 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from python-dateutil>=2.7.3->pandas->datasets[audio]) (1.16.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from scikit-learn>=0.19.1->librosa->datasets[audio]) (3.1.0)
Requirement already satisfied: cffi>=1.0 in ./.virtualenvs/hubert/lib/python3.7/site-packages (from soundfile>=0.10.2->librosa->datasets[audio]) (1.15.0)
Requirement already satisfied: pycparser in ./.virtualenvs/hubert/lib/python3.7/site-packages (from cffi>=1.0->soundfile>=0.10.2->librosa->datasets[audio]) (2.21)

### reload
```
python3 -c "from datasets import load_dataset; print(load_dataset('ami', 'headset-single', split='validation')[0])"
```

### log
Downloading and preparing dataset ami/headset-single (download: 10.71 GiB, generated: 49.99 MiB, post-processed: Unknown size, total: 10.76 GiB) to /home/neo/.cache/huggingface/datasets/ami/headset-single/1.6.2/2accdf810f7c0585f78f4bcfa47684fbb980e35d29ecf126e6906dbecb872d9e...
AMI corpus cannot be downloaded using multi-processing. Setting number of downloaded processes `num_proc` to 1.
100%|██████████████████████████████████████████████████████| 136/136 [00:00<00:00, 33542.59it/s]
100%|█████████████████████████████████████████████████████████| 136/136 [00:06<00:00, 22.28it/s]
100%|████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 21558.39it/s]
100%|█████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 2996.41it/s]
100%|████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 23431.87it/s]
100%|█████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 2697.52it/s]
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/load.py", line 1707, in load_dataset
    use_auth_token=use_auth_token,
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/builder.py", line 595, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/builder.py", line 690, in _download_and_prepare
    ) from None
OSError: Cannot find data file.
Original error:
sndfile library not found

### just to double-check as per your docs
```
pip3 install librosa torchaudio
```

### logs
Requirement already satisfied: librosa in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (0.9.1)
Requirement already satisfied: torchaudio in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (0.11.0+cu113)
Requirement already satisfied: audioread>=2.1.5 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from librosa) (2.1.9)
Requirement already satisfied: joblib>=0.14 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from librosa) (1.1.0)
Requirement already satisfied: packaging>=20.0 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from librosa) (21.3)
Requirement already satisfied: scikit-learn>=0.19.1 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from librosa) (1.0.2)
Requirement already satisfied: scipy>=1.2.0 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from librosa) (1.7.3)
Requirement already satisfied: decorator>=4.0.10 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from librosa) (5.1.1)
Requirement already satisfied: resampy>=0.2.2 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from librosa) (0.2.2)
Requirement already satisfied: pooch>=1.0 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from librosa) (1.6.0)
Requirement already satisfied: numpy>=1.17.0 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from librosa) (1.21.5)
Requirement already satisfied: soundfile>=0.10.2 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from librosa) (0.10.3.post1)
Requirement already satisfied: numba>=0.45.1 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from librosa) (0.55.1)
Requirement already satisfied: torch==1.11.0 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from torchaudio) (1.11.0+cu113)
Requirement already satisfied: typing-extensions in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from torch==1.11.0->torchaudio) (4.1.1)
Requirement already satisfied: setuptools in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from numba>=0.45.1->librosa) (60.10.0)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from numba>=0.45.1->librosa) (0.38.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from packaging>=20.0->librosa) (3.0.7)
Requirement already satisfied: requests>=2.19.0 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from pooch>=1.0->librosa) (2.27.1)
Requirement already satisfied: appdirs>=1.3.0 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from pooch>=1.0->librosa) (1.4.4)
Requirement already satisfied: six>=1.3 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from resampy>=0.2.2->librosa) (1.16.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from scikit-learn>=0.19.1->librosa) (3.1.0)
Requirement already satisfied: cffi>=1.0 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from soundfile>=0.10.2->librosa) (1.15.0)
Requirement already satisfied: pycparser in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from cffi>=1.0->soundfile>=0.10.2->librosa) (2.21)
Requirement already satisfied: charset-normalizer~=2.0.0 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from requests>=2.19.0->pooch>=1.0->librosa) (2.0.12)
Requirement already satisfied: certifi>=2017.4.17 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from requests>=2.19.0->pooch>=1.0->librosa) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from requests>=2.19.0->pooch>=1.0->librosa) (1.26.9)
Requirement already satisfied: idna<4,>=2.5 in /home/neo/.virtualenvs/hubert/lib/python3.7/site-packages (from requests>=2.19.0->pooch>=1.0->librosa) (3.3)

### try loading again
```
python3 -c "from datasets import load_dataset; print(load_dataset('ami', 'headset-single', split='validation')[0])"
```

### same error
Downloading and preparing dataset ami/headset-single (download: 10.71 GiB, generated: 49.99 MiB, post-processed: Unknown size, total: 10.76 GiB) to /home/neo/.cache/huggingface/datasets/ami/headset-single/1.6.2/2accdf810f7c0585f78f4bcfa47684fbb980e35d29ecf126e6906dbecb872d9e...
AMI corpus cannot be downloaded using multi-processing. Setting number of downloaded processes `num_proc` to 1.
100%|██████████████████████████████████████████████████████| 136/136 [00:00<00:00, 33542.59it/s]
100%|█████████████████████████████████████████████████████████| 136/136 [00:06<00:00, 22.28it/s]
100%|████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 21558.39it/s]
100%|█████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 2996.41it/s]
100%|████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 23431.87it/s]
100%|█████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 2697.52it/s]
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/load.py", line 1707, in load_dataset
    use_auth_token=use_auth_token,
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/builder.py", line 595, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/builder.py", line 690, in _download_and_prepare
    ) from None
OSError: Cannot find data file.
Original error:
sndfile library not found


load_dataset error: sndfile library not found
 ## Describe the bug
Can't load ami dataset

## Steps to reproduce the bug
```
python3 -c "from datasets import load_dataset; print(load_dataset('ami', 'headset-single', split='validation')[0])"
```

## Expected results

## Actual results
Downloading and preparing dataset ami/headset-single (download: 10.71 GiB, generated: 49.99 MiB, post-processed: Unknown size, total: 10.76 GiB) to /home/neo/.cache/huggingface/datasets/ami/headset-single/1.6.2/2accdf810f7c0585f78f4bcfa47684fbb980e35d29ecf126e6906dbecb872d9e...
AMI corpus cannot be downloaded using multi-processing. Setting number of downloaded processes `num_proc` to 1.
100%|██████████████████████████████████████████████████████| 136/136 [00:00<00:00, 36004.88it/s]
100%|█████████████████████████████████████████████████████████| 136/136 [00:01<00:00, 79.10it/s]
100%|████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 25343.23it/s]
100%|█████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 2874.78it/s]
100%|████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 27950.38it/s]
100%|█████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 2892.25it/s]
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/load.py", line 1707, in load_dataset
    use_auth_token=use_auth_token,
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/builder.py", line 595, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/builder.py", line 690, in _download_and_prepare
    ) from None
OSError: Cannot find data file.
Original error:
sndfile library not found

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.18.3
- Platform: Linux-4.19.0-18-cloud-amd64-x86_64-with-debian-10.11
- Python version: 3.7.3
- PyArrow version: 7.0.0

 Hi @i-am-neo, thanks again for your detailed report.

Our `datasets` library support for audio relies on a third-party Python library called `librosa`, which is installed when you do:
```shell
pip install datasets[audio]
```

However, the `librosa` library has a dependency on `soundfile`; and `soundfile` depends on a non-Python package called `sndfile`.

On Linux (which is your case), this must be installed manually using your operating system package manager, for example:
```shell
sudo apt-get install libsndfile1
```

Please, let me know if this works and if so, I will update our docs with all this information.


load_dataset error: sndfile library not found
 ## Describe the bug
Can't load ami dataset

## Steps to reproduce the bug
```
python3 -c "from datasets import load_dataset; print(load_dataset('ami', 'headset-single', split='validation')[0])"
```

## Expected results

## Actual results
Downloading and preparing dataset ami/headset-single (download: 10.71 GiB, generated: 49.99 MiB, post-processed: Unknown size, total: 10.76 GiB) to /home/neo/.cache/huggingface/datasets/ami/headset-single/1.6.2/2accdf810f7c0585f78f4bcfa47684fbb980e35d29ecf126e6906dbecb872d9e...
AMI corpus cannot be downloaded using multi-processing. Setting number of downloaded processes `num_proc` to 1.
100%|██████████████████████████████████████████████████████| 136/136 [00:00<00:00, 36004.88it/s]
100%|█████████████████████████████████████████████████████████| 136/136 [00:01<00:00, 79.10it/s]
100%|████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 25343.23it/s]
100%|█████████████████████████████████████████████████████████| 18/18 [00:00<00:00, 2874.78it/s]
100%|████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 27950.38it/s]
100%|█████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 2892.25it/s]
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/load.py", line 1707, in load_dataset
    use_auth_token=use_auth_token,
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/builder.py", line 595, in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
  File "/home/neo/.virtualenvs/hubert/lib/python3.7/site-packages/datasets/builder.py", line 690, in _download_and_prepare
    ) from None
OSError: Cannot find data file.
Original error:
sndfile library not found

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.18.3
- Platform: Linux-4.19.0-18-cloud-amd64-x86_64-with-debian-10.11
- Python version: 3.7.3
- PyArrow version: 7.0.0

 @albertvillanova thanks, all good.  The key is ```libsndfile1``` - it may help others to note that in your docs.  I had installed libsndfile previously.


Audio.encode_example() throws an error when writing example from array
 ## Describe the bug
When trying to do `Audio().encode_example()` with preexisting array (see [this line](https://github.com/huggingface/datasets/blob/master/src/datasets/features/audio.py#L73)), `sf.write()` throws you an error:
`TypeError: No format specified and unable to get format from file extension: <_io.BytesIO object at 0x7f4218c0db30>`

## Steps to reproduce the bug
### Sample code to reproduce the bug
```python
# download sample file
!wget https://huggingface.co/datasets/polinaeterna/test_encode_example/resolve/main/common_voice_vi_21824030.mp3

arr, sr = librosa.load("common_voice_vi_21824030.mp3")

Audio().encode_example({
    "path": "common_voice_vi_21824030.mp3",
    "array": arr,
    "sampling_rate":sr
})
```

## Expected results
An encoded example (`{"bytes": b'....', "path": 'path'}`)

## Actual results
```python
TypeError                                 Traceback (most recent call last)
Input In [3], in <module>
      1 arr, sr = librosa.load("common_voice_vi_21824030.mp3")
----> 3 Audio().encode_example({
      4     "path": "common_voice_vi_21824030.mp3",
      5     "array": arr,
      6     "sampling_rate":sr
      7 })

File ~/workspace/datasets/src/datasets/features/audio.py:75, in Audio.encode_example(self, value)
     73 elif isinstance(value, dict) and "array" in value:
     74     buffer = BytesIO()
---> 75     sf.write(buffer, value["array"], value["sampling_rate"])
     76     return {"bytes": buffer.getvalue(), "path": value.get("path")}
     77 elif value.get("bytes") is not None or value.get("path") is not None:

File ~/miniconda3/envs/datasets/lib/python3.8/site-packages/soundfile.py:314, in write(file, data, samplerate, subtype, endian, format, closefd)
    312 else:
    313     channels = data.shape[1]
--> 314 with SoundFile(file, 'w', samplerate, channels,
    315                subtype, endian, format, closefd) as f:
    316     f.write(data)

File ~/miniconda3/envs/datasets/lib/python3.8/site-packages/soundfile.py:627, in SoundFile.__init__(self, file, mode, samplerate, channels, subtype, endian, format, closefd)
    625 mode_int = _check_mode(mode)
    626 self._mode = mode
--> 627 self._info = _create_info_struct(file, mode, samplerate, channels,
    628                                  format, subtype, endian)
    629 self._file = self._open(file, mode_int, closefd)
    630 if set(mode).issuperset('r+') and self.seekable():
    631     # Move write position to 0 (like in Python file objects)

File ~/miniconda3/envs/datasets/lib/python3.8/site-packages/soundfile.py:1416, in _create_info_struct(file, mode, samplerate, channels, format, subtype, endian)
   1414 original_format = format
   1415 if format is None:
-> 1416     format = _get_format_from_filename(file, mode)
   1417     assert isinstance(format, (_unicode, str))
   1418 else:

File ~/miniconda3/envs/datasets/lib/python3.8/site-packages/soundfile.py:1457, in _get_format_from_filename(file, mode)
   1455     pass
   1456 if format.upper() not in _formats and 'r' not in mode:
-> 1457     raise TypeError("No format specified and unable to get format from "
   1458                     "file extension: {0!r}".format(file))
   1459 return format

TypeError: No format specified and unable to get format from file extension: <_io.BytesIO object at 0x7fd8daf88180>
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: datasets master
- Platform: Ubuntu 20.04
- Python version: python 3.8.12
- PyArrow version: 6.0.1

## Solution

I guess we just need to add `format` arg in [this line](https://github.com/huggingface/datasets/blob/master/src/datasets/features/audio.py#L75) like this:
```python
sf.write(buffer, value["array"], value["sampling_rate"], format="wav")
```

BTW discovered this when trying to decode audio in mp3 format without torchaudio (would be useful for TensorFlow users), like this:
```python
from datasets import load_dataset, Features, Audio

ds = load_dataset("common_voice", "vi", split="test")
ds = ds.remove_columns("audio")
ds.select(range(3))  # 3 samples just for testing

def load_mp3_with_librosa(example):
    arr, sr = librosa.load(example["path"])
    example["audio"] = {
        "path": example["path"],
        "array": arr,
        "sampling_rate": sr
    }
    return example

updated_dataset = ds.map(lambda example: load_mp3_with_librosa(example),
                         features=Features(
                             {"audio": Audio(decode=False)}
                         ))
```

@lhoestq @mariosasko @albertvillanova am I right in my logic? do we agree that we can set wav as the format? 🤗
 Thanks @polinaeterna for reporting this issue.

In relation to the decoding of MP3 audio files without torchaudio, I remember Patrick made some tests and these had quite bad performance. That is why he proposed to support MP3 files only with torchaudio. But yes, nice to give an alternative to non-torchaudio users (with a big warning on performance).


Audio.encode_example() throws an error when writing example from array
 ## Describe the bug
When trying to do `Audio().encode_example()` with preexisting array (see [this line](https://github.com/huggingface/datasets/blob/master/src/datasets/features/audio.py#L73)), `sf.write()` throws you an error:
`TypeError: No format specified and unable to get format from file extension: <_io.BytesIO object at 0x7f4218c0db30>`

## Steps to reproduce the bug
### Sample code to reproduce the bug
```python
# download sample file
!wget https://huggingface.co/datasets/polinaeterna/test_encode_example/resolve/main/common_voice_vi_21824030.mp3

arr, sr = librosa.load("common_voice_vi_21824030.mp3")

Audio().encode_example({
    "path": "common_voice_vi_21824030.mp3",
    "array": arr,
    "sampling_rate":sr
})
```

## Expected results
An encoded example (`{"bytes": b'....', "path": 'path'}`)

## Actual results
```python
TypeError                                 Traceback (most recent call last)
Input In [3], in <module>
      1 arr, sr = librosa.load("common_voice_vi_21824030.mp3")
----> 3 Audio().encode_example({
      4     "path": "common_voice_vi_21824030.mp3",
      5     "array": arr,
      6     "sampling_rate":sr
      7 })

File ~/workspace/datasets/src/datasets/features/audio.py:75, in Audio.encode_example(self, value)
     73 elif isinstance(value, dict) and "array" in value:
     74     buffer = BytesIO()
---> 75     sf.write(buffer, value["array"], value["sampling_rate"])
     76     return {"bytes": buffer.getvalue(), "path": value.get("path")}
     77 elif value.get("bytes") is not None or value.get("path") is not None:

File ~/miniconda3/envs/datasets/lib/python3.8/site-packages/soundfile.py:314, in write(file, data, samplerate, subtype, endian, format, closefd)
    312 else:
    313     channels = data.shape[1]
--> 314 with SoundFile(file, 'w', samplerate, channels,
    315                subtype, endian, format, closefd) as f:
    316     f.write(data)

File ~/miniconda3/envs/datasets/lib/python3.8/site-packages/soundfile.py:627, in SoundFile.__init__(self, file, mode, samplerate, channels, subtype, endian, format, closefd)
    625 mode_int = _check_mode(mode)
    626 self._mode = mode
--> 627 self._info = _create_info_struct(file, mode, samplerate, channels,
    628                                  format, subtype, endian)
    629 self._file = self._open(file, mode_int, closefd)
    630 if set(mode).issuperset('r+') and self.seekable():
    631     # Move write position to 0 (like in Python file objects)

File ~/miniconda3/envs/datasets/lib/python3.8/site-packages/soundfile.py:1416, in _create_info_struct(file, mode, samplerate, channels, format, subtype, endian)
   1414 original_format = format
   1415 if format is None:
-> 1416     format = _get_format_from_filename(file, mode)
   1417     assert isinstance(format, (_unicode, str))
   1418 else:

File ~/miniconda3/envs/datasets/lib/python3.8/site-packages/soundfile.py:1457, in _get_format_from_filename(file, mode)
   1455     pass
   1456 if format.upper() not in _formats and 'r' not in mode:
-> 1457     raise TypeError("No format specified and unable to get format from "
   1458                     "file extension: {0!r}".format(file))
   1459 return format

TypeError: No format specified and unable to get format from file extension: <_io.BytesIO object at 0x7fd8daf88180>
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: datasets master
- Platform: Ubuntu 20.04
- Python version: python 3.8.12
- PyArrow version: 6.0.1

## Solution

I guess we just need to add `format` arg in [this line](https://github.com/huggingface/datasets/blob/master/src/datasets/features/audio.py#L75) like this:
```python
sf.write(buffer, value["array"], value["sampling_rate"], format="wav")
```

BTW discovered this when trying to decode audio in mp3 format without torchaudio (would be useful for TensorFlow users), like this:
```python
from datasets import load_dataset, Features, Audio

ds = load_dataset("common_voice", "vi", split="test")
ds = ds.remove_columns("audio")
ds.select(range(3))  # 3 samples just for testing

def load_mp3_with_librosa(example):
    arr, sr = librosa.load(example["path"])
    example["audio"] = {
        "path": example["path"],
        "array": arr,
        "sampling_rate": sr
    }
    return example

updated_dataset = ds.map(lambda example: load_mp3_with_librosa(example),
                         features=Features(
                             {"audio": Audio(decode=False)}
                         ))
```

@lhoestq @mariosasko @albertvillanova am I right in my logic? do we agree that we can set wav as the format? 🤗
 >  I remember Patrick made some tests and these had quite bad performance. That is why he proposed to support MP3 files only with torchaudio.

Yeah, I know, but as far as I understand, some users just categorically don't want to have torchaudio in their environment. Anyway, it's just a more or less random example, they can use any library they like following the same logic (I'm just not a big expert in decoding utils so if you can give me some presentation / resources about that I would really appreciate it 🤗)


Streaming dataset + interleave + DataLoader hangs with multiple workers
 ## Describe the bug
Interleaving multiple iterable datasets that use `load_dataset` on streaming mode hangs when passed to `torch.utils.data.DataLoader` with multiple workers.

## Steps to reproduce the bug
```python
from datasets import interleave_datasets, load_dataset
from torch.utils.data import DataLoader

en_dataset = load_dataset('oscar', "unshuffled_deduplicated_en", split='train', streaming=True)
fr_dataset = load_dataset('oscar', "unshuffled_deduplicated_fr", split='train', streaming=True)
it_dataset = load_dataset('oscar', "unshuffled_deduplicated_it", split='train', streaming=True)
de_dataset = load_dataset('oscar', "unshuffled_deduplicated_de", split='train', streaming=True)
multilingual_dataset = interleave_datasets([en_dataset, fr_dataset, de_dataset, it_dataset])
multilingual_dataset = multilingual_dataset.with_format('torch')

next(iter(multilingual_dataset))  # works fairly fast

dataloader = DataLoader(multilingual_dataset, batch_size=8, num_workers=4)
for batch in dataloader:
    print(len(batch))  # prints nothing after 30 min of waiting

dataloader = DataLoader(multilingual_dataset, batch_size=8, num_workers=0)
for batch in dataloader:
    print(len(batch))  # prints right away

```

## Expected results
It should be able to iterate the dataset with multiple workers.

## Actual results
Prints with results with `next(iter(multilingual_dataset)) ` and `num_workers=0`  but it prints nothing with `num_workers=4` or any number above 0.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.0.1.dev0
- `pytorch` version: 1.10.0+cu113
- Python version: 3.7
- PyArrow version: 6.0.1

 Hi ! Thanks for reporting, could this be related to https://github.com/huggingface/datasets/issues/3950 ?

Currently streaming datasets only works in single process, but we're working on having in work in distributed setups as well :)


Image column is not decoded in map when using with with_transform
 ## Describe the bug
Image column is not _decoded_ in **map** when using with `with_transform`

## Steps to reproduce the bug
```python
from datasets import Image, Dataset

def add_C(batch):
    batch["C"] = batch["A"]
    return batch

ds = Dataset.from_dict({"A": ["image.png"]}).cast_column("A",  Image())
ds = ds.with_transform(lambda x: x)  # <= This line causes the problem
ds = ds.map(add_C, batched=True)
print(ds[0])
```

## Expected results
```
{'C': <PIL.PngImagePlugin.PngImageFile>,  ...}
```

## Actual results
```
{'C': {'bytes': None, 'path': 'image.png'}, ...}
```

If we remove the `with_transform` line, we get the expected result.

## Environment info
- `datasets` version: 2.0.0
- Platform: Mac OSX
- Python version: 3.8.12
- PyArrow version: 7.0.0

 Hi! This behavior stems from this line: https://github.com/huggingface/datasets/blob/799b817d97590ddc97cbd38d07469403e030de8c/src/datasets/arrow_dataset.py#L1919
Basically, the `Image`/`Audio` columns are decoded only if the `format_type` attribute is `None` (`set_format`/`with_format` and `set_transform`/`with_transform` assign a non-`None` value to it) and the  `input_columns` param is not specified (see https://github.com/huggingface/datasets/issues/3756). We will remove these limitations soon.


Improve AutomaticSpeechRecognition task template
 **Is your feature request related to a problem? Please describe.**
[AutomaticSpeechRecognition task template](https://github.com/huggingface/datasets/blob/master/src/datasets/tasks/automatic_speech_recognition.py) is outdated as it uses path to audiofile as an audio column instead of a Audio feature itself (I guess it's because Audio feature didn't exist at the time this template was created).

**Describe the solution you'd like**
Change audio columns from string path to Audio feature.

 There is an open PR to do that: #3364. I just haven't had time to finish it...


Improve AutomaticSpeechRecognition task template
 **Is your feature request related to a problem? Please describe.**
[AutomaticSpeechRecognition task template](https://github.com/huggingface/datasets/blob/master/src/datasets/tasks/automatic_speech_recognition.py) is outdated as it uses path to audiofile as an audio column instead of a Audio feature itself (I guess it's because Audio feature didn't exist at the time this template was created).

**Describe the solution you'd like**
Change audio columns from string path to Audio feature.

 > There is an open PR to do that: #3364. I just haven't had time to finish it...

😬 thanks...


Dataset loads indefinitely after modifying default cache path (~/.cache/huggingface)
 ## Describe the bug
Dataset loads indefinitely after modifying cache path (~/.cache/huggingface)
If none of the environment variables are set, this custom dataset loads fine ( json-based dataset with custom dataset load script)
** Update: Transformer modules faces the same issue as well during loading

## A clear and concise description of what the bug is.

Issue:
- Dataset loading stalls / freezes indefinitely when HF_HOME is changed to a custom directory
- No error code, had to terminate the process
- There are some files created in the cache directory:

```
custom_cache_dir
  | -- modules
            | -- __init__.py
            | -- datasets_modules
                      | -- __init__.py
                      | -- datasets
                             | -- __init__.py
                             | -- script.py (Dataset loading script)
                             | -- script.lock

```

There's no error nor any logs thrown so I'm out of ideas of how to to debug this. The custom dataset works fine if the default ~/.cache dir is used, but unfortunately it's out of space and we do not have permissions to modify the disk.

## Steps to reproduce the bug

What I've tried:
- Modifying HF_HOME  (https://github.com/huggingface/transformers/issues/8703)
- Modifying HF_DATASETS_CACHE (https://huggingface.co/docs/datasets/v1.12.0/cache.html)
- Modifying cache_dir param during runtime
```python
>>> from datasets import load_dataset
>>> dataset = load_dataset('test_dataset', cache_dir='/path/to/new/cache')
```
- Disabling dataset cache
```python
>>> from datasets import set_caching_enabled
>>> set_caching_enabled(False)
```

## Expected results
Datasets should load / cache as usual with the only exception that cache directory is different

## Actual results

Any actions taken above to change the cache directory results in loading indefinitely without terminating.

## Environment info

- `transformers` version: 4.18.0.dev0
- Platform: Linux-4.15.0-54-generic-x86_64-with-glibc2.10
- Python version: 3.8.8
- Huggingface_hub version: 0.4.0
- PyTorch version (GPU?): 1.8.1+cu102 (True)
- Tensorflow version (GPU?): 2.4.1 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

 Hi ! I didn't managed to reproduce the issue. When you kill the process, is there any stacktrace that shows at what point in the code python is hanging ?


Dataset loads indefinitely after modifying default cache path (~/.cache/huggingface)
 ## Describe the bug
Dataset loads indefinitely after modifying cache path (~/.cache/huggingface)
If none of the environment variables are set, this custom dataset loads fine ( json-based dataset with custom dataset load script)
** Update: Transformer modules faces the same issue as well during loading

## A clear and concise description of what the bug is.

Issue:
- Dataset loading stalls / freezes indefinitely when HF_HOME is changed to a custom directory
- No error code, had to terminate the process
- There are some files created in the cache directory:

```
custom_cache_dir
  | -- modules
            | -- __init__.py
            | -- datasets_modules
                      | -- __init__.py
                      | -- datasets
                             | -- __init__.py
                             | -- script.py (Dataset loading script)
                             | -- script.lock

```

There's no error nor any logs thrown so I'm out of ideas of how to to debug this. The custom dataset works fine if the default ~/.cache dir is used, but unfortunately it's out of space and we do not have permissions to modify the disk.

## Steps to reproduce the bug

What I've tried:
- Modifying HF_HOME  (https://github.com/huggingface/transformers/issues/8703)
- Modifying HF_DATASETS_CACHE (https://huggingface.co/docs/datasets/v1.12.0/cache.html)
- Modifying cache_dir param during runtime
```python
>>> from datasets import load_dataset
>>> dataset = load_dataset('test_dataset', cache_dir='/path/to/new/cache')
```
- Disabling dataset cache
```python
>>> from datasets import set_caching_enabled
>>> set_caching_enabled(False)
```

## Expected results
Datasets should load / cache as usual with the only exception that cache directory is different

## Actual results

Any actions taken above to change the cache directory results in loading indefinitely without terminating.

## Environment info

- `transformers` version: 4.18.0.dev0
- Platform: Linux-4.15.0-54-generic-x86_64-with-glibc2.10
- Python version: 3.8.8
- Huggingface_hub version: 0.4.0
- PyTorch version (GPU?): 1.8.1+cu102 (True)
- Tensorflow version (GPU?): 2.4.1 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

 Hi @lhoestq , I've traced the issue back to file locking. It's similar to this thread, using Lustre filesystem as well. https://github.com/huggingface/datasets/issues/329 . In this case the user was able to modify and add -o flock option while mounting and it solved the problem.
However in other cases such as mine, we do not have the permissions to modify the commands while mounting. I'm still trying to figure out a workaround. Any ideas how can we use a mounted Lustre filesystem with no flock option?


Local and automatic tests fail

Local and automatic tests fail
 ## Describe the bug
Running the tests from CircleCI on a PR or locally fails, even with no changes. Tests seem to fail on `test_metric_common.py`

## Steps to reproduce the bug
```shell
git clone https://huggingface/datasets.git
cd datasets
```
```python
python -m pip install -e .
pytest
```

## Expected results
All tests passing

## Actual results
```
tests/test_metric_common.py:91:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../.pyenv/versions/3.8.5/lib/python3.8/doctest.py:1336: in __run
    exec(compile(example.source, filename, "single",
<doctest datasets_modules.metrics.ter.c0cfb5adedac7eb15ffa47bba6a70fabd80f3eb906ee508abf5e1906285d1155.ter.Ter[3]>:1: in <module>
    ???
../datasets/src/datasets/metric.py:430: in compute
    output = self._compute(**inputs, **compute_kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = Metric(name: "ter", features: {'predictions': Value(dtype='string', id='sequence'), 'references': Sequence(feature=Val...ences=references)
    >>> print(results)
    {'score': 0.0, 'num_edits': 0, 'ref_length': 6.5}
""", stored examples: 0)
predictions = ['hello there general kenobi', 'foo bar foobar']
references = [['hello there general kenobi', 'hello there !'], ['foo bar foobar', 'foo bar foobar']]
normalized = False, no_punct = False, asian_support = False, case_sensitive = False

    def _compute(
        self,
        predictions,
        references,
        normalized: bool = False,
        no_punct: bool = False,
        asian_support: bool = False,
        case_sensitive: bool = False,
    ):
        references_per_prediction = len(references[0])
        if any(len(refs) != references_per_prediction for refs in references):
            raise ValueError("Sacrebleu requires the same number of references for each prediction")
        transformed_references = [[refs[i] for refs in references] for i in range(references_per_prediction)]

>       sb_ter = TER(normalized, no_punct, asian_support, case_sensitive)
E       TypeError: __init__() takes 2 positional arguments but 5 were given

/tmp/pytest-of-markussagen/pytest-1/cache/modules/datasets_modules/metrics/ter/c0cfb5adedac7eb15ffa47bba6a70fabd80f3eb906ee508abf5e1906285d1155/ter.py:130: TypeError
------------------------------ Captured stdout call -------------------------------
Trying:
    predictions = ["hello there general kenobi", "foo bar foobar"]
Expecting nothing
ok
Trying:
    references = [["hello there general kenobi", "hello there !"], ["foo bar foobar", "foo bar foobar"]]
Expecting nothing
ok
Trying:
    ter = datasets.load_metric("ter")
Expecting nothing
ok
Trying:
    results = ter.compute(predictions=predictions, references=references)
Expecting nothing
================================ warnings summary =================================
../.pyenv/versions/3.8.5/envs/huggingface/lib/python3.8/site-packages/hdfs/config.py:15
  /home/markussagen/.pyenv/versions/3.8.5/envs/huggingface/lib/python3.8/site-packages/hdfs/config.py:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    from imp import load_source

../datasets/src/datasets/commands/test.py:35
  /home/markussagen/datasets/src/datasets/commands/test.py:35: PytestCollectionWarning: cannot collect test class 'TestCommand' because it has a __init__ constructor (from: tests/commands/test_test.py)
    class TestCommand(BaseDatasetsCLICommand):

tests/commands/test_test.py:33
  /home/markussagen/mydataset/tests/commands/test_test.py:33: PytestCollectionWarning: cannot collect test class 'TestCommandArgs' because it has a __new__ constructor (from: tests/commands/test_test.py)
    class TestCommandArgs:

tests/test_arrow_dataset.py: 760 warnings
tests/test_formatting.py: 60 warnings
tests/test_search.py: 31 warnings
tests/features/test_array_xd.py: 117 warnings
  /home/markussagen/datasets/src/datasets/formatting/formatting.py:197: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    (isinstance(x, np.ndarray) and (x.dtype == np.object or x.shape != array[0].shape))

tests/test_arrow_dataset.py: 154 warnings
tests/features/test_array_xd.py: 1 warning
  /home/markussagen/datasets/src/datasets/formatting/formatting.py:201: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    return np.array(array, copy=False, **{**self.np_array_kwargs, "dtype": np.object})

tests/test_arrow_dataset.py: 60 warnings
  /home/markussagen/datasets/src/datasets/arrow_dataset.py:3105: DeprecationWarning: `np.str` is a deprecated alias for the builtin `str`. To silence this warning, use `str` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.str_` here.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    elif np.issubdtype(values.dtype, np.str):

tests/test_arrow_dataset.py: 138 warnings
tests/test_formatting.py: 21 warnings
  /home/markussagen/datasets/src/datasets/formatting/tf_formatter.py:69: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    data_struct.dtype == np.object

tests/test_arrow_dataset.py: 240 warnings
tests/test_formatting.py: 20 warnings
  /home/markussagen/datasets/src/datasets/formatting/torch_formatter.py:49: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    if data_struct.dtype == np.object:  # pytorch tensors cannot be instantied from an array of objects

tests/test_arrow_dataset.py: 12 warnings
tests/test_search.py: 2 warnings
tests/features/test_array_xd.py: 6 warnings
tests/features/test_image.py: 4 warnings
  /home/markussagen/datasets/src/datasets/features/features.py:1129: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    [0] + [len(arr) for arr in l_arr], dtype=np.object

tests/test_dataset_common.py::LocalDatasetTest::test_builder_class_banking77
  /tmp/pytest-of-markussagen/pytest-1/cache/modules/datasets_modules/datasets/banking77/aec0289529599d4572d76ab00c8944cb84f88410ad0c9e7da26189d31f62a55b/banking77.py:24: DeprecationWarning: invalid escape sequence \~
    _CITATION = """\

tests/test_dataset_common.py::LocalDatasetTest::test_builder_class_universal_dependencies
  /tmp/pytest-of-markussagen/pytest-1/cache/modules/datasets_modules/datasets/universal_dependencies/065e728dfe9a8371434a6e87132c2386a6eacab1a076d3a12aa417b994e6ef7d/universal_dependencies.py:6: DeprecationWarning: invalid escape sequence \=
    _CITATION = """\

tests/test_filesystem.py: 105 warnings
  /home/markussagen/.pyenv/versions/3.8.5/envs/huggingface/lib/python3.8/site-packages/responses/__init__.py:398: DeprecationWarning: stream argument is deprecated. Use stream parameter in request directly
    warn(

tests/test_formatting.py::FormatterTest::test_jax_formatter
tests/test_formatting.py::FormatterTest::test_jax_formatter
tests/test_formatting.py::FormatterTest::test_jax_formatter
tests/test_formatting.py::FormatterTest::test_jax_formatter
tests/test_formatting.py::FormatterTest::test_jax_formatter_np_array_kwargs
tests/test_formatting.py::FormatterTest::test_jax_formatter_np_array_kwargs
tests/test_formatting.py::FormatterTest::test_jax_formatter_np_array_kwargs
tests/test_formatting.py::FormatterTest::test_jax_formatter_np_array_kwargs
  /home/markussagen/datasets/src/datasets/formatting/jax_formatter.py:57: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    if data_struct.dtype == np.object:  # jax arrays cannot be instantied from an array of objects

tests/test_formatting.py::FormatterTest::test_jax_formatter
tests/test_formatting.py::FormatterTest::test_jax_formatter
tests/test_formatting.py::FormatterTest::test_jax_formatter
  /home/markussagen/.pyenv/versions/3.8.5/envs/huggingface/lib/python3.8/site-packages/jax/_src/numpy/lax_numpy.py:3567: UserWarning: Explicitly requested dtype <class 'jax._src.numpy.lax_numpy.int64'> requested in array is not available, and will be truncated to dtype int32. To enable more dtypes, set the jax_enable_x64 configuration option or the JAX_ENABLE_X64 shell environment variable. See https://github.com/google/jax#current-gotchas for more.
    lax._check_user_dtype_supported(dtype, "array")

tests/test_metric_common.py::LocalMetricTest::test_load_metric_frugalscore
  /home/markussagen/.pyenv/versions/3.8.5/envs/huggingface/lib/python3.8/site-packages/apscheduler/util.py:95: PytzUsageWarning: The zone attribute is specific to pytz's interface; please migrate to a new time zone provider. For more details on how to do so, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
    if obj.zone == 'local':

tests/test_upstream_hub.py::TestPushToHub::test_push_dataset_to_hub_custom_features
_audio
  /home/markussagen/.pyenv/versions/3.8.5/envs/huggingface/lib/python3.8/site-packages/librosa/core/constantq.py:1059: DeprecationWarning: `np.complex` is a deprecated alias for the builtin `complex`. To silence this warning, use `complex` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.complex128` here.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    dtype=np.complex,

tests/features/test_array_xd.py::test_array_xd_with_none
  /home/markussagen/mydataset/tests/features/test_array_xd.py:338: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    assert isinstance(arr, np.ndarray) and arr.dtype == np.object and arr.shape == (3,)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
============================= short test summary info =============================
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_bleurt - I...
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_chrf - Att...
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_code_eval
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_comet - Im...
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_competition_math
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_coval - Im...
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_frugalscore
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_perplexity
FAILED tests/test_metric_common.py::LocalMetricTest::test_load_metric_ter - Type...
```

## Environment info
- `datasets` version: 2.0.1.dev0
- Platform: Linux-5.16.11-76051611-generic-x86_64-with-glibc2.33
- Python version: 3.8.5
- PyArrow version: 5.0.0

 Hi ! To be able to run the tests, you need to install all the test dependencies and additional ones with
```
pip install -e .[tests]
pip install -r additional-tests-requirements.txt --no-deps
```

In particular, you probably need to `sacrebleu`. It looks like it wasn't able to instantiate `sacrebleu.TER` properly.


Infinitely attempting lock

Infinitely attempting lock
 I am trying to run one of the examples of the `transformers` repo, which makes use of `datasets`.
Important to note is that I am trying to run this via a Databricks notebook, and all the files reside in the Databricks Filesystem (DBFS).

```
%sh
python /dbfs/transformers/examples/pytorch/summarization/run_summarization.py \
    --model_name_or_path t5-small \
    --do_train \
    --do_eval \
    --dataset_name cnn_dailymail \
    --dataset_config "3.0.0" \
    --source_prefix "summarize: " \
    --output_dir /dbfs/transformers/tmp/tst-summarization \
    --per_device_train_batch_size=4 \
    --per_device_eval_batch_size=4 \
    --overwrite_output_dir \
    --predict_with_generate \
    --log_level debug \
    --cache_dir /dbfs/transformers/cache
```

All goes well until acquiring a lock --

```
03/21/2022 17:53:19 - DEBUG - datasets.utils.filelock - Attempting to acquire lock 140386484514192 on /dbfs/transformers/cache/_dbfs_transformers_cache_cnn_dailymail_3.0.0_3.0.0_3cb851bf7cf5826e45d49db2863f627cba583cbc32342df7349dfe6c38060234.lock
03/21/2022 17:53:19 - DEBUG - datasets.utils.filelock - Lock 140386484514192 not acquired on /dbfs/transformers/cache/_dbfs_transformers_cache_cnn_dailymail_3.0.0_3.0.0_3cb851bf7cf5826e45d49db2863f627cba583cbc32342df7349dfe6c38060234.lock, waiting 0.05 seconds ...
03/21/2022 17:53:20 - DEBUG - datasets.utils.filelock - Attempting to acquire lock 140386484514192 on /dbfs/transformers/cache/_dbfs_transformers_cache_cnn_dailymail_3.0.0_3.0.0_3cb851bf7cf5826e45d49db2863f627cba583cbc32342df7349dfe6c38060234.lock
03/21/2022 17:53:20 - DEBUG - datasets.utils.filelock - Lock 140386484514192 not acquired on /dbfs/transformers/cache/_dbfs_transformers_cache_cnn_dailymail_3.0.0_3.0.0_3cb851bf7cf5826e45d49db2863f627cba583cbc32342df7349dfe6c38060234.lock, waiting 0.05 seconds ...
03/21/2022 17:53:20 - DEBUG - datasets.utils.filelock - Attempting to acquire lock 140386484514192 on /dbfs/transformers/cache/_dbfs_transformers_cache_cnn_dailymail_3.0.0_3.0.0_3cb851bf7cf5826e45d49db2863f627cba583cbc32342df7349dfe6c38060234.lock
03/21/2022 17:53:20 - DEBUG - datasets.utils.filelock - Lock 140386484514192 not acquired on /dbfs/transformers/cache/_dbfs_transformers_cache_cnn_dailymail_3.0.0_3.0.0_3cb851bf7cf5826e45d49db2863f627cba583cbc32342df7349dfe6c38060234.lock, waiting 0.05 seconds ...
03/21/2022 17:53:20 - DEBUG - datasets.utils.filelock - Attempting to acquire lock 140386484514192 on /dbfs/transformers/cache/_dbfs_transformers_cache_cnn_dailymail_3.0.0_3.0.0_3cb851bf7cf5826e45d49db2863f627cba583cbc32342df7349dfe6c38060234.lock
03/21/2022 17:53:20 - DEBUG - datasets.utils.filelock - Lock 140386484514192 not acquired on /dbfs/transformers/cache/_dbfs_transformers_cache_cnn_dailymail_3.0.0_3.0.0_3cb851bf7cf5826e45d49db2863f627cba583cbc32342df7349dfe6c38060234.lock, waiting 0.05 seconds ...
03/21/2022 17:53:20 - DEBUG - datasets.utils.filelock - Attempting to acquire lock 140386484514192 on /dbfs/transformers/cache/_dbfs_transformers_cache_cnn_dailymail_3.0.0_3.0.0_3cb851bf7cf5826e45d49db2863f627cba583cbc32342df7349dfe6c38060234.lock
03/21/2022 17:53:20 - DEBUG - datasets.utils.filelock - Lock 140386484514192 not acquired on /dbfs/transformers/cache/_dbfs_transformers_cache_cnn_dailymail_3.0.0_3.0.0_3cb851bf7cf5826e45d49db2863f627cba583cbc32342df7349dfe6c38060234.lock, waiting 0.05 seconds ...
03/21/2022 17:53:20 - DEBUG - datasets.utils.filelock - Attempting to acquire lock 140386484514192 on /dbfs/transformers/cache/_dbfs_transformers_cache_cnn_dailymail_3.0.0_3.0.0_3cb851bf7cf5826e45d49db2863f627cba583cbc32342df7349dfe6c38060234.lock
03/21/2022 17:53:20 - DEBUG - datasets.utils.filelock - Lock 140386484514192 not acquired on /dbfs/transformers/cache/_dbfs_transformers_cache_cnn_dailymail_3.0.0_3.0.0_3cb851bf7cf5826e45d49db2863f627cba583cbc32342df7349dfe6c38060234.lock, waiting 0.05 seconds ...
```

and so on.

I imagine this has to do with DBFS -- is there a way to tackle this?
 Hi ! Thanks for reporting. We're using `py-filelock` as our locking mechanism.

Can you try deleting the .lock file mentioned in the logs and try again ? Make sure that no other process is generating the `cnn_dailymail` dataset.

If it doesn't work, could you try to set up a lock using  the latest version of `py-filelock` and see if it works ?

```
pip install filelock
```
here is a code example from the `py-filelock` documentation that you can try:
```python
from filelock import Timeout, FileLock

lock = FileLock("high_ground.txt.lock")
with lock:
    with open("high_ground.txt", "a") as f:
        f.write("You were the chosen one.")
```


I can't view HFcallback dataset for ASR Space
 ## Dataset viewer issue for '*Urdu-ASR-flags*'

**Link:** *[link to the dataset viewer page](https://huggingface.co/datasets/kingabzpro/Urdu-ASR-flags)*

*I think dataset should show some thing and if you want me to add script, please show me the documentation. I thought this was suppose to be automatic task.*

Am I the one who added this dataset ? Yes

 the dataset viewer is working on this dataset. I imagine the issue is that we would expect to be able to listen to the audio files in the `Please Record Your Voice file` column, right?

maybe @lhoestq or @albertvillanova could help

<img width="1019" alt="Capture d’écran 2022-03-24 à 17 36 20" src="https://user-images.githubusercontent.com/1676121/159966006-57dcf8f7-b65f-4200-ac8c-66859318a8bb.png">


I can't view HFcallback dataset for ASR Space
 ## Dataset viewer issue for '*Urdu-ASR-flags*'

**Link:** *[link to the dataset viewer page](https://huggingface.co/datasets/kingabzpro/Urdu-ASR-flags)*

*I think dataset should show some thing and if you want me to add script, please show me the documentation. I thought this was suppose to be automatic task.*

Am I the one who added this dataset ? Yes

 The structure of the dataset is not supported. Only the CSV file is parsed and the audio files are ignored.

We're working on supporting audio datasets with a specific structure in #3963


ConnectionError  and SSLError

ConnectionError  and SSLError
 code
```
from datasets import load_dataset
dataset = load_dataset('oscar', 'unshuffled_deduplicated_it')
```
bug report
```
---------------------------------------------------------------------------
ConnectionError                           Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_29788/2615425180.py in <module>
----> 1 dataset = load_dataset('oscar', 'unshuffled_deduplicated_it')

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, keep_in_memory, save_infos, revision, use_auth_token, task, streaming, **config_kwargs)
   1658
   1659     # Create a dataset builder
-> 1660     builder_instance = load_dataset_builder(
   1661         path=path,
   1662         name=name,

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in load_dataset_builder(path, name, data_dir, data_files, cache_dir, features, download_config, download_mode, revision, use_auth_token, **config_kwargs)
   1484         download_config = download_config.copy() if download_config else DownloadConfig()
   1485         download_config.use_auth_token = use_auth_token
-> 1486     dataset_module = dataset_module_factory(
   1487         path,
   1488         revision=revision,

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in dataset_module_factory(path, revision, download_config, download_mode, force_local_path, dynamic_modules_path, data_dir, data_files, **download_kwargs)
   1236                         f"Couldn't find '{path}' on the Hugging Face Hub either: {type(e1).__name__}: {e1}"
   1237                     ) from None
-> 1238                 raise e1 from None
   1239     else:
   1240         raise FileNotFoundError(

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in dataset_module_factory(path, revision, download_config, download_mode, force_local_path, dynamic_modules_path, data_dir, data_files, **download_kwargs)
   1173             if path.count("/") == 0:  # even though the dataset is on the Hub, we get it from GitHub for now
   1174                 # TODO(QL): use a Hub dataset module factory instead of GitHub
-> 1175                 return GithubDatasetModuleFactory(
   1176                     path,
   1177                     revision=revision,

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in get_module(self)
    531         revision = self.revision
    532         try:
--> 533             local_path = self.download_loading_script(revision)
    534         except FileNotFoundError:
    535             if revision is not None or os.getenv("HF_SCRIPTS_VERSION", None) is not None:

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in download_loading_script(self, revision)
    511         if download_config.download_desc is None:
    512             download_config.download_desc = "Downloading builder script"
--> 513         return cached_path(file_path, download_config=download_config)
    514
    515     def download_dataset_infos_file(self, revision: Optional[str]) -> str:

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\utils\file_utils.py in cached_path(url_or_filename, download_config, **download_kwargs)
    232     if is_remote_url(url_or_filename):
    233         # URL, so get it from the cache (downloading if necessary)
--> 234         output_path = get_from_cache(
    235             url_or_filename,
    236             cache_dir=cache_dir,

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\utils\file_utils.py in get_from_cache(url, cache_dir, force_download, proxies, etag_timeout, resume_download, user_agent, local_files_only, use_etag, max_retries, use_auth_token, ignore_url_params, download_desc)
    580         _raise_if_offline_mode_is_enabled(f"Tried to reach {url}")
    581         if head_error is not None:
--> 582             raise ConnectionError(f"Couldn't reach {url} ({repr(head_error)})")
    583         elif response is not None:
    584             raise ConnectionError(f"Couldn't reach {url} (error {response.status_code})")

ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/2.0.0/datasets/oscar/oscar.py (SSLError(MaxRetryError("HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /huggingface/datasets/2.0.0/datasets/oscar/oscar.py (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1129)')))")))
```
It may be caused by Caused by SSLError(in China?) because it works well on google colab.
So how can I download this dataset manually?

 Hi ! You can download the `oscar.py` file from this repository at `/datasets/oscar/oscar.py`.

Then you can load the dataset by passing the local path to `oscar.py` to `load_dataset`:
```python
load_dataset("path/to/oscar.py", "unshuffled_deduplicated_it")
```


ConnectionError  and SSLError

ConnectionError  and SSLError
 code
```
from datasets import load_dataset
dataset = load_dataset('oscar', 'unshuffled_deduplicated_it')
```
bug report
```
---------------------------------------------------------------------------
ConnectionError                           Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_29788/2615425180.py in <module>
----> 1 dataset = load_dataset('oscar', 'unshuffled_deduplicated_it')

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, keep_in_memory, save_infos, revision, use_auth_token, task, streaming, **config_kwargs)
   1658
   1659     # Create a dataset builder
-> 1660     builder_instance = load_dataset_builder(
   1661         path=path,
   1662         name=name,

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in load_dataset_builder(path, name, data_dir, data_files, cache_dir, features, download_config, download_mode, revision, use_auth_token, **config_kwargs)
   1484         download_config = download_config.copy() if download_config else DownloadConfig()
   1485         download_config.use_auth_token = use_auth_token
-> 1486     dataset_module = dataset_module_factory(
   1487         path,
   1488         revision=revision,

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in dataset_module_factory(path, revision, download_config, download_mode, force_local_path, dynamic_modules_path, data_dir, data_files, **download_kwargs)
   1236                         f"Couldn't find '{path}' on the Hugging Face Hub either: {type(e1).__name__}: {e1}"
   1237                     ) from None
-> 1238                 raise e1 from None
   1239     else:
   1240         raise FileNotFoundError(

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in dataset_module_factory(path, revision, download_config, download_mode, force_local_path, dynamic_modules_path, data_dir, data_files, **download_kwargs)
   1173             if path.count("/") == 0:  # even though the dataset is on the Hub, we get it from GitHub for now
   1174                 # TODO(QL): use a Hub dataset module factory instead of GitHub
-> 1175                 return GithubDatasetModuleFactory(
   1176                     path,
   1177                     revision=revision,

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in get_module(self)
    531         revision = self.revision
    532         try:
--> 533             local_path = self.download_loading_script(revision)
    534         except FileNotFoundError:
    535             if revision is not None or os.getenv("HF_SCRIPTS_VERSION", None) is not None:

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in download_loading_script(self, revision)
    511         if download_config.download_desc is None:
    512             download_config.download_desc = "Downloading builder script"
--> 513         return cached_path(file_path, download_config=download_config)
    514
    515     def download_dataset_infos_file(self, revision: Optional[str]) -> str:

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\utils\file_utils.py in cached_path(url_or_filename, download_config, **download_kwargs)
    232     if is_remote_url(url_or_filename):
    233         # URL, so get it from the cache (downloading if necessary)
--> 234         output_path = get_from_cache(
    235             url_or_filename,
    236             cache_dir=cache_dir,

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\utils\file_utils.py in get_from_cache(url, cache_dir, force_download, proxies, etag_timeout, resume_download, user_agent, local_files_only, use_etag, max_retries, use_auth_token, ignore_url_params, download_desc)
    580         _raise_if_offline_mode_is_enabled(f"Tried to reach {url}")
    581         if head_error is not None:
--> 582             raise ConnectionError(f"Couldn't reach {url} ({repr(head_error)})")
    583         elif response is not None:
    584             raise ConnectionError(f"Couldn't reach {url} (error {response.status_code})")

ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/2.0.0/datasets/oscar/oscar.py (SSLError(MaxRetryError("HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /huggingface/datasets/2.0.0/datasets/oscar/oscar.py (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1129)')))")))
```
It may be caused by Caused by SSLError(in China?) because it works well on google colab.
So how can I download this dataset manually?

 it works,but another error occurs.
```
ConnectionError: Couldn't reach https://s3.amazonaws.com/datasets.huggingface.co/oscar/1.0/unshuffled/deduplicated/it/it_sha256.txt (SSLError(MaxRetryError("HTTPSConnectionPool(host='s3.amazonaws.com', port=443): Max retries exceeded with url: /datasets.huggingface.co/oscar/1.0/unshuffled/deduplicated/it/it_sha256.txt (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1129)')))")))
```
I can access `https://s3.amazonaws.com/datasets.huggingface.co/oscar/1.0/unshuffled/deduplicated/it/it_sha256.txt` and `https://aws.amazon.com/cn/s3/` directly, so why it reports a SSLError, should I need tomodify the host file？


ConnectionError  and SSLError

ConnectionError  and SSLError
 code
```
from datasets import load_dataset
dataset = load_dataset('oscar', 'unshuffled_deduplicated_it')
```
bug report
```
---------------------------------------------------------------------------
ConnectionError                           Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_29788/2615425180.py in <module>
----> 1 dataset = load_dataset('oscar', 'unshuffled_deduplicated_it')

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, keep_in_memory, save_infos, revision, use_auth_token, task, streaming, **config_kwargs)
   1658
   1659     # Create a dataset builder
-> 1660     builder_instance = load_dataset_builder(
   1661         path=path,
   1662         name=name,

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in load_dataset_builder(path, name, data_dir, data_files, cache_dir, features, download_config, download_mode, revision, use_auth_token, **config_kwargs)
   1484         download_config = download_config.copy() if download_config else DownloadConfig()
   1485         download_config.use_auth_token = use_auth_token
-> 1486     dataset_module = dataset_module_factory(
   1487         path,
   1488         revision=revision,

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in dataset_module_factory(path, revision, download_config, download_mode, force_local_path, dynamic_modules_path, data_dir, data_files, **download_kwargs)
   1236                         f"Couldn't find '{path}' on the Hugging Face Hub either: {type(e1).__name__}: {e1}"
   1237                     ) from None
-> 1238                 raise e1 from None
   1239     else:
   1240         raise FileNotFoundError(

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in dataset_module_factory(path, revision, download_config, download_mode, force_local_path, dynamic_modules_path, data_dir, data_files, **download_kwargs)
   1173             if path.count("/") == 0:  # even though the dataset is on the Hub, we get it from GitHub for now
   1174                 # TODO(QL): use a Hub dataset module factory instead of GitHub
-> 1175                 return GithubDatasetModuleFactory(
   1176                     path,
   1177                     revision=revision,

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in get_module(self)
    531         revision = self.revision
    532         try:
--> 533             local_path = self.download_loading_script(revision)
    534         except FileNotFoundError:
    535             if revision is not None or os.getenv("HF_SCRIPTS_VERSION", None) is not None:

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in download_loading_script(self, revision)
    511         if download_config.download_desc is None:
    512             download_config.download_desc = "Downloading builder script"
--> 513         return cached_path(file_path, download_config=download_config)
    514
    515     def download_dataset_infos_file(self, revision: Optional[str]) -> str:

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\utils\file_utils.py in cached_path(url_or_filename, download_config, **download_kwargs)
    232     if is_remote_url(url_or_filename):
    233         # URL, so get it from the cache (downloading if necessary)
--> 234         output_path = get_from_cache(
    235             url_or_filename,
    236             cache_dir=cache_dir,

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\utils\file_utils.py in get_from_cache(url, cache_dir, force_download, proxies, etag_timeout, resume_download, user_agent, local_files_only, use_etag, max_retries, use_auth_token, ignore_url_params, download_desc)
    580         _raise_if_offline_mode_is_enabled(f"Tried to reach {url}")
    581         if head_error is not None:
--> 582             raise ConnectionError(f"Couldn't reach {url} ({repr(head_error)})")
    583         elif response is not None:
    584             raise ConnectionError(f"Couldn't reach {url} (error {response.status_code})")

ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/2.0.0/datasets/oscar/oscar.py (SSLError(MaxRetryError("HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /huggingface/datasets/2.0.0/datasets/oscar/oscar.py (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1129)')))")))
```
It may be caused by Caused by SSLError(in China?) because it works well on google colab.
So how can I download this dataset manually?

 you are so wise!
it report [ConnectionError] in python 3.9.7
and works well in python 3.8.12

I need you help again: how can I specify the path for download files?
the data is too large and my C hardware is not enough


ConnectionError  and SSLError

ConnectionError  and SSLError
 code
```
from datasets import load_dataset
dataset = load_dataset('oscar', 'unshuffled_deduplicated_it')
```
bug report
```
---------------------------------------------------------------------------
ConnectionError                           Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_29788/2615425180.py in <module>
----> 1 dataset = load_dataset('oscar', 'unshuffled_deduplicated_it')

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, keep_in_memory, save_infos, revision, use_auth_token, task, streaming, **config_kwargs)
   1658
   1659     # Create a dataset builder
-> 1660     builder_instance = load_dataset_builder(
   1661         path=path,
   1662         name=name,

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in load_dataset_builder(path, name, data_dir, data_files, cache_dir, features, download_config, download_mode, revision, use_auth_token, **config_kwargs)
   1484         download_config = download_config.copy() if download_config else DownloadConfig()
   1485         download_config.use_auth_token = use_auth_token
-> 1486     dataset_module = dataset_module_factory(
   1487         path,
   1488         revision=revision,

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in dataset_module_factory(path, revision, download_config, download_mode, force_local_path, dynamic_modules_path, data_dir, data_files, **download_kwargs)
   1236                         f"Couldn't find '{path}' on the Hugging Face Hub either: {type(e1).__name__}: {e1}"
   1237                     ) from None
-> 1238                 raise e1 from None
   1239     else:
   1240         raise FileNotFoundError(

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in dataset_module_factory(path, revision, download_config, download_mode, force_local_path, dynamic_modules_path, data_dir, data_files, **download_kwargs)
   1173             if path.count("/") == 0:  # even though the dataset is on the Hub, we get it from GitHub for now
   1174                 # TODO(QL): use a Hub dataset module factory instead of GitHub
-> 1175                 return GithubDatasetModuleFactory(
   1176                     path,
   1177                     revision=revision,

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in get_module(self)
    531         revision = self.revision
    532         try:
--> 533             local_path = self.download_loading_script(revision)
    534         except FileNotFoundError:
    535             if revision is not None or os.getenv("HF_SCRIPTS_VERSION", None) is not None:

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\load.py in download_loading_script(self, revision)
    511         if download_config.download_desc is None:
    512             download_config.download_desc = "Downloading builder script"
--> 513         return cached_path(file_path, download_config=download_config)
    514
    515     def download_dataset_infos_file(self, revision: Optional[str]) -> str:

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\utils\file_utils.py in cached_path(url_or_filename, download_config, **download_kwargs)
    232     if is_remote_url(url_or_filename):
    233         # URL, so get it from the cache (downloading if necessary)
--> 234         output_path = get_from_cache(
    235             url_or_filename,
    236             cache_dir=cache_dir,

D:\DataScience\PythonSet\IDES\anaconda\lib\site-packages\datasets\utils\file_utils.py in get_from_cache(url, cache_dir, force_download, proxies, etag_timeout, resume_download, user_agent, local_files_only, use_etag, max_retries, use_auth_token, ignore_url_params, download_desc)
    580         _raise_if_offline_mode_is_enabled(f"Tried to reach {url}")
    581         if head_error is not None:
--> 582             raise ConnectionError(f"Couldn't reach {url} ({repr(head_error)})")
    583         elif response is not None:
    584             raise ConnectionError(f"Couldn't reach {url} (error {response.status_code})")

ConnectionError: Couldn't reach https://raw.githubusercontent.com/huggingface/datasets/2.0.0/datasets/oscar/oscar.py (SSLError(MaxRetryError("HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /huggingface/datasets/2.0.0/datasets/oscar/oscar.py (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1129)')))")))
```
It may be caused by Caused by SSLError(in China?) because it works well on google colab.
So how can I download this dataset manually?

 Cool ! And you can specify the path for download files with to the `cache_dir` parameter:
```python
from datasets import load_dataset
dataset = load_dataset('oscar', 'unshuffled_deduplicated_it', cache_dir='path/to/directory')