IsADirectoryError when running Huggingface Transformers on Azure ML Notebook : [Errno 21] Is a directory: '\n <div>\n \n <progress value....
I am running this notebook
https://github.com/openai/openai-cookbook/blob/main/articles/gpt-oss/fine-tune-transfomers.ipynb
On an Azure ML instance with one H100 GPU. I changed some parameters to make my GPU memory use less, but I am getting this error:
IsADirectoryError: [Errno 21] Is a directory: '\n <div>\n \n <progress value='2' max='899' style='width:300px; height:20px; vertical-align: middle;'></progress>\n [ 2/899 : < :, Epoch 0.00/1]\n </div>\n <table border="1" class="dataframe">\n <thead>\n <tr style="text-align: left;">\n <th>Step</th>\n <th>Training Loss</th>\n </tr>\n </thead>\n <tbody>\n </tbody>\n</table><p>'
When I use
trainer.remove_callback(NotebookProgressCallback)
The error disappears, but then I can't see the model's training progress.
This appears to be a known issue: hugging face works on other platforms but not on Azure ML notebooks.