CoderFunda
  • Home
  • About us
    • Contact Us
    • Disclaimer
    • Privacy Policy
    • About us
  • Home
  • Php
  • HTML
  • CSS
  • JavaScript
    • JavaScript
    • Jquery
    • JqueryUI
    • Stock
  • SQL
  • Vue.Js
  • Python
  • Wordpress
  • C++
    • C++
    • C
  • Laravel
    • Laravel
      • Overview
      • Namespaces
      • Middleware
      • Routing
      • Configuration
      • Application Structure
      • Installation
    • Overview
  • DBMS
    • DBMS
      • PL/SQL
      • SQLite
      • MongoDB
      • Cassandra
      • MySQL
      • Oracle
      • CouchDB
      • Neo4j
      • DB2
      • Quiz
    • Overview
  • Entertainment
    • TV Series Update
    • Movie Review
    • Movie Review
  • More
    • Vue. Js
    • Php Question
    • Php Interview Question
    • Laravel Interview Question
    • SQL Interview Question
    • IAS Interview Question
    • PCS Interview Question
    • Technology
    • Other

03 June, 2024

Pytorch - sending dataset to cuda breaks the dataloader iterator - TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu()

 Programing Coderfunda     June 03, 2024     No comments   

I am trying to speed up my pytorch training by following the advice from here:



https://discuss.pytorch.org/t/cpu-faster-than-gpu/25343/12 />

So now, I am sending my trainingdata.data and .targets to cuda before starting training. What I am confused about it how to then use the Dataloader made off of the trainingdata in my train function, as when I try the way I had before (when I was sending individual batches to cuda), I get this error on the iterator of the dataloader:
TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.



It is erroring on the following line (loader is the dataloader here):
for data, target in loader:



What am I doing wrong? How can I still use the dataloader while also sending the full dataset over to the gpu?


Python version is 3.12, pytorch is 2.3.0+cu121


Also, the error is the same if I dont include the pin_memory var to the dataloader


Full error trace:
Traceback (most recent call last):
File "C:\Program Files\JetBrains\PyCharm Community Edition 2024.1.2\plugins\python-ce\helpers\pydev\pydevd.py", line 1537, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\JetBrains\PyCharm Community Edition 2024.1.2\plugins\python-ce\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:\Users\me\PycharmProjects\NueralNetTests\TorchTests.py", line 181, in
main()
File "C:\Users\me\PycharmProjects\NueralNetTests\TorchTests.py", line 169, in main
train(epoch, model, loaders, device, optimizer, lossFN)
File "C:\Users\me\PycharmProjects\NueralNetTests\TorchTests.py", line 41, in train
for data, target in loaders['train']:
File "C:\Users\me\PycharmProjects\NueralNetTests\venv\Lib\site-packages\torch\utils\data\dataloader.py", line 631, in __next__
data = self._next_data()
^^^^^^^^^^^^^^^^^
File "C:\Users\me\PycharmProjects\NueralNetTests\venv\Lib\site-packages\torch\utils\data\dataloader.py", line 675, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\me\PycharmProjects\NueralNetTests\venv\Lib\site-packages\torch\utils\data\_utils\fetch.py", line 51, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
~~~~~~~~~~~~^^^^^
File "C:\Users\me\PycharmProjects\NueralNetTests\venv\Lib\site-packages\torchvision\datasets\mnist.py", line 143, in __getitem__
img = Image.fromarray(img.numpy(), mode="L")
^^^^^^^^^^^
TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.



and The code:
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

_transform = Compose([
lambda img: rotate(img, -90),
lambda img: hflip(img),
ToTensor()
])

trainData = datasets.EMNIST(
root='data',
train=True,
transform=_transform,
download=True,
split='letters'
)
testData = datasets.EMNIST(
root='data',
train=False,
transform=_transform,
download=True,
split='letters'
)

trainLoader = DataLoader(trainData,
batch_size=100,
shuffle=True,
pin_memory=True
)

testLoader = DataLoader(testData,
batch_size=100,
shuffle=True,
pin_memory=True
)

trainData.data = trainData.data.to(device)
trainData.targets = trainData.targets.to(device)
testData.data = testData.data.to(device)
testData.targets = testData.targets.to(device)

model = CNN().to(device)
optimizer = optim.Adam(model.parameters(), lr=0.001)
lossFN = nn.CrossEntropyLoss()

for epoch in range(1, 2):
train(model, trainLoader, device, optimizer, lossFN)
test(model, testLoader, device, lossFN)

def train(model, loader, device, optimizer, lossFN):
model.train()
for data, target in loader:
optimizer.zero_grad()
output = model(data)
loss = lossFN(output, target)
loss.backward()
optimizer.step()

class CNN(nn.Module):

def __init__(self):
super(CNN, self).__init__()

self.conv1 = nn.Conv2d(1, 10, kernel_size=5)
self.conv2 = nn.Conv2d(10, 20, kernel_size=5)
self.conv2Drop = nn.Dropout2d()
self.fc1 = nn.Linear(320, 50)
self.fc2 = nn.Linear(50, 27)

def forward(self, x):
x = F.leaky_relu(F.max_pool2d(self.conv1(x), 2))
x = F.leaky_relu(F.max_pool2d(self.conv2Drop(self.conv2(x)), 2))
x = x.view(-1, 320)
x = F.leaky_relu(self.fc1(x))
x = F.dropout(x, training=self.training)
x = self.fc2(x)

return F.softmax(x)
  • Share This:  
  •  Facebook
  •  Twitter
  •  Google+
  •  Stumble
  •  Digg
Email ThisBlogThis!Share to XShare to Facebook
Newer Post Older Post Home

0 comments:

Post a Comment

Thanks

Meta

Popular Posts

  • Writing and debugging Eloquent queries with Tinkerwell
    In this article, let's look into the options that you can use with Tinkerwell to write and debug Eloquent queries easier. The post Wr...
  • The token request was rejected by the remote server
    error:invalid_granterror_description:The token request was rejected by the remote server.error_uri: https://documentation.openiddict.com/err...
  • Sitaare Zameen Par Full Movie Review
     Here’s a  complete Vue.js tutorial for beginners to master level , structured in a progressive and simple way. It covers all essential topi...
  • Vue.js Tutorial
      Vue.js Installation Compatibility Check Before going to install and use Vue.js in your project, you should check the compatibility issues....
  • JqueryUI Tutorial
    JqueryUI Tutorial    JqueryUI is the most popular front end frameworks currently. It is sleek, intuitive, and powerful mobile first fr...

Categories

  • Ajax (26)
  • Bootstrap (30)
  • DBMS (42)
  • HTML (12)
  • HTML5 (45)
  • JavaScript (10)
  • Jquery (34)
  • Jquery UI (2)
  • JqueryUI (32)
  • Laravel (1017)
  • Laravel Tutorials (23)
  • Laravel-Question (6)
  • Magento (9)
  • Magento 2 (95)
  • MariaDB (1)
  • MySql Tutorial (2)
  • PHP-Interview-Questions (3)
  • Php Question (13)
  • Python (36)
  • RDBMS (13)
  • SQL Tutorial (79)
  • Vue.js Tutorial (69)
  • Wordpress (150)
  • Wordpress Theme (3)
  • codeigniter (108)
  • oops (4)
  • php (853)

Social Media Links

  • Follow on Twitter
  • Like on Facebook
  • Subscribe on Youtube
  • Follow on Instagram

Pages

  • Home
  • Contact Us
  • Privacy Policy
  • About us

Blog Archive

  • July (4)
  • September (100)
  • August (50)
  • July (56)
  • June (46)
  • May (59)
  • April (50)
  • March (60)
  • February (42)
  • January (53)
  • December (58)
  • November (61)
  • October (39)
  • September (36)
  • August (36)
  • July (34)
  • June (34)
  • May (36)
  • April (29)
  • March (82)
  • February (1)
  • January (8)
  • December (14)
  • November (41)
  • October (13)
  • September (5)
  • August (48)
  • July (9)
  • June (6)
  • May (119)
  • April (259)
  • March (122)
  • February (368)
  • January (33)
  • October (2)
  • July (11)
  • June (29)
  • May (25)
  • April (168)
  • March (93)
  • February (60)
  • January (28)
  • December (195)
  • November (24)
  • October (40)
  • September (55)
  • August (6)
  • July (48)
  • May (2)
  • January (2)
  • July (6)
  • June (6)
  • February (17)
  • January (69)
  • December (122)
  • November (56)
  • October (92)
  • September (76)
  • August (6)

Loading...

Laravel News

Loading...

Copyright © CoderFunda | Powered by Blogger
Design by Coderfunda | Blogger Theme by Coderfunda | Distributed By Coderfunda