CoderFunda
  • Home
  • About us
    • Contact Us
    • Disclaimer
    • Privacy Policy
    • About us
  • Home
  • Php
  • HTML
  • CSS
  • JavaScript
    • JavaScript
    • Jquery
    • JqueryUI
    • Stock
  • SQL
  • Vue.Js
  • Python
  • Wordpress
  • C++
    • C++
    • C
  • Laravel
    • Laravel
      • Overview
      • Namespaces
      • Middleware
      • Routing
      • Configuration
      • Application Structure
      • Installation
    • Overview
  • DBMS
    • DBMS
      • PL/SQL
      • SQLite
      • MongoDB
      • Cassandra
      • MySQL
      • Oracle
      • CouchDB
      • Neo4j
      • DB2
      • Quiz
    • Overview
  • Entertainment
    • TV Series Update
    • Movie Review
    • Movie Review
  • More
    • Vue. Js
    • Php Question
    • Php Interview Question
    • Laravel Interview Question
    • SQL Interview Question
    • IAS Interview Question
    • PCS Interview Question
    • Technology
    • Other

03 August, 2024

Very slow NET I/O speeds between docker containers when indexing to elasticsearch

 Programing Coderfunda     August 03, 2024     No comments   

I have spent a lot of time figuring out this but I cant find the way to fix it. I have one computer that has 48gb assigned to docker(31 to Elastic and everything else to other containers). Other computer has xeon 4410y silver and 128gb ram total and same way assigned to docker. This xeon pc is rocky linux, first pc is windows with WSL2. I have tried a lot of things but none of them work. I am trying to index db data to elasticsearch but it takes centuries on wsl2, on linux its like 1m entries in few minutes, wsl is 100k in 1 hour or more. Here are things I have tried(with ES settings, wsl configuration) that was told as fix somewhere in web, but did not work -



wsl -d docker-desktop && sysctl -w vm.max_map_count=262144



in wslconfig -



kernelCommandLine=ipv6.disable=1



es settings -
{
"index": {
"refresh_interval": "60s"
}
}

{
"index": {
"number_of_replicas": 0
}
}




sudo apt install ethtool


eth0 | grep 'tcp-segmentation-offload'


sudo ethtool -K eth0 tso off sudo ethtool -k

CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
47748c962aac INDEX 0.00% 15.43MiB / 47.05GiB 0.03% 16.5MB / 29.8MB 0B / 0B 7
d05d2eb764da DB 16.42% 267.6MiB / 47.05GiB 0.56% 2.84MB / 16.3MB 0B / 0B 11
3e013b08fd13 ES 0.12% 31.13GiB / 47.05GiB 66.16% 27MB / 195kB 0B / 0B 86



Here you can see on wsl that the NET I/O are very low numbers transfered over around 1hour. On Xeon it would be already many MB. Overall WSL pc took 18+ hours to index, xeon took around 40 minutes. The db has ~58m entries. CPU usage is very low. What is going on here? How I can fix this or what I can try?


Thanks
  • Share This:  
  •  Facebook
  •  Twitter
  •  Google+
  •  Stumble
  •  Digg
Email ThisBlogThis!Share to XShare to Facebook
Newer Post Older Post Home

0 comments:

Post a Comment

Thanks

Meta

Popular Posts

  • Laravel auth check login
          <?php     Laravel auth check login     use Illuminate\Support\Facades\ Auth ;     if ( Auth :: check()) {         // The use...
  • JqueryUI Tutorial
    JqueryUI Tutorial    JqueryUI is the most popular front end frameworks currently. It is sleek, intuitive, and powerful mobile first fr...
  • window.location.replace() is not working
    just wanna ask why does window.location.replace is not working in my page. I've been working on it for weeks. It works fine on my other ...
  • Comments in PHP in Hindi
    PHP में comments programmer को code समझने के लिए दिया जाता है | ये Comment lines server द्वारा read नहीं की जाती | PHP में दो प्रकार से comm...
  • Inertia and React or Vue
    Hi just checking your thoughts on whether to learn React or Vue, I want to learn React as it may be better to find work and it has a larger ...

Categories

  • Ajax (26)
  • Bootstrap (30)
  • DBMS (42)
  • HTML (12)
  • HTML5 (45)
  • JavaScript (10)
  • Jquery (34)
  • Jquery UI (2)
  • JqueryUI (32)
  • Laravel (1017)
  • Laravel Tutorials (23)
  • Laravel-Question (6)
  • Magento (9)
  • Magento 2 (95)
  • MariaDB (1)
  • MySql Tutorial (2)
  • PHP-Interview-Questions (3)
  • Php Question (13)
  • Python (36)
  • RDBMS (13)
  • SQL Tutorial (79)
  • Vue.js Tutorial (69)
  • Wordpress (150)
  • Wordpress Theme (3)
  • codeigniter (108)
  • oops (4)
  • php (853)

Social Media Links

  • Follow on Twitter
  • Like on Facebook
  • Subscribe on Youtube
  • Follow on Instagram

Pages

  • Home
  • Contact Us
  • Privacy Policy
  • About us

Blog Archive

  • July (4)
  • September (100)
  • August (50)
  • July (56)
  • June (46)
  • May (59)
  • April (50)
  • March (60)
  • February (42)
  • January (53)
  • December (58)
  • November (61)
  • October (39)
  • September (36)
  • August (36)
  • July (34)
  • June (34)
  • May (36)
  • April (29)
  • March (82)
  • February (1)
  • January (8)
  • December (14)
  • November (41)
  • October (13)
  • September (5)
  • August (48)
  • July (9)
  • June (6)
  • May (119)
  • April (259)
  • March (122)
  • February (368)
  • January (33)
  • October (2)
  • July (11)
  • June (29)
  • May (25)
  • April (168)
  • March (93)
  • February (60)
  • January (28)
  • December (195)
  • November (24)
  • October (40)
  • September (55)
  • August (6)
  • July (48)
  • May (2)
  • January (2)
  • July (6)
  • June (6)
  • February (17)
  • January (69)
  • December (122)
  • November (56)
  • October (92)
  • September (76)
  • August (6)

Loading...

Laravel News

Loading...

Copyright © CoderFunda | Powered by Blogger
Design by Coderfunda | Blogger Theme by Coderfunda | Distributed By Coderfunda