CoderFunda
  • Home
  • About us
    • Contact Us
    • Disclaimer
    • Privacy Policy
    • About us
  • Home
  • Php
  • HTML
  • CSS
  • JavaScript
    • JavaScript
    • Jquery
    • JqueryUI
    • Stock
  • SQL
  • Vue.Js
  • Python
  • Wordpress
  • C++
    • C++
    • C
  • Laravel
    • Laravel
      • Overview
      • Namespaces
      • Middleware
      • Routing
      • Configuration
      • Application Structure
      • Installation
    • Overview
  • DBMS
    • DBMS
      • PL/SQL
      • SQLite
      • MongoDB
      • Cassandra
      • MySQL
      • Oracle
      • CouchDB
      • Neo4j
      • DB2
      • Quiz
    • Overview
  • Entertainment
    • TV Series Update
    • Movie Review
    • Movie Review
  • More
    • Vue. Js
    • Php Question
    • Php Interview Question
    • Laravel Interview Question
    • SQL Interview Question
    • IAS Interview Question
    • PCS Interview Question
    • Technology
    • Other

03 August, 2024

Very slow NET I/O speeds between docker containers when indexing to elasticsearch

 Programing Coderfunda     August 03, 2024     No comments   

I have spent a lot of time figuring out this but I cant find the way to fix it. I have one computer that has 48gb assigned to docker(31 to Elastic and everything else to other containers). Other computer has xeon 4410y silver and 128gb ram total and same way assigned to docker. This xeon pc is rocky linux, first pc is windows with WSL2. I have tried a lot of things but none of them work. I am trying to index db data to elasticsearch but it takes centuries on wsl2, on linux its like 1m entries in few minutes, wsl is 100k in 1 hour or more. Here are things I have tried(with ES settings, wsl configuration) that was told as fix somewhere in web, but did not work -



wsl -d docker-desktop && sysctl -w vm.max_map_count=262144



in wslconfig -



kernelCommandLine=ipv6.disable=1



es settings -
{
"index": {
"refresh_interval": "60s"
}
}

{
"index": {
"number_of_replicas": 0
}
}




sudo apt install ethtool


eth0 | grep 'tcp-segmentation-offload'


sudo ethtool -K eth0 tso off sudo ethtool -k

CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
47748c962aac INDEX 0.00% 15.43MiB / 47.05GiB 0.03% 16.5MB / 29.8MB 0B / 0B 7
d05d2eb764da DB 16.42% 267.6MiB / 47.05GiB 0.56% 2.84MB / 16.3MB 0B / 0B 11
3e013b08fd13 ES 0.12% 31.13GiB / 47.05GiB 66.16% 27MB / 195kB 0B / 0B 86



Here you can see on wsl that the NET I/O are very low numbers transfered over around 1hour. On Xeon it would be already many MB. Overall WSL pc took 18+ hours to index, xeon took around 40 minutes. The db has ~58m entries. CPU usage is very low. What is going on here? How I can fix this or what I can try?


Thanks
  • Share This:  
  •  Facebook
  •  Twitter
  •  Google+
  •  Stumble
  •  Digg
Email ThisBlogThis!Share to XShare to Facebook
Newer Post Older Post Home

0 comments:

Post a Comment

Thanks

Meta

Popular Posts

  • Blade Component to Render Markdown in Laravel
      Laravel Markdown   is a highly configurable markdown renderer and Blade component for Laravel by the folks at   Spatie : The package featu...
  • There are two solutions for one board, using different chips. But one of their i2c address is the same. How to resolve conflict in one dts?
    Two chips A and B conflict with 0x62 on i2c bus 10. If A@62 and B@62 are configured on dts, Linux loads the driver of A. If B@62 is in front...
  • Real-Time Chat Package for Laravel
      Chatify   is a Laravel package by   Munaf Aqeel Mahdi   that adds a complete real-time chat system to your application without any additio...
  • Using Laravel Model Factories in your tests
    Laravel Model factories are one of the best features you can use in your application when it comes to testing. They provide a way to define ...
  • Detecting .NET8 with Inno Setup and InnoDependenciyInstaller
    I am looking at using InnoDependencyInstaller and it has a function that it uses under the hood: Dependency_AddDotNet80 procedure Depend...

Categories

  • Ajax (26)
  • Bootstrap (30)
  • DBMS (42)
  • HTML (12)
  • HTML5 (45)
  • JavaScript (10)
  • Jquery (34)
  • Jquery UI (2)
  • JqueryUI (32)
  • Laravel (1017)
  • Laravel Tutorials (23)
  • Laravel-Question (6)
  • Magento (9)
  • Magento 2 (95)
  • MariaDB (1)
  • MySql Tutorial (2)
  • PHP-Interview-Questions (3)
  • Php Question (13)
  • Python (36)
  • RDBMS (13)
  • SQL Tutorial (79)
  • Vue.js Tutorial (69)
  • Wordpress (150)
  • Wordpress Theme (3)
  • codeigniter (108)
  • oops (4)
  • php (853)

Social Media Links

  • Follow on Twitter
  • Like on Facebook
  • Subscribe on Youtube
  • Follow on Instagram

Pages

  • Home
  • Contact Us
  • Privacy Policy
  • About us

Blog Archive

  • July (4)
  • September (100)
  • August (50)
  • July (56)
  • June (46)
  • May (59)
  • April (50)
  • March (60)
  • February (42)
  • January (53)
  • December (58)
  • November (61)
  • October (39)
  • September (36)
  • August (36)
  • July (34)
  • June (34)
  • May (36)
  • April (29)
  • March (82)
  • February (1)
  • January (8)
  • December (14)
  • November (41)
  • October (13)
  • September (5)
  • August (48)
  • July (9)
  • June (6)
  • May (119)
  • April (259)
  • March (122)
  • February (368)
  • January (33)
  • October (2)
  • July (11)
  • June (29)
  • May (25)
  • April (168)
  • March (93)
  • February (60)
  • January (28)
  • December (195)
  • November (24)
  • October (40)
  • September (55)
  • August (6)
  • July (48)
  • May (2)
  • January (2)
  • July (6)
  • June (6)
  • February (17)
  • January (69)
  • December (122)
  • November (56)
  • October (92)
  • September (76)
  • August (6)

Loading...

Laravel News

Loading...

Copyright © CoderFunda | Powered by Blogger
Design by Coderfunda | Blogger Theme by Coderfunda | Distributed By Coderfunda