CoderFunda
  • Home
  • About us
    • Contact Us
    • Disclaimer
    • Privacy Policy
    • About us
  • Home
  • Php
  • HTML
  • CSS
  • JavaScript
    • JavaScript
    • Jquery
    • JqueryUI
    • Stock
  • SQL
  • Vue.Js
  • Python
  • Wordpress
  • C++
    • C++
    • C
  • Laravel
    • Laravel
      • Overview
      • Namespaces
      • Middleware
      • Routing
      • Configuration
      • Application Structure
      • Installation
    • Overview
  • DBMS
    • DBMS
      • PL/SQL
      • SQLite
      • MongoDB
      • Cassandra
      • MySQL
      • Oracle
      • CouchDB
      • Neo4j
      • DB2
      • Quiz
    • Overview
  • Entertainment
    • TV Series Update
    • Movie Review
    • Movie Review
  • More
    • Vue. Js
    • Php Question
    • Php Interview Question
    • Laravel Interview Question
    • SQL Interview Question
    • IAS Interview Question
    • PCS Interview Question
    • Technology
    • Other

03 August, 2024

Very slow NET I/O speeds between docker containers when indexing to elasticsearch

 Programing Coderfunda     August 03, 2024     No comments   

I have spent a lot of time figuring out this but I cant find the way to fix it. I have one computer that has 48gb assigned to docker(31 to Elastic and everything else to other containers). Other computer has xeon 4410y silver and 128gb ram total and same way assigned to docker. This xeon pc is rocky linux, first pc is windows with WSL2. I have tried a lot of things but none of them work. I am trying to index db data to elasticsearch but it takes centuries on wsl2, on linux its like 1m entries in few minutes, wsl is 100k in 1 hour or more. Here are things I have tried(with ES settings, wsl configuration) that was told as fix somewhere in web, but did not work -



wsl -d docker-desktop && sysctl -w vm.max_map_count=262144



in wslconfig -



kernelCommandLine=ipv6.disable=1



es settings -
{
"index": {
"refresh_interval": "60s"
}
}

{
"index": {
"number_of_replicas": 0
}
}




sudo apt install ethtool


eth0 | grep 'tcp-segmentation-offload'


sudo ethtool -K eth0 tso off sudo ethtool -k

CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
47748c962aac INDEX 0.00% 15.43MiB / 47.05GiB 0.03% 16.5MB / 29.8MB 0B / 0B 7
d05d2eb764da DB 16.42% 267.6MiB / 47.05GiB 0.56% 2.84MB / 16.3MB 0B / 0B 11
3e013b08fd13 ES 0.12% 31.13GiB / 47.05GiB 66.16% 27MB / 195kB 0B / 0B 86



Here you can see on wsl that the NET I/O are very low numbers transfered over around 1hour. On Xeon it would be already many MB. Overall WSL pc took 18+ hours to index, xeon took around 40 minutes. The db has ~58m entries. CPU usage is very low. What is going on here? How I can fix this or what I can try?


Thanks
  • Share This:  
  •  Facebook
  •  Twitter
  •  Google+
  •  Stumble
  •  Digg
Email ThisBlogThis!Share to XShare to Facebook

Related Posts:

  • JqueryUI - AccordionJqueryUI - Accordion Accordion Widget in jQueryUI is a jQuery based expandable and collapsible content holder that is broken into sections and probab… Read More
  • JqueryUI - AutocompleteJqueryUI - AutocompleteAuto completion is a mechanism frequently used in modern websites to provide the user with a list of suggestions for the begin… Read More
  • JqueryUI - Sortable, JqueryJqueryUI - Sortable, Jquery jQueryUI provides sortable() method to reorder elements in list or grid using the mouse. This method performs sortability… Read More
  • JqueryUI - ButtonJqueryUI - ButtonjQueryUI provides button() method to transform the HTML elements (like buttons, inputs and anchors) into themeable buttons, with au… Read More
  • JqueryUI - DatepickerJqueryUI - DatepickerDatepickers in jQueryUI allow users to enter dates easily and visually. You can customize the date format and language, restrict… Read More
Newer Post Older Post Home

0 comments:

Post a Comment

Thanks

Meta

Popular Posts

  • Spring boot app (error: method getFirst()) failed to run at local machine, but can run on server
    The Spring boot app can run on the online server. Now, we want to replicate the same app at the local machine but the Spring boot jar file f...
  • Log activity in a Laravel app with Spatie/Laravel-Activitylog
      Requirements This package needs PHP 8.1+ and Laravel 9.0 or higher. The latest version of this package needs PHP 8.2+ and Laravel 8 or hig...
  • Laravel auth login with phone or email
          <?php     Laravel auth login with phone or email     <? php     namespace App \ Http \ Controllers \ Auth ;         use ...
  • Failed to install 'cordova-plugin-firebase': CordovaError: Uh oh
    I had follow these steps to install an configure firebase to my cordova project for cloud messaging. https://medium.com/@felipepucinelli/how...
  • Cashier package and Blade files
    I'm a little confused about this Cashier package. I installed it using the Laravel website (with composer), but noticed there's no...

Categories

  • Ajax (26)
  • Bootstrap (30)
  • DBMS (42)
  • HTML (12)
  • HTML5 (45)
  • JavaScript (10)
  • Jquery (34)
  • Jquery UI (2)
  • JqueryUI (32)
  • Laravel (1017)
  • Laravel Tutorials (23)
  • Laravel-Question (6)
  • Magento (9)
  • Magento 2 (95)
  • MariaDB (1)
  • MySql Tutorial (2)
  • PHP-Interview-Questions (3)
  • Php Question (13)
  • Python (36)
  • RDBMS (13)
  • SQL Tutorial (79)
  • Vue.js Tutorial (68)
  • Wordpress (150)
  • Wordpress Theme (3)
  • codeigniter (108)
  • oops (4)
  • php (853)

Social Media Links

  • Follow on Twitter
  • Like on Facebook
  • Subscribe on Youtube
  • Follow on Instagram

Pages

  • Home
  • Contact Us
  • Privacy Policy
  • About us

Blog Archive

  • September (100)
  • August (50)
  • July (56)
  • June (46)
  • May (59)
  • April (50)
  • March (60)
  • February (42)
  • January (53)
  • December (58)
  • November (61)
  • October (39)
  • September (36)
  • August (36)
  • July (34)
  • June (34)
  • May (36)
  • April (29)
  • March (82)
  • February (1)
  • January (8)
  • December (14)
  • November (41)
  • October (13)
  • September (5)
  • August (48)
  • July (9)
  • June (6)
  • May (119)
  • April (259)
  • March (122)
  • February (368)
  • January (33)
  • October (2)
  • July (11)
  • June (29)
  • May (25)
  • April (168)
  • March (93)
  • February (60)
  • January (28)
  • December (195)
  • November (24)
  • October (40)
  • September (55)
  • August (6)
  • July (48)
  • May (2)
  • January (2)
  • July (6)
  • June (6)
  • February (17)
  • January (69)
  • December (122)
  • November (56)
  • October (92)
  • September (76)
  • August (6)

  • Failed to install 'cordova-plugin-firebase': CordovaError: Uh oh - 9/21/2024
  • pyspark XPath Query Returns Lists Omitting Missing Values Instead of Including None - 9/20/2024
  • SQL REPL from within Python/Sqlalchemy/Psychopg2 - 9/20/2024
  • MySql Explain with Tobias Petry - 9/20/2024
  • How to combine information from different devices into one common abstract virtual disk? [closed] - 9/20/2024

Laravel News

  • Auto-translate Application Strings with Laratext - 5/16/2025
  • Simplify Factory Associations with Laravel's UseFactory Attribute - 5/13/2025
  • Improved Installation and Frontend Hooks in Laravel Echo 2.1 - 5/15/2025
  • Filter Model Attributes with Laravel's New except() Method - 5/13/2025
  • Arr::from() Method in Laravel 12.14 - 5/14/2025

Copyright © 2025 CoderFunda | Powered by Blogger
Design by Coderfunda | Blogger Theme by Coderfunda | Distributed By Coderfunda