CoderFunda
  • Home
  • About us
    • Contact Us
    • Disclaimer
    • Privacy Policy
    • About us
  • Home
  • Php
  • HTML
  • CSS
  • JavaScript
    • JavaScript
    • Jquery
    • JqueryUI
    • Stock
  • SQL
  • Vue.Js
  • Python
  • Wordpress
  • C++
    • C++
    • C
  • Laravel
    • Laravel
      • Overview
      • Namespaces
      • Middleware
      • Routing
      • Configuration
      • Application Structure
      • Installation
    • Overview
  • DBMS
    • DBMS
      • PL/SQL
      • SQLite
      • MongoDB
      • Cassandra
      • MySQL
      • Oracle
      • CouchDB
      • Neo4j
      • DB2
      • Quiz
    • Overview
  • Entertainment
    • TV Series Update
    • Movie Review
    • Movie Review
  • More
    • Vue. Js
    • Php Question
    • Php Interview Question
    • Laravel Interview Question
    • SQL Interview Question
    • IAS Interview Question
    • PCS Interview Question
    • Technology
    • Other

05 December, 2023

Find a value from an array column in a dictionary Pyspark

 Programing Coderfunda     December 05, 2023     No comments   

I have this dataframe in Pyspark:
data = [("definitely somewhere",), ("Las Vegas",), ("其他",), (None,), ("",), ("Pucela Madrid Langreo, España",), ("Trenches, With Egbon Adugbo",)]
df = spark.createDataFrame(data, ["address"])
city_country = {
'las vegas': 'US',
'lagos': 'NG',
'España': 'ES'
}
cities_name_to_code = spark.sparkContext.broadcast(city_country )
df_with_codes = df.withColumn('cities_array', F.lower(F.col('address'))) \
.withColumn('cities_array', F.split(F.col('cities_array'), ', '))



I want to find in cities_array all the keys from cities_name_to_code for each element (get an array of values).
The problem is that I don't want to use UDF.
  • Share This:  
  •  Facebook
  •  Twitter
  •  Google+
  •  Stumble
  •  Digg
Email ThisBlogThis!Share to XShare to Facebook

Related Posts:

  • Validate Console Command Input With the Command Validator Package--- The Command Validator package by Andrea Marco Sartori makes validating the input of console commands a cinch using Laravel's beloved Validato… Read More
  • Windows 10 Intellij 2024.2.1 Spring boot WebFlux Unit test failed normally it's work fine. I have a exit code -1@ExtendWith(MockitoExtension.class) class XXXXXXXTest { private YYYYRule rule; @Mock private ZZZZService zzzzService; @BeforeEach void setUp() { ru… Read More
  • window.location.replace() is not workingjust wanna ask why does window.location.replace is not working in my page. I've been working on it for weeks. It works fine on my other pages, althoug… Read More
  • My first Laravel package - Translation checker for LaravelHey everyone! I have created my first Laravel package, Translation Checker! It's designed to simplify the process of managing translations in your l… Read More
  • Asking random PHP questions at Laraconsubmitted by /u/pronskiy [link] [comments]… Read More
Newer Post Older Post Home

0 comments:

Post a Comment

Thanks

Meta

Popular Posts

  • Spring boot app (error: method getFirst()) failed to run at local machine, but can run on server
    The Spring boot app can run on the online server. Now, we want to replicate the same app at the local machine but the Spring boot jar file f...
  • Log activity in a Laravel app with Spatie/Laravel-Activitylog
      Requirements This package needs PHP 8.1+ and Laravel 9.0 or higher. The latest version of this package needs PHP 8.2+ and Laravel 8 or hig...
  • Laravel auth login with phone or email
          <?php     Laravel auth login with phone or email     <? php     namespace App \ Http \ Controllers \ Auth ;         use ...
  • Vue3 :style backgroundImage not working with require
    I'm trying to migrate a Vue 2 project to Vue 3. In Vue 2 I used v-bind style as follow: In Vue 3 this doesn't work... I tried a...
  • Failed to install 'cordova-plugin-firebase': CordovaError: Uh oh
    I had follow these steps to install an configure firebase to my cordova project for cloud messaging. https://medium.com/@felipepucinelli/how...

Categories

  • Ajax (26)
  • Bootstrap (30)
  • DBMS (42)
  • HTML (12)
  • HTML5 (45)
  • JavaScript (10)
  • Jquery (34)
  • Jquery UI (2)
  • JqueryUI (32)
  • Laravel (1017)
  • Laravel Tutorials (23)
  • Laravel-Question (6)
  • Magento (9)
  • Magento 2 (95)
  • MariaDB (1)
  • MySql Tutorial (2)
  • PHP-Interview-Questions (3)
  • Php Question (13)
  • Python (36)
  • RDBMS (13)
  • SQL Tutorial (79)
  • Vue.js Tutorial (68)
  • Wordpress (150)
  • Wordpress Theme (3)
  • codeigniter (108)
  • oops (4)
  • php (853)

Social Media Links

  • Follow on Twitter
  • Like on Facebook
  • Subscribe on Youtube
  • Follow on Instagram

Pages

  • Home
  • Contact Us
  • Privacy Policy
  • About us

Blog Archive

  • September (100)
  • August (50)
  • July (56)
  • June (46)
  • May (59)
  • April (50)
  • March (60)
  • February (42)
  • January (53)
  • December (58)
  • November (61)
  • October (39)
  • September (36)
  • August (36)
  • July (34)
  • June (34)
  • May (36)
  • April (29)
  • March (82)
  • February (1)
  • January (8)
  • December (14)
  • November (41)
  • October (13)
  • September (5)
  • August (48)
  • July (9)
  • June (6)
  • May (119)
  • April (259)
  • March (122)
  • February (368)
  • January (33)
  • October (2)
  • July (11)
  • June (29)
  • May (25)
  • April (168)
  • March (93)
  • February (60)
  • January (28)
  • December (195)
  • November (24)
  • October (40)
  • September (55)
  • August (6)
  • July (48)
  • May (2)
  • January (2)
  • July (6)
  • June (6)
  • February (17)
  • January (69)
  • December (122)
  • November (56)
  • October (92)
  • September (76)
  • August (6)

  • Failed to install 'cordova-plugin-firebase': CordovaError: Uh oh - 9/21/2024
  • pyspark XPath Query Returns Lists Omitting Missing Values Instead of Including None - 9/20/2024
  • SQL REPL from within Python/Sqlalchemy/Psychopg2 - 9/20/2024
  • MySql Explain with Tobias Petry - 9/20/2024
  • How to combine information from different devices into one common abstract virtual disk? [closed] - 9/20/2024

Laravel News

  • Validate Controller Requests with the Laravel Data Package - 5/19/2025
  • Deployer - 5/18/2025
  • Transform JSON into Typed Collections with Laravel's AsCollection::of() - 5/18/2025
  • Auto-translate Application Strings with Laratext - 5/16/2025
  • Simplify Factory Associations with Laravel's UseFactory Attribute - 5/13/2025

Copyright © 2025 CoderFunda | Powered by Blogger
Design by Coderfunda | Blogger Theme by Coderfunda | Distributed By Coderfunda