CoderFunda
  • Home
  • About us
    • Contact Us
    • Disclaimer
    • Privacy Policy
    • About us
  • Home
  • Php
  • HTML
  • CSS
  • JavaScript
    • JavaScript
    • Jquery
    • JqueryUI
    • Stock
  • SQL
  • Vue.Js
  • Python
  • Wordpress
  • C++
    • C++
    • C
  • Laravel
    • Laravel
      • Overview
      • Namespaces
      • Middleware
      • Routing
      • Configuration
      • Application Structure
      • Installation
    • Overview
  • DBMS
    • DBMS
      • PL/SQL
      • SQLite
      • MongoDB
      • Cassandra
      • MySQL
      • Oracle
      • CouchDB
      • Neo4j
      • DB2
      • Quiz
    • Overview
  • Entertainment
    • TV Series Update
    • Movie Review
    • Movie Review
  • More
    • Vue. Js
    • Php Question
    • Php Interview Question
    • Laravel Interview Question
    • SQL Interview Question
    • IAS Interview Question
    • PCS Interview Question
    • Technology
    • Other

08 September, 2023

Some AWS lamba execution have more than 1 S3event key

 Programing Coderfunda     September 08, 2023     No comments   

I'm having an issue when passing events to the AWS lambda function. I have to sync 60k data in 1 json file to a remote server. My solution is to divide the file into many small files (2000 data/ file) since the function cannot process 60k data in 15 minutes. After breaking the files, I will upload them one by one to S3 (time interval is 10 seconds), when a new file is uploaded, it will send a S3Event notification to trigger the function, and that function execution will take the newest uploaded file to process. The speed is impressive, but when the number of data reaches 60k, it does not stop, it continues adding more 4k items (I don't know where it came from). When I check the newest CloudWatch log, it seems there are some executions that have more than 1 S3Events like below "Records": [ { "eventVersion": "2.1", "eventSource": "aws:s3", "awsRegion": "ap-southeast-1", "eventTime": "2023-09-08T06:26:38.873Z", "eventName": "ObjectCreated:Put", "userIdentity": { "principalId": "AWS:AIDA4EODLYGLIMIHEIAVB" }, "object": { "key": "test_4.json", "size": 846900, "eTag": "44768b0689b2acd120c7683d8f0ce236", "sequencer": "0064FABE9E323A01CD" } } } ] "Records": [ { "eventVersion": "2.1", "eventSource": "aws:s3", "awsRegion": "ap-southeast-1", "eventTime": "2023-09-08T06:28:06.642Z", "eventName": "ObjectCreated:Put", "userIdentity": { "principalId": "AWS:AIDA4EODLYGLIMIHEIAVB" }, "object": { "key": "test_12.json", "size": 845141, "eTag": "05008d0d12413f98c3aefa101bbbd5eb", "sequencer": "0064FABEF5B003C033" } } } ] This is so weird, This is the 1st time I have worked with AWS so can someone help me explain the reason? Also, Is it possible to make all execution run synchronously instead of asynchronously? I think if they are running in order, it will be easily to debug then.
  • Share This:  
  •  Facebook
  •  Twitter
  •  Google+
  •  Stumble
  •  Digg
Email ThisBlogThis!Share to XShare to Facebook

Related Posts:

  • Left join laravel “left join laravel” Coderfunda Code Answer’slaravel join231Inner Join : ->join('contacts', 'users.id', '=', 'contacts.user_id')2Left Jo… Read More
  • Laravel if elseon join “laravel if elseon join” Coderfunda Code Answerlaravel if elseon join01md asif… Read More
  • join in laravel “join in laravel” Coderfunda Code Answer’slaravel join231Inner Join : ->join('contacts', 'users.id', '=', 'contacts.user_id')2Left Join… Read More
  • How to add a additional condition in a join using query builder laravel 5.3? “How to add a additional condition in a join using query builder laravel 5.3?” Coderfunda Code Answer01->leftJoin('table3 AS c', functio… Read More
  • Select selected fields in laravel join“select selected fields in laravel join” Coderfunda Code Answerselect selected fields in laravel join11->select('accounts.*', 'users.id as uid… Read More
Newer Post Older Post Home

0 comments:

Post a Comment

Thanks

Meta

Popular Posts

  • Write API Integrations in Laravel and PHP Projects with Saloon
    Write API Integrations in Laravel and PHP Projects with Saloon Saloon  is a Laravel/PHP package that allows you to write your API integratio...
  • Features CodeIgniter
    Features CodeIgniter There is a great demand for the CodeIgniter framework in PHP developers because of its features and multiple advan...
  • Laravel Breeze with PrimeVue v4
    This is an follow up to my previous post about a "starter kit" I created with Laravel and PrimeVue components. The project has b...
  • Credit card validation in laravel
      Validation rules for credit card using laravel-validation-rules/credit-card package in laravel Install package laravel-validation-rules/cr...
  • Fast Excel Package for Laravel
      Fast Excel is a Laravel package for importing and exporting spreadsheets. It provides an elegant wrapper around Spout —a PHP package to ...

Categories

  • Ajax (26)
  • Bootstrap (30)
  • DBMS (42)
  • HTML (12)
  • HTML5 (45)
  • JavaScript (10)
  • Jquery (34)
  • Jquery UI (2)
  • JqueryUI (32)
  • Laravel (1017)
  • Laravel Tutorials (23)
  • Laravel-Question (6)
  • Magento (9)
  • Magento 2 (95)
  • MariaDB (1)
  • MySql Tutorial (2)
  • PHP-Interview-Questions (3)
  • Php Question (13)
  • Python (36)
  • RDBMS (13)
  • SQL Tutorial (79)
  • Vue.js Tutorial (68)
  • Wordpress (150)
  • Wordpress Theme (3)
  • codeigniter (108)
  • oops (4)
  • php (853)

Social Media Links

  • Follow on Twitter
  • Like on Facebook
  • Subscribe on Youtube
  • Follow on Instagram

Pages

  • Home
  • Contact Us
  • Privacy Policy
  • About us

Blog Archive

  • September (100)
  • August (50)
  • July (56)
  • June (46)
  • May (59)
  • April (50)
  • March (60)
  • February (42)
  • January (53)
  • December (58)
  • November (61)
  • October (39)
  • September (36)
  • August (36)
  • July (34)
  • June (34)
  • May (36)
  • April (29)
  • March (82)
  • February (1)
  • January (8)
  • December (14)
  • November (41)
  • October (13)
  • September (5)
  • August (48)
  • July (9)
  • June (6)
  • May (119)
  • April (259)
  • March (122)
  • February (368)
  • January (33)
  • October (2)
  • July (11)
  • June (29)
  • May (25)
  • April (168)
  • March (93)
  • February (60)
  • January (28)
  • December (195)
  • November (24)
  • October (40)
  • September (55)
  • August (6)
  • July (48)
  • May (2)
  • January (2)
  • July (6)
  • June (6)
  • February (17)
  • January (69)
  • December (122)
  • November (56)
  • October (92)
  • September (76)
  • August (6)

  • Failed to install 'cordova-plugin-firebase': CordovaError: Uh oh - 9/21/2024
  • pyspark XPath Query Returns Lists Omitting Missing Values Instead of Including None - 9/20/2024
  • SQL REPL from within Python/Sqlalchemy/Psychopg2 - 9/20/2024
  • MySql Explain with Tobias Petry - 9/20/2024
  • How to combine information from different devices into one common abstract virtual disk? [closed] - 9/20/2024

Laravel News

  • PayHere for Laravel - 6/23/2025
  • Dividing Collections with Laravel's splitIn Helper - 6/21/2025
  • Lightning Fast Schedule Management for Laravel - 6/20/2025
  • Reset Rate Limits Dynamically with Laravel's clear Method - 6/18/2025
  • Manipulate Image URLs in Laravel with the Image Transform Package - 6/19/2025

Copyright © 2025 CoderFunda | Powered by Blogger
Design by Coderfunda | Blogger Theme by Coderfunda | Distributed By Coderfunda