CoderFunda
  • Home
  • About us
    • Contact Us
    • Disclaimer
    • Privacy Policy
    • About us
  • Home
  • Php
  • HTML
  • CSS
  • JavaScript
    • JavaScript
    • Jquery
    • JqueryUI
    • Stock
  • SQL
  • Vue.Js
  • Python
  • Wordpress
  • C++
    • C++
    • C
  • Laravel
    • Laravel
      • Overview
      • Namespaces
      • Middleware
      • Routing
      • Configuration
      • Application Structure
      • Installation
    • Overview
  • DBMS
    • DBMS
      • PL/SQL
      • SQLite
      • MongoDB
      • Cassandra
      • MySQL
      • Oracle
      • CouchDB
      • Neo4j
      • DB2
      • Quiz
    • Overview
  • Entertainment
    • TV Series Update
    • Movie Review
    • Movie Review
  • More
    • Vue. Js
    • Php Question
    • Php Interview Question
    • Laravel Interview Question
    • SQL Interview Question
    • IAS Interview Question
    • PCS Interview Question
    • Technology
    • Other

08 March, 2024

Using UPDATE ... SET arr[idx] = ... to aggregate rows into arrays

 Programing Coderfunda     March 08, 2024     No comments   

I currently have a database schema like the following table:
CREATE TABLE Measures(
expId SERIAL,
iteration INT NOT NULL,
value float4 NOT NULL,
PRIMARY KEY(expId, iteration)
);




So, a table of various measurements, repeated for n iterations.
Though, because we have more data than originally expected, I want to move to a new table layout that instead uses an array column, which overall gives better performance (already tested and benchmarked):
CREATE TABLE TmpMeasures(
expId SERIAL PRIMARY KEY,
values float4[] NOT NULL
);



My problem now is how to get the old data into the new format.
In the simplest case, the data may look something like this:
INSERT INTO Measures (expId, iteration, value)
VALUES
(1, 1, 1.1),
(1, 2, 2.1),
(1, 3, 3.1),
(2, 1, 1.2),
(3, 1, 1.3);



And conversion could be done with a two step process, roughly like this, to first create the array for an experiment, and then populate the iteration values:
INSERT INTO TmpMeasures(expId, values)
SELECT expId, '{}'::float4[]
FROM Measures
ON CONFLICT DO NOTHING;

UPDATE TmpMeasures tm
SET values[iteration] = m.value
FROM Measures m WHERE tm.expId = m.expId;



Though, my problem now is that the UPDATE actually only ever seems to take the first iteration, i.e., iteration = 1.
I am not quite understanding why that is the case.


I suspect, alternative approaches to values[iteration] would try to group by expId, and order by iteration and aggregate that into an array.


Unfortunately, the data isn't perfect, but iterations should line up.


So, the following seems to work, but it's extremely slow, and I don't quite understand why it's needed in the first place.
DO
$do$
BEGIN
FOR i IN 1..(SELECT max(iteration) FROM Measures m) LOOP
UPDATE TmpMeasures tm
SET values[i] = m.value
FROM Measures m
WHERE
tm.expId = m.expId AND
m.iteration=i;
END LOOP;
END
$do$;



Why does the "normal" update statement not suffice?
  • Share This:  
  •  Facebook
  •  Twitter
  •  Google+
  •  Stumble
  •  Digg
Email ThisBlogThis!Share to XShare to Facebook
Newer Post Older Post Home

0 comments:

Post a Comment

Thanks

Meta

Popular Posts

  • Blade Component to Render Markdown in Laravel
      Laravel Markdown   is a highly configurable markdown renderer and Blade component for Laravel by the folks at   Spatie : The package featu...
  • There are two solutions for one board, using different chips. But one of their i2c address is the same. How to resolve conflict in one dts?
    Two chips A and B conflict with 0x62 on i2c bus 10. If A@62 and B@62 are configured on dts, Linux loads the driver of A. If B@62 is in front...
  • Real-Time Chat Package for Laravel
      Chatify   is a Laravel package by   Munaf Aqeel Mahdi   that adds a complete real-time chat system to your application without any additio...
  • Using Laravel Model Factories in your tests
    Laravel Model factories are one of the best features you can use in your application when it comes to testing. They provide a way to define ...
  • Detecting .NET8 with Inno Setup and InnoDependenciyInstaller
    I am looking at using InnoDependencyInstaller and it has a function that it uses under the hood: Dependency_AddDotNet80 procedure Depend...

Categories

  • Ajax (26)
  • Bootstrap (30)
  • DBMS (42)
  • HTML (12)
  • HTML5 (45)
  • JavaScript (10)
  • Jquery (34)
  • Jquery UI (2)
  • JqueryUI (32)
  • Laravel (1017)
  • Laravel Tutorials (23)
  • Laravel-Question (6)
  • Magento (9)
  • Magento 2 (95)
  • MariaDB (1)
  • MySql Tutorial (2)
  • PHP-Interview-Questions (3)
  • Php Question (13)
  • Python (36)
  • RDBMS (13)
  • SQL Tutorial (79)
  • Vue.js Tutorial (69)
  • Wordpress (150)
  • Wordpress Theme (3)
  • codeigniter (108)
  • oops (4)
  • php (853)

Social Media Links

  • Follow on Twitter
  • Like on Facebook
  • Subscribe on Youtube
  • Follow on Instagram

Pages

  • Home
  • Contact Us
  • Privacy Policy
  • About us

Blog Archive

  • July (4)
  • September (100)
  • August (50)
  • July (56)
  • June (46)
  • May (59)
  • April (50)
  • March (60)
  • February (42)
  • January (53)
  • December (58)
  • November (61)
  • October (39)
  • September (36)
  • August (36)
  • July (34)
  • June (34)
  • May (36)
  • April (29)
  • March (82)
  • February (1)
  • January (8)
  • December (14)
  • November (41)
  • October (13)
  • September (5)
  • August (48)
  • July (9)
  • June (6)
  • May (119)
  • April (259)
  • March (122)
  • February (368)
  • January (33)
  • October (2)
  • July (11)
  • June (29)
  • May (25)
  • April (168)
  • March (93)
  • February (60)
  • January (28)
  • December (195)
  • November (24)
  • October (40)
  • September (55)
  • August (6)
  • July (48)
  • May (2)
  • January (2)
  • July (6)
  • June (6)
  • February (17)
  • January (69)
  • December (122)
  • November (56)
  • October (92)
  • September (76)
  • August (6)

Loading...

Laravel News

Loading...

Copyright © CoderFunda | Powered by Blogger
Design by Coderfunda | Blogger Theme by Coderfunda | Distributed By Coderfunda