Thom's memory leak

In my previous blog post about horizontal scalability changes in PostrgreSQL 9.6, I covered pushing sorts and joins to the remote server, but Robert Haas (EnterpriseDB) has just committed a change that allows DML (that's UPDATEs and DELETEs) to be pushed to the remote server too. This is thanks to the work of Etsuro Fujita (NTT) with advice and testing from Rushabh Lathia (EnterpriseDB), Shigeru Hanada, Robert Haas (EnterpriseDB), Kyotaro Horiguchi (NTT), Albe Laurenz and myself (EnterpriseDB). So this calls for an appendix to my original post! DML pushdown The problem previously was that an update or a delete would mean fetching a batch of rows from the remote server, and sending individual UPDATE commands to the remote server. So if we ran this on the local server (where remote.big_table is a remote table which resides on the remote server): UPDATE remote.big_table SET content = content || '.'; The remote server would receive the following request from the local s...

This is part 2 in a 4 part blog series on improvements coming in PostgreSQL 9.6. Part 1 - Horizontal Scalability Join Pushdown Sort Pushdown "Walk and chew gum at the same time" Part 2 - Monitoring VACUUM progress monitoring Wait monitoring Part 3 - Parallelism Part 4 - Vertical Scalability VACUUM progress monitoring When you run VACUUM on a large table, do you know how much it's done after 5 minutes? Do you know how much more there is to go? The answer is, no. You can't know, at least not accurately or easily. In PostgreSQL 9.6, you'll be able to monitor just this (although not VACUUM FULL), thanks to the work of Amit Langote (NTT) and Robert Haas (EnterpriseDB), which was built upon the work done by Rahila Syed (EnterpriseDB) and Vinayak Pokale (NTT). We now have a system view called pg_stat_progress_vacuum. This is the first in, hopefully, several progress monitoring views to come in future. Here's an example of what it shows you (shown in...

PostgreSQL 9.6 is shaping up to be an impressive release, so I thought I'd try to summarise some of the big features and improvements that it will bring. This certainly won't be exhaustive, but it will touch upon areas I've looked at closely. I'm looking at breaking this down into the following areas: Part 1 - Horizontal Scalability Part 2 - Monitoring Part 3 - Parallelism Part 4 - Vertical Scalability So without further ado, here's Part 1 - Horizontal Scalability I guess I should start with the question some might be asking: "What is 'Horizontal Scalability'?" This is just a fancy way of describing the ability to distribute workload among several clusters/servers/locations etc. So rather than putting all the burden on one server, you can delegate at least some of the work to other servers to speed things up. For 9.6, I'll specifically be covering foreign tables (i.e. tables on remote servers which are being queried as if they wer...