Fix primary crash continually with invalid checkpoint after promote

[Zhao Rui]

Newly promoted primary may leave an invalid checkpoint.

In function CreateRestartPoint, control file is updated and old wals are 
removed. But in some situations, control file is not updated, old wals are 
still removed. Thus produces an invalid checkpoint with nonexistent wal. 
Crucial log: "invalid primary checkpoint record", "could not locate a valid 
checkpoint record".




The following timeline reproduces above situation:

tl1: standby begins to create restart point (time or wal triggered).

tl2: standby promotes and control file state is updated to DB_IN_PRODUCTION. 
Control file will not update (xlog.c:9690). But old wals are still removed 
(xlog.c:9719).

tl3: standby becomes primary. primary may crash before the next complete 
checkpoint (OOM in my situation). primary will crash continually with invalid 
checkpoint.




The attached patch reproduces this problem using standard postgresql perl test, 
you can run with 

./configure --enable-tap-tests; make -j; make -C src/test/recovery/ 
check PROVE_TESTS=t/027_invalid_checkpoint_after_promote.pl

The attached patch also fixes this problem by ensuring that remove old wals 
only after control file is updated.

0001-Fix-primary-crash-continually-with-invalid-checkpoint-after-promote.patch
Description: Binary data

Re:Improving RLS planning

[Zhao Rui]

This causes wrong index scan with RLS. Maybe function 
restriction_is_securely_promotable is too strict?


You can reproduce in this way:


create table abc (a integer, b text);
insert into abc select (random()*(10^4))::integer, (random()*(10^4))::text from 
generate_series(1,10);

create index on abc(a, lower(b));


ALTER TABLE abc enable ROW LEVEL SECURITY;
ALTER TABLE abc FORCE  ROW LEVEL SECURITY;

CREATE POLICY abc_id_iso_ply on abc to CURRENT_USER USING (a = 
(current_setting('app.a'::text))::int);



# for bypass user, index scan works fine
explain analyse select * from abc where a=1 and lower(b)='1234';
     Index Scan using abc_a_lower_idx on abc

         Index Cond: ((a = 1) AND (lower(b) = 
'1234'::text))



# for RLS user, index scan can only use column a, and filter by lower(b)
set app.a=1;
explain analyse select * from abc where a=1 and lower(b)='1234';
     Index Scan using abc_a_lower_idx on abc

         Index Cond: (a = 1)
         Filter: (lower(b) = '1234'::text)



This only occurs when using non-leak-proof functional index. Everything works 
fine in following way: 
create index on abc(a, b);
explain analyse select * from abc where a=1 and b='1234';


I think crucial function is restriction_is_securely_promotable. Maybe it is too 
strict to reject normal clause match. 
Could you please recheck RLS with functional index?


regards, 
Mark Zhao




-- Original --
From:   
 "Tom Lane" 
   
http://www.postgresql.org/mailpref/pgsql-hackers

Re:Improving RLS planning

[Zhao Rui]

This patch causes wrong index scan plan with RLS. Maybe function 
restriction_is_securely_promotable is too strict?


You can reproduce in this way:


create table abc (a integer, b text);
insert into abc select (random()*(10^4))::integer, (random()*(10^4))::text from 
generate_series(1,10);

create index on abc(a, lower(b));


ALTER TABLE abc enable ROW LEVEL SECURITY;
ALTER TABLE abc FORCE  ROW LEVEL SECURITY;

CREATE POLICY abc_id_iso_ply on abc to CURRENT_USER USING (a = 
(current_setting('app.a'::text))::int);



# for bypass user, index scan works fine
explain analyse select * from abc where a=1 and lower(b)='1234';
     Index Scan using abc_a_lower_idx on abc

         Index Cond: ((a = 1) AND (lower(b) = 
'1234'::text))



# for RLS user, index scan can only use column a, and filter by lower(b)
set app.a=1;
explain analyse select * from abc where a=1 and lower(b)='1234';
     Index Scan using abc_a_lower_idx on abc

         Index Cond: (a = 1)
         Filter: (lower(b) = '1234'::text)



This only occurs when using non-leak-proof functional index. Everything works 
fine in following way: 
create index on abc(a, b);
explain analyse select * from abc where a=1 and b='1234';


I think crucial function is restriction_is_securely_promotable. Maybe it is too 
strict to reject normal clause match. 
Could you please recheck RLS with functional index?


regards, 
Mark Zhao




-- Original --
From:   
 "Tom Lane" 
   
http://www.postgresql.org/mailpref/pgsql-hackers