i found issue, but this needs more testing and i only have allwinner h3 here
with hw.userspace_allow_phys_counter=1 time doesn't jump forward and back (!) anymore i tracked this down to this commit here: 7ad28b73ec1f - arm: Add a userspace physical timer check perhaps there are more people out there with current on 32bit arm...