There are new results for local and network disk (checkout large code base):
Local disk: svn.r1387070 - 256.8 sec svn.patch - 152.2 sec Network disk: svn.r1387070 - 5717 sec svn.patch - 2790 sec All systems use Windows 7. On Wed, Sep 19, 2012 at 10:18 PM, Vasily Tunegov <vasily.tune...@gmail.com>wrote: > Hi, Bert > > I understand what WAL mode is not suitable for svn. But my patch uses > PERSIST mode which give very good results, e.g. for NTFS on Windows as I > shown in tests. > > One problem, as says Philip Martin, is reduced performance on NFS disks. > But I don't understand why. The number of file operations in PERSIST mode > greatly reduced. Unfortunaly I don't have NFS disk, if it means Network > File System developed by Sun Microsystems. Windows is using SMB protocol to > access files via network. I will test SBM and put result here later. > > Best regards, > Vasily Tunegov. > > On Wed, Sep 19, 2012 at 7:30 PM, Bert Huijben <b...@vmoo.com> wrote: > >> systems like NTFS on Windows >> MIME-Version: 1.0 >> Content-Type: multipart/alternative; boundary=20cf3074b2a0fd54e504ca0ed837 >> >> --20cf3074b2a0fd54e504ca0ed837 >> Content-Type: text/plain; charset="utf-8" >> Content-Transfer-Encoding: 7bit >> >> For your own install you can enable the WAL on the file and other users >> of the db will use it. I don't think we can enable it as default (like >> you said) >> >> Bert Huijben (Cell phone) >> From: Vasily Tunegov >> Sent: 19-9-2012 7:30 >> To: Philip Martin >> Cc: dev@subversion.apache.org >> Subject: Re: Subject: [PATCH] WC rep cache optimization for some file >> systems like NTFS on Windows >> Hi, Philip >> >> I'm using SQLite 3.7.14. >> I tried to test checkout for our project code base, hosted on svn 1.7.6 >> server (93 984 Files, 3 204 Folders, 1.97GB). There are results: >> >> svn co snv://server/project/branches/branche src -q >> >> Standard disk: >> svn.r1387070 - 256.8 sec >> svn.patch - 152.2 sec >> >> My research shows that PERSIST journal mode greatly reducing the number of >> I/O operations, such as creating, opening and closing SQLite journal files >> on the disk. The best option here is WAL journal mode. But it works only >> if >> SQLite database is used from one computer (it use memory mapped files >> API). >> I don't know is this restriction is important to svn client. >> >> Best regards, >> Vasily Tunegov. >> >> On Wed, Sep 19, 2012 at 2:30 PM, Philip Martin >> <philip.mar...@wandisco.com>wrote: >> >> > Vasily Tunegov <vasily.tune...@gmail.com> writes: >> > >> > > Please review my patch (see attached file patch.txt). It improve svn >> > client >> > > performance on file systems like NTFS on Windows. Maybe on some other >> > file >> > > systems, e.g. NFS, but I didn't test it. This patch switch rep cache >> db >> > > journal mode from DELETE (by default) to PERSISTENT. For more info >> about >> > > journal modes in SQLite see >> > > http://www.sqlite.org/pragma.html#pragma_journal_mode. >> > > >> > > I have tested this patch on my desktop (Core i7-2600, 16Gb, Windows 7 >> x64 >> > > Enterprise) for two disk drives: standard and ssd. For tests I used >> > > Subversion Benchmark Tool, see >> > > >> > >> https://ctf.open.collab.net/sf/frs/do/listReleases/projects.csvn/frs.subversion_benchmark_tool >> > . >> > > For tested application I used svn 1.8.0 from trunk, revision 1387070. >> > > >> > > Subversion Benchmark Tool tests, total results: >> > > >> > > Standard disk: >> > > svn.r1387070 - 3:05.895 >> > > svn.patch - 2:06.548 >> > > >> > > SSD disk: >> > > svn.r1387070 - 2:11.504 >> > > svn.patch - 1:38.397 >> > >> > Which version of SQLite? >> > >> > On my Linux systems this gives a small improvement in checkout >> > performance for working copies on local disks, unfortunately it also >> > reduces checkout performance for working copies on NFS disks. >> > >> > I'm using SQLite 3.7.12.1 and 3.7.13. >> > >> > -- >> > Certified & Supported Apache Subversion Downloads: >> > http://www.wandisco.com/subversion/download >> > >> >> --20cf3074b2a0fd54e504ca0ed837 >> Content-Type: text/html; charset="utf-8" >> Content-Transfer-Encoding: quoted-printable >> >> <html><head><meta content=3D"text/html; charset=3Dutf-8" >> http-equiv=3D"Cont= >> ent-Type"></head><body><div><div style=3D"font-family: >> Calibri,sans-serif; = >> font-size: 11pt;">For your own install you can enable the WAL on the file >> a= >> nd other users of the db will use it. I don't think we can enable it as >> def= >> ault (like you said)<br><br>Bert Huijben (Cell >> phone)<br></div></div><hr><s= >> pan style=3D"font-family: Tahoma,sans-serif; font-size: 10pt; >> font-weight: = >> bold;">From: </span><span style=3D"font-family: Tahoma,sans-serif; >> font-siz= >> e: 10pt;">Vasily Tunegov</span><br><span style=3D"font-family: >> Tahoma,sans-= >> serif; font-size: 10pt; font-weight: bold;">Sent: </span><span >> style=3D"fon= >> t-family: Tahoma,sans-serif; font-size: 10pt;">19-9-2012 >> 7:30</span><br><sp= >> an style=3D"font-family: Tahoma,sans-serif; font-size: 10pt; font-weight: >> b= >> old;">To: </span><span style=3D"font-family: Tahoma,sans-serif; >> font-size: = >> 10pt;">Philip Martin</span><br><span style=3D"font-family: >> Tahoma,sans-seri= >> f; font-size: 10pt; font-weight: bold;">Cc: </span><span >> style=3D"font-fami= >> ly: Tahoma,sans-serif; font-size: 10pt;">dev@subversion.apache.org >> </span><b= >> r><span style=3D"font-family: Tahoma,sans-serif; font-size: 10pt; >> font-weig= >> ht: bold;">Subject: </span><span style=3D"font-family: Tahoma,sans-serif; >> f= >> ont-size: 10pt;">Re: Subject: [PATCH] WC rep cache optimization for some >> fi= >> le systems like NTFS on Windows</span><br><br></body></html><div >> style=3D"c= >> >> olor:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-c= >> olor:rgb(255,255,255)">Hi,=C2=A0Philip</div><div >> style=3D"color:rgb(34,34,3= >> >> 4);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255= >> ,255)"> >> <br></div><span >> style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;f= >> ont-size:13px;background-color:rgb(255,255,255)">I'm using SQLite >> 3.7.1= >> 4.=C2=A0</span><div >> style=3D"color:rgb(34,34,34);font-family:arial,sans-ser= >> if;font-size:13px;background-color:rgb(255,255,255)"> >> I tried to test checkout for our project code base, hosted on svn 1.7.6 >> ser= >> ver (93 984 Files, 3 204 Folders, 1.97GB). There are results:</div><div >> sty= >> >> le=3D"color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;backg= >> round-color:rgb(255,255,255)"> >> <br></div><div >> style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;fo= >> nt-size:13px;background-color:rgb(255,255,255)">svn co >> snv://server/project= >> /branches/branche src -q</div><div >> style=3D"color:rgb(34,34,34);font-family= >> :arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)"> >> <br></div><div >> style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;fo= >> nt-size:13px;background-color:rgb(255,255,255)">Standard >> disk:<br>svn.r1387= >> 070 - 256.8 sec<br>svn.patch - 152.2 sec<br><br>My research shows that >> PERS= >> IST journal mode greatly reducing the number of I/O operations, such as >> cre= >> ating, opening and closing SQLite journal files on the disk. The best >> optio= >> n here is WAL journal mode. But it works only if SQLite database is used >> fr= >> om one computer (it use memory mapped files API). I don't know is >> this = >> restriction is important to svn client.</div> >> <div >> style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13= >> px;background-color:rgb(255,255,255)"><br></div><div >> style=3D"color:rgb(34,= >> >> 34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255= >> ,255,255)"> >> Best regards,</div><div >> style=3D"color:rgb(34,34,34);font-family:arial,sans= >> -serif;font-size:13px;background-color:rgb(255,255,255)">Vasily >> Tunegov.</d= >> iv><br><div class=3D"gmail_quote">On Wed, Sep 19, 2012 at 2:30 PM, Philip >> M= >> artin <span dir=3D"ltr"><<a href=3D"mailto:philip.mar...@wandisco.com"; >> t= >> arget=3D"_blank">philip.mar...@wandisco.com</a>></span> wrote:<br> >> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 >> .8ex;border-left:1p= >> x #ccc solid;padding-left:1ex"><div class=3D"im">Vasily Tunegov <<a >> href= >> =3D"mailto:vasily.tune...@gmail.com";>vasily.tune...@gmail.com</a>> >> write= >> s:<br> >> >> <br> >> > Please review my patch (see attached file patch.txt). It improve svn >> c= >> lient<br> >> > performance on file systems like NTFS on Windows. Maybe on some >> other = >> file<br> >> > systems, e.g. NFS, but I didn't test it. This patch switch rep >> cac= >> he db<br> >> > journal mode from DELETE (by default) to PERSISTENT. For more info >> abo= >> ut<br> >> > journal modes in SQLite see<br> >> > <a href=3D"http://www.sqlite.org/pragma.html#pragma_journal_mode"; >> targ= >> et=3D"_blank">http://www.sqlite.org/pragma.html#pragma_journal_mode >> </a>.<br= >> > >> ><br> >> > I have tested this patch on my desktop (Core i7-2600, 16Gb, Windows >> 7 = >> x64<br> >> > Enterprise) for two disk drives: standard and ssd. For tests I >> used<br= >> > >> > Subversion Benchmark Tool, see<br> >> > <a href=3D" >> https://ctf.open.collab.net/sf/frs/do/listReleases/projects= >> .csvn/frs.subversion_benchmark_tool" target=3D"_blank"> >> https://ctf.open.col= >> lab.net/sf/frs/do/listReleases/projects.csvn/frs.subversion_benchmark_tool >> <= >> /a>.<br> >> >> > For tested application I used svn 1.8.0 from trunk, revision >> 1387070.<= >> br> >> ><br> >> > Subversion Benchmark Tool tests, total results:<br> >> ><br> >> > Standard disk:<br> >> > svn.r1387070 - 3:05.895<br> >> > svn.patch - 2:06.548<br> >> ><br> >> > SSD disk:<br> >> > svn.r1387070 - 2:11.504<br> >> > svn.patch - 1:38.397<br> >> <br> >> </div>Which version of SQLite?<br> >> <br> >> On my Linux systems this gives a small improvement in checkout<br> >> performance for working copies on local disks, unfortunately it also<br> >> reduces checkout performance for working copies on NFS disks.<br> >> <br> >> I'm using SQLite 3.7.12.1 and 3.7.13.<br> >> <span class=3D"HOEnZb"><font color=3D"#888888"><br> >> --<br> >> Certified & Supported Apache Subversion Downloads:<br> >> <a href=3D"http://www.wandisco.com/subversion/download"; >> target=3D"_blank">h= >> ttp://www.wandisco.com/subversion/download</a><br> >> </font></span></blockquote></div><br> >> >> --20cf3074b2a0fd54e504ca0ed837-- >> > >
