Why not merge https://go-review.googlesource.com/c/go/+/51670 <https://go-review.googlesource.com/c/go/+/51670> since 2017? I tell you the truth: Workers of commercial companies dislike the community pull request, this will wast their free time or vacation time. They only worked for capitalists.
> On Mar 11, 2020, at 20:58, 'Benjamin' via golang-nuts > <golang-nuts@googlegroups.com> wrote: > > I think that’s why go should be driven by the community but not by Bell labs > nor Google labs. > Works in Bell Labs or Google Labs may leave the job, and may not know > everything. > But the community have enough Human Resources and various of knowledges. > > Some scientists and reaserchers worked for commercial company for whole of > their life, but never do a things owned by themselves. Like Anders Hejlsberg, > the architect of Delphi and C#. What a pity. > > I think go programming language should also belongs to the community but not > Google. One day Bell Lab or Google are closed, the community still exists. > > >> On Mar 11, 2020, at 20:43, Nicola Murino <nicola.mur...@gmail.com >> <mailto:nicola.mur...@gmail.com>> wrote: >> >> Hi all, >> >> I want to share the performances analysis I recently did for SFTPGo, the >> fully featured and highly configurable SFTP server written in Go >> (https://github.com/drakkan/sftpgo <https://github.com/drakkan/sftpgo>). >> >> When I decided to write an SFTP server I evaluated the available libraries >> and I did some quick performance tests too. >> >> Golang's SSH stack and pkg/sftp were able to easily saturate a Gigabit >> connection and this seemed enough. I noticed that OpenSSH was faster, but I >> didn't investigate further. >> >> So I chose Golang for SFTPGo. >> >> The project is growing fast and one of the users also noticed that SFTPGo >> has lower performance than OpenSSH. He opened an issue providing some stats >> when using a 40Gb Ethernet card. >> >> I did some more profiling and discovered that the main bottlenecks are, >> unsurprisingly, the cipher used and the message authentication. So we can >> have a huge performance boost using a fast cipher with implicit messages >> authentication, for example aes128-...@openssh.com >> <mailto:aes128-...@openssh.com>, however this cipher is not widely supported. >> >> The most used cipher, and the one used in the user's tests is AES-CTR, and >> the Golang implementations seems quite slow. >> >> He noticed that an unmerged patch is available for Golang, greatly improving >> AES-CTR performance: >> >> https://go-review.googlesource.com/c/go/+/51670 >> <https://go-review.googlesource.com/c/go/+/51670> >> >> I applied this patch and, while performance improved, the AES-CTR SFTP >> transfers were still slower than the AES-GCM ones. The bottleneck is now the >> MAC computation. >> >> The tested hardware supports Intel SHA extensions but Golang's SHA256 >> implementation only uses the AVX2 extension. >> >> Again, I was lucky: I can simply use minio/sha256-simd as a drop-in >> replacement for Golang's SHA256 implementation: >> >> https://github.com/drakkan/crypto/commit/17ef3d252b1c9d6124fa17889033e11eaa5c9ddb >> >> <https://github.com/drakkan/crypto/commit/17ef3d252b1c9d6124fa17889033e11eaa5c9ddb> >> >> The performance improved again, but OpenSSH was still faster. >> >> To my great surprise I noticed that my very simple SCP implementation >> (https://github.com/drakkan/sftpgo/blob/master/sftpd/scp.go >> <https://github.com/drakkan/sftpgo/blob/master/sftpd/scp.go>) was now as >> fast as OpenSSH! >> >> So this time I have to look at pkg/sftp: I found some extraneous >> copies/allocations in critical paths and I sent some pull requests that are >> now merged in the master branch. Still, SFTP transfers were slower than >> OpenSSH ones. >> >> Compared to my SCP implementation the main difference is that pkg/sftp >> allocates a new slice for each SFTP packet, while my SCP implementation >> allocates a slice once and then reuses this slice. >> >> Basically for each SFTP packet pkg/sftp does something like this: >> >> data := make([]byte, size) >> >> So I wrote a proof of concept allocator that tries to avoid all these extra >> allocations reusing the previously allocated slices: >> >> https://github.com/drakkan/sftp/blob/master/allocator.go >> <https://github.com/drakkan/sftp/blob/master/allocator.go> >> >> And bingo! Now SFTPGo performance is very close to OpenSSH! You can find the >> full benchmark results here: >> >> https://github.com/drakkan/sftpgo/blob/master/docs/performance.md >> <https://github.com/drakkan/sftpgo/blob/master/docs/performance.md> >> >> Conclusion: I see several complaints about Go performance, especially >> compared to Rust, but, at least in my use case, Go can be as fast as a C >> project (such as OpenSSH). But some special attention is required and thus >> this improved performance is not by default available to all the users. >> >> Now some questions: >> >> 1) for the pkg/sftp allocator I'm discussing with pkg/sftp maintainers to >> find a way to get it included. Do you have smarter ideas to avoid these >> allocations? >> 2) There is a patch available for AES-CTR in Golang (since 2017): I wonder >> why it is not yet merged? >> 3) To improve SHA computation performance, I cannot find anything usable in >> Golang itself. Is there any plan to have support for Intel SHA Extensions >> and AVX512 directly in Golang anytime soon? >> >> Thank you for this great programming language, it makes it really simple to >> add new features to SFTPGo! >> >> regards, >> Nicola >> >> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golang-nuts+unsubscr...@googlegroups.com >> <mailto:golang-nuts+unsubscr...@googlegroups.com>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/7cfdd60d-7415-4c2f-bd7d-9d0372461a3c%40googlegroups.com >> >> <https://groups.google.com/d/msgid/golang-nuts/7cfdd60d-7415-4c2f-bd7d-9d0372461a3c%40googlegroups.com?utm_medium=email&utm_source=footer>. > > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com > <mailto:golang-nuts+unsubscr...@googlegroups.com>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/A06728A5-EBC1-4B2A-B5A0-D2DE526ACE30%40icloud.com > > <https://groups.google.com/d/msgid/golang-nuts/A06728A5-EBC1-4B2A-B5A0-D2DE526ACE30%40icloud.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/FE66D0AE-EE2F-4DEF-9022-9E5FD4CFC2E5%40icloud.com.
