If the replication traffic is responsible for this raising network traffic for only 1.200.000 objects, how much traffic I can expect if I have 100.000.000 objects stored? The average size of my mailobjects are 120 kB. It's planned to use all 12 hard drive slots of my DELL R720xd with 4 TB drives. I have 5 storage nodes and 2 balanced proxy nodes. Will the replication traffic kill my system?
Here is a small part of my object-server-replicator log: Jul 9 06:48:16 storage-node1 object-replicator Starting object replication pass. Jul 9 06:49:13 storage-node1 object-replicator 9830/9830 (100.00%) partitions replicated in 57.56s (170.78/sec, 0s remaining) Jul 9 06:49:13 storage-node1 object-replicator 1234597 suffixes checked - 0.00% hashed, 0.00% synced Jul 9 06:49:13 storage-node1 object-replicator Partition times: max 0.0279s, min 0.0068s, med 0.0104s Jul 9 06:49:13 storage-node1 object-replicator Object replication complete. (0.96 minutes) Jul 9 06:49:43 storage-node1 object-replicator Starting object replication pass. Jul 9 06:50:41 storage-node1 object-replicator 9830/9830 (100.00%) partitions replicated in 57.69s (170.39/sec, 0s remaining) Jul 9 06:50:41 storage-node1 object-replicator 1234643 suffixes checked - 0.00% hashed, 0.00% synced Jul 9 06:50:41 storage-node1 object-replicator Partition times: max 0.0365s, min 0.0068s, med 0.0104s Jul 9 06:50:41 storage-node1 object-replicator Object replication complete. (0.96 minutes) Jul 9 06:51:11 storage-node1 object-replicator Starting object replication pass. Jul 9 06:52:09 storage-node1 object-replicator 9830/9830 (100.00%) partitions replicated in 58.31s (168.58/sec, 0s remaining) Jul 9 06:52:09 storage-node1 object-replicator 1234688 suffixes checked - 0.00% hashed, 0.00% synced Jul 9 06:52:09 storage-node1 object-replicator Partition times: max 0.0348s, min 0.0069s, med 0.0106s Jul 9 06:52:09 storage-node1 object-replicator Object replication complete. (0.97 minutes) -----Ursprüngliche Nachricht----- Von: Pete Zaitcev [mailto:zait...@redhat.com] Gesendet: Montag, 8. Juli 2013 19:22 An: Klaus Schürmann Cc: openstack@lists.launchpad.net Betreff: Re: [Openstack] [SWIFT] raising network traffic o the storage node On Mon, 8 Jul 2013 13:50:38 +0000 Klaus Schürmann <klaus.schuerm...@mediabeam.com> wrote: > I use a swift storage as a mail-store. Now I have about 1.000.000 objects > stored in the cluster. > Traffic Storagenode: http://www.schuermann.net/temp/storagenode.png > Traffic Proxyserver: http://www.schuermann.net/temp/proxyserver.png > > Can someone explain such behavior? At a guess, raising number of objects makes number of partitions to increase, and that increases the replication traffic, specifically the number of sent MD5s for partitions. It would be interesting to corellate the number of objects and number of non-empty and empty partitions with the amounts of traffic. If the increasing transfer of hashes is the reason, you could also verify by graphing the traffic to port 873 separately. Swift never replicates object bodies through its own HTTP interface, so this splits control traffic from data traffic for you. Data traffic should be driven by customer and node failures, not consistency checking. Be prepared to split up storage nodes, however. Even if there's no bug with replication, its aggregate traffic increases with the increases in object counts. -- Pete _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
