16 Feb 10 Walking the Talk: Cloud Transfer Tests

Last week, I bemoaned the issue of “proof” and data provenance behind scientific work and publication and, to that end, I’ve made an effort to change, at least in the way in which my work is performed.

To that end, I’ve posted the methodology to the tests I’m currently running and writing about here. If you are at all interested in the results (posted subsequently) please take a moment to read the methodology so that you can interpret the results with understanding. Further, in the interest of improving the work and resultant technology, I’m very interested in (constructive) critique of the methodology and tools (source code is available per the methodology link).

I’ve also posted the results of a few of the tests:

• Cloud Transfer Test: Consumer Connection 01: Tests file transfers of various sizes originating from a residential connection and targeting both Amazon’s S3 and Azure’s Blob storage.
• Cloud Transfer Test: Research Center Conn 01: Tests file transfers of various sizes originating from a research institution and targeting both Amazon’s S3 and Azure’s Blob storage
• Cloud Transfer Test: Research Center Conn 02: Tests file transfers of various sizes originating from a research institution and comparing Azure’s US North Central data center and Azure’s US South Central data center

These are introductory and represent the state prior to any optimizations in transfer approaches. It will be interesting to see how these values change as we experiment with different techniques.