Skip to Main content Skip to Navigation
Conference papers

JetStream: Enabling High Performance Event Streaming across Cloud Data-Centers

Abstract : The easily-accessible computation power offered by cloud infrastructures coupled with the revolution of Big Data are expanding the scale and speed at which data analysis is per-formed. In their quest for finding the Value in the 3 Vs of Big Data, applications process larger data sets, within and across clouds. Enabling fast data transfers across geograph-ically distributed sites becomes particularly important for applications which manage continuous streams of events in real time. Scientific applications (e.g. the Ocean Observa-tory Initiative or the ATLAS experiment) as well as com-mercial ones (e.g. Microsoft's Bing and Office 365 large-scale services) operate on tens of data-centers around the globe and follow similar patterns: they aggregate monitoring data, assess the QoS or run global data mining queries based on in-ter site event stream processing. In this paper, we propose a set of strategies for efficient transfers of events between cloud data-centers and we introduce JetStream: a prototype implementing these strategies as a high performance batch-based streaming middleware. JetStream is able to self-adapt to the streaming conditions by modeling and monitoring a set of context parameters. It further aggregates the available bandwidth by enabling multi-route streaming across cloud sites. The prototype was validated on tens of nodes from US and Europe data-centers of the Windows Azure cloud using synthetic benchmarks and with application code from the context of the Alice experiment at CERN. The results show an increase in transfer rate of 250 times over individual event streaming. Besides, introducing an adaptive transfer strategy brings an additional 25% gain. Finally, the transfer rate can further be tripled thanks to the use of multi-route streaming.
Document type :
Conference papers
Complete list of metadata

Cited literature [40 references]  Display  Hide  Download
Contributor : Radu Tudoran Connect in order to contact the contributor
Submitted on : Friday, December 19, 2014 - 4:58:38 PM
Last modification on : Thursday, January 20, 2022 - 5:33:32 PM
Long-term archiving on: : Saturday, April 15, 2017 - 1:49:53 AM


DEBS2014 Camera Ready.pdf
Files produced by the author(s)



Radu Tudoran, Olivier Nano, Ivo Santos, Alexandru Costan, Hakan Soncu, et al.. JetStream: Enabling High Performance Event Streaming across Cloud Data-Centers. DEBS'14 - The 8th ACM International Conference on Distributed Event-Based Systems, May 2014, Mumbai, India. pp.23 - 34, ⟨10.1145/2611286.2611298⟩. ⟨hal-01090281⟩



Record views


Files downloads