{"id":2913,"date":"2017-01-31T14:17:10","date_gmt":"2017-01-31T12:17:10","guid":{"rendered":"https:\/\/voxxeddays.com\/bucharest\/?p=2913"},"modified":"2017-04-12T16:43:42","modified_gmt":"2017-04-12T14:43:42","slug":"big-data-streaming-recipe","status":"publish","type":"post","link":"https:\/\/voxxeddays.com\/greece\/2017\/01\/31\/big-data-streaming-recipe\/","title":{"rendered":"A Big Data Streaming Recipe"},"content":{"rendered":"<h2>A Big Data Streaming Recipe<\/h2>\n<p><img loading=\"lazy\" class=\"image-medalion-post alignright wp-image-1971\" src=\"\/\/incremental.community\/wp-content\/uploads\/2017\/01\/konstantin-gregor.png\" alt=\"Konstantin Gregor\" width=\"130\" height=\"130\" \/><\/p>\n<blockquote>\n<p class=\"p1\"><span class=\"s1\">I am <a href=\"https:\/\/www.linkedin.com\/in\/kgregor\" target=\"_blank\">Konstantin Gregor<\/a>, a developer and software consultant with a background in mathematics and machine learning. Since two years, I work for TNG Technology Consulting in Munich, Germany, where I help our clients develop big data applications with a main focus on real time streaming applications and I always enjoy sharing my knowledge of this awesome field of IT with other developers.<\/span><\/p>\n<\/blockquote>\n<p>There is a lot to consider when setting up a big data streaming application: How much data will we need to handle? How important are \u201creal time\u201d results? What about constraints on data quality? And how can we deal with various failure scenarios? The open-source world offers numerous big data frameworks that can help process unbounded data, each with its own mechanisms to tackle these problems. I want to show you these frameworks and explain their mechanisms in order to give you some insights on which ingredients you should add to build a big data streaming application that suits your needs.<\/p>\n<p><a href=\"https:\/\/youtu.be\/60kTLG37m5c\">CLICK HERE is the video recording<\/a><\/p>\n<p><strong>Video 23:49 Erratum:<\/strong> Spilling does not happen in the Flink DataStream-API. However you can handle state that does not fit in memory with the RocksDB state backend<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A Big Data Streaming Recipe I am Konstantin Gregor, a developer and software consultant with a background in mathematics and machine learning. Since two years, I work for TNG Technology Consulting in Munich, Germany, where I help our clients develop big data applications with a main focus on real time streaming applications and I always [&hellip;]<\/p>\n","protected":false},"author":7816,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"video","meta":[],"categories":[1],"tags":[],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v15.3 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>A Big Data Streaming Recipe - Voxxed Days Greece<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/voxxeddays.com\/greece\/2017\/01\/31\/big-data-streaming-recipe\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"A Big Data Streaming Recipe - Voxxed Days Greece\" \/>\n<meta property=\"og:description\" content=\"A Big Data Streaming Recipe I am Konstantin Gregor, a developer and software consultant with a background in mathematics and machine learning. Since two years, I work for TNG Technology Consulting in Munich, Germany, where I help our clients develop big data applications with a main focus on real time streaming applications and I always [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/voxxeddays.com\/greece\/2017\/01\/31\/big-data-streaming-recipe\/\" \/>\n<meta property=\"og:site_name\" content=\"Voxxed Days Greece\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/voxxedbucharest\/\" \/>\n<meta property=\"article:published_time\" content=\"2017-01-31T12:17:10+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2017-04-12T14:43:42+00:00\" \/>\n<meta property=\"og:image\" content=\"\/\/incremental.community\/wp-content\/uploads\/2017\/01\/konstantin-gregor.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@VoxxedBucharest\" \/>\n<meta name=\"twitter:site\" content=\"@VoxxedBucharest\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\">\n\t<meta name=\"twitter:data1\" content=\"alexproca\">\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\">\n\t<meta name=\"twitter:data2\" content=\"1 minute\">\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"https:\/\/voxxeddays.com\/greece\/#website\",\"url\":\"https:\/\/voxxeddays.com\/greece\/\",\"name\":\"Voxxed Days Greece\",\"description\":\"Another Voxxed event\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":\"https:\/\/voxxeddays.com\/greece\/?s={search_term_string}\",\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/voxxeddays.com\/greece\/2017\/01\/31\/big-data-streaming-recipe\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"\/\/incremental.community\/wp-content\/uploads\/2017\/01\/konstantin-gregor.png\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/voxxeddays.com\/greece\/2017\/01\/31\/big-data-streaming-recipe\/#webpage\",\"url\":\"https:\/\/voxxeddays.com\/greece\/2017\/01\/31\/big-data-streaming-recipe\/\",\"name\":\"A Big Data Streaming Recipe - Voxxed Days Greece\",\"isPartOf\":{\"@id\":\"https:\/\/voxxeddays.com\/greece\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/voxxeddays.com\/greece\/2017\/01\/31\/big-data-streaming-recipe\/#primaryimage\"},\"datePublished\":\"2017-01-31T12:17:10+00:00\",\"dateModified\":\"2017-04-12T14:43:42+00:00\",\"author\":{\"@id\":\"https:\/\/voxxeddays.com\/greece\/#\/schema\/person\/3610fff738a03b155237ab4707e803eb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/voxxeddays.com\/greece\/2017\/01\/31\/big-data-streaming-recipe\/\"]}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/voxxeddays.com\/greece\/#\/schema\/person\/3610fff738a03b155237ab4707e803eb\",\"name\":\"alexproca\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/voxxeddays.com\/greece\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/fb9bbcf4c88be28b95644b3697985fcb?s=96&d=mm&r=g\",\"caption\":\"alexproca\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","_links":{"self":[{"href":"https:\/\/voxxeddays.com\/greece\/wp-json\/wp\/v2\/posts\/2913"}],"collection":[{"href":"https:\/\/voxxeddays.com\/greece\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/voxxeddays.com\/greece\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/voxxeddays.com\/greece\/wp-json\/wp\/v2\/users\/7816"}],"replies":[{"embeddable":true,"href":"https:\/\/voxxeddays.com\/greece\/wp-json\/wp\/v2\/comments?post=2913"}],"version-history":[{"count":0,"href":"https:\/\/voxxeddays.com\/greece\/wp-json\/wp\/v2\/posts\/2913\/revisions"}],"wp:attachment":[{"href":"https:\/\/voxxeddays.com\/greece\/wp-json\/wp\/v2\/media?parent=2913"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/voxxeddays.com\/greece\/wp-json\/wp\/v2\/categories?post=2913"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/voxxeddays.com\/greece\/wp-json\/wp\/v2\/tags?post=2913"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}