{"id":1062,"date":"2018-06-14T16:11:44","date_gmt":"2018-06-14T14:11:44","guid":{"rendered":"https:\/\/voxxeddays.com\/microservices2019\/?p=1062"},"modified":"2018-06-14T16:11:44","modified_gmt":"2018-06-14T14:11:44","slug":"gunnar-morling-on-data-streaming-with-debezium","status":"publish","type":"post","link":"https:\/\/voxxeddays.com\/microservices\/2018\/06\/14\/gunnar-morling-on-data-streaming-with-debezium\/","title":{"rendered":"Gunnar Morling on Data Streaming with Debezium"},"content":{"rendered":"<h3>Hi Gunnar, tell us who you are and what lead you into microservices?<\/h3>\n<p>I\u2019m an open-source enthusiast and software engineer working at Red Hat, where I\u2019m leading the Debezium project. Debezium is a platform for change data capture (CDC) based on Apache Kafka, allowing you to react to all the inserts, updates and deletes in your database in near real-time. That\u2019s a very powerful tool in the box when working with microservice architectures, e.g. for propagating data between different services or when extracting new microservices from existing monolithic applications.<\/p>\n<p>Before working on Debezium, I was the spec lead for the Bean Validation 2.0 spec (JSR 380) and contributed towards the MicroProfile Config API. Both are very useful APIs for implementing microservices; after all, you should make sure the data you receive from other services is valid and also there\u2019s plenty of need for configuring things like service endpoints etc. Overall, microservices relate to many parts of my work.<\/p>\n<h3>What will you be talking about at Voxxed Days Microservices?<\/h3>\n<p>I\u2019ll talk about streaming data changes out of your database (e.g. MySQL, Postgres or MongoDB) with Debezium. We\u2019ll discuss different use cases for CDC &#8211; e.g. synchronizing data between microservices, but also updating caches, full-text search indexes and others &#8211; and how it can be implemented using Debezium and Kafka. In my demo, I\u2019ll show how to set up Debezium, stream change events out of MySQL and push life updates to a web application via WebSockets. That is, if the demo gods are in my favor \ud83d\ude42<\/p>\n<h3>Debezium implements the change data capture pattern, right? Is it the only pattern we can use in a microservice architecture to synchronize data?<\/h3>\n<p>It\u2019s not the only one, but certainly one I recommend very much. E.g. you could consider doing (synchronous) REST calls from one service to another, but this easily leads to a high coupling between services. What to do for instance if the called service isn\u2019t available? Should you buffer and retry the requests? Fail the original request of the invoking system? These kinds of issues can be prevented by setting up change data streams to which interested services can subscribe, allowing them to keep their own local (and optimized) view of other services data.<\/p>\n<h3>Good, see you soon then<\/h3>\n<p>Thanks; I\u2019m really looking forward to the inaugural Voxxed Days Microservices and am super-excited to be part of it!<\/p>\n<p><strong>#dataStreaming #changeDataCapture #dataPropagation<\/strong><\/p>\n<p>Twitter: <a href=\"https:\/\/twitter.com\/gunnarmorling\">https:\/\/twitter.com\/gunnarmorling<\/a><br \/>\nBlog: <a href=\"http:\/\/in.relation.to\/gunnar-morling\">http:\/\/in.relation.to\/gunnar-morling<\/a><br \/>\nGitHub: <a href=\"http:\/\/github.com\/gunnarmorling\">http:\/\/github.com\/gunnarmorling<\/a><\/p>\n<h3><a href=\"https:\/\/vxdms2018.confinabox.com\/talks\/conf\">Check the other conferences<\/a><\/h3>\n","protected":false},"excerpt":{"rendered":"<p>Hi Gunnar, tell us who you are and what lead you into microservices? I\u2019m an open-source enthusiast and software engineer working at Red Hat, where I\u2019m leading the Debezium project. Debezium is a platform for change data capture (CDC) based on Apache Kafka, allowing you to react to all the inserts, updates and deletes in [&hellip;]<\/p>\n","protected":false},"author":1301,"featured_media":1063,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[12],"tags":[24,25,23],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v15.3 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Gunnar Morling on Data Streaming with Debezium : Voxxed Days Microservices 2019<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/voxxeddays.com\/microservices\/2018\/06\/14\/gunnar-morling-on-data-streaming-with-debezium\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Gunnar Morling on Data Streaming with Debezium : Voxxed Days Microservices 2019\" \/>\n<meta property=\"og:description\" content=\"Hi Gunnar, tell us who you are and what lead you into microservices? I\u2019m an open-source enthusiast and software engineer working at Red Hat, where I\u2019m leading the Debezium project. Debezium is a platform for change data capture (CDC) based on Apache Kafka, allowing you to react to all the inserts, updates and deletes in [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/voxxeddays.com\/microservices\/2018\/06\/14\/gunnar-morling-on-data-streaming-with-debezium\/\" \/>\n<meta property=\"og:site_name\" content=\"Voxxed Days Microservices 2019\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/vxdmicroservice\/\" \/>\n<meta property=\"article:published_time\" content=\"2018-06-14T14:11:44+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/voxxeddays.com\/microservices\/wp-content\/uploads\/sites\/54\/2018\/05\/GunnarMorlingBanner.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"513\" \/>\n<meta property=\"fb:app_id\" content=\"vxdmicroservice\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@agoncal\" \/>\n<meta name=\"twitter:site\" content=\"@vxdmicroservice\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\">\n\t<meta name=\"twitter:data1\" content=\"Antonio Goncalves\">\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\">\n\t<meta name=\"twitter:data2\" content=\"2 minutes\">\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https:\/\/voxxeddays.com\/microservices\/#organization\",\"name\":\"Voxxed Days Microservices\",\"url\":\"https:\/\/voxxeddays.com\/microservices\/\",\"sameAs\":[\"https:\/\/www.facebook.com\/vxdmicroservice\/\",\"https:\/\/www.instagram.com\/voxxeddaysmicroservices\",\"https:\/\/www.linkedin.com\/in\/voxxed-days-microservices-906115164\/\",\"https:\/\/www.youtube.com\/playlist?list=PLRsbF2sD7JVrhlYIKTI--IU6xcfUjdNt2\",\"https:\/\/twitter.com\/vxdmicroservice\"],\"logo\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/voxxeddays.com\/microservices\/#logo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/voxxeddays.com\/microservices\/wp-content\/uploads\/sites\/54\/2019\/06\/large-Voxxed_Days_Microservices_date_2019-01-2.png\",\"width\":1000,\"height\":387,\"caption\":\"Voxxed Days Microservices\"},\"image\":{\"@id\":\"https:\/\/voxxeddays.com\/microservices\/#logo\"}},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/voxxeddays.com\/microservices\/#website\",\"url\":\"https:\/\/voxxeddays.com\/microservices\/\",\"name\":\"Voxxed Days Microservices 2019, Conference & workshops\",\"description\":\"A Developer Conference on Microservices\",\"publisher\":{\"@id\":\"https:\/\/voxxeddays.com\/microservices\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":\"https:\/\/voxxeddays.com\/microservices\/?s={search_term_string}\",\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/voxxeddays.com\/microservices\/2018\/06\/14\/gunnar-morling-on-data-streaming-with-debezium\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/voxxeddays.com\/microservices\/wp-content\/uploads\/sites\/54\/2018\/05\/GunnarMorlingBanner.png\",\"width\":1024,\"height\":513,\"caption\":\"Gunnar Morling\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/voxxeddays.com\/microservices\/2018\/06\/14\/gunnar-morling-on-data-streaming-with-debezium\/#webpage\",\"url\":\"https:\/\/voxxeddays.com\/microservices\/2018\/06\/14\/gunnar-morling-on-data-streaming-with-debezium\/\",\"name\":\"Gunnar Morling on Data Streaming with Debezium : Voxxed Days Microservices 2019\",\"isPartOf\":{\"@id\":\"https:\/\/voxxeddays.com\/microservices\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/voxxeddays.com\/microservices\/2018\/06\/14\/gunnar-morling-on-data-streaming-with-debezium\/#primaryimage\"},\"datePublished\":\"2018-06-14T14:11:44+00:00\",\"dateModified\":\"2018-06-14T14:11:44+00:00\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/voxxeddays.com\/microservices\/2018\/06\/14\/gunnar-morling-on-data-streaming-with-debezium\/\"]}]},{\"@type\":\"Article\",\"@id\":\"https:\/\/voxxeddays.com\/microservices\/2018\/06\/14\/gunnar-morling-on-data-streaming-with-debezium\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/voxxeddays.com\/microservices\/2018\/06\/14\/gunnar-morling-on-data-streaming-with-debezium\/#webpage\"},\"author\":{\"@id\":\"https:\/\/voxxeddays.com\/microservices\/#\/schema\/person\/871a4bfea3b1bf117a77bc3fbfeb5f14\"},\"headline\":\"Gunnar Morling on Data Streaming with Debezium\",\"datePublished\":\"2018-06-14T14:11:44+00:00\",\"dateModified\":\"2018-06-14T14:11:44+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/voxxeddays.com\/microservices\/2018\/06\/14\/gunnar-morling-on-data-streaming-with-debezium\/#webpage\"},\"publisher\":{\"@id\":\"https:\/\/voxxeddays.com\/microservices\/#organization\"},\"image\":{\"@id\":\"https:\/\/voxxeddays.com\/microservices\/2018\/06\/14\/gunnar-morling-on-data-streaming-with-debezium\/#primaryimage\"},\"keywords\":\"changeDataCapture,dataPropagation,dataStreaming\",\"articleSection\":\"all\",\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/voxxeddays.com\/microservices\/#\/schema\/person\/871a4bfea3b1bf117a77bc3fbfeb5f14\",\"name\":\"Antonio Goncalves\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/voxxeddays.com\/microservices\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/1fc6aa04011b2f0f9721df913b0fd415?s=96&d=mm&r=g\",\"caption\":\"Antonio Goncalves\"},\"description\":\"I am a senior software architect living in Paris. I work as a consultant in helping customers to set up their software architecture. My specialities are Java \/ Java EE and Open Source frameworks that evolve around it. As a former BEA consultant I have a great expertise in application servers (Weblogic, JBoss and GlassFish). So I decided to write a few books about it, create the Paris JUG and Devoxx France.\",\"sameAs\":[\"https:\/\/plus.google.com\/101641663893160877351\",\"https:\/\/twitter.com\/agoncal\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","_links":{"self":[{"href":"https:\/\/voxxeddays.com\/microservices\/wp-json\/wp\/v2\/posts\/1062"}],"collection":[{"href":"https:\/\/voxxeddays.com\/microservices\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/voxxeddays.com\/microservices\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/voxxeddays.com\/microservices\/wp-json\/wp\/v2\/users\/1301"}],"replies":[{"embeddable":true,"href":"https:\/\/voxxeddays.com\/microservices\/wp-json\/wp\/v2\/comments?post=1062"}],"version-history":[{"count":0,"href":"https:\/\/voxxeddays.com\/microservices\/wp-json\/wp\/v2\/posts\/1062\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/voxxeddays.com\/microservices\/wp-json\/wp\/v2\/media\/1063"}],"wp:attachment":[{"href":"https:\/\/voxxeddays.com\/microservices\/wp-json\/wp\/v2\/media?parent=1062"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/voxxeddays.com\/microservices\/wp-json\/wp\/v2\/categories?post=1062"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/voxxeddays.com\/microservices\/wp-json\/wp\/v2\/tags?post=1062"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}