Extending DevOps to Big Data Applications with Kubernetes

Extending DevOps to Big Data Applications with Kubernetes

Nicola Ferraro

Nicola Ferraro is a senior software engineer at Red Hat. I’m an Apache Camel committer and a contributor of the Fabric8 project, a microservice development platform based on Docker and Kubernetes. I’ve a long background on Big Data systems, having built applications based on Spark, HBase, Kafka and Hadoop for years. I’m also the author of some open source projects related to Big Data application development.

DevOps, continuous delivery and modern architectural trends can incredibly speed up the software development process. Big Data applications cannot be an exception and need to keep the same pace. In this talk, Nicola Ferraro (Red Hat) will show an overview of the next generation cloud-native Big Data systems based on Docker and Kubernetes. Switching from static Big Data platforms to infrastructure as code allows releasing robust applications in minutes by minimizing the gap between all the environments (development, testing, staging and production), a typical problem in Big Data application development. The presentation goes through the full release cycle of a Spark application alongside a fleet of microservices exchanging data in near real time. A fully featured continuous delivery pipeline, provided by the Fabric8 platform, will automate every step of the build and release process. Spark clusters will come to life from nowhere and disappear when they are no longer needed…

Leave A Comment

Your email address will not be published. Required fields are marked *

back to top