Tag: Big Data

Big Data

Categories
Cloud & Big Data

Continuous Delivery at Docker age

Continuous Delivery at Docker age

adrien-blind

Adrien is a former consultant from Octo Technology, a french IT consulting firm. He joined Societe Generale as a DevOps coach and Infrastructure as Code product owner, driving the developers new needs toward infrastructure and helping them to leverage on those new principles and tools. Fond of Docker, he’s also co-organizer of the Paris Docker Meetup.

Reduced time-to-market, calmer deployments or possible source of savings … Promises of continued deployment are numerous.

To achieve this, the IT department can rely on a wide range of practices such as Agile, collaboration and automation preached by the DevOps culture, or modern architecture patterns such as micro-services. All of this is also backed by an entire ecosystem of appropriate tools, among which we find particularly Docker.

In this session, I propose to explore a fully automated continuous deployment process, leveraging on Docker containers. I’ll also present some recommendations for for the application architecture, and discuss roles and responsibilities shifts, which often accompanies such an approach.

Categories
Cloud & Big Data

Galder Zamarreño interview

Q. You’re speaking at Voxxed Days Bucharest in March. Tell us a bit about your session.

The session is about how to use Java 8 Stream API to process and analyse data stored in distributed Infinispan data grid instances. The talk will start with a short presentation about Infinispan and Java 8 Stream API and then it will follow with a demonstration on how to combine both to do distributed data processing.

Q. Why is the subject matter important?

The Java 8 Stream API is a very popular API to get started with processing data in a single node environment. Infinispan has extended the Java 8 Stream API so that lambda functions are shipped to where data is located, rather than bringing all data to a single node, and hence it can take full advantage of the parallel data processing capabilities of multiple nodes.

Q. Who should attend your session?

This is a beginner talk for Java developers and architects interested in data processing and distributed computing. Previous knowledge of Java 8 Streams and Map/Reduce is desirable but not mandatory.

Q. What are the key things attendees will take away from your session?

The key message is that the Java 8 Stream API can be extended to process large data sets in a distributed environment, taking advantage of the parallelism offered by multi-node environments. The attendees will also learn about the differences of Java 8 Stream API and other data processing APIs exposed by Infinispan.

Q. Aside from speaking at Voxxed Days Bucharest, what else are you excited about for 2016?

I’m very excited about enhancements we’ll be doing to a new Functional Map API we designed for Infinispan 8 which takes advantage of lambdas functions to define how data is modified and manipulated. Being able to distributed the functions that lead to a particular state as opposed to distributed latest values is a very powerful concept, enabling strong eventual consistent data structures such as conflict-free replicated data type (CRDT).

I’m also very excited with the work we’ve been doing at Infinispan to integrate with other data processing projects at Apache, such as Hadoop, Spark, etc., so you can use Infinispan as an alternative backend for the data. These integrations expose Infinispan’s in-memory data grid capabilities to a new audience expanding our user base.

Categories
Cloud & Big Data

DuyHai Doan Interview

Q. You’re speaking at Voxxed Days Bucharest in March. Tell us a bit about your session.

My session is about Apache Zeppelin, an open-source web-based notebook for your Big Data (or Small Data) technologies.
In this talk, I will show all Zeppelin features, highlight its modular architecture and show you how to develop your own interpreter/plugin for your technology to work with Zeppelin, if it doesn’t already exist!

Q. Why is the subject matter important?

With the rise of Big Data, the need for a easy-to-use front-end is more and more a must-have. And Zeppelin is gaining momentum right now, being used a lot for live coding demo but also as a tool for Data Scientists working with Spark.

Q. Who should attend your session?

Anyone working in the Big Data space, developer or end-user (consumer) of Big Data projects

Q. What are the key things attendees will take away from your session?

a. Zeppelin offers a free front-end for your big data project
b. Zeppelin architecture is modular enough so that you can add your own interpreter/plugin
c. The community is growing and proposing an impressive list of plugins for all major Big Data technologies (Cassandra, ElasticSearch, Flink, …)

Q. Aside from speaking at Voxxed Days Bucharest, what else are you excited about for 2016?

I’m excited about new features to come in Apache Zeppelin that will make it the default choice front-end for many projects

Categories
Cloud & Big Data

Modular Infrastructure for IoT Applications

Modular Infrastructure for IoT Applications

Michal - Modular Infrastructure for IOT Applications

Michal joined IBM in 2013 as a Java Senior Developer, after earning over 5 years of experience in Java development, working across multiple technologies for companies with projects dedicated to transportation and entertainment industry. Currently he is working on Home Automation project for internationally leading provider of integral lighting solutions in Austria. He is a passionate engineer with a wide range of interests from electric circuits and microcontrollers to Internet of Things and cloud solutions.

Internet of Things (IoT) is a buzzword which has had everyone talking over the last couple of years. My presentation will demonstrate how to build a simply extensible and widely configurable gateway allowing protocols translation (HTTP with Apache Http Components, Serial and I2C with Pi4J and libbulldog). Compared with other existing products on the market, the proposed solution brings great performance even on multicore ARM processors using Fork/Join Framework, but also a simple manner to write and manage your own extensions via RESTfull API (by integrating Jetty-Jersey-Genson). The iGate [IotaGate] resource is not just one-purpose tool, but a reusable solution to integrate M2M and SCADA offerings with web portals or other tools such as IBM Bluemix, IBM Intelligent Operation for Transportation or custom complex solutions based on IoT data acquiring and control.

Categories
Workshops

Everybody likes Redis

Everybody likes Redis

liviu Everybody likes Redis

Liviu is currently leading the developing efforts of a company called Biz Pro Technologies. They are enthusiasts about combining SQL and NoSQL solutions and he had the privillege to introduce Redis to their stack which has been live for more than 1 year and a half now with no major incidents.

We are going to discover how Redis can help your current stack, from being a simple cache server and up to a real database. We are going to explore its more advanced features and see real examples of how to use them. It is not a silver bullet, but use it right and it can really speed things up.

Categories
Cloud & Big Data

ToroDB: NoSQL & relational Java database with SQL analytics

ToroDB: NoSQL & relational Java database with SQL analytics

alvaro ToroDB: NoSQL & relational Java database with SQL analytics

Álvaro Hernandez is a 36 year-old IT entrepreneur, based in Madrid, Spain. Founder and CTO at 8Kdata, a database R&D company, he spends most of his time working on the ToroDB project, the first NoSQL-on-SQL database, a MongoDB-compatible database that runs on top of PostgreSQL. He is a passionate software developer and open source advocate. Álvaro is a Java software developer, member of JavaSpecialists.eu, but also a DBA, trainer and frequent lecturer at international conferences. He also founded the PostgreSQL Spanish User Group, one of the largest PUGs in the world, with more than 500 members.

How do you cope with the rise of unstructured data? Do you have different systems for your NoSQL and your SQL databases? How do you perform analytics of NoSQL data? ToroDB is an open-source NoSQL & relational database written in Java. ToroDB transforms NoSQL documents to relational structures and stores them in relational databases like PostgreSQL. ToroDB speaks the MongoDB protocol, being compatible with MongoDB programs and tools. With ToroDB, you can use either SQL or the MongoDB API for queries. ToroDB automatically structures and partitions your NoSQL data. ToroDB also speaks the MongoDB protocol and can replicate live from it. And you can do analytics in pure SQL against Massively Parallel Databases like Greenplum or CitusDB. Join this talk to understand how ToroDB works, what advantages it may bring to your Big Data architecture, and how to hack ToroDB’s Java open source code.

 

Categories
Cloud & Big Data

Distributed Data Processing with Infinispan and Java Streams

Distributed Data Processing with Infinispan and Java Streams

zamareno Distributed Data Processing with Infinispan and Java Streams

Galder Zamarreño is a core R&D engineer at JBoss, a division of Red Hat. He is one of the founding engineers of Infinispan, Red Hat’s distributed, in-memory key-value store and he currently spends most of his time developing Infinispan’s Functional Map API as well as other data grid and caching functionality. He is very keen on functional programming and has been developing in Scala since 2009. Galder has previously worked with JBoss customers helping them build highly distributed and massively scalable Application Server clusters based on technologies such as JGroups and JBoss Cache. Prior to joining Red Hat, Galder worked in the Retail industry where he was a software developer involved in the development of an EFT software switch solution based on JBoss technologies. The love for distributed systems and open source software comes from his days at ESIDE faculty at University of Deusto (Bilbao, Spain) where he studied a master’s degree in Computer Science.

Infinispan is a distributed in-memory key/value data store capable accelerating data processing using Hadoop, Spark and home-grown Map/Reduce APIs. Starting with Infinispan 8, you can now also use the Java 8 Stream API to process, transform and analyse the data stored in the grid, without burdening the architecture with external platforms. Processing can be applied to keys and/or values and it uses Infinispan’s data partitioning logic to distribute operations to nodes where data lives so that they can be executed locally. In this talk you’ll learn about this new extension to Java 8’s Stream class to process data in Infinispan and how it compares with existing APIs.

Categories
Cloud & Big Data

Apache Zeppelin, the missing GUI for your Big Data back-end

Apache Zeppelin, the missing GUI for your Big Data back-end

Apache Zeppelin, the missing GUI for your Big Data back-end

DuyHai Doan is a Cassandra technical advocate. He spends his time between technical presentations/meetups on Cassandra, coding on open source projects to support the community and helping all companies using Cassandra to make their project successful. Previously he was working as a freelance Java/Cassandra consultant.

If you are interested in Big Data, you surely already know about Apache Spark or Apache Cassandra, but do you know about Apache Zeppelin? Do you know that it is possible to draw out beautiful graph using an user-friendly interface out of your Spark RDD and Cassandra queries? In this session, I will introduce Zeppelin by live coding example and highlight its modular architecture which allows you to plug-in any interpreter for the back-end of your choice (AsciiDoc, Cassandra, etc.).

Zeppelin is a young but very promising Apache project. Its modular and interpreter-based architecture makes it suitable for any kind of back-end, not just Spark (like Spark Notebooks for example). The community is vivid and a dozen of pull requests have been created to integrate interpreters for different technologies (Apache HBase, Apache Lens, Apache Ignite, etc.). The community is in a building phase and my talk aims at presenting Zeppelin to the developers and to extend Zeppelin community. The talk itself will be done using Zeppelin. Yes, it is possible to execute Markdown, AngularJS or just plain HTML code using Zeppelin directly.