Apache® Spark™ News

Databricks and IBM Collaborate to Enhance Apache Spark Machine Learning

At today’s Spark Summit, Databricks and IBM announced a joint effort to contribute key machine learning capabilities to the Apache Spark Project.  Over the course of the next few months, Databricks and IBM will collaborate to expand Spark’s machine learning capabilities. The companies plan to introduce new domain specific algorithms to the Spark ecosystem and add new machine learning primitives in the Apache Spark Project. IBM and Databricks will also collaborate to integrate IBM’s SystemML – a robust machine-learning engine for large-scale data, with the Spark platform.