Biggish Blue will allocate 3,500 of its researchers and developers to Spark-related projects and promised a Spark Technology Centre in San Francisco, California.

It will also open source, its SystemML family machine-learning libraries.

Spark was created by the University of California at Berkeley in 2009 and donated to Apache in 2013. It is written in Java, Scala and Python and is an in-memory system for processing large data sets.

Until last year, Spark had only 465 contributors but with Big Blue backing it, it is more likely to be taken very seriously.

Announcing its backing for Apache's Spark IBM said it was going to use it as a platform for data and analytics, the analogy being Linux – which IBM also contributes to – as a platform for apps.