NEC Hackathon: HPC (SX-Aurora TSUBASA)

To read more about Aurora, please click here.

SPARK + AI SUMMIT 2019

Accelerating Spark MLlib and DataFrame with Vector Processor "SX-Aurora TSUBASA"

Supercomputing 2018 (SC18) HandsOn Resources

Resources at Supercomputing 2018 hands-on about NEC SX-Aurora Tsubasa Vectorizing Compiler

Aurora Vectorization Training

Machine Learning Library:

The participants can use machine learning library called Frovedis which specially developed for SX-Aurura TSUBASA.

• URL for Frovedis

The top README.md tells you what Frovedis is, how to install it in your system.

• Tutorial of Python/Spark interface

There is Python/Spark interface for Frovedis functions.

The tutorial guides you to use the interface quickly. It includes how to set environmental variables before execution, how to write code with python/spark interface to perform matrix operation, machine learning, dataframe functionalities supported by Frovedis

Python

Spark

• Manual of Python/Spark API. We make python interface similar to scikit-learn, spark interface to spark MLlib. These are the API documentation which explains how to use classes, the member functions in detail.

Python

Spark

• Performance benchmark and tips to improve performance.

There are scripts to measure the time to run machine learning with Frovedis and scikit-learn. Also it has tips to improve the runtime performance. Click here