Yuhao Yang Interview
summary
A library called BigDL runs deep learning on Spark clusters using Scala to give JVM shops model training and inference without moving to Python frameworks. It aims for feature parity with TensorFlow and Caffe, scales out natively on Spark, and reports network transfer of large models and unstable or slow clusters as the main bottleneck. The implementation targets production use and claims practical success across fraud detection, recommendation, speech and vision workloads.
AI-written summary · about
chapters (4)
- 0:00Start
- 1:23What are some scaling issues you deal with?
- 1:48What interesting scalability issues do you see in the future?
- 2:54What do you love about the scale by the bay conference?
related talks
40mScala and Spark in the New Deep Learning Era
39mMachine Learning with Scala on Spark
06mUnleashing intelligence and data analytics at scale
29mAchieve real-time inference at scale on AWS with Deep Java Library
17mMachine Learning at the Speed of Light with Spark
51mDeep Learning with DL4J and Apache Spark?
35mJVM based DeepLearing on IoT data with Apache Spark
04mThe machine-learning renaissance