New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

一个问题，就是Spark是不是能把所有传入作为参数的函数都分布式进行计算？对吗？ #42

Open

guotong1988 opened this issue Oct 11, 2015 · 1 comment

guotong1988 commented Oct 11, 2015

传入一个函数f，Spark会把数据分块，分到各个Worker节点，每个Worker节点都计算这个函数f，那么数据又如何汇总/归并？

Owner

JerryLead commented Oct 30, 2015

一些action()，比如collect(), count()之类会将每个worker节点上task的计算结果收集到driver端，并进行计算。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment