java - 为什么 map 变换是窄的？

Question

我知道 RDD 的窄转换和宽转换之间的区别。我的问题是证明 map 、平面图为窄的参数是什么？为什么这些转变不广泛？对 RDD 学习 Material 的任何建议表示赞赏。

Answer 1

Here is one explanation

– Narrow dependency: RDD operations like map, union, filter can operate on a single partition and map the data of that partition to resulting single partition. These kind of operations which maps data from one to one partition are referred as Narrow operations. Narrow operations doesn’t required to distribute the data across the partitions.

– Wide dependency: RDD operations like groupByKey, distinct, join may require to map the data across the partitions in new RDD. These kind of operations which maps data from one to many partitions are referred as Wide operations

另一种思考方式是这样的。子 RDD 的任何行都将仅依赖于父 RDD 的一行。由于每个子行都可以指向它所依赖的 1 个父行，因此存在窄依赖性。