google-cloud-dataflow - 使用 DirectRunner 测试数据流并获得大量 verifyUnmodifiedThrowingCheckedExceptions

Question

我正在使用 Mac 上的 DirectRunner 测试我的数据流管道，并收到了很多这样的“警告”消息，我是否知道如何摆脱它们，因为太多了，我什至看不到我的调试消息。

谢谢

Apr 05, 2018 2:14:48 PM org.apache.beam.sdk.util.MutationDetectors$CodedValueMutationDetector verifyUnmodifiedThrowingCheckedExceptions
WARNING: Coder of type class org.apache.beam.sdk.coders.SerializableCoder has a #structuralValue method which does not return true when the encoding of the elements is equal. 
Element com.apigee.analytics.platform.core.service.schema.EventRow@4a590d0b

Answer 1

确保所有序列化值都具有正确的 equals() 可能会有所帮助。自 SerializableCoder 以来的实现expects他们:

The structural value of the object is the object itself. The SerializableCoder should be only used for objects with a proper Object#equals implementation.

您可以实现自己的 Coder为您的 POJO。根据 docs，SerializableCoder 不保证确定性编码:

SerializableCoder does not guarantee a deterministic encoding, as Java serialization may produce different binary encodings for two equivalent objects.

This article explains custom coders in details .