how Spark + Kafka streaming NoClassDefFoundError kafka/serializer/StringDecoder
I’m trying to send message from my kafka producer and stream it in spark streaming. But I’m getting the following error when I run my application on spark submit.
Application code is as follows:
Couldn’t find a solution for this error. Any help would be appreciated.
More specifically the part:
Whereas your pom.xml clearly shows that the jar you are building is without the dependencies. That’s why spark-submit cannot find the class kafka.serializer.StringDecoder.
What you might want to use to solve such a problem is a plugin that include your dependencies inside your jar, the maven assembly plugin can help you with this