tl;dr If you want to run Kafka 2.x use 2.1.1rc1 or later.

So someone started to update from Kafka 1.1.1 to 2.1.0 yesterday and it kept crashing every other hour. It pretty much looks like https://issues.apache.org/jira/browse/KAFKA-7697, so we're now trying out 2.1.1rc1 because we missed the rc2 at http://home.apache.org/~cmccabe/kafka-2.1.1-rc2/. So ideally you go with rc2 which has a few more fixes for unrelated issues.

Beside of that, be aware that the update to 2.1.0 is a one way street! Read http://kafka.apache.org/21/documentation.html#upgrade carefully. There is a schema change in the consumer offset topic which is internally used to track your consumer offsets since those moved out of Zookeeper some time ago.

For us the primary lesson is that we've to put way more synthetic traffic on our testing environments, because 2.1.0 was running in the non production environments for several days without an issue, and the relevant team hit the deadlock in production within hours.