-
Two excellent tools in one blog post.
Visidata “is a commandline tool to work with data in all sorts of formats, including from stdin”; in this example it’s taking lines of JSONL and producing an instant histogram of values from the stream:
Once visidata is open, use the arrow keys to move to the column on which you want to build a histogram and press Shift-F. Since it works with pipes if you leave the -e off the kafkacat argument you get a live stream of messages from the Kafka topic and the visidata will continue to update as messages arrive (although I think you need to replot the histogram if you want it to refresh).
On top of that, there’s kcat, “netcat for Kafka”, “a swiss-army knife of tools for inspecting and creating data in Kafka”, even supporting on-the-fly decode of Avro messages. https://github.com/edenhill/kcat