In Kafka (Python), you can do:

from kafka import KafkaConsumer
import cv2

consumer = KafkaConsumer('topic')
for msg in consumer:
    img = extract_img(msg)  # Placeholder function to get image from Kafka
    cv2.imshow('img', img) 

Producer side:

import cv2
from kafka import KafkaProducer

producer = KafkaProducer()
cap = cv2.VideoCapture(0)

while True:
    _, img = cap.read()
    encoded = encode_img(img)  # Encode to bytes for Kafka
    producer.send('topic', encoded)

This works, but each frame is stored as an uncompressed entry in Kafka. This is too much overhead (esp. for the network) for real-time streaming.

Is there a tool like Kafka where the interface is the same, but compression is done under the hood?

for msg in consumer:  # Same API, compression under the hood

How do projects like Amazon Go and self-driving cars implement this?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.