1

I'm working on a part of a project, that detects fraud from sensors' data using deep learning. Current steps are:

  1. Receiving sensors' data (I'm currently receiving data from Kafka).
  2. Predicting anomalies of data (Tensorflow + PySpark structured streaming + Pandas UDF - large latency in my opinion).
  3. Send the result back (to Kafka - using PySpark structured streaming).

The results are monitored by Kibana, but another team work on that part.

I don't think my solution is a good one, so I'm asking for suggestions on which platform/framework/tool should be used at each step. Any help/advice is appreciated.

1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.