I am working on a project that needs to store data at a rate of 10 Gbps (packet capture files and metadata about each file). Once this data is stored i need to access this data (make querries to get certain files from the stored data). These querries are based on multiple datapoints in metadata files. I have been exploring hadoop (big data anylytics ) and ceph (purely storage) as possible solutions to suffice my use case but i am confused about them as hadoop is something made for big data analytics but provides with additional tools that can work for and Ceph is purely for storage. I need some recommendation about which one should i use If i go for hadoop, would it be enable me to store data at 10 Gbps and then access a chunk of this data say 30 Gbs in seconds? and if i go for ceph it enables me to store data at 10 Gbps but i will have to implement the querrying and retrieval mechanism myself. or is there any better solution available that fulfills my use case. (any recomendations)?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.