I need to share the datasets among various (software) servers (developed by other teams). Server A for instance can do text normalization, another server can do e.g. sentiment analysis, yet another server can (will) do analytics and visualization.
Sure I could implement this myself, but why reinvent the wheel if the wheel is actually invented?
I would like to use a project that is:
- implement basic Web UI for users to upload, share and delete their datasets, whatever they are.
- open source
- can handle at least gigabyte sized datasets
- users can allow particular servers to access indicated datasets they own
- servers can access the datasets via REST API or Python, Java client
I was thinking about something like Dremio, but simpler and fully open source.