Software Recommendations

Questions tagged [hadoop]

Ask Question

The tag has no usage guidance.

21 questions
Newest
Active
Bountied
Unanswered
Filter by
Sorted by
Tagged with
0
votes
0answers
67 views

Distributing partitioned data across different geographic regions

Is there any distributed database, file system or storage service that supports the following use-case that I have? It must be possible to replicate data (files) over 3 servers in different ...
0
votes
0answers
7 views

Bigdata DB JSON objects via Kafka / Spark / HDFS?

I am doing a concept for a bigdata DB and would love to hear your advise to solve the following issue. Requirements: fast async write calls to the db mandatory high horizontal scalability no issues ...
0
votes
0answers
6 views

Can I integrate GitLab and Hadoop for CI?

I am devoloping an app using Shiny R, hadoop (local sever) and sparklyr. I would like to use Gitlab CI/CD. Do you have any idea how could I use Gitlab CI/CD if my app is using files from Hadoop? ...
0
votes
0answers
10 views

Storage Solutions (Hadoop, Ceph. Gluster FS)

I am working on a project that needs to store data at a rate of 10 Gbps (packet capture files and metadata about each file). Once this data is stored i need to access this data (make querries to get ...
0
votes
0answers
8 views

Software to display hive /hadoop sql results

I am looking for a software that displays my hive query results in a tabular format.We have to convert each query into csv and then transfer the csv to local in order to get the results. Previously ...
0
votes
0answers
3 views

Bigquery to Hadoop

i want to automatically (daily) exporte a table from Bigquery into HDFS. Is there a direct way to do that witouth transitioning by GCS(Google Cloud Storage)? Thanks
0
votes
0answers
27 views

How to compare same table between hive , Teradata

We have a situation where we copy same tables to Hive , Teradata and vertica . We have to test and confirm whether the tables were copied exactly between all. As of now we use python to download the ...
0
votes
1answer
24 views

Distributed data storage recommendations in mixed operating system environment for Python applications (Hadoop vs Databricks vs others)

I'm working on a big data project and have several old on-prem servers running a mix of operating systems (Ubuntu, CentOS, Windows 2012, Windows 10). One of the big reasons I would like a distributed ...
1
vote
0answers
13 views

Connect Excel to HDFS and update data real time

I am trying to figure out a method or software for updating data into HDFS real time. My requirement is as follows: I will have an Excel in a server. That Excel data needs to be loaded into HDFS. IF ...
1
vote
1answer
38 views

Operating system for Personal & Enterprise application for Hadoop

I ain't sure if this is a right forum to ask this question. Please let me know which SE site I have to use, in case I am on the wrong forum. I have started learning Hadoop & I have tried ...
0
votes
0answers
155 views

Suggestions for Distributed File System

Current Configuration: No of machines: 7-8 Hard Disk in each: 400 GB Memory in each machine: 60 GB Requirements: Each program can access many files. Since program is in c++, some interface in ...
0
votes
0answers
181 views

Alternative for HBase to store and query small files

I'm using Hadoop on my system to store my files (each file is very large: ~4GB). I also have small sqlite3 db files (~50MB) and I would like to store them (somewhere - I don't have shared storage) and ...
2
votes
1answer
49 views

Deep Learning: is there any open-source library that can be integrated with Hadoop streaming and MapReduce?

Google search popped out quite a few open source deep learning frameworks. Here is a collected list Google TensorFlow Theano mxnet keras Pylearn2 Blocks Lasagne chainer scikit-neuralnetwork theano-...
1
vote
0answers
547 views

Python Django with MongoDB with Hadoop

For my Final year project I have to make a big data application. So far the sketch of the overall project includes these three ( Django, MongoDb, Hadoop ). But I am not sure they will work together ...
2
votes
0answers
714 views

Is there any alternative software for JSON Studio?

Is there any alternative software for JSON Studio that would be free to use or probably opensource? I have to use Jsonstudio for its interactivity handling json documents stored in MongoDb. It also ...
1
vote
0answers
19 views

What SQL-on-hadoop will be the best solution for my case?

There are so many open source frameworks that help doing analysis by SQL-like query on distributed system. We are currently looking for good solutions for our case such as Apache Hive, Tajo, Tez, ...
8
votes
2answers
779 views

Database for large volume of metrics (key, value, timestamp)

I'm looking for a Database for large amounts of metrics, and i'm confused that there is nothing out there to: store large amounts (terrabytes) of metrics data in the form: key (string) value (...
2
votes
1answer
86 views

video storing and synchronous streaming software

I need some software solution for video storing. I will have a few IP cameras which has to stream into disks. This so called database records those streams.On demand I should be able to stream any of ...
4
votes
0answers
265 views

Which SQL engine atop Hadoop?

Since Hadoop came out, there have been various commercial and/or open-source attempts to expose some compatibility with SQL. I am seeking one which is good for low-latency querying, and supports the ...
1
vote
1answer
222 views

Hadoop JSON Data to Data Warehouse

I'm interested in building a system that allows efficient loading of JSON stored in a Hadoop\EMR-based system to a relational data warehouse. I've already collected large amounts of JSON data in the ...
2
votes
1answer
71 views

MapReduce or Hadoop plugin for Eclipse

I am looking for a plugin in Eclipse that can automate some of the work when creating a MapReduce job using Java. For example: Auto create Mapper - Reducer classes(implementing the interfaces). ...