Questions tagged [hadoop]
The hadoop tag has no usage guidance.
21
questions
0
votes
0answers
58 views
Distributing partitioned data across different geographic regions
Is there any distributed database, file system or storage service that supports the following use-case that I have?
It must be possible to replicate data (files) over 3 servers in different ...
0
votes
0answers
7 views
Bigdata DB JSON objects via Kafka / Spark / HDFS?
I am doing a concept for a bigdata DB and would love to hear your advise to solve the following issue.
Requirements:
fast async write calls to the db mandatory
high horizontal scalability
no issues ...
0
votes
0answers
6 views
Can I integrate GitLab and Hadoop for CI?
I am devoloping an app using Shiny R, hadoop (local sever) and sparklyr. I would like to use Gitlab CI/CD. Do you have any idea how could I use Gitlab CI/CD if my app is using files from Hadoop?
...
0
votes
0answers
10 views
Storage Solutions (Hadoop, Ceph. Gluster FS)
I am working on a project that needs to store data at a rate of 10 Gbps (packet capture files and metadata about each file). Once this data is stored i need to access this data (make querries to get ...
0
votes
0answers
8 views
Software to display hive /hadoop sql results
I am looking for a software that displays my hive query results in a tabular format.We have to convert each query into csv and then transfer the csv to local in order to get the results.
Previously ...
0
votes
0answers
3 views
Bigquery to Hadoop
i want to automatically (daily) exporte a table from Bigquery into HDFS. Is there a direct way to do that witouth transitioning by GCS(Google Cloud Storage)?
Thanks
0
votes
0answers
25 views
How to compare same table between hive , Teradata
We have a situation where we copy same tables to Hive , Teradata and vertica .
We have to test and confirm whether the tables were copied exactly between all.
As of now we use python to download the ...
0
votes
1answer
24 views
Distributed data storage recommendations in mixed operating system environment for Python applications (Hadoop vs Databricks vs others)
I'm working on a big data project and have several old on-prem servers running a mix of operating systems (Ubuntu, CentOS, Windows 2012, Windows 10). One of the big reasons I would like a distributed ...
1
vote
0answers
13 views
Connect Excel to HDFS and update data real time
I am trying to figure out a method or software for updating data into HDFS real time.
My requirement is as follows:
I will have an Excel in a server. That Excel data needs to be loaded into HDFS. IF ...
1
vote
1answer
37 views
Operating system for Personal & Enterprise application for Hadoop
I ain't sure if this is a right forum to ask this question. Please let me know which SE site I have to use, in case I am on the wrong forum.
I have started learning Hadoop & I have tried ...
0
votes
0answers
154 views
Suggestions for Distributed File System
Current Configuration:
No of machines: 7-8
Hard Disk in each: 400 GB
Memory in each machine: 60 GB
Requirements:
Each program can access many files. Since program is in c++, some interface in ...
0
votes
0answers
180 views
Alternative for HBase to store and query small files
I'm using Hadoop on my system to store my files (each file is very large: ~4GB). I also have small sqlite3 db files (~50MB) and I would like to store them (somewhere - I don't have shared storage) and ...
2
votes
1answer
49 views
Deep Learning: is there any open-source library that can be integrated with Hadoop streaming and MapReduce?
Google search popped out quite a few open source deep learning frameworks. Here is a collected list
Google TensorFlow
Theano
mxnet
keras
Pylearn2
Blocks
Lasagne
chainer
scikit-neuralnetwork
theano-...
1
vote
0answers
536 views
Python Django with MongoDB with Hadoop
For my Final year project I have to make a big data application. So far the sketch of the overall project includes these three ( Django, MongoDb, Hadoop ).
But I am not sure they will work together ...
2
votes
0answers
704 views
Is there any alternative software for JSON Studio?
Is there any alternative software for JSON Studio that would be free to use or probably opensource? I have to use Jsonstudio for its interactivity handling json documents stored in MongoDb. It also ...
1
vote
0answers
19 views
What SQL-on-hadoop will be the best solution for my case?
There are so many open source frameworks that help doing analysis by SQL-like query on distributed system.
We are currently looking for good solutions for our case such as Apache Hive, Tajo, Tez, ...
8
votes
2answers
778 views
Database for large volume of metrics (key, value, timestamp)
I'm looking for a Database for large amounts of metrics, and i'm confused that there is nothing out there to:
store large amounts (terrabytes) of metrics data in the form:
key (string)
value (...
2
votes
1answer
86 views
video storing and synchronous streaming software
I need some software solution for video storing. I will have a few IP cameras which has to stream into disks. This so called database records those streams.On demand I should be able to stream any of ...
4
votes
0answers
265 views
Which SQL engine atop Hadoop?
Since Hadoop came out, there have been various commercial and/or open-source attempts to expose some compatibility with SQL.
I am seeking one which is good for low-latency querying, and supports the ...
1
vote
1answer
221 views
Hadoop JSON Data to Data Warehouse
I'm interested in building a system that allows efficient loading of JSON stored in a Hadoop\EMR-based system to a relational data warehouse. I've already collected large amounts of JSON data in the ...
2
votes
1answer
71 views
MapReduce or Hadoop plugin for Eclipse
I am looking for a plugin in Eclipse that can automate some of the work when creating a MapReduce job using Java.
For example:
Auto create Mapper - Reducer classes(implementing the interfaces).
...
