Questions tagged [distributed]

is for questions dealing with distributed systems, programs and programming.

Filter by
Sorted by
Tagged with
1
vote
0answers
22 views

Distributed FS for storing data in the single “master” node and duplicating that data, which is used on “worker” nodes

I am looking for a DFS for unified data access and caching used files within a small cluster. We are currently using NFS to RAID node, and it seems not the best way to further scaling. So, I am ...
1
vote
0answers
4 views

Framework/Tool for real-time inference with large input flow using Tensorflow

I'm working on a part of a project, that detects fraud from sensors' data using deep learning. Current steps are: Receiving sensors' data (I'm currently receiving data from Kafka). Predicting ...
0
votes
1answer
16 views

looking for the distributed / parallel processing software?

I have 1 PC, 1 iMac and 5 laptops and use only 1 or 2 at a time. My hobby is developing web, video/audio editing. So when encoding videos, I would like to assign other laptops for parallel processing ...
1
vote
0answers
12 views

Distributed arrays in Python

I am looking for a minimal interface to distributed ndarrays. I'd like to chunk my data in arbitrary blocks, scatter them across a cluster, run computations on the local numpy array, rechunk my data ...
1
vote
0answers
28 views

Multi-leader or leaderless replication distributed DB (in-memory)

I am trying to find out few examples of multi-leader or leaderless distributed DB. It seems Redis is there but only enterprise version is having multi master support. Any distributed DB which satisfy ...
1
vote
0answers
19 views

Alternative analytics database for memsql and crate.io

Have been looking for real-time analytics database as a managed service that could scale horizontally and have column storage and support distributed joins. Data is not too big (around 2 billion rows ...
1
vote
2answers
35 views

Distributed Database Systems

I am looking for an open source distributed SQL database system which will distribute the data across nodes but so that there is data redundancy (if one node fails, the data is still available without ...
2
votes
0answers
120 views

Are there free tools/setups for cached, distributed C++ compiling with CMake on Windows?

This problem touches too many topics outside of the CMake, C++, Windows world I am familiar with. I kind of hit a wall of way to many options to explore each one. Actually I am hoping for someone ...
0
votes
0answers
12 views

parallel processing

We asked our dear friends, there are several idle systems that we want to use with our network of other active systems, 4 weighing software applications, in parallel processing and maximum processing ...
1
vote
0answers
32 views

Can someone suggest me a NoSQL database which is good for frequent updates?

I have heard that NoSQL databases like Cassandra, Couchbase etc are not good for frequent updates of data. They behave well when the data inserted doesn't change often. My use case is to save/update ...
0
votes
1answer
26 views

Distributed data storage recommendations in mixed operating system environment for Python applications (Hadoop vs Databricks vs others)

I'm working on a big data project and have several old on-prem servers running a mix of operating systems (Ubuntu, CentOS, Windows 2012, Windows 10). One of the big reasons I would like a distributed ...
0
votes
0answers
23 views

Publish/subscribe software to handle at least 10k requests/sec and 10k simultaneous connection

I am on the verge to start making a "Large distributed application", but i'm struggling to decide what software is the most adequate to do so. My needs: A publish/subscribe application, which ...
1
vote
0answers
39 views

How to store 1.5 GB CSV files?

I am searching a solution to store files of 1.5 GB. There is a decade of file per day. I am searching a solution to store those files. I have already done some searches on HDFS and MongoDB. However, ...
0
votes
0answers
19 views

Leader Election with Customization

I have multiple micro-service instances of same type. Each service instance will have connections to some of remote services. In below diagram, I have shown 3 service instance and 3 remote service. ...
1
vote
0answers
22 views

Shareable List of Links With Notifications

So basically I am creating Tinycards decks for the vocabulary of each Duolingo Hebrew level that feature transliterations to practice learning not only the words and their meanings but also their ...
2
votes
0answers
18 views

How can i run my service on a clustered environment?

How can i run my service on a clustered environment ? Any products out there to help me out. This product should take care of managing the cluster, cluster failover, rebalancing in case of node ...
1
vote
0answers
24 views

Automated system testing and performance measurement for distributed system

I'm looking for a tool/combination of tools which would allow me to do system testing for application spanning multiple hosts. What I'm looking for: ability to spawn variable number of application ...
1
vote
0answers
148 views

Distributed build system for Rust

I'm looking for a distributed build system for Rust that: Is free, gratis and opensource Is cross-platform and cross-compiles (from ARMv6, ARMv7, i386 and amd64 Linux and amd64 Windows for amd64 ...
4
votes
2answers
147 views

A distributed (on-demand) build system

I'm tired of long build times … I'm interested in a distributed build system, either at the lower-level of running the equivalent of make, and not knowing anything about anything other than ...
0
votes
0answers
169 views

Suggestions for Distributed File System

Current Configuration: No of machines: 7-8 Hard Disk in each: 400 GB Memory in each machine: 60 GB Requirements: Each program can access many files. Since program is in c++, some interface in ...
0
votes
0answers
188 views

Alternative for HBase to store and query small files

I'm using Hadoop on my system to store my files (each file is very large: ~4GB). I also have small sqlite3 db files (~50MB) and I would like to store them (somewhere - I don't have shared storage) and ...
2
votes
0answers
36 views

Distributed, indexed regular expression search

I'm looking for a distributed solution to do regular expression searches over a large directory structure with many nested directories and files. Ideally it will be also able to search file/directory ...
2
votes
0answers
178 views

Cluster computing drop-in replacement for Python multiprocessing Pool

Python has a very handy multiprocessing built-in module. Particularly its Pool class allows simple parallelization. However, it only supports a shared-memory multiprocessing model, not a distributed ...
4
votes
1answer
51 views

Distributed updates over LAN for Windows

Does software exist that would make it easy to distribute software updates for a variety of programs over LAN? My job involves supporting numerous low-bandwidth offices with very expensive internet ...
2
votes
1answer
291 views

Linear algebra C/C++ library for distributed computing

My goal is to invert a large dense matrix in a distributed environment. I would like a library which would be "easily" integrated in a C++ application, thus it would be ideal that the library has a C/...
1
vote
2answers
105 views

Version control system on distributed files or database?

Do you know of any "version control" or sync tool able to work transparently on files stored on distributed file systems (such as Hadoop) or using their own database to keep the files? I'm looking ...
1
vote
0answers
58 views

Distributed job queue with task locality?

Context At present i have one server, which knows some tasks need to be scheduled at recurring intervals. So one process pushes such tasks into a message queue on a timer; and then other processes ...
5
votes
0answers
199 views

Free distributed wiki software with full-text search, working even offline

I'm looking for a way to organize knowledge in my company. Since we have to work disconnected from the company net many times, a server-client solution alone won't suffice. The requirements are: ...