Questions tagged [distributed]

is for questions dealing with distributed systems, programs and programming.

Filter by
Sorted by
Tagged with
0 votes
0 answers
9 views

Any languages with serializable / mobile runtime?

I am developing a system that executes arbitrary jobs. A job is a piece of code in some programming language (e.g. Javascript), it can run sub-jobs and wait for their completion as it is shown in the ...
ischenko's user avatar
0 votes
0 answers
88 views

Software for distributing algorithms and flowcharts

I am looking for software that I can use to distribute instructions to other users in the form of algorithms or flowcharts. Absolute requirements: The app runs on either Android OR Apple devices ...
allrik's user avatar
  • 1
0 votes
0 answers
10 views

How to train huge ML model on multiple budget machines on web?

Is there a way to train huge models (1+ TiB of parameter data) being distributed on web? For example, I have a several hundreds of budget machines, each equipped with 2-4 GiB of RAM and with no ...
ababo's user avatar
  • 111
0 votes
1 answer
44 views

MQTT broker for fault-tolerance

Looking for a distributed MQTT broker that remains operational when one of its nodes fails for interdevice comunication using low-bandwith in time-sensitive scenarios. Its a low volume of messages and ...
Luis Miguel Silva's user avatar
1 vote
0 answers
9 views

is there some profiling and coverage library for rust?

This question is related to a common criticism of the model for sharing computing resources on blockchains (often seen on economics), based on accounting of participation of various users on the ...
jeferson lemos's user avatar
3 votes
1 answer
36 views

Software with a database to lookup hashes of files

Sometimes it happens that an open source project adds files with a wrong license. For example the C code of a game is GPL-2, but by accident a sound file from a proprietary source was copied to the ...
Jonas Stein's user avatar
0 votes
0 answers
28 views

A distributed file system with interactive conflict resolution?

I have searched high and low and cannot find anything suitable for my use case: I’m looking for a solution that offers me to use the same files on both my mobile, laptop, desktop, home server and ...
user avatar
1 vote
0 answers
22 views

Distributed FS for storing data in the single "master" node and duplicating that data, which is used on "worker" nodes

I am looking for a DFS for unified data access and caching used files within a small cluster. We are currently using NFS to RAID node, and it seems not the best way to further scaling. So, I am ...
Sergei Voloboev's user avatar
1 vote
0 answers
8 views

Framework/Tool for real-time inference with large input flow using Tensorflow

I'm working on a part of a project, that detects fraud from sensors' data using deep learning. Current steps are: Receiving sensors' data (I'm currently receiving data from Kafka). Predicting ...
Tung's user avatar
  • 11
0 votes
1 answer
42 views

looking for the distributed / parallel processing software?

I have 1 PC, 1 iMac and 5 laptops and use only 1 or 2 at a time. My hobby is developing web, video/audio editing. So when encoding videos, I would like to assign other laptops for parallel processing ...
zhtway's user avatar
  • 101
2 votes
0 answers
83 views

Distributed arrays in Python

I am looking for a minimal interface to distributed ndarrays. I'd like to chunk my data in arbitrary blocks, scatter them across a cluster, run computations on the local numpy array, rechunk my data ...
dPol's user avatar
  • 21
1 vote
0 answers
136 views

Multi-leader or leaderless replication distributed DB (in-memory)

I am trying to find out few examples of multi-leader or leaderless distributed DB. It seems Redis is there but only enterprise version is having multi master support. Any distributed DB which satisfy ...
Rahul's user avatar
  • 111
1 vote
0 answers
28 views

Alternative analytics database for memsql and crate.io

Have been looking for real-time analytics database as a managed service that could scale horizontally and have column storage and support distributed joins. Data is not too big (around 2 billion rows ...
Hyperion's user avatar
1 vote
2 answers
51 views

Distributed Database Systems

I am looking for an open source distributed SQL database system which will distribute the data across nodes but so that there is data redundancy (if one node fails, the data is still available without ...
rhobro's user avatar
  • 11
7 votes
1 answer
1k views

Are there free tools/setups for cached, distributed C++ compiling with CMake on Windows?

This problem touches too many topics outside of the CMake, C++, Windows world I am familiar with. I kind of hit a wall of way to many options to explore each one. Actually I am hoping for someone ...
Warmy's user avatar
  • 71
0 votes
0 answers
15 views

parallel processing

We asked our dear friends, there are several idle systems that we want to use with our network of other active systems, 4 weighing software applications, in parallel processing and maximum processing ...
Alireza Taaki's user avatar
1 vote
0 answers
208 views

Can someone suggest me a NoSQL database which is good for frequent updates?

I have heard that NoSQL databases like Cassandra, Couchbase etc are not good for frequent updates of data. They behave well when the data inserted doesn't change often. My use case is to save/update ...
Amit Jayee's user avatar
0 votes
1 answer
37 views

Distributed data storage recommendations in mixed operating system environment for Python applications (Hadoop vs Databricks vs others)

I'm working on a big data project and have several old on-prem servers running a mix of operating systems (Ubuntu, CentOS, Windows 2012, Windows 10). One of the big reasons I would like a distributed ...
dan's user avatar
  • 1
0 votes
0 answers
28 views

Publish/subscribe software to handle at least 10k requests/sec and 10k simultaneous connection

I am on the verge to start making a "Large distributed application", but i'm struggling to decide what software is the most adequate to do so. My needs: A publish/subscribe application, which ...
Fouzi Otoman's user avatar
1 vote
0 answers
42 views

How to store 1.5 GB CSV files?

I am searching a solution to store files of 1.5 GB. There is a decade of file per day. I am searching a solution to store those files. I have already done some searches on HDFS and MongoDB. However, ...
Thomas R's user avatar
  • 111
0 votes
0 answers
22 views

Leader Election with Customization

I have multiple micro-service instances of same type. Each service instance will have connections to some of remote services. In below diagram, I have shown 3 service instance and 3 remote service. ...
defender's user avatar
  • 101
2 votes
0 answers
23 views

Shareable List of Links With Notifications

So basically I am creating Tinycards decks for the vocabulary of each Duolingo Hebrew level that feature transliterations to practice learning not only the words and their meanings but also their ...
Reginald Greyling's user avatar
2 votes
0 answers
19 views

How can i run my service on a clustered environment?

How can i run my service on a clustered environment ? Any products out there to help me out. This product should take care of managing the cluster, cluster failover, rebalancing in case of node ...
Ankit Srivastava's user avatar
1 vote
0 answers
31 views

Automated system testing and performance measurement for distributed system

I'm looking for a tool/combination of tools which would allow me to do system testing for application spanning multiple hosts. What I'm looking for: ability to spawn variable number of application ...
Artur Biesiadowski's user avatar
1 vote
0 answers
192 views

Distributed build system for Rust

I'm looking for a distributed build system for Rust that: Is free, gratis and opensource Is cross-platform and cross-compiles (from ARMv6, ARMv7, i386 and amd64 Linux and amd64 Windows for amd64 ...
wb9688's user avatar
  • 1,240
4 votes
2 answers
216 views

A distributed (on-demand) build system

I'm tired of long build times … I'm interested in a distributed build system, either at the lower-level of running the equivalent of make, and not knowing anything about anything other than ...
einpoklum's user avatar
  • 3,189
0 votes
0 answers
197 views

Alternative for HBase to store and query small files

I'm using Hadoop on my system to store my files (each file is very large: ~4GB). I also have small sqlite3 db files (~50MB) and I would like to store them (somewhere - I don't have shared storage) and ...
galbru's user avatar
  • 101
2 votes
0 answers
43 views

Distributed, indexed regular expression search

I'm looking for a distributed solution to do regular expression searches over a large directory structure with many nested directories and files. Ideally it will be also able to search file/directory ...
Kevin Wheeler's user avatar
2 votes
0 answers
192 views

Cluster computing drop-in replacement for Python multiprocessing Pool

Python has a very handy multiprocessing built-in module. Particularly its Pool class allows simple parallelization. However, it only supports a shared-memory multiprocessing model, not a distributed ...
TheBlackCat's user avatar
4 votes
1 answer
58 views

Distributed updates over LAN for Windows

Does software exist that would make it easy to distribute software updates for a variety of programs over LAN? My job involves supporting numerous low-bandwidth offices with very expensive internet ...
Nahum G's user avatar
  • 61
2 votes
2 answers
350 views

Dense Linear Algebra C/C++ library for distributed computing

My goal is to invert a large dense matrix in a distributed environment. I would like a library which would be "easily" integrated in a C++ application, thus it would be ideal that the library has a C/...
gsamaras's user avatar
  • 135
1 vote
2 answers
112 views

Version control system on distributed files or database?

Do you know of any "version control" or sync tool able to work transparently on files stored on distributed file systems (such as Hadoop) or using their own database to keep the files? I'm looking ...
skan's user avatar
  • 171
1 vote
0 answers
60 views

Distributed job queue with task locality?

Context At present i have one server, which knows some tasks need to be scheduled at recurring intervals. So one process pushes such tasks into a message queue on a timer; and then other processes ...
mappu's user avatar
  • 21
5 votes
0 answers
217 views

Free distributed wiki software with full-text search, working even offline

I'm looking for a way to organize knowledge in my company. Since we have to work disconnected from the company net many times, a server-client solution alone won't suffice. The requirements are: ...
Onur's user avatar
  • 301