Questions tagged [distributed]

is for questions dealing with distributed systems, programs and programming.

Filter by
Sorted by
Tagged with
1
vote
0answers
6 views

Distributed arrays in Python

I am looking for a minimal interface to distributed ndarrays. I'd like to chunk my data in arbitrary blocks, scatter them across a cluster, run computations on the local numpy array, rechunk my data ...
1
vote
0answers
11 views

Multi-leader or leaderless replication distributed DB (in-memory)

I am trying to find out few examples of multi-leader or leaderless distributed DB. It seems Redis is there but only enterprise version is having multi master support. Any distributed DB which satisfy ...
1
vote
0answers
15 views

Alternative analytics database for memsql and crate.io

Have been looking for real-time analytics database as a managed service that could scale horizontally and have column storage and support distributed joins. Data is not too big (around 2 billion rows ...
0
votes
0answers
76 views

Distributing partitioned data across different geographic regions

Is there any distributed database, file system or storage service that supports the following use-case that I have? It must be possible to replicate data (files) over 3 servers in different ...
0
votes
0answers
6 views

Looking for lightweight library which caters functionality of distributed logging with persistence primarily in publisher / subscriber fashion

I am looking into IOT World Solution where we have MQTT Broker. This MQTT Broker Sits in one of the Data Center. Reduced operational cost is our primary lookout. We are doing lot of Alert and Alarm ...
1
vote
2answers
33 views

Distributed Database Systems

I am looking for an open source distributed SQL database system which will distribute the data across nodes but so that there is data redundancy (if one node fails, the data is still available without ...
2
votes
0answers
42 views

Are there free tools/setups for cached, distributed C++ compiling with CMake on Windows?

This problem touches too many topics outside of the CMake, C++, Windows world I am familiar with. I kind of hit a wall of way to many options to explore each one. Actually I am hoping for someone ...
0
votes
0answers
23 views

data flow / processing flow framework

Data flow is probably a much more commonly known term to define the category of software I'm looking for. However, data flow basically describes the path a piece of data takes through the system. What ...
0
votes
0answers
4 views

Is there any open source Asynchronous and Distributed Multi-Agent System (MAS)?

For a research project, I need to create an evolutionary simulation with 100 000 agents. Thus, I am looking for a distributed and asynchronous multi-agent system (MAS). The only project I found is no ...
0
votes
0answers
11 views

parallel processing

We asked our dear friends, there are several idle systems that we want to use with our network of other active systems, 4 weighing software applications, in parallel processing and maximum processing ...
1
vote
0answers
14 views

Can someone suggest me a NoSQL database which is good for frequent updates?

I have heard that NoSQL databases like Cassandra, Couchbase etc are not good for frequent updates of data. They behave well when the data inserted doesn't change often. My use case is to save/update ...
0
votes
1answer
24 views

Distributed data storage recommendations in mixed operating system environment for Python applications (Hadoop vs Databricks vs others)

I'm working on a big data project and have several old on-prem servers running a mix of operating systems (Ubuntu, CentOS, Windows 2012, Windows 10). One of the big reasons I would like a distributed ...
0
votes
0answers
16 views

Publish/subscribe software to handle at least 10k requests/sec and 10k simultaneous connection

I am on the verge to start making a "Large distributed application", but i'm struggling to decide what software is the most adequate to do so. My needs: A publish/subscribe application, which ...
1
vote
0answers
38 views

How to store 1.5 GB CSV files?

I am searching a solution to store files of 1.5 GB. There is a decade of file per day. I am searching a solution to store those files. I have already done some searches on HDFS and MongoDB. However, ...
0
votes
0answers
19 views

Leader Election with Customization

I have multiple micro-service instances of same type. Each service instance will have connections to some of remote services. In below diagram, I have shown 3 service instance and 3 remote service. ...
1
vote
0answers
20 views

Shareable List of Links With Notifications

So basically I am creating Tinycards decks for the vocabulary of each Duolingo Hebrew level that feature transliterations to practice learning not only the words and their meanings but also their ...
0
votes
0answers
15 views

Service that sends system information (and IP) periodically to central server

As part of my project I need to have a central server that multiple devices would periodically ping and send some system information (for example OS type/version, nothing too special). The purpose of ...
2
votes
0answers
18 views

How can i run my service on a clustered environment?

How can i run my service on a clustered environment ? Any products out there to help me out. This product should take care of managing the cluster, cluster failover, rebalancing in case of node ...
1
vote
0answers
23 views

Automated system testing and performance measurement for distributed system

I'm looking for a tool/combination of tools which would allow me to do system testing for application spanning multiple hosts. What I'm looking for: ability to spawn variable number of application ...
1
vote
0answers
133 views

Distributed build system for Rust

I'm looking for a distributed build system for Rust that: Is free, gratis and opensource Is cross-platform and cross-compiles (from ARMv6, ARMv7, i386 and amd64 Linux and amd64 Windows for amd64 ...
4
votes
2answers
140 views

A distributed (on-demand) build system

I'm tired of long build times … I'm interested in a distributed build system, either at the lower-level of running the equivalent of make, and not knowing anything about anything other than ...
0
votes
0answers
162 views

Suggestions for Distributed File System

Current Configuration: No of machines: 7-8 Hard Disk in each: 400 GB Memory in each machine: 60 GB Requirements: Each program can access many files. Since program is in c++, some interface in ...
0
votes
0answers
185 views

Alternative for HBase to store and query small files

I'm using Hadoop on my system to store my files (each file is very large: ~4GB). I also have small sqlite3 db files (~50MB) and I would like to store them (somewhere - I don't have shared storage) and ...
2
votes
0answers
36 views

Distributed, indexed regular expression search

I'm looking for a distributed solution to do regular expression searches over a large directory structure with many nested directories and files. Ideally it will be also able to search file/directory ...
2
votes
0answers
173 views

Cluster computing drop-in replacement for Python multiprocessing Pool

Python has a very handy multiprocessing built-in module. Particularly its Pool class allows simple parallelization. However, it only supports a shared-memory multiprocessing model, not a distributed ...
4
votes
1answer
50 views

Distributed updates over LAN for Windows

Does software exist that would make it easy to distribute software updates for a variety of programs over LAN? My job involves supporting numerous low-bandwidth offices with very expensive internet ...
2
votes
1answer
285 views

Linear algebra C/C++ library for distributed computing

My goal is to invert a large dense matrix in a distributed environment. I would like a library which would be "easily" integrated in a C++ application, thus it would be ideal that the library has a C/...
1
vote
2answers
104 views

Version control system on distributed files or database?

Do you know of any "version control" or sync tool able to work transparently on files stored on distributed file systems (such as Hadoop) or using their own database to keep the files? I'm looking ...
1
vote
0answers
56 views

Distributed job queue with task locality?

Context At present i have one server, which knows some tasks need to be scheduled at recurring intervals. So one process pushes such tasks into a message queue on a timer; and then other processes ...
5
votes
0answers
198 views

Free distributed wiki software with full-text search, working even offline

I'm looking for a way to organize knowledge in my company. Since we have to work disconnected from the company net many times, a server-client solution alone won't suffice. The requirements are: ...