Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

Questions tagged [distributed]

is for questions dealing with distributed systems, programs and programming.

Filter by
Sorted by
Tagged with
0
votes
0answers
3 views

Is there any open source Asynchronous and Distributed Multi-Agent System (MAS)?

For a research project, I need to create an evolutionary simulation with 100 000 agents. Thus, I am looking for a distributed and asynchronous multi-agent system (MAS). The only project I found is no ...
0
votes
0answers
11 views

parallel processing

We asked our dear friends, there are several idle systems that we want to use with our network of other active systems, 4 weighing software applications, in parallel processing and maximum processing ...
0
votes
0answers
25 views

Decentralized database for server-list to get rid of masterservers

I am searching for technology or library for a decentralized Database. The Problem Currently game developers are hosting their own masterservers to distribute server lists for community servers. A ...
0
votes
0answers
5 views

Distributed process control

Today there is the supervisor who plays his role very well. I would like to implement a more distributed type of supervisor among several servers. For example, for each client I need some service ...
1
vote
0answers
13 views

Can someone suggest me a NoSQL database which is good for frequent updates?

I have heard that NoSQL databases like Cassandra, Couchbase etc are not good for frequent updates of data. They behave well when the data inserted doesn't change often. My use case is to save/update ...
0
votes
0answers
13 views

Metrics, monitoring or Distributed tracing?

Ours is a enterprise based SaaS application which is hosted on AWS. We use a lot of AWS resources including RDS, Dynamo, Kinesis, Memcache, SQS. Our application has accounts and each account can have ...
0
votes
1answer
20 views

Distributed data storage recommendations in mixed operating system environment for Python applications (Hadoop vs Databricks vs others)

I'm working on a big data project and have several old on-prem servers running a mix of operating systems (Ubuntu, CentOS, Windows 2012, Windows 10). One of the big reasons I would like a distributed ...
0
votes
0answers
16 views

Publish/subscribe software to handle at least 10k requests/sec and 10k simultaneous connection

I am on the verge to start making a "Large distributed application", but i'm struggling to decide what software is the most adequate to do so. My needs: A publish/subscribe application, which ...
0
votes
0answers
12 views

Simple tools to use for distributed stateful processing

I'm wondering which simple tools could help me implementing a standard Master <-> N Workers stateful job dispatch in Java. Master would be polling jobs from a broker and distributing it to worker ...
1
vote
0answers
38 views

How to store 1.5 GB CSV files?

I am searching a solution to store files of 1.5 GB. There is a decade of file per day. I am searching a solution to store those files. I have already done some searches on HDFS and MongoDB. However, ...
0
votes
0answers
9 views

Lightweight Python framework for I/O intensive, multicore, multimachine pipeline that utilises all the resources via spooling/scheduling/scaling

I have a pipeline of independent Python programs. They load their models on start (expensive operation, and not threadsafe) then wait for input. The input can be splitted into chunks which the program ...
0
votes
0answers
17 views

Leader Election with Customization

I have multiple micro-service instances of same type. Each service instance will have connections to some of remote services. In below diagram, I have shown 3 service instance and 3 remote service. ...
0
votes
0answers
13 views

Need a globally distributed FS with fast non-blocking writes

I am researching distributed file systems for a particular use-case. Our use case involves high-latency between nodes with the requirement of fast writes. The options I've seen so far do not seem ...
1
vote
0answers
20 views

Shareable List of Links With Notifications

So basically I am creating Tinycards decks for the vocabulary of each Duolingo Hebrew level that feature transliterations to practice learning not only the words and their meanings but also their ...
0
votes
0answers
15 views

Service that sends system information (and IP) periodically to central server

As part of my project I need to have a central server that multiple devices would periodically ping and send some system information (for example OS type/version, nothing too special). The purpose of ...
2
votes
0answers
18 views

How can i run my service on a clustered environment?

How can i run my service on a clustered environment ? Any products out there to help me out. This product should take care of managing the cluster, cluster failover, rebalancing in case of node ...
1
vote
0answers
23 views

Automated system testing and performance measurement for distributed system

I'm looking for a tool/combination of tools which would allow me to do system testing for application spanning multiple hosts. What I'm looking for: ability to spawn variable number of application ...
1
vote
0answers
113 views

Distributed build system for Rust

I'm looking for a distributed build system for Rust that: Is free, gratis and opensource Is cross-platform and cross-compiles (from ARMv6, ARMv7, i386 and amd64 Linux and amd64 Windows for amd64 ...
4
votes
2answers
132 views

A distributed (on-demand) build system

I'm tired of long build times … I'm interested in a distributed build system, either at the lower-level of running the equivalent of make, and not knowing anything about anything other than ...
0
votes
0answers
148 views

Suggestions for Distributed File System

Current Configuration: No of machines: 7-8 Hard Disk in each: 400 GB Memory in each machine: 60 GB Requirements: Each program can access many files. Since program is in c++, some interface in ...
0
votes
0answers
177 views

Alternative for HBase to store and query small files

I'm using Hadoop on my system to store my files (each file is very large: ~4GB). I also have small sqlite3 db files (~50MB) and I would like to store them (somewhere - I don't have shared storage) and ...
2
votes
0answers
36 views

Distributed, indexed regular expression search

I'm looking for a distributed solution to do regular expression searches over a large directory structure with many nested directories and files. Ideally it will be also able to search file/directory ...
2
votes
0answers
167 views

Cluster computing drop-in replacement for Python multiprocessing Pool

Python has a very handy multiprocessing built-in module. Particularly its Pool class allows simple parallelization. However, it only supports a shared-memory multiprocessing model, not a distributed ...
4
votes
1answer
49 views

Distributed updates over LAN for Windows

Does software exist that would make it easy to distribute software updates for a variety of programs over LAN? My job involves supporting numerous low-bandwidth offices with very expensive internet ...
2
votes
1answer
271 views

Linear algebra C/C++ library for distributed computing

My goal is to invert a large dense matrix in a distributed environment. I would like a library which would be "easily" integrated in a C++ application, thus it would be ideal that the library has a C/...
1
vote
2answers
101 views

Version control system on distributed files or database?

Do you know of any "version control" or sync tool able to work transparently on files stored on distributed file systems (such as Hadoop) or using their own database to keep the files? I'm looking ...
1
vote
0answers
56 views

Distributed job queue with task locality?

Context At present i have one server, which knows some tasks need to be scheduled at recurring intervals. So one process pushes such tasks into a message queue on a timer; and then other processes ...
5
votes
0answers
196 views

Free distributed wiki software with full-text search, working even offline

I'm looking for a way to organize knowledge in my company. Since we have to work disconnected from the company net many times, a server-client solution alone won't suffice. The requirements are: ...