1

Is there a software that allows me to 'connect' my local network computers (say a workstation, with high processing power) with a public hosted server and have them communicate pretty effortlessly?

I have docker containers deployed on a public server instance. This containers have very little processing power. I am looking to deploy some heavy data analytic power for the server without paying for dedicated GPUs. I want to set up a system whereby the workstation can be leveraged for processing if it is available, otherwise use the server.

I am not sure that kubernetes fits this need, although I could be wrong. I keep hearing about openstack, but can not seem to find anything concrete on it.

Any software out that allows this, is open source, and is security minded. By the way, I use Python for programming. I could leverage Python to SSH into the computer, process and return the info to the server, but that does not seem well integrated with my stack (docker containers) and seems more hacky. Something that could be used cleanly with a programming language (in my case python, even it is just json passing like we see will containers, kubelets, etc)

Improve this question
7
  • I can see why you'd want to do this and you should be able to -- in a better world. In this one, in my experience, people rarely do this, and some of the reasons are security and reliable connectivity. The latter may not matter for batch processing, but the former is very hard to fix, IMO.
    – Croad Langshan
    Commented Oct 19, 2020 at 23:05
  • @CroadLangshan what is the next best alternative?
    – David Frick
    Commented Oct 20, 2020 at 0:14
  • I don't know if I misread your question yesterday or I'm just feeling a more optimistic person today. I don't have a solid answer for you, but think were I looking for a solution I'd look for one that did not for example directly expose your workstation as a kubernetes ("k8s") node. Instead I'd split the problem (because security) into two: 1. Special-purpose highly restricted thing to fetch processing jobs (including the data needed to run them) and push job results back. 2. Potentially general-purpose tool to execute the jobs. k8s may be overkill for 2, but I have not used it myself.
    – Croad Langshan
    Commented Oct 20, 2020 at 19:54
  • Are you saing that k8s might work. I have heard of that so I might look into it. As for saying doing the batch job. I could always set up a python ssh program but then make sure I have to provide email and text confirmation for it to be allowed to connect. I definitely can see what you are saying!
    – David Frick
    Commented Oct 20, 2020 at 21:17
  • There are many tools "like this" in multiple software categories. I don't have a good overview, but my impression is that k8s has a reputation for complexity and configuration effort appropriate to major websites but not a small project. You might look at something like docs.celeryproject.org and configure it to use maybe sqlite as its backend?
    – Croad Langshan
    Commented Oct 22, 2020 at 20:12

1 Answer 1

Reset to default
0

Setup wireguard vpn. That can put your vps and local network into same network and they can communicate with each other as if they are in the same network.

Improve this answer
2
  • This "connect two networks" seems to be a feature of all VPN's. Is there a reason you mentioned wireguard specifically for this usecase?
    – DeveloperACE
    Commented Jun 2, 2021 at 18:31
  • No specific reason. Just that wireguard is a modern and easier to setup vpn
    – balki
    Commented Jun 3, 2021 at 13:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.