I want to create a code or a program that I will use in order to investigate market opportunities and system optimization. What I want the program to be able to do is among other (but not limited to) the following:

  1. Collect and analyze a huge set of data points (market signals that are updated maybe once every 5 seconds, i.e. too big for excel). Preferably it can collect this data and update the data set + result continuously and automatically
  2. From this data set it decides the volume of the commodity that is collected as every signal means a specific action, i.e. data signal 3 ALWAYS means add 3 worth of bla bla to your stack while -3 ALWAYS means remove 3 worth of bla bla from the stack (depending on the speed of the system, however each system reacts the same way to the same signal every time).
  3. The stack has a maximum and minimum volume. This is custom for each market and stack so it has to be able to change.
  4. The speed of which the commodity can be bought is also a subject to case to case basis, however the speed is fixed to the signal from the system.
  5. These signals shall be able to pair up with another set of data points that tells how much the commodity cost or what the income is, depending on the commodity.
  6. This shall be compared to the system cost that is subject to change in speed etc in order to optimize the system to the revenue.

There are some other things to take into account, i.e. losses in the system etc., however this should not be necessary to know for my question I think.

My question is: is there a software I can use for this type of service? As I would do this on my free time to begin with I would like to know especially if there is one that is free to begin with, and if I get the program to work I would be able to pay for it. If there is no software for this, what programming language is optimal to use in order to create something like this yourself? I'm asking so that I will be able to look for the right person to help me create it.

If anything is unclear please don't hesitate to ask so that I perhaps can clarify it all.

Thank you in advance!

  • 1
    Can you specify "huge set of data points"? Does "More than Excel can handle" mean more than 65k which get updated every 5 seconds? Some of your requirements sound like you want a database, but it's not clear how often how much gets updated and how fast the reaction needs to be. – kutschkem Nov 13 '19 at 14:34
  • Python with the Pandas library. – Eric Shain Nov 15 '19 at 2:00
  • Hello, sorry for disappearing for a while. Thank you for the clarification questions. Technically data measurements could be made an infinite amount of times as it is a continuously changing value, however it would be sufficient with maybe once every 1-5 seconds, however it is 24/7/365. Some of the measurements can be made at site, but some would be required to be downloaded continuously online. – Mikael Nov 26 '19 at 13:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.