zosoforever replies to: General tech-talk

Continuation from:
HI there again,

I am desperate to get out of the MT4/5 retail brokerage space and go the FIX route. I am zero’ng on Python to replace my EA automation so research to build a solution in parallel has begun. Is there a better alternative to Python? I came across this but haven’t played around much.

I would also like some GUI frontend to make things easier. My order execution piece is pretty straightforward but the administrative piece (management functions like DD, margin, trade, RM, exposure etc) is quite advanced and complicated to transform that into a brand new language. I am looking for something that comes with a “building block” approach that I can plug and play. I am looking for a quicker alternative as I have no desire to spend next two years replacing mt4. I am looking for a multi-broker, multi-market/exchanges/DMA type solution….away from the likes of Oanda/Forex.com/retail servers.

EDIT: My automation is extremely CPU intensive… it goes exponential the more pairs are trade and goes even higher the more orders I have open. MT4 (demo) simply clocks when I reach the 200+ open trades

  1. Post 18
  2. Cleanup
  3. Quote
  4. Aug 22, 22:09 (2 hr 24 min ago)
  1. https://cdn-assets.forexfactory.net/…r459520_11.gif masterx584
  2. | Joined Apr 2016 | Status: LiquidatorSoft | 16 Posts | Invisible

Quoting VEEFX
{quote} HI there again, I am desperate to get out of the MT4/5 retail brokerage space and go the FIX route. I am zero’ng on Python to replace my EA automation so research to build a solution in parallel has begun. Is there a better alternative to Python? I came across this but haven’t played around much. I would also like some GUI frontend to make things easier. My order execution piece is pretty straightforward but the administrative piece (management functions like DD, margin, trade, RM, exposure…
The entire MQ4 code can be converted to C++.On my system under C++ the CPU runs 0% with 350 threads at 1ms sleep time.MT4 is killing CPU’s because using WinApi for draw functions not because of the math calculations and isn’t even allowing multithreading.More colors text or draw on your charts more CPU needed.
Quantitative algorithmic to build smart data packets.
Liquidator Return This Week: 1.8% AUD/USD

  1. Post 19
  2. Cleanup
  3. Quote
  4. Aug 22, 22:16 (2 hr 17 min ago)
  1. VEEFX
  2. | Joined Jun 2006 | Status: Member | 1,713 Posts | Invisible

Quoting masterx584
{quote} The entire MQ4 code can be converted to C++.On my system under C++ the CPU runs 0% with 350 threads and 1ms sleep time.MT4 is killing CPU’s because using WinApi for draw functions not because of the math calculations and isn’t even allowing multithreading.More colors text or draw on your charts more CPU needed.
I agree. It’s the single threading limitation… and I use no charts and no technical indicators so I know it is not my graphics adapter. I learnt C in the 80s in high school and then some more for my post-graduate project to develop a compiler for a language. Is there a mq4 to C++ convertor that I can play with?

EDIT: Its the 9 levels deep nested recursive logic which hogs my CPU on every tick
Attached Image
https://www.forexfactory.com/attachm…1&d=1534973088

  1. Post 20
  2. Cleanup
  3. Quote
  4. Aug 22, 22:47 (1 hr 46 min ago) | Edited Aug 22, 23:01
  1. emmzett
  2. | Joined Apr 2008 | Status: Member | 58 Posts

Quoting VEEFX
{quote} HI there again, I am desperate to get out of the MT4/5 retail brokerage space and go the FIX route…
OK, this is going now completely off-topic here but I’ll try to give some general answers.

Language is irrelevant, simplicity and ROI are important (KISS). If you’r already familiar/professional with one language don’t switch (as long as it’s not VB or Excel). Plan and measure in *your* time, that’s the resource you can control.

FIX is the standard for institutions but it’s an unimportant detail for retail. Good to know but not important. You don’t need FIX to become consistently profitable and you will not increase your chances to do so by using it. The opposite is true. What really matters is what stage you are on with your trading.

Becoming broker/platform independant is a natural wish, especially as MetaQuotes will cease to exist as we knew it. What you need for this depends on your strategy (if you have one). The options we have in the future in retail space are platforms like cTrader, Quantower etc. You can setup a broker independant MT4 for charting and market analysis if you want to, even if there’s no MetaQuotes anymore. But if you don’t need charting but only market scanning it will be completely different. May be you understand now that there can be no general answer as it really depends on your needs. Let’s say we don’t have a strategy and need a good charting module. OK, we might get used to MT5 for this (this is all MT5 is good for). Retail more than others needs to be able to adapt.

The real issue with retail automation is not finding a profitable strategy. The real issue is managing the infrastructure and potential infrastructure errors. A realistic backtest showing 7% return per month over 5 years will not work out as expected in a live account, as your account will crash once or twice a year because you have a network outage or made a stupid coding mistake. There is not a single code example on mql5.com or forexfactory.com with proper error handling. Think about this if you have experience in IT. On the other hand there is always room for unexpected error. Problem here: you can test your strategy but you cannot test in a backtester (whatever platform) for those unexpected errors.

So in summary the real issue is not what platform/language do I use. The real issue is how do I prepare my trading for the…let’s call it Black Swan events. They will always come and they are the real threat. It might be your fault, it might be a 15th of January 2015. It might turn out that I better control such risk by manual trading (this is what institutions do). Manual trading also makes you independant of platform changes. But it requires completely different skills which a retail trader can aquire only in a very difficult way.

So focus on what you already can do and simplify that. Don’t complicate it by introducing more abstractions (e.g. FIX). At the current moment in time there is no need for FIX in the retail space. I say “need”…

1

  1. Post 21
  2. Cleanup
  3. Quote
  4. Aug 22, 22:50 (1 hr 43 min ago)
  1. https://cdn-assets.forexfactory.net/…r459520_11.gif masterx584
  2. | Joined Apr 2016 | Status: LiquidatorSoft | 16 Posts | Invisible

Quoting VEEFX
{quote} I agree. It’s the single threading limitation… and I use no charts and no technical indicators so I know it is not my graphics adapter. I learnt C in the 80s in high school and then some more for my post-graduate project to develop a compiler for a language. Is there a mq4 to C++ convertor that I can play with?
Only the hard way as i know.U need to add Winsock for connections , a draw engine for C++ environment ,to rewrite all the algorithms or to create new multithreaded ones.. to allocate memory etc. c++ have alot of elasticity and can compile assembly language inside ,i have a few functions in machine code for more speed.It’s easy for me to speak now but i worked almost 2 years on the entire thing …mostly on my multithreaded algorithms.Anyway if u start to build your C++ system i can help u with things like multithreaded clients,graphics ,how to build charts or how to transform functions like NormalizeDouble which is pain..u can almost create a copy of MT4 …how to completely eliminate the netwotk lag.
Quantitative algorithmic to build smart data packets.
Liquidator Return This Week: 1.8% AUD/USD

  1. Post 22
  2. Cleanup
  3. Quote
  4. Aug 22, 22:50 (1 hr 42 min ago)
  1. emmzett
  2. | Joined Apr 2008 | Status: Member | 58 Posts

Quoting VEEFX
Its the 9 levels deep nested recursive logic which hogs my CPU on every tick
Never do this. A 9 level deep loop will kill every performance, even if it’s written in Assembler.

  1. Post 23
  2. Cleanup
  3. Quote
  4. Aug 22, 22:56 (1 hr 36 min ago) | Edited Aug 22, 23:26
  1. https://cdn-assets.forexfactory.net/…r459520_11.gif masterx584
  2. | Joined Apr 2016 | Status: LiquidatorSoft | 16 Posts | Invisible

Quoting VEEFX
{quote} I agree. It’s the single threading limitation… and I use no charts and no technical indicators so I know it is not my graphics adapter. I learnt C in the 80s in high school and then some more for my post-graduate project to develop a compiler for a language. Is there a mq4 to C++ convertor that I can play with? EDIT: Its the 9 levels deep nested recursive logic which hogs my CPU on every tick {image}
U have a 9 level big problem …break it into multiple small problems …go multithreading.
Attached Image (click to enlarge)
https://www.forexfactory.com/attachm…1&d=1534975667

Right now i am with only 123 threads ,8 clients..16 threads *client,used to connect 20 clients before and 340+ threads.I calculate tick by tick data no timed bars.My threads are setted to 1ms sleep.They keep pushing.The algorithm alone divided in multiple cells running through threads is 3000+ code lines totally.It’s more about how u divide & conquer.
Quantitative algorithmic to build smart data packets.
Liquidator Return This Week: 1.8% AUD/USD

  1. Post 24
  2. Cleanup
  3. Quote
  4. Aug 22, 23:31 (1 hr 2 min ago)
  1. emmzett
  2. | Joined Apr 2008 | Status: Member | 58 Posts

OK, we already hijacked the thread. Sorry to the OP.

Key to becoming better/more independant/more successful is not improving our platform or an indicator or a strategy.

Key is awareness. Awareness *why* the natural consequence of our trading approach as a retail trader is for the vast majority failure. If we gain that awareness we are able to make the required changes. Without that awareness we are doomed. A platform or language will not help us.
Attached Image (click to enlarge)
https://www.forexfactory.com/attachm…1&d=1534976458

As Morad Askar says: The statistics about 99% of failing retail traders are correct. But it is a useless statistic. Online trading has no entry barriers, a 14 year old can open an account, upload the ID of his sister which lays around in her room all day long and fund the account with the credit card of his mother which lays around in the kitchen.

Take a chirurg with proper study and years of experience. Take 100 people you find randomly in the streets and give each one of them a scalpell. What percentage of the 100 people do you expect to become successfull chirurgs?

None. 100 percent failure. Is this statistic correct? Yes. Does it correctly depict your chances to become a successfull chirurg? Of course not.

It’s the same with trading. Awareness is key.
Inserted Video

  1. Post 25
  2. Cleanup
  3. Quote
  4. Edit
  5. Aug 22, 23:47 (45 min ago)
  1. https://cdn-assets.forexfactory.net/…atar2942_7.gif Nicholishen
  2. Joined Jul 2005 | Status: Member | 997 Posts | Invisible

Quoting masterx584
{quote} U have a 9 level big problem …break it into multiple small problems …go multithreading. {image} Right now i am with only 123 threads ,8 clients..16 threads *client,used to connect 20 clients before and 340+ threads.I calculate tick by tick data no timed bars.My threads are setted to 1ms sleep.They keep pushing.The algorithm alone divided in multiple cells running through threads is 3000+ code lines totally.It’s more about how u divide & conquer.
Multithreading aids in IO bound, flat, and asynchronous operation. For CPU intensive tasks, multiprocessing is required. If the algorithm (actually) requires nested loops then neither would be useful since the each inner loop is interdependent on the iteration outcome of its outer loop. I’ve rarely ran up against a problem that couldn’t be flattened, but I won’t presume to know the logic in question, and assume that OP knows what he/she is doing.

  1. Post 26
  2. Cleanup
  3. Quote
  4. Aug 22, 23:48 (45 min ago)
  1. https://cdn-assets.forexfactory.net/…r459520_11.gif masterx584
  2. | Joined Apr 2016 | Status: LiquidatorSoft | 16 Posts | Invisible

Quoting emmzett
OK, we already hijacked the thread. Sorry to the OP. Key to becoming better/more independant/more successful is not improving our platform or an indicator or a strategy. Key is awareness. Awareness *why* the natural consequence of our trading approach as a retail trader is for the vast majority failure. If we gain that awareness we are able to make the required changes. Without that awareness we are doomed. A platform or language will not help us. {image} As Morad Askar says: The statistics about 99% of failing retail traders are correct. But it…
hahah, u think i was so stupid to loose 2 years to build and put my algos into a completely isolated and secured network if they weren’t valid?..away from any broker spy plugins … with OP hijacked most of the services disabled completely invisible against Kali Linux or Parrot Security tested in the same network…i think who will work harder will be more succesfull in life.Who want to take it professionally need to cut brokers from this ecuation with all the services they offering and all the preparation or webinars…What u put is what u get…as allways.. and i am able to take trades without never watching the Bid…only the spread.Peace .
Quantitative algorithmic to build smart data packets.
Liquidator Return This Week: 1.8% AUD/USD

  1. Post 27
  2. Cleanup
  3. Quote
  4. Aug 23, 0:03 (29 min ago)
  1. VEEFX
  2. | Joined Jun 2006 | Status: Member | 1,713 Posts | Invisible

Quoting emmzett
{quote} Never do this. A 9 level deep loop will kill every performance, even if it’s written in Assembler.
Thanks emmzett and masterx for the responses. I really appreciate it. Level 9 deep is only on one pair….. the depth also exponentially increases depending on when my signal confluence happens which triggers a recursive crawl if you know what I mean but I have capped it at 32 levels. It’s is designed as a sequential crawl and cannot be done via parallel processing so to mimic multi-thread, I run it across several terminals plus one terminal to handle all portfolio/trade mgmt functions but still my CPU clocks continuously non stop at 50%-60%. Ultimately, I need to use Spot to trade regulated futures, indices, sub-sector ETFs etc…. any market that is a derivative of underlying instruments as this model is quite expandable to other markets (because it is purely data driven and contained within the weekly candle so not a whole lot of volume of data involved) so yeah… i do have a strategy. I just do not want reliance on a broker-provided platform. Order Execution Speed/latency/slippage etc is not my concern at all as they have no or minimum impact on my overall outcome. What’s important is the timely processing (computations) to complete within one minute for the signal to remain valid.

  1. Post 28
  2. Cleanup
  3. Quote
  4. Aug 23, 0:09 (24 min ago)
  1. zosoforever
  2. | Joined Sep 2010 | Status: Member | 55 Posts | Invisible

Hi Guys,
This must be very interesting conversation, but it is not related to FIX API.
It would be more beneficiary for you to start your own thread. Also for us who want to discuss FIX API.
Thanks

  1. Post 29
  2. Cleanup
  3. Quote
  4. Aug 23, 0:14 (19 min ago) | Edited Aug 23, 0:26
  1. https://cdn-assets.forexfactory.net/…r459520_11.gif masterx584
  2. | Joined Apr 2016 | Status: LiquidatorSoft | 16 Posts | Invisible

Quoting VEEFX
{quote} Thanks emmzett and masterx for the responses. I really appreciate it. Level 9 deep is only on one pair….. the depth also exponentially increases depending on when my signal confluence happens which triggers a recursive crawl if you know what I mean but I have capped it at 32 levels. It’s is designed as a sequential crawl and cannot be done via parallel processing so to mimic multi-thread, I run it across several terminals plus one terminal to handle all portfolio/trade mgmt functions but still my CPU clocks continuously non stop at 50%-60%….
My initial computation takes 30 sec too but i am able to pack that data in a way that allows me to save it on HDD & to reutilize it with the new arrived ticks as it is without any other computations.I am only making ticks based calculations no timeframed bars.I create my own “smart packets” that can be utilised as it is.I pack and create my own history … don’t know the mkt price for none of my trading pairs… am only based on the market value *strength, no price action,high lows or whatever.I don’t care what price it has but only if the market is into a value*strength accumulation range or is going to explode after being ranging for a while.I can wait 2-3 weeks to take a pair outside of the range going to replace a top trending one who’s weakening and ready to get into accumulation.Is allways an exchange.
Quantitative algorithmic to build smart data packets.
Liquidator Return This Week: 1.8% AUD/USD

1

  1. Post 30
  2. Cleanup
  3. Quote
  4. Last Post: Aug 23, 0:30 (3 min ago)
  1. https://cdn-assets.forexfactory.net/…r459520_11.gif masterx584
  2. | Joined Apr 2016 | Status: LiquidatorSoft | 16 Posts | Invisible

Quoting zosoforever
Hi Guys, This must be very interesting conversation, but it is not related to FIX API. It would be more beneficiary for you to start your own thread. Also for us who want to discuss FIX API. Thanks
That’s true.I’m very sorry.I’m done.See u guys.

Leave a Reply

Your email address will not be published. Required fields are marked *