Category Archives: cryptocoins

14 Feb

rewriting the trading algorithm

I have the script now running at a realistic 1.14% per day return, taking into account the average number of times that I’ve had to pay a trading fee, the average offset of what price a trade happens vs what price the algorithm wanted it to happen, and other unpredictable things.

The script as it is, is very hard to train in any way other than almost a brute force fashion. Because most of the variables (length of EMA long tail, number of bars to measure for ATR, etc) are integers, I can’t hone into the right values using a momentum-based trainer, so I need to do it using long, laborious loops.

There is one improvement that I could do, which would probably double or more the return, without drastically changing the script. At the moment, there is a lot of “down time”, where the script has some cash sitting in the wallet, and is waiting for a good opportunity to jump on a deal. If the script were to consider other currencies at the same time, then it would be able to use that down time to buy in those currencies.

On second thought, I think the returns would be much higher, because when the script currently makes a BUY trade, its usually sold again within about 30 minutes. That means that it could potentially make 48 BUY trades in a day, each with an average of maybe a .5% return. That’s about a 27% return ((1+(.5/100))48 = 1.27).

That would be nice, but it’s impossible with the GDAX market, because the GDAX market only caters to four cryptocurrencies at the moment. Also, while the money aspect is nice, I’m actually doing this more for the puzzle-solving aspect. I get a real thrill out of adding in the real-life toils and troubles (fees, unpredictable trades, etc) and coming up with solutions that return interest despite those. So, I’m not going to refine the hell out of the script. Instead, I’m going to start working on another version.

Before I started on the current version, I had made a naive attempt at market prediction using neural networks. While some of the results looked very realistic, none of them stood up to scrutiny. I failed. But, I also learned a lot.

I’m going to make another neural-net-based attempt, using a different approach.

The idea I have is that instead of trying to predict the upcoming market values, I’ll simply try to predict whether I should buy or sell. This reduces the complexity a lot, because instead of trying to predict an arbitrary number, I’m only outputting a 1 (buy), 0 (nothing), or -1 (sell). This can then be checked by running the market from day 1 of the test data using each iteration of the network, and then adjusting the weights of the network based on whether the end result was higher or lower than the last time it was run.

I noticed that the best tunings from my current script only made a very few trades, on the very few days where the market basically dropped like a stone and then rebounded. But I can’t assume those will happen very often, so I like to see a few trades happen every day. With the neural network, I can increase the number of trades by simply adjusting the output function that decides whether a result is -1, 0, or 1 (this is usually done by converting the inputs into a value between -1 and 1, then rounding to an integer based on whether the value is above/below 0.7/-0.7. That value can be adjusted easily to provide more 1/-1 hits).

The current approach also involves a lot of math to measure ATR (average true range), running EMA (exponential moving average), etc. With the neural network approach, I will just need to squish the numbers down to form a pattern between -1 and 1 (where 0 is the current market price) and run directly against those numbers.

Because neural networks are a lot more “touchy-feely” than the current EMA/stop-gain/stop-loss approach I’m using, I will be able to “hone in” on good values – something I cannot do at the moment because of the step-like nature of integers.

I won’t be using any off-the-shell networks, as I can’t imagine how to write a good trainer for them. Instead, I’ll write my own, using a cascade-correlation approach.

Cascade-correlation is an approach to neural networks which allows a network to “grow” and gradually learn features of the input data. You train a network layer against every single input, until the output stops improving. You then “freeze” that network layer so is not adjusted anymore. You then create a second layer which trains against every single input and the previously trained network. You continue adding layers until there is no more noticeable improvement.

The point of this is that the first training layer will notice a feature in the data that produces a good result, and will train itself to recognise that feature very well. The second layer will then be able to ignore that feature (because it’s already being checked for), and find another one that improves the results. It’s like how you decide what animal is in a picture – does it have 6 legs (level one), is it red (level two), does it have a stinger (level three) – it’s a scorpion! Instead of trying to learn all the features of a successful sale at once, the algorithm picks up a new one at each level.

Around Christmas, I was playing with the FANN version of cascade correlation, and I think it’s very limited. It appears to create its new levels based on all inputs, but only the last feature-detection layer. Using the above example, this would make it difficult to recognise a black scorpion, as it would not be red. I believe that ideally, each feature layer should be treated as a separate new input, letting the end output make decisions based on multiple parallel features, not a single linear feature decision.

29 Jan

using LIMIT trades to reduce fees

I made two enhancements to my Litecoin market script over the weekend.

The first one was based on something I did last week, where it occured to me that if you buy when the ATR is low, you will probably lose money through fees because the market value will not shift enough for there to be a profit.

For example, if the value is 200 when you buy and 201 when you sell, then you will lose money. Let’s say you you €10 of coins. It will increase in value to €10.05 by the time you sell them, but you will pay just over €0.06 in fees.

An ATR limit large enough to discourage the script from buying when the margin is so small would stop this issue.

However, I realised that a limit that worked at around a value of 200 would not be effective at 400.

The solution was to make a new limit which is inversely proportional to the market value. Let’s say the number is 50. Then it would look for an ATR of 0.25 if the market value was 200, and an ATR of 0.125 if the value was 400.

This made a remarkable difference in my simulation. It now estimates a 3.15% return per day based on the configuration figures I found.

Last week’s version ended up with about 14 buys in a 50 day period, which meant that there was only about one buy every 4 days, making it look like it wasn’t doing anything.

Now, it has what looks like about 32 events per day. A lot of them are repeats, where a sell signal might pop up from a chandelier exit followed by another from exponential moving average or simple moving average, but it’s still a lot more, making the script feel a lot more alive.

This is helped by me changing the trade method.

I had it on MARKET trades, which are virtually guaranteed sales/buys, but also are guaranteed 0.3% fees.

I’ve changed that to LIMIT trades that work in a way that might not trade at exactly what was requested, but should not trigger a fee at all (at least, I haven’t had a single fee yet in my tests!).

How it works: let’s say the market value is 200 right now. The script will check the order books, which might currently have an “ask” of 200.01 and a “bid” of 199.99 (for example). If we are trying to sell, then we add an sell/ask order of 200.02 (current+0.01). If the market value goes down, then we cancel that order and create a new one based on whatever the new value is.

And vice versa with buys/bids.

This means we probably won’t get exactly what we want, because we are relying on market jitter to make the trade happen. But at least we won’t have a fee to worry about!

25 Jan

using ATR to restrict market purchases

After re-checking the simulations I’d done recently, I realised I had the wrong broker fee marked in, which meant that even though the simulation said I was making a 1% return every day, I was actually losing money in reality.

The script was originally making LIMIT buys and sells, which my experience showed rarely caused a fee, so I’d marked that as a 0.1% fee in the simulation vs the normal 0.3% that a MARKET trade has. But the simulation didn’t take into account that LIMIT trades sometimes don’t get filled. The market might be at 147, for example, so you try to sell your 4 Litecoin (eg) for 147, but by the time you get the order uploaded, the market might have shifted down. Now you’re stuck with an unfulfilled order.

To solve this, I changed the script to only use MARKET trades, which are guaranteed to sell/buy, but also are guaranteed to incur a fee of 0.3%.

When I plugged 0.3% into my simulation, I was suddenly not making any money at all. Instead, I was losing money. In fact, the simulation showed I was losing money so badly that if I started with €1,000,000 46 days ago, I would be down to €14 today. That’s bad.

Looking at a 6hr chart of the Litecoin market, we can see where the money vanished.

On the 6hr chart, the highlighted area looks very flat, like nothing is happening there. So you would not expect any trade to happen. But if you zoom into the area, you can see that trades are still happening, even though they value isn’t changed very much in absolute terms.

Let’s zoom in even further, to the area that’s highlighted:

You can see that the value is rising and falling vigorously, but this is an illusion. If you look at the figures on the right, you see that the LTC is oscillating between about €142 per coin to €150 per coin. That’s about a 5% range. Remember that the fee we’re trying to avoid is 0.3%, so this /might/ be okay to trade with.

But the script trades minute by minute, not hour by hour. So let’s look at what the market does in that range by zooming in even further:

That’s one hour of data. In the preceding image, the above image is what’s contained in the single bar all the way on the right. In it, the price of Litecoin rises steadily from €144 to €146.

Worth buying?

144 to 146 is less than 1.4% of a rise. If you buy €100 worth of LTC at the beginning of this rise, and sell it at the end, it will cost you €0.30 to buy (so you’ll actually spend €100.30), the value will increase from €100 to €101.39, and then when you sell, there will be a fee of €0.31.

So even though the price went up, from €100 to €101.39, you will only have made €0.78.

This is still a profit, but if the rise was less, it might have been a loss.

Example, let’s say it was 200 to 201

You invest €100 and pay your €0.30 fee. The value increases to €100.50. You then cash out. Now you’ve just lost money, because the fees add up to €0.61, but the price increase was only €0.50.

Buying when the increase is so small can be dangerous, because if the increase isn’t enough, then you will lose money.

So how to solve this?

The problem has to do with the volatility of the market. For the last few days in Litecoin land, there pretty-much hasn’t been any!

After realising the above, I made a small adjustment to my script so that it refused to allow any trades at all if the volatility was too small. I did this by measuring the ATR (average true range), and if it was below a certain range (1.5 or so), then even if the script signalled a buy or sell, it was stopped in its tracks.

This had an immediate and amazing effect on the returns.

Beforehand, with a 0.1% fee, I was getting a 1.25% return daily in my simulations, but now I’m getting a 2.8% return daily on a 0.3% fee. That’s HUGE! In money terms, if I invested €20 46 days ago on the 9th of December (chosen by finding the oldest data point which had a value higher than the most recent data point), then that €20 would now mbe worth €73.80.

You can check these figures for yourself.

Here’s the list of buys and sells that it came up with for that time. Compare that with the GDAX charts to see where it was making its decisions:

DateTotal HoldingsEuroLitecoinDecision
2017-12-09 06:16:0020200sell (EMA)
2017-12-09 18:19:0019.940636021120.152643061120.1467952buy (SMA)
2017-12-12 03:31:0029.36742425154629.0685282375460.0014974sell (EMA)
2017-12-12 03:48:0029.3042030636560.2218644596560.1352604buy (EMA)
2017-12-12 15:14:0038.52175170325638.1299169032560.0013797sell (EMA)
2017-12-12 15:31:0038.4582437192560.2910117192560.1192726buy (EMA)
2017-12-12 16:15:0035.48967840693635.1295769729360.0012166sell (EMA)
2017-12-13 19:09:0035.3404169406610.2681059496610.1354091buy (SMA)
2017-12-19 21:14:0039.43499144845439.0342915164540.0013812sell (Chandelier exit)
2017-12-20 07:04:0039.2997793092440.2979209192440.1412599buy (SMA)
2017-12-20 13:56:0040.35716812173439.9473329347340.0014409sell (SMA)
2017-12-22 04:12:0040.1758308880540.3048847450540.1655289buy (SMA)
2017-12-22 05:22:0038.59355655705438.2018477570540.0016884sell (EMA)
2017-12-22 07:54:0038.4303411583720.2915521483720.188349buy (SMA)
2017-12-22 08:14:0035.96596461087435.6009942468740.0019212sell (EMA)
2017-12-22 08:37:0035.8814845811740.2717014811740.1771631buy (EMA)
2017-12-22 11:24:0039.40210318013439.0017763171340.0018071sell (EMA)
2017-12-22 15:15:0039.2239389743340.2976636743340.2081619buy (SMA)
2017-12-22 23:36:0049.65572643277849.1507632267780.0021233sell (EMA)
2017-12-24 03:46:0049.5232776772680.3751258122680.2013031buy (SMA)
2017-12-24 03:53:0048.58655807989448.0933348868940.0020533sell (EMA)
2017-12-24 12:22:0048.4228433058940.3670573058940.2089382buy (SMA)
2017-12-25 02:34:0051.42157858321450.8992640872140.0021312sell (Chandelier exit)
2017-12-28 19:25:0051.2085454123310.3884559393310.2352673buy (SMA)
2018-01-06 05:27:0053.76221736848153.2161668764810.0023998sell (SMA)
2018-01-16 22:55:0053.3952865952710.4061476172710.3772543buy (SMA)
2018-01-17 01:04:0058.59785130425258.0025272242520.003848sell (EMA)
2018-01-17 16:11:0058.3115950181180.4426687601180.4627293buy (SMA)
2018-01-17 22:56:0073.79815697631873.0476928763180.0047199sell (EMA)

 

Notice that the script has not made any decisions in the last few days (today’s the 25th). That’s because the last few days, there’s been nothing interesting happening in the market so it’s holding on to what it has.

Here’s a pretty picture showing the above in line format

21 Jan

automated script for the Litecoin market

Over Christmas, I started looking into how stock markets work and decided to give it a shot. The simplest way I found to start off on it was actually the cryptocoin market. I chose Coinbase’s GDAX market as the one I’d work on.

At first, I had a naive idea that I just needed to watch the numbers on the market for times when the latest figure is the lowest in a long time. Then you buy. And then the opposite happens -if the latest bar is the highest in a long time, sell.

It turns out that doesn’t work. I wrote a testing application that downloaded 6 months of per-minute data about the LTC-EUR market and ran simulations against it to figure out what would happen if I was to trade based on my algorithms. The first one (above) sucked.

So I started looking a bit further into how traders actually do it themselves.

It turns out it’s pretty simple, if you’re willing to put the testing time in and come up with some good configuration numbers.

The first thing I checked out was called “MACD” (Moving Average Convergence Divergence). That uses a simple moving average (SMA) of the market value to generate two lines – a “long” average based on 26 figures, and a “short” average based on 12 figures. When the short average crosses over the long, it signals an action. For example, if the current short value is higher than the current long, and the last calculations were the opposite (short under long), then that indicates you should Buy, because it looks like there is an upwards trend. The opposite happens when the crossover shows the short going under the long. Sell.

The 12 and 26 figures are traditional. You could work based on them, but my tests showed that there are different figures that can give you better results. I guess it depends on the market. My current settings here are 25/43 instead of 12/26.

The next thing I worked on was a “Chandelier Exit”. This is a strategy for cutting your losses when the market suddenly drops more than usual. To do this, you measure the “ATR” (average true range) for the last n periods (traditionally 22). You then multiply the ATR by a volatility value (traditionally 3), subtract that from the current High value, and if the current market value is below that, Sell. My current values for this are a volatility of 5.59 based on an ATR of 18 bars.

I then looked at exponential moving average based MACD. The standard moving average is a straightforward average of n numbers, but the EMA puts more weight on the more recent numbers, so it reacts quicker to changes in the market.

After trying to tune the EMA for a while, I found that if I use EMA instead of SMA, then I get worse results, because the script would buy quickly when it saw an upward trend, but that might turn out to be just jitter and you’ll lose it all immediately afterwards. It’s safe to sell when the market drops, it’s not safe to buy when the market looks like it’s just starting to rise. it’s better to take your time.

So, I added a switch to my code so that I could decide whether to use SMA or EMA for buys and sells, etc.

I found that the combination that gives the best results uses only SMA for buys, but then uses all of SMA, EMA and Chandelier exits to signal a sell. Oh – EMA of 40 and 80.

Doing this, I’ve been able to come up with a configuration of my script that gives an average return of about 1.1%. This means that if you were to invest €5000, then there would be “interest” of about €55 per day. Or if you can keep your money in the game, it starts to grow. €50 invested for 365 days at 1.1% interest per day is €2711.

If you’re interested in going the script a shot, you can download it from here.

I keep on having more ideas on how to improve this.