I’ve always been fascinated by technology. I love building all types of electronic projects and automating things. Recently I ran across a guy who discussed building a battery backup using recycled scooter batteries that contained 18650 rechargeable batteries. The same ones that you find in Tesla cars and Powerwalls.
I started thinking. I wonder if it would be possible to trickle charge a battery during low-cost energy hours (6PM-3PM) and then dump it back into the grid when the electricity was expensive(3PM-6PM). Or at least offset your peak hour usage with battery power. So I created some formulas and crunched some numbers. Before you read any further I want to say this is a complete hypothetical experiment. I would never connect this to my SRP system and risk them terminating my services.
Here is a list of SRP’s different electricity rates. You can see the greatest offset is in the months of May, June, July, August, September, and October. The variance in winter is not that great and you really can’t save that much using this method.
Analyzing the ROI this setup using a simple 3kWh battery and a 1kW inverter we can yield 18.27% with a completely paid for system in 5.47 years. Not a bad return for someone with a little bit of money to invest.
A popular strategy that I have followed in my investments is crowdsourcing hedge fund manager investments. Many people don’t know this but any hedge fund that manages over $100 million dollars must report their stock holdings on a form called a 13f. This is filed with the SEC. There is a website that conveniently maps all of these filings so you can scan through your favorite investment manager and see what stocks they are buying and selling.
For instance, if you’re a fan of Warren Buffett and Charlie Munger you can take a look at Berkshire Hathaway’s holdings – https://whalewisdom.com/filer/berkshire-hathaway-inc Here is a list of their holdings which are over 1% of their portfolio
Or may be you’re a fan of Ray Dalio and you want to see what Bridgewater is holding? Here is a list of all their holdings that represent more than 1% of their portfolio.
Let’s take a look at Jim Simon’s fund Renaissance Technologies.
You get the point. You are able to get some pretty good insight as to what massive hedge funds are buying.
What I like about Whale Wisdom is they categorize the most successful hedge fund managers using what they refer to as a WhaleIndex.They then put together a list of 30 stocks based on successful fund managers. Some of their requirements are as follows.
Between 5 and 750 holdings in their 13F filing
At least 3 consecutive years of quarterly 13F filings
Hold no fewer than five stocks in its portfolio
Manage more than $100 million in marketable securities
Hold at least 20% of its portfolio in its top 20 stocks
Managers considered to be a bank, trust, pension, or insurance company are excluded
The top 40 managers who have maintained an average WhaleScore over the past 5-years higher than the five-year average WhaleScore of the S&P 500 is used in the WhaleIndex. Based on the holdings disclosed on their SEC filings, WhaleWisdom identifies the 100 stocks most commonly held among the respective managers’ 13F holdings.
The Whale Fund 2.0 is the one I follow. Since 2006 this strategy has yielded 1,345% – https://whalewisdom.com/whaleindex/portfolio_2_0
A few words of caution
13f filings come out 45 days after the quarter ends. This means the data is somewhat stale. It should also be noted that the fund could have purchased that stock at any time during the quarter. Meaning the data could be as old as 135 days. Secondly, funds are not required to report short positions or hedged positions. So you should not assume you know exactly what their portfolio consists of.
You’ll want to find a broker that allows for fractional investing if you don’t have enough money to buy full shares. Here are a couple for reference.
Despite its downfalls, Whale Wisdom’s returns are still solid. While I wouldn’t recommend this strategy for your entire portfolio this is a good strategy to deploy a fixed percentage of your portfolio. There is also a book that written about this concept which you can find here – https://www.amazon.com/Invest-With-The-House-Hacking-ebook/dp/B01A3L1VEO and also an ETF that was created named VIP ETF
At times when creating trading strategies using big data you need access to historical economic data. One of the best sources of data is the Economic Research branch of the St. Louis Federal Reserve or FRED. Today I’m going to show you how to pull that data into a dataframe so that you can analyze it using machine learning or AI.
When analyzing historical time frame data in machine learning it needs to be normalized. In this code example I will show how to get S&P data then convert it to a percent of daily increase/decrease as well as a logarithmic daily increase/decrease.
Colab can be found here – https://colab.research.google.com/drive/1Wvh7mRX0PUthBhXw1HD5tJAC8mQr6DYH?usp=sharing
The first part of this code will use yfinance as our datasource.
#we're going to use yfinance as our data source
!pip install yfinance --upgrade --no-cache-dir
import pandas as pd
import numpy as np
import yfinance as yf
Next we’re going to create a dataframe called df and download SPY data from 2000 to it. Finally we’ll print the result of df so you can get an idea of what is inside of it.
#Here we're creating a dataframe for spy data from 2000-current
df = yf.download('spy',
#actions='inline',) #adjust for stock splits and dividends
#print the dataframe to see what lives in it
We’re going to drop all the columns except Adj Close. Then we’ll rename it adj_close. Next we’ll create a column labeled simple_rtn. This is the daily simple return or percent increase/decrease. The next line of code creates a logarithmic increase/decrease. Logarithmic gives equal bearing to the Y axis and can be defined as follows, “A logarithmic price scale uses the percentage of change to plot data points, so, the scale prices are not positioned equidistantly. A linear price scale uses an equal value between price scales providing an equal distance between values.”
A few years ago Facebook decided to open source Prophet. This is their analytics algorithm that uses an additive model to fit non-linear data with seasonality. I started to wonder, “If this algorithm were in place in March when the stock market’s crashed what would it have advised?” So I decided to give it a spin.
Let’s assume you had a significant amount of money invested in S&P index funds on March 23, 2020. Since the beginning of the year, you would have lost 31% of your money. At this point you might be thinking, “Oh shit, what do I do, sell, buy, hold?”. A lot of investors would panic and sell. The market thrives and two psychologies, fear and greed.
But let’s take an analytics approach to this problem. What would Facebook’s Prophet algorithm advise you do to? Here is how you can approach that problem.
The first thing I did was fire up Google Colab. The entire notebook can be found here.
The first part of this code uses a Python DataReader to pull SPY from Yahoo Finance. I created an end date of 3/23/2020. This only gives Facebook’s Prophet access to data up until this point. We are then going to have it predict where it thinks the price would be today 8/20/2020 without feeding it any future data.
import pandas as pd
from fbprophet import Prophet
from pandas_datareader import data as web
import pandas as pd
import matplotlib.pyplot as plt
stock = 'spy'
endDate = datetime.datetime(2020, 3, 23)
#start_date = (datetime.datetime.now() - datetime.timedelta(days=2000)).strftime("%m-%d-%Y")
start_date = (endDate - datetime.timedelta(days=2000)).strftime("%m-%d-%Y")
df = web.DataReader(stock, data_source='yahoo', start=start_date,end=endDate)
#date is the index so you need to make it a column
df["Date"] = df.index
import matplotlib.pyplot as plt
The next part of code renames the imported columns from “Date” to “ds” and “Close” to “y”. DS and Y are the two variables that Prophet will be looking for.
This next part of the code starts to set up Prophet. The only variable you should be concerned with is 151. This is telling the algorithm to look out 151 days. Then it forecasts three variables yhat, yhat_lower, and yhat_upper. Yhat is the predicted price with upper and lower being the bounds in which it assumes the price will fall in.
You can see that it predicts the closing price for tomorrow will be 331.52. Remember it is only using data from March 23rd to make this calculation. Given the wild gyrations in the market, this is extremely close to being accurate. The SPY closed today at 338. Prophet predicted it would close at 331. It was off by 2%. This is using a 5-month look ahead forecasting model.
Here is the visual representation of what that looks like.
from fbprophet.plot import plot_plotly, plot_components_plotly
And more charts…
Finally here is the visualization of the predicted price vs. the actual price chart.
If you’re working with a default installation of MT4 or Metatrader chances are you modeling quality is not very good. I have composed this post to show how I import data into MT4 for better backtesting performance.
1. You will want to download the file that pertains to the pair you will be trading.
2. The next step is unzipping the file using a simple Zip program.
3. Before proceeding to import your new data you will want to go ahead and delete all of your old data first. So that you are not mixing the two up. If you are running Windows7 or Windows Vista you have to look in two locations for the .hst files and delete all of them. If however you are running Windows XP you need to only look in one. These locations can vary greatly depending what broker you download your platform from. I use Interbank but you will have to look and mind and figure out where yours are. Or you can just search your entire hard drive for *.hst files. C:\Program Files (x86)\Interbank FX Trader 4 – B\history C:\Users\%username%\AppData\Local\VirtualStore\Program Files (x86)\Interbank FX Trader 4 – B\history\IBFX-MT4 Mini-2
4. Now that you have unzipped the files they should be in .CSV format. From here you can now open your Metatrader platform. Then go to tools>options click on the chart tab and change the “max bars in history” and “max bars in chart” to 999999999. Once you close this window Metatrader will adjust to the highest possible setting automatically.
5. You will now want to go to tools>history center. Double click your currency pair that you are trying to import. Then double click on the 1m timeframe underneath it. Now click on the import button. Browse to the location of the .txt file that you extracted in step 2. You may have to change the rows setting to 0. If it works correctly you will be able to see every row from the file that you are importing. Click ok and all of your 1m data is now complete.
6. The next step is to generate your 5m 10m 30m 60m 240m 1440m and 10080m data. This will get all of the commonly used timeframes. You can also do custom ones if you wish.
7. To do this you need to go to navigator from Metatrader terminal. Select the scripts from the navigator window and look for one named “period_converter”.
8. Open a 1m chart of the currency pair that you just imported.
9. Now double click on period_converter
10. It’s going to ask you for an input labeled “ExtPeriodMultiplier”. The default value is 3. Change this to 5 to generate your 5 minute chart data.
11. If you have your terminal window open in Metatrader select the experts tab. You will see something that says period_converter GBPUSDm,M1: 726279 record(s) written. This is how you know it is complete.
12. Double click period converter again. This time select 10 for 10 minute chart. Continue doing steps 10 and 11 until you have created your 5m 10m 30m 60m 240m 1440m and 10080m data. You are now ready to move on to your next currency pair.
If you are a currency trader it can be quite annoying when you cannot access volume data as you can in other markets. This can be instrumental to ones success if your strategy from another insrument outside of Forex requires some type of volume indicator. Although it is not perfect you can use the futures market COT or Commitment of Traders report to gauge this.
Since 2000 the COT reports are released weekly. However, you must keep in mind the reports are released on Friday but reflect data from Tuesday. As the reports are closed off on Tuesday but not published until 3 days later.
When it comes to the COT report there are a few different sections that are valuable:
Commercial: These are organizations that use the futures market to hedge their positions. They are not traders by nature. They only use futures to guarantee their profits for the future. Non-Commercial: These are the speculators who trade the futures market for a living. They include hedge funds and other individuals who speculate the market and make the requirements that are set by the CFTC. Reportable: Positions held above specific reporting levels set by the CFTC. Non-Reportable: These are individuals or organizations who do not meet the minimum requirements set forth by the CFTC to be categorized. Open Interest: The open interest is the amount of futures or options contracts which are not offset by another transaction.
The Whittaker COT Index Variant-A is an MT4 indicator. Basically what it will do is go through the data from the COT report and extract the above sections. It will then as a percentage let you know how many traders are long in each category.
Use the image below to make sure you’re downloading the correct link. Save this file to C:\Program Files (x86)\Interbank FX Trader 4\experts\files.
Once you download this file it must be converted to CSV format for MetaTrader to read it. You can do this by opening the file located at C:\Program Files (x86)\Interbank FX Trader 4\experts\files\annual.xls with Microsoft Excel. Once it is open in Excel go to file>save as. From the save as type window select CSV(Comma Delimited) (*.csv). You will want to name the file annual.xls.csv this by default is what Whitaker COT Index will be looking for.
The actual Whittaker COT Index variant-A can be downloaded here. Save the file to C:\Program Files (x86)\Interbank FX Trader 4\experts Once the file is downloaded simply add it to your chart.
I wanted to share with everyone an indicator that I programmed and I find very useful. It is called the Whittaker Volatility Breakout Indicator or WVBI. The purpose of this indicator is to represent the range of a bar in percentage terms rather than in points. I think the flaw of most traders or programmers is they often write code and optimize it for pips of a given pair. This I truely believe is one of the core fundamental flaws of sustainability of a strategy. When people start programming static values. Whether it be in money management, entries, exits, or take profit levels.
Displays as a histogram the percentage of change from the average.
Takes an average of the percentage of change from the average.
Allows you to set a horizontal line at your preference so you can see when the average is breaking out.
This indicator has 3 inputs:
“ATRPeriod” which allows you to specify the amount of bars to average out for the Average True Range of that period
“WVBIAverage” is a setting that allows you to specify the amount of bars to which will be the average of the percentage of the ATR.
The last setting is pretty straightforward it is “TooFar”. This simply will draw a horizontal line across the indicator so that you can visually see where the percentage is in comparison to other bars. Its purpose is to simply put things in perspective.
All of this might seem confusing at first. However, I assure you this a very useful indicator for detecting volatility in a market.