Traditionally, there have been two general ways of analyzing market data:
- fundamental analysis – focused on underlying fundamental data
- technical analysis – focused on charts and price movements
In recent years, computer science and mathematics revolutionized trading, it has become dominated by computers helping to analyze vast amounts of available data. Algorithms are responsible for making trading decisions faster than any human being could. Machine learning and data mining techniques are growing in popularity, all that falls under one broad category called ‘quantitative trading’ or ‘algorithmic trading’.
Below, I intend to provide you with basic tools for handling and analyzing market data with aim of generating profit from buying and selling financial instruments.
Python Programming Language
Currently, among the hottest programming languages for finance, you’ll find R and Python, alongside languages such as C++, C# and Java. I think Python or R is the right choice for many traders today. In this post, I assume you’re more or less starting from scratch or with very basic knowledge of Python, which by the way is one of the more approachable languages.
It is good to get the feeling of general Python programming before moving on with application to trading, there is a number of books and tutorials most available free or almost free:
- A Byte of Python by Swaroop C H
- Python for Everybody – Prof.Charles Severance
- Python Programming by Wikibooks
- Think Python: How would you Think Like a Computer Scientist by Allen Downey
- Dive Into Python 3 by Mark Pilgrim
All blow examples of code are for Python 3.5 with Anaconda distribution available there – www.continuum.io
The whole point of trading is to predict with certain probability what will be market behavior in future and take advantage of that. Very often it can be as simple as ‘go long’ while expecting market prices to go up or ‘go short’ while expecting market prices to drop.
Defining our ‘view’ on market or expectations about future price changes usually takes some kind of market data analysis, to do it we need data first.
There are many ways to import data to python, one of most common is using pandas-datareader package (starting with Pandas 0.19 on), it allows to import data from multiple sources like Yahoo Finance, Google Finance, Quandl, World Bank or OECD.
The easiest way to install it is:
pip install pandas-datareader
conda install -c anaconda pandas-datareader
When pandas-datareader is installed getting historical data takes only a few inputs, start and end date of a period we require the data for, ticker symbol and source of data.
Something like this:
import pandas_datareader.data as web import datetime start = datetime.datetime(2000, 1, 1) end = datetime.datetime(2017, 1, 1) data = web.DataReader('AAPL', 'yahoo', start, end)
Let’s check what we have:
Yahoo Finance gives back:
- Date – quotation date
- Open – open price
- High – highest price for the day
- Low – lowest price of the day
- Close – close price
- AdjClose – close rice with adjustments eg.stock split or dividend.
- Volume – trade volume for the day
Let’s check Google Finance, we need as above Apple for a period of 01/01/2000 to 01/01/2017.
import pandas_datareader.data as web import datetime start = datetime.datetime(2000, 1, 1) end = datetime.datetime(2017, 1, 1) data = web.DataReader("AAPL", 'google', start, end) print (data.tail())
We get an exact same set of data, except AdjClose value, so stock splits or dividends are not included.
Another very valuable source of financial/economic data can be Quandl.com.
import pandas_datareader.data as web symbol = 'WIKI/AAPL' # or 'AAPL.US' data = web.DataReader(symbol, 'quandl', "2000-01-01", "2017-01-01") print(data.tail())
Again just like Yahoo Finance, Quandl delivers AdjClose information. There is plenty of other sources accessible via pandas-datareader, for more detailed information and examples pelase go there:
Pandas-datareader is very useful and offers plenty of options, although not the only solution, you can also use libraries like Quandl.
Running this line of code installs the package:
pip install quandl
Getting data is very similar to Pandas-datareader:
import quandl data = quandl.get("WIKI/AAPL", start_date="2000-10-01", end_date="2017-01-01") print(data.tail())
After downloading the data it is always useful to save a local copy for further work, as generating online query every time data is required may be very time consuming for larger data sets eg. 100 tickers.
import pandas_datareader.data as web import datetime start = datetime.datetime(2000, 1, 1) end = datetime.datetime(2017, 1, 1) data = web.DataReader('AAPL', 'yahoo', start, end) data.to_csv('data.csv')
After securing an own copy of the data, we can quickly read it in using pandas:
import pandas as pd data = pd.read_csv('data.csv', index_col='Date', parse_dates=True) print(data.tail())
Working with Data
Having all data saved, we can start looking at them in more detail, for purpose of this intro, we will use Adjusted Close values only. Let’s select that from the whole dataset:
AdjClose=data['Adj Close'] print(AdjClose.tail())
Next thing useful for trading would be to try and plot it, we can do it using pandas:
As we are about to use quantitative methods, let’s see some statistics about the Adjusted Close price values:
For a variety of reasons that are out of scope for this text, it is better to work with daily returns rather than nominal prices of financial instruments. There is a very simple way to returns using pandas:
Rets = AdjClose.pct_change(1)
We can also plot returns:
Sometimes instead of simple returns, we may like to use log returns, it is easy to do it using Numpy:
import numpy as np LogRets = np.log(AdjClose.pct_change()+1)
Lets plot log returns:
In next part of this introduction, we will move on to code first trading strategy.
Was the above useful? Please share with others on social media.
Recommended reading list:
|Pairs Trading: Quantitative Methods and Analysis
The first in-depth analysis of pairs trading
Pairs trading is a market-neutral strategy in its most simple form. The strategy involves being long (or bullish) one asset and short (or bearish) another. If properly performed, the investor will gain if the market rises or falls. Pairs Trading reveals the secrets of this rigorous quantitative analysis program to provide individuals and investment houses with the tools they need to successfully implement and profit from this proven trading methodology. Pairs Trading contains specific and tested formulas for identifying and investing in pairs, and answers important questions such as what ratio should be used to construct the pairs properly.
Ganapathy Vidyamurthy (Stamford, CT) is currently a quantitative software analyst and developer at a major New York City hedge fund.
|Machine Trading: Deploying Computer Algorithms to Conquer the Markets (Wiley Trading)
Dive into algo trading with step-by-step tutorials and expert insight
Machine Trading is a practical guide to building your algorithmic trading business. Written by a recognized trader with major institution expertise, this book provides step-by-step instruction on quantitative trading and the latest technologies available even outside the Wall Street sphere. You'll discover the latest platforms that are becoming increasingly easy to use, gain access to new markets, and learn new quantitative strategies that are applicable to stocks, options, futures, currencies, and even bitcoins. The companion website provides downloadable software codes, and you'll learn to design your own proprietary tools using MATLAB. The author's experiences provide deep insight into both the business and human side of systematic trading and money management, and his evolution from proprietary trader to fund manager contains valuable lessons for investors at any level.
Algorithmic trading is booming, and the theories, tools, technologies, and the markets themselves are evolving at a rapid pace. This book gets you up to speed, and walks you through the process of developing your own proprietary trading operation using the latest tools.
Utilize the newer, easier algorithmic trading platforms
Access markets previously unavailable to systematic traders
Adopt new strategies for a variety of instruments
Gain expert perspective into the human side of trading
The strength of algorithmic trading is its versatility. It can be used in any strategy, including market-making, inter-market spreading, arbitrage, or pure speculation; decision-making and implementation can be augmented at any stage, or may operate completely automatically. Traders looking to step up their strategy need look no further than Machine Trading for clear instruction and expert solutions.
|Applied Quantitative Methods for Trading and Investment
This much-needed book, from a selection of top international experts, fills a gap by providing a manual of applied quantitative financial analysis. It focuses on advanced empirical methods for modelling financial markets in the context of practical financial applications.
Data, software and techniques specifically aligned to trading and investment will enable the reader to implement and interpret quantitative methodologies covering various models.
The unusually wide-ranging methodologies include not only the 'traditional' financial econometrics but also technical analysis systems and many nonparametric tools from the fields of data mining and artificial intelligence. However, for those readers wishing to skip the more theoretical developments, the practical application of even the most advanced techniques is made as accessible as possible.
The book will be read by quantitative analysts and traders, fund managers, risk managers; graduate students in finance and MBA courses.
|Quantitative Technical Analysis: An integrated approach to trading system development and trading management
This book, the fifth by Dr. Howard Bandy, discusses an integrated approach to trading system development and trading management.
It begins with a discussion and quantification of the several aspects of risk.
1. The trader's personal tolerance for risk.
2. The risk inherent in the price fluctuations of the issue to be traded.
3. The risk added by the trading system rules.
4. The trade-by-trade risk experienced during trading.
An original objective function, called "CAR25," based on risk-normalized profit potential is developed and explained. CAR25 is as near a universal objective function as I have found.
The importance of recognizing the non-stationary characteristics of financial data, and techniques for handling it, are discussed.
There is a general discussion of trading system development, including design, testing, backtesting, optimization, and walk forward analysis. That is followed by two parallel development paths -- one using traditional trading system development platform and the second machine learning.
Recognizing the importance of position sizing in managing trading, an original technique based on empirical Bayesian analysis, called "dynamic position sizing" and quantified in a metric called "safe-f," is introduced. Computer code implementing dynamic position sizing is included in the book.
56 fully disclosed, ready-to-run, and downloadable programs are included.
|Finding Alphas: A Quantitative Approach to Building Trading Strategies
Design more successful trading systems with this practical guide to identifying alphas
Finding Alphas seeks to teach you how to do one thing and do it well: design alphas. Written by experienced practitioners from WorldQuant, including its founder and CEO Igor Tulchinsky, this book provides detailed insight into the alchemic art of generating trading signals, and gives you access to the tools you need to practice and explore. Equally applicable across regions, this practical guide provides you with methods for uncovering the hidden signals in your data. A collection of essays provides diverse viewpoints to show the similarities, as well as unique approaches, to alpha design, covering a wide variety of topics, ranging from abstract theory to concrete technical aspects. You'll learn the dos and don'ts of information research, fundamental analysis, statistical arbitrage, alpha diversity, and more, and then delve into more advanced areas and more complex designs. The companion website, www.worldquantchallenge.com, features alpha examples with formulas and explanations. Further, this book also provides practical guidance for using WorldQuant's online simulation tool WebSim® to get hands-on practice in alpha design.
Alpha is an algorithm which trades financial securities. This book shows you the ins and outs of alpha design, with key insight from experienced practitioners.
Learn the seven habits of highly effective quants
Understand the key technical aspects of alpha design
Use WebSim® to experiment and create more successful alphas
Finding Alphas is the detailed, informative guide you need to start designing robust, successful alphas.
|Inside the Black Box: A Simple Guide to Quantitative and High Frequency Trading
New edition of book that demystifies quant and algo trading
In this updated edition of his bestselling book, Rishi K Narang offers in a straightforward, nontechnical style—supplemented by real-world examples and informative anecdotes—a reliable resource takes you on a detailed tour through the black box. He skillfully sheds light upon the work that quants do, lifting the veil of mystery around quantitative trading and allowing anyone interested in doing so to understand quants and their strategies. This new edition includes information on High Frequency Trading.
Offers an update on the bestselling book for explaining in non-mathematical terms what quant and algo trading are and how they work
Provides key information for investors to evaluate the best hedge fund investments
Explains how quant strategies fit into a portfolio, why they are valuable, and how to evaluate a quant manager
This new edition of Inside the Black Box explains quant investing without the jargon and goes a long way toward educating investment professionals.
|Automated Trading with R: Quantitative Research and Platform Development
Learn to trade algorithmically with your existing brokerage, from data management, to strategy optimization, to order execution, using free and publicly available data. Connect to your brokerage’s API, and the source code is plug-and-play.
Automated Trading with R explains automated trading, starting with its mathematics and moving to its computation and execution. You will gain a unique insight into the mechanics and computational considerations taken in building a back-tester, strategy optimizer, and fully functional trading platform.
The platform built in this book can serve as a complete replacement for commercially available platforms used by retail traders and small funds. Software components are strictly decoupled and easily scalable, providing opportunity to substitute any data source, trading algorithm, or brokerage. This book will:
Provide a flexible alternative to common strategy automation frameworks, like Tradestation, Metatrader, and CQG, to small funds and retail traders
Offer an understanding of the internal mechanisms of an automated trading system
Standardize discussion and notation of real-world strategy optimization problems
What You Will Learn
Understand machine-learning criteria for statistical validity in the context of time-series
Optimize strategies, generate real-time trading decisions, and minimize computation time while programming an automated strategy in R and using its package library
Best simulate strategy performance in its specific use case to derive accurate performance estimates
Understand critical real-world variables pertaining to portfolio management and performance assessment, including latency, drawdowns, varying trade size, portfolio growth, and penalization of unused capital
Who This Book Is For
Traders/practitioners at the retail or small fund level with at least an undergraduate background in finance or computer science; graduate level finance or data science students
|Quantitative Trading with R: Understanding Mathematical and Computational Tools from a Quant's Perspective
Quantitative Finance with R offers a winning strategy for devising expertly-crafted and workable trading models using the R open source programming language, providing readers with a step-by-step approach to understanding complex quantitative finance problems and building functional computer code.
|Quantitative Momentum: A Practitioner's Guide to Building a Momentum-Based Stock Selection System (Wiley Finance)
The individual investor's comprehensive guide to momentum investing
Quantitative Momentum brings momentum investing out of Wall Street and into the hands of individual investors. In his last book, Quantitative Value, author Wes Gray brought systematic value strategy from the hedge funds to the masses; in this book, he does the same for momentum investing, the system that has been shown to beat the market and regularly enriches the coffers of Wall Street's most sophisticated investors. First, you'll learn what momentum investing is not: it's not 'growth' investing, nor is it an esoteric academic concept. You may have seen it used for asset allocation, but this book details the ways in which momentum stands on its own as a stock selection strategy, and gives you the expert insight you need to make it work for you. You'll dig into its behavioral psychology roots, and discover the key tactics that are bringing both institutional and individual investors flocking into the momentum fold.
Systematic investment strategies always seem to look good on paper, but many fall down in practice. Momentum investing is one of the few systematic strategies with legs, withstanding the test of time and the rigor of academic investigation. This book provides invaluable guidance on constructing your own momentum strategy from the ground up.
Learn what momentum is and is not
Discover how momentum can beat the market
Take momentum beyond asset allocation into stock selection
Access the tools that ease DIY implementation
The large Wall Street hedge funds tend to portray themselves as the sophisticated elite, but momentum investing allows you to 'borrow' one of their top strategies to enrich your own portfolio. Quantitative Momentum is the individual investor's guide to boosting market success with a robust momentum strategy.
|Quantitative Trading: Algorithms, Analytics, Data, Models, Optimization
The first part of this book discusses institutions and mechanisms of algorithmic trading, market microstructure, high-frequency data and stylized facts, time and event aggregation, order book dynamics, trading strategies and algorithms, transaction costs, market impact and execution strategies, risk analysis, and management. The second part covers market impact models, network models, multi-asset trading, machine learning techniques, and nonlinear filtering. The third part discusses electronic market making, liquidity, systemic risk, recent developments and debates on the subject.
|Quantitative Trading: How to Build Your Own Algorithmic Trading Business
While institutional traders continue to implement quantitative (or algorithmic) trading, many independent traders have wondered if they can still challenge powerful industry professionals at their own game? The answer is "yes," and in Quantitative Trading, Dr. Ernest Chan, a respected independent trader and consultant, will show you how. Whether you're an independent "retail" trader looking to start your own quantitative trading business or an individual who aspires to work as a quantitative trader at a major financial institution, this practical guide contains the information you need to succeed.
|Algorithmic Trading and DMA: An introduction to direct access trading strategies
Algorithmic trading and Direct Market Access (DMA) are important tools helping both buy and sell-side traders to achieve best execution (Note: the focus is on institutional sized orders, not those of individuals/retail traders).
This book starts from the ground up to provide detailed explanations of both these techniques:
An introduction to the different types of execution is followed by a review of market microstructure theory. Throughout the book examples from empirical studies bridge the gap between the theory and practice of trading.
Orders are the fundamental building blocks for any strategy. Market, limit, stop, hidden, iceberg, peg, routed and immediate-or-cancel orders are all described with illustrated examples.
Trading algorithms are explained and compared using charts to show potential trading patterns. TWAP, VWAP, Percent of Volume, Minimal Impact, Implementation Shortfall, Adaptive Shortfall, Market On Close and Pairs trading algorithms are all covered, together with common variations.
Transaction costs can have a significant effect on investment returns. An in-depth example shows how these may be broken down into constituents such as market impact, timing risk, spread and opportunity cost and other fees.
Coverage includes all the major asset classes, from equities to fixed income, foreign exchange and derivatives. Detailed overviews for each of the world's major markets are provided in the appendices.
Order placement and execution tactics are covered in more detail, as well as potential enhancements (such as short-term forecasts), for those interested in the specifics of implementing these strategies.
Cutting edge applications such as portfolio and multi-asset trading are also considered, as are handling news and data mining/artificial intelligence.
|Python for Finance: Analyze Big Financial Data
The financial industry has adopted Python at a tremendous rate recently, with some of the largest investment banks and hedge funds using it to build core trading and risk management systems. This hands-on guide helps both developers and quantitative analysts get started with Python, and guides you through the most important aspects of using Python for quantitative finance.
Using practical examples through the book, author Yves Hilpisch also shows you how to develop a full-fledged framework for Monte Carlo simulation-based derivatives and risk analytics, based on a large, realistic case study. Much of the book uses interactive IPython Notebooks, with topics that include:
Fundamentals: Python data structures, NumPy array handling, time series analysis with pandas, visualization with matplotlib, high performance I/O operations with PyTables, date/time information handling, and selected best practices
Financial topics: mathematical techniques with NumPy, SciPy and SymPy such as regression and optimization; stochastics for Monte Carlo simulation, Value-at-Risk, and Credit-Value-at-Risk calculations; statistics for normality tests, mean-variance portfolio optimization, principal component analysis (PCA), and Bayesian regression
Special topics: performance Python for financial algorithms, such as vectorization and parallelization, integrating Python with Excel, and building financial applications based on Web technologies
|A Guide to Creating A Successful Algorithmic Trading Strategy (Wiley Trading)
Turn insight into profit with guru guidance toward successful algorithmic trading
A Guide to Creating a Successful Algorithmic Trading Strategy provides the latest strategies from an industry guru to show you how to build your own system from the ground up. If you're looking to develop a successful career in algorithmic trading, this book has you covered from idea to execution as you learn to develop a trader's insight and turn it into profitable strategy. You'll discover your trading personality and use it as a jumping-off point to create the ideal algo system that works the way you work, so you can achieve your goals faster. Coverage includes learning to recognize opportunities and identify a sound premise, and detailed discussion on seasonal patterns, interest rate-based trends, volatility, weekly and monthly patterns, the 3-day cycle, and much more—with an emphasis on trading as the best teacher. By actually making trades, you concentrate your attention on the market, absorb the effects on your money, and quickly resolve problems that impact profits.
Algorithmic trading began as a "ridiculous" concept in the 1970s, then became an "unfair advantage" as it evolved into the lynchpin of a successful trading strategy. This book gives you the background you need to effectively reap the benefits of this important trading method.
Navigate confusing markets
Find the right trades and make them
Build a successful algo trading system
Turn insights into profitable strategies
Algorithmic trading strategies are everywhere, but they're not all equally valuable. It's far too easy to fall for something that worked brilliantly in the past, but with little hope of working in the future. A Guide to Creating a Successful Algorithmic Trading Strategy shows you how to choose the best, leave the rest, and make more money from your trades.
|Building Winning Algorithmic Trading Systems, + Website: A Trader's Journey From Data Mining to Monte Carlo Simulation to Live Trading (Wiley Trading)
Develop your own trading system with practical guidance and expert advice
In Building Algorithmic Trading Systems: A Trader's Journey From Data Mining to Monte Carlo Simulation to Live Training, award-winning trader Kevin Davey shares his secrets for developing trading systems that generate triple-digit returns. With both explanation and demonstration, Davey guides you step-by-step through the entire process of generating and validating an idea, setting entry and exit points, testing systems, and implementing them in live trading. You'll find concrete rules for increasing or decreasing allocation to a system, and rules for when to abandon one. The companion website includes Davey's own Monte Carlo simulator and other tools that will enable you to automate and test your own trading ideas.
A purely discretionary approach to trading generally breaks down over the long haul. With market data and statistics easily available, traders are increasingly opting to employ an automated or algorithmic trading system—enough that algorithmic trades now account for the bulk of stock trading volume. Building Algorithmic Trading Systems teaches you how to develop your own systems with an eye toward market fluctuations and the impermanence of even the most effective algorithm.
Learn the systems that generated triple-digit returns in the World Cup Trading Championship
Develop an algorithmic approach for any trading idea using off-the-shelf software or popular platforms
Test your new system using historical and current market data
Mine market data for statistical tendencies that may form the basis of a new system
Market patterns change, and so do system results. Past performance isn't a guarantee of future success, so the key is to continually develop new systems and adjust established systems in response to evolving statistical tendencies. For individual traders looking for the next leap forward, Building Algorithmic Trading Systems provides expert guidance and practical advice.
|Artificial Intelligence in Financial Markets: Cutting Edge Applications for Risk Management, Portfolio Optimization and Economics (New Developments in Quantitative Trading and Investment)
As technology advancement has increased, so to have computational applications for forecasting, modelling and trading financial markets and information, and practitioners are finding ever more complex solutions to financial challenges. Neural networking is a highly effective, trainable algorithmic approach which emulates certain aspects of human brain functions, and is used extensively in financial forecasting allowing for quick investment decision making.
This book presents the most cutting-edge artificial intelligence (AI)/neural networking applications for markets, assets and other areas of finance. Split into four sections, the book first explores time series analysis for forecasting and trading across a range of assets, including derivatives, exchange traded funds, debt and equity instruments. This section will focus on pattern recognition, market timing models, forecasting and trading of financial time series. Section II provides insights into macro and microeconomics and how AI techniques could be used to better understand and predict economic variables. Section III focuses on corporate finance and credit analysis providing an insight into corporate structures and credit, and establishing a relationship between financial statement analysis and the influence of various financial scenarios. Section IV focuses on portfolio management, exploring applications for portfolio theory, asset allocation and optimization.
This book also provides some of the latest research in the field of artificial intelligence and finance, and provides in-depth analysis and highly applicable tools and techniques for practitioners and researchers in this field.
|Optimal Trading Strategies: Quantitative Approaches for Managing Market Impact and Trading Risk
"The decisions that investment professionals and fund managers make have a direct impact on investor return. Unfortunately, the best implementation methodologies are not widely disseminated throughout the professional community, compromising the best interests of funds, their managers, and ultimately the individual investor. But now there is a strategy that lets professionals make better decisions. This valuable reference answers crucial questions such as: * How do I compare strategies? * Should I trade aggressively or passively? * How do I estimate trading costs, "slice" an order, and measure performance? and dozens more. Optimal Trading Strategies is the first book to give professionals the methodology and framework they need to make educated implementation decisions based on the objectives and goals of the funds they manage and the clients they serve."