PTWhich model forecasts the exchange rate best?
Since 1983, economists have tried to beat a simple guess: next period's exchange rate will equal today's. The thesis tested economic models and a neural network on five currencies against the dollar, 1995 to 2020. No model won every time, but from two quarters ahead on, the simple guess dropped out of the best group. Explained in 12 screens.
Each block is a model in the best group by error (Table 9 of the thesis), from 1 to 6 quarters ahead.
Two yardsticks for judging a forecast
The hardest guess to beat is repeating the last value: next quarter's dollar will be the same as today's. Economists call this the random walk. In 1983, Meese and Rogoff showed that it erred less than the economic models of the time.
The thesis judges the models by two yardsticks: how far off the value they are and whether they get the direction right (up or down), which matters to investors. Try it with the real Brazilian real, quarter by quarter.
Approximate values, read by us from the thesis chart (Figure 11): Brazilian reais per dollar at the end of each quarter.
Almost no one looked at the currencies of emerging countries
The author gathered 775 Web of Science articles on exchange rate models, measured who publishes and who is cited, and read in full the 69 that compare forecasting models.
Of the 69, only 4 include currencies of emerging countries, and the Brazilian real appears in just 2. She also found few studies that use several kinds of model and several yardsticks at once. The thesis grows out of these gaps.
One dot per article read. For the slices the thesis gives as percentages (approach, frequency, period), the dots are our approximation.
An article from 1983 is still the most cited
Meese and Rogoff's study has 1,543 citations in the database. It created the "puzzle": why don't models based on the economy (interest rates, inflation, money, output) beat the guess of repeating the last value?
Shared citations split the studies into three groups:
The 10 most cited articles among the 775 (Table 4 of the thesis). Per year, Patton (2006), on how currencies move together, takes the lead.
Five currencies and 104 quarters
The Japanese yen, a core currency of the world market, and four currencies of emerging countries: the Brazilian real and the Mexican, Chilean and Colombian pesos, all against the dollar, from 1995 (start of Brazil's Real Plan) to 2020.
The models learn from the first 75% of the quarters and are tested on the final 25%, from 1 to 6 quarters ahead. At each step, they receive the actual value before forecasting the next one.
Quarterly data from the IMF, the OECD and FRED (Federal Reserve Bank of St. Louis); CDS from Bloomberg. The split into learning and test quarters is our calculation from the thesis's 75% and 25%.
Eighteen recipes made with ingredients from the economy
Each economic model forecasts the exchange rate from differences between the country and the US: money in circulation, GDP, interest rates, inflation, foreign trade. The thesis's novelty is one more ingredient, the CDS, a thermometer of the country's default risk.
Each recipe ran in 6 versions, and the forecasts were also combined with one another:
Table 5 of the thesis. Tap a row to see the recipe. VAR and VEC forecast several series together; VEC also uses the long-run equilibrium between them. The 18th recipe, the Taylor rule, uses GDP, interest and inflation gaps computed with five filters (BK, BW, CF, HP and HM). The 150 best models for each country went on to the final comparison.
The simple guess was among the best only one quarter ahead
A statistical test (the Model Confidence Set) picked out, for each country and each horizon, the group of models that erred least and tied with one another. "Tomorrow = today" made that group only one quarter ahead, in three countries. From the 2nd to the 6th quarter, it was left out everywhere.
Table 9 of the thesis. Each block is a model in the best group; tap a stack to see which ones. The counts by family are ours.
By direction, several hit rates were close to luck
Among the tied models, the thesis picked the one that most often got right whether the exchange rate would rise or fall. By this yardstick, the models beat "tomorrow = today" in every quarter except the 1st in Chile and Colombia.
There is a catch: "tomorrow = today" never says "up" or "down", so it gets 0% of the directions right. The fair comparison is with a coin toss: 50%. Switch the yardstick.
Our reading. Direction hit rates from Table 11 of the thesis ("= today": the thesis indicates "tomorrow = today" itself). The test uses the final 25%, about 26 quarters. With 26 tosses, a fair coin gets between 31% and 69% right more than 95% of the time (blue band). The thesis does not test whether the hit rates differ from chance.
The risk thermometer did not improve the forecasts
The thesis's bet was to add the five-year CDS, the price of insurance against a government default, as a measure of uncertainty. Earlier studies link the CDS to the exchange rate, especially in crises. In the result, models with the CDS appeared among the best, but they did not prove superior to the others.
Start of the CDS series: Table 7 of the thesis. Quarters with CDS through the end of 2020: our count.
A larger neural network erred less than the simple guess
In the third article, the author trained an LSTM neural network, a type of network with memory, using only the exchange rate series itself, to forecast one quarter ahead. She tested seven sizes, from 1 to 100 neurons in 3 layers. Drag the slider.
Logarithmic scale: each step in height multiplies the error by 10. The blue sheet is the error of "tomorrow = today"; below it, the network erred less. The number on the bar is the network's average error divided by the simple guess's, our calculation from Table 13. Japan is left out: in Table 13, its values repeat Colombia's (equal or divided by a thousand), which does not fit a yen between 100 and 125 per dollar.
See the thesis's errors (Table 13)
The network tracked the real closely until the pandemic
With 100 neurons, the one-quarter-ahead forecast almost merges with the actual exchange rate from 2014 to 2019. In 2020, the real shot up and the network fell behind. The author points to this sensitivity to shocks in emerging countries.
To explain it, she cites two kinds of uncertainty (Gaba, Hogarth and Makridakis, 2010). Tap them:
Approximate values, read by us from Figure 11 of the thesis (3rd quarter of 2014 to 3rd quarter of 2020, reais per dollar).
What the research changes and what it does not say
For those who hedge currency risk
Companies that use futures contracts (hedging) want to measure risk better. From 2 to 6 quarters ahead, economic models and combinations of models erred less than repeating the last value, in all five countries.
Abstract and Table 9For those who build currency portfolios
For the author, getting the direction right is what matters to investors, and the models did that better than the simple guess. In several cases, however, the hit rate was close to a coin toss.
Section 5.3; caveat: our readingFor researchers
Look at the same problem with several models, several techniques and several yardsticks, and include currencies of emerging countries, which are still little studied.
Sections 2.2.3 and 7.1What this research does not say
- No margin of error for the forecasts. The thesis did not compute confidence intervals for the projections; the author suggests this for future studies. · according to the author
- Direction not tested against chance. With about 26 test quarters, hit rates between 31% and 69% are within reach of luck. · our observation
- A neural network with one horizon and one ingredient. The LSTM forecast only one quarter ahead and only from the exchange rate itself, with no economic data. · our observation
- Japan's data in Table 13. Japan's errors repeat Colombia's; that is why screen 10 shows four countries. · our observation
Credits
GENIN, Camila de Souza Vasconcelos. Modelos de projeção de câmbio: uma investigação múltipla com séries de tempo, modelos estruturais e model selection. Doctoral thesis (Business Administration), Mackenzie Presbyterian University, São Paulo, 2023. Advisor: Prof. Dr. Eli Hadad Junior.
Research supported by CAPES (the Brazilian federal agency for graduate education) and the Mackenzie Presbyterian Institute. Data from the IMF, the OECD, FRED and Bloomberg; analyses in Stata and OxMetrics. This version simplifies the work for a general audience; the equations and full tables are in the original text.