1. Intuition of Synthetic Control Approach

To understand the economic impact of Brexit, we've adopted a method called the synthetic control approach. This method helps create a "virtual" control group by combining data from multiple similar units that did not experience the event in question. This synthetic group is constructed in such a way that it closely resembles the group that did experience the event before it happened.

For example, let's consider the region of West Midlands in the UK. To measure the economic impact of Brexit on this region, we compile data from several similar regions outside the UK, such as in Germany, France, and Spain which did not undergo Brexit. By aggregating data from these regions, we create a synthetic West Midlands that mirrors its economic trends before Brexit. This synthetic region then acts as a benchmark to compare against the actual post-Brexit economic performance of the West Midlands.

2. Potential Concerns with Synthetic Control Approach

One major concern with the synthetic control approach is the accuracy of the synthetic control group. Specifically, whether it truly represents what would have happened to the treated unit in the absence of the event. If the regions used to create the synthetic control are not sufficiently similar to the treated unit in key characteristics, the comparison can be misleading. Another issue arises if other major events occur in the control regions around the time of Brexit, making it difficult to isolate Brexit’s impact.

In the example of the West Midlands, if the selected regions in Germany, France, the US or Spain have different economic structures or trends compared to the West Midlands, the synthetic control might not be a perfect match. Additionally, if these regions were affected by other significant economic events around the time of Brexit, this could confound our results, making it harder to attribute economic changes in the West Midlands specifically to Brexit.

3. Input data quality differences

Subnational economic data is often quite poor quality as statistical capacity varies across countries. Further, the agencies tasked with producing subnational accounts may have limited access to high quality administrative or private data. This can produce noisy figures that may be subject to revision. We would expect that estimates at coarser granularity, such as the region estimates we present separately, on average, are more accurate. The accuracy of district level estimate strongly depends on the underlying economic structure and data quality. Methodologically, a lot of innovation in improving (sub) national economic accounts is presently underway.

4. Robustness of Our Approach to Tackling Concerns

We have implemented several robustness checks to address these concerns. Firstly, we carefully selected the set of regions and countries outside the UK for which data was leveraged. That is: we are not mechnically constraining the synthetic control counterfactual to only rely on data from Europe as comparator, but rather, allow the inclusion of data from other countries that may be perceived to be growing faster than reference regions in Europe. Secondly, we employed various statistical methods to ensure the synthetic control group closely aligns with the West Midlands in key economic metrics. Lastly, we conducted placebo tests by pretending Brexit happened in other regions outside the UK, assessing if the created synthetic controls in those regions would yield similar outcomes to further validate our method.

For instance, by comparing the West Midlands to carefully chosen regions in Germany, France, and Spain with similar economic trends before Brexit, we enhance the accuracy of our synthetic control. Conducting placebo tests - for example, applying the synthetic control method to a region in Germany as if it experienced Brexit - helps to ensure that our findings are not specific to the United Kingdom but would generalize similarly in a comparable context.