Abstract

We examine the potential for selection bias in voluntarily reported hedge fund performance data. We construct a set of hedge fund returns that have never been reported to a commercial hedge fund database. These returns allow a direct comparison of performance between funds that choose to report to commercial databases and funds that do not. We find that funds that report their performance to commercial databases significantly outperform nonreporting funds. Our results suggest that the voluntarily reported performance in commercial databases suffers from a selection bias that may exaggerate the average skill of the universe of hedge fund managers.

As private entities, hedge funds have not been historically required to report their performance to any regulatory agency. However, some hedge funds voluntarily report their returns and other fund characteristics to commercial database providers (e.g., Lipper TASS) in an effort to document their performance and attract potential investors. These self-reported databases have been an important resource used to examine the performance of the hedge fund industry. It is well known that voluntary reporting can lead to data biases, and there is a growing body of literature documenting the potential for hedge funds to take advantage of strategic incentives to withhold or misreport their performance to commercial databases.1 In this paper, we examine the performance of a unique sample of hedge funds that do not report their performance to commercial databases. By comparing nonreporting funds to those that choose to report their performance, we are able to better estimate the potential biases in commercial databases that may arise from voluntary disclosure.

Given the challenges inherent in studying voluntarily reported hedge fund data, there is a lack of consensus among academics as to the appropriate methodology to evaluate hedge fund performance. As such, there remains a continuing controversy as to whether hedge funds earn superior returns for their investors. Many studies using commercial databases document significant alpha in hedge fund performance, with estimates ranging from 3% to 5% annually (e.g., Ibbotson, Chen, and Zhu 2011; Kosowski, Naik, and Teo 2007). However, some studies find that alpha estimates can be highly sensitive to methodological and database choices, and find either small or inconsistent alphas for hedge fund managers (e.g., Aragon 2007; Fung et al. 2008). An issue that is central to this controversy is that researchers do not observe the universe of hedge funds; rather, they (typically) observe a subset of funds that choose to report to commercial databases.

If reporting funds are fundamentally different than nonreporting funds, commercial databases could suffer from a selection bias. By selection bias, we refer to the idea that because hedge funds can choose not to report their performance to databases, the performance of the observable sample of funds may not be representative of the performance of the universe of funds (Fung and Hsieh 2000). Because hedge funds have little incentive to advertise poor returns, commercial databases may be missing the worst-performing hedge funds. On the other hand, some have argued that hedge fund databases may be missing the best hedge funds, as these funds are likely closed to new investment and have little to gain from further advertising their performance (Fung and Hsieh 1997). The direction and economic magnitude of the selection bias in commercial databases is controversial because very little is known about the funds that choose not to report their performance.

Selection bias in commercial databases would not necessarily invalidate or marginalize the findings of hedge fund research that use this data, but it raises questions about the generalizability of those results to the universe of hedge funds. For instance, if database-reporting hedge funds are comprised of a subset of relatively highly skilled managers, then studies of hedge fund performance using commercial databases may be internally valid but could reflect biased estimates of the performance of the hedge fund industry as a whole.

Testing for selection bias is inherently difficult, as it requires examining returns from hedge funds that have chosen not to report their performance to commercial databases. Recently, some researchers have attempted to circumvent this issue by analyzing the equity holdings of hedge fund advisors disclosed in their required quarterly 13F filings to the SEC.2Griffin and Xu (2009) use the equity holdings of hedge funds and find little evidence of stock-picking ability. On the other hand, Aragon and Martin (2012) find that hedge funds have skill in selecting equity options and Agarwal et al. (forthcoming) find that the confidentially held equity positions of hedge funds generate superior performance. The advantage of studying 13F filings is that they are mandatory for large institutional investors and thus do not suffer from the selection issues inherent in commercial hedge fund databases. However, one of the drawbacks of using 13F filings is that they contain only quarterly snapshots of long equity-based positions aggregated at the hedge fund advisor, rather than the fund, level. Also, performance implied from these filings ignores the impacts of short-selling, illiquid securities, and intraquarter trading, leading to an imprecise estimate of hedge fund returns. Griffin and Xu (2009) find a mean correlation of 0.55 between the implicit returns calculated from 13F filings and the actual returns reported to a commercial database, suggesting that a substantial portion of hedge fund returns are unexplained by the returns of their long equity holdings. Moreover, hedge fund advisors with less than $100 million in assets are not required to report their holdings to the SEC, implying that smaller hedge funds are not included in these studies.

In this paper, we avoid the issues inherent in 13F filings and test for selection bias in commercial databases using the actual fund-level returns earned by a sample of hedge fund investors, registered funds-of-funds (FoFs).3 We are able to obtain the returns for 1,445 distinct hedge funds, comprising 10,126 quarterly returns over the 2004–2009 period. We match the names of these funds to the union of five major hedge fund databases: Lipper TASS, HFR, BarclayHedge, Morningstar, and EurekaHedge. We determine that approximately one-half of these returns are not disclosed to any of the five major commercial hedge fund databases.

We find that the performance of these funds that do not report to a commercial database (nondatabase funds) is significantly worse on average than the performance of reporting funds (database funds). For example, using the Fung and Hsieh (2004) 7-factor model to adjust for risk, we find that the alpha of database funds is a statistically significant 120 basis points per quarter (bps/quarter), similar in magnitude to the 3%–5% annual alpha found in prior studies of database hedge funds (see, e.g., Stulz 2007). In contrast, the alpha of nondatabase funds is a statistically insignificant 5 bps/quarter ($$t$$-statistic = 0.39). Combining the database and nondatabase samples results in an alpha estimate that is over 60% lower than the alpha estimated using database funds alone.

We find that the majority of the difference in average performance between database and nondatabase funds is driven by what appears to be a truncated left tail in the returns distribution of database funds. The fifth percentile of nondatabase returns is 511 bps/quarter lower than the fifth percentile of database returns. Further, we find that differences in alpha between the database and nondatabase samples are particularly pronounced in the left tail of the returns distribution, suggesting the commercial databases are missing the worst performers in the hedge fund universe.

We also find that a number of the worst-performing nondatabase funds previously reported to commercial databases at some point in their history but subsequently chose to delist. Funds that report to commercial databases have considerable discretion over the timing of their disclosure and can delist from a database at any time. The flexibility to delay disclosure could give fund managers the opportunity to delist rather than advertise negative performance shocks. This delisting decision could lead to a particular form of selection bias, commonly known as a delisting bias.4 A delisting bias could be particularly concerning because, unlike mutual funds, hedge fund investments are illiquid, and traditional investors cannot simply liquidate their positions immediately after a fund delists.

We identify 205 hedge funds that have delisted from the commercial databases and are subsequently present in our sample of returns. Funds that have delisted, that is, dead funds, continue to operate for some time after the last database return is recorded, as 85% (48%) of dead funds have at least two (eight) quarters of returns subsequent to the delisting date. On average, the returns of dead funds are −16 bps/quarter, 184 bps/quarter lower than the returns of funds that continue to report to commercial databases. Additionally, we find that the pre-delisting performance of dead funds predicts their post-delisting performance. This evidence is indicative of a delisting bias, whereby the poor returns of delisted funds are missing from the commercial databases.

Taken together, our results support the notion that commercial hedge fund databases suffer from a positive selection bias, suggesting that managerial skill estimates from commercial databases are likely higher than the true managerial skill of the hedge fund universe. However, this does not mean that past evidence of managerial ability can be explained by a hedge fund's decision to report returns to databases. It could be the case that talented managers need to certify to investors that they have skill prior to reporting and initiate reporting once they have demonstrated a track record of success (Aggarwal and Jorion 2010). Managers that start reporting may have strong incentives to continue reporting to satisfy an implicit commitment of continued disclosure to investors. Thus, managers who report to commercial databases could represent a reliably talented subset of the manager universe. Though our tests neither confirm nor reject this possibility, our results do suggest that the true hedge fund investor experience is not as positive as the performance records found in commercial databases would imply. The poor performance of funds that delist appears to drive a wedge between investor experience and observable aggregate hedge fund performance.

Our findings have implications for the broad literature examining the risk-return characteristics of hedge funds and contribute to earlier work documenting hedge fund database biases. These biases include, but are not limited to, a backfilling bias (Fung and Hsieh 2000), a survivorship bias (Ackermann, McEnally, and Ravenscraft 1999), and return smoothing (Getmansky, Lo, and Makarov 2004; Bollen and Pool 2009). In particular, our paper is related to Fung and Hsieh (2000), who examine aggregate FoF returns to infer the possibility of selection bias by assuming FoF portfolios may be composed of some proportion of database and nondatabase funds. They found little evidence of selection bias, though their methodology relied on strong assumptions concerning the fees, cash balances, and portfolio composition of the FoFs in their sample. This paper is also similar in spirit to Agarwal, Fos, and Jiang (forthcoming), who study selection bias via the performance of equity-based holdings of hedge fund advisors from 13F filings. They also find evidence of a delisting bias but only marginal evidence of an overall selection bias. However, as mentioned previously, inferring hedge fund performance from 13F positions has its own set of limitations and is arguably imprecise. In contrast, we employ a more precise and direct measure of the performance of nondatabase hedge funds, allowing for more powerful tests of performance differences between database and nondatabase funds.5

In addition, our paper contributes to the recent literature documenting the prevalence of strategic reporting behavior of hedge funds. Similar to the earnings management research, hedge funds report more small positive than small negative monthly returns (Bollen and Pool 2009) and inflated performance at year-end (Agarwal, Daniel, and Naik 2011). Further, hedge funds strategically choose whether or not to continue reporting their returns each month and may decide to go back to restate prior returns (Patton, Ramadorai, and Streatfield 2012; Aragon and Nanda 2011). Cici, Kempf, and Puetz (2011) examine the reported equity valuations in 13F filings and find deviations from historic market prices that reflect opportunistic reporting behavior by hedge fund advisers. Our findings suggest that hedge funds respond to strategic reporting incentives, as some hedge funds appear unwilling to report extremely poor performance to commercial databases.

The remainder of the paper is organized as follows. Section 1 describes our data and return methodology. Section 2 reports our results. Section 3 addresses selection bias in our data and tests for robustness. Section 4 presents our conclusions.

1. Data

1.1 Registered funds-of-funds

Here, we describe the process for hand-collecting our set of hedge fund returns. Our data come from a set of registered funds-of-funds (FoFs) that have received little, if any, attention from the literature. These funds register with the Securities and Exchange Commission (SEC) pursuant to the Investment Company Act of 1940 (40 Act). With similar filing requirements to that of a mutual fund, these registered funds allow the researcher the unique opportunity to use regulatory filings to study FoFs. In Section 3 of the paper, we provide a detailed discussion of potential selection biases regarding the use of registered FoF data.

A registered FoF is organized as a closed-end investment company. Unlike the traditional closed-end fund, however, it is usually not listed on an exchange. Rather, the fund typically offers interest to investors on a periodic basis (usually monthly or quarterly). These funds have an average (median) minimum investment of $528,559 ($100,000). Similar to that of a typical FoF, the registered funds are marketed to qualified investors and often charge both fixed and performance-based fees. Additionally, investors typically have their assets “locked-up” for periods of up to a year and receive liquidity when fund management agrees to redeem interests through a regular tender offer.

The primary benefit to the fund in registering with the SEC is to allow the manager access to greater distribution channels. Nonregistered FoFs typically offer interests under Regulation D of the Securities Act of 1933. Although this allows the fund to avoid registration and sell interests in its fund through private placements, it explicitly limits the fund's ability to market or advertise. A registered fund faces no such restrictions. A registered fund files an offering prospectus with the SEC that allows it to actively advertise or market the fund to potential investors, including dedicated distribution platforms through investment advisers.

The advantage of using registered FoFs to the researcher is that SEC filing requirements mandate the fund disclose a detailed account of the hedge fund investments in their underlying portfolio. These portfolio snapshots enable us to construct the set of hedge fund returns we use in this paper. Beginning in 2004, funds were required to report portfolio holdings on a quarterly basis. This allows for the estimation of quarterly returns. Thus, we restrict our analysis to the holdings of registered funds between 2004 and 2009 where quarterly holdings are available.

Unfortunately, the SEC does not provide a distinctive classification mechanism that allows for an easy identification of all registered FoFs. We employ a set of search algorithms that comb through SEC filings to identify registered FoFs. We use NSAR forms from the SEC to identify all funds that file as a closed-end fund but do not list a closing price for the fund.6 This results in a sample of 132 possible FoFs from 2004 to 2009. We believe this to be the universe of registered FoFs. We eliminate fifteen funds that either registered and never raised any funds or held primarily venture capital or private equity investments. We then eliminate funds that simply duplicate the holdings of another fund from the same management company (such as feeder funds or funds with special tax treatment that cross-register). This screen eliminated thirty-eight funds, which yields our final sample of seventy-nine registered FoFs.

Table 1, Panel A, details the years our sample FoFs filed their initial registration statement with the SEC (Form N-2). A few funds existed prior to 2000, but the bulk of our sample began reporting in years 2001–2004. The funds typically register as de novo funds seeking startup capital from initial investors. However, established asset management companies operate most of the funds. The managers of our sample of FoFs represent some of the largest money managers and financial institutions, including Morgan Stanley, J.P. Morgan, Credit Suisse, UBS, and Blackrock. We have included a detailed case study of a representative fund from our sample of registered FoFs in an online appendix (Exhibit A1), describing the fund's history, organizational form, and contract characteristics.7

Table 1

SEC registered funds-of-funds

Panel A: First reporting year for sample of registered FoFs
 
Year Number of funds initiating reporting 
1998 
1999 
2000 
2001 11 
2002 23 
2003 11 
2004 17 
2005 
2006 
2007 
Panel A: First reporting year for sample of registered FoFs
 
Year Number of funds initiating reporting 
1998 
1999 
2000 
2001 11 
2002 23 
2003 11 
2004 17 
2005 
2006 
2007 
Panel B: Comparing registered FoFs to database FoFs (FoF/Qtr)
 
  Registered FoF
 
Database FoF
 
Differences
 
Mean Median Mean Median Means Medians 
Quarterly Net Return (%) 996 0.89 1.71 84,698 0.82 1.73 0.08 −0.01% 
AUM ($MM) 1,008 273.0 113.0 84,698 208.0 54.4 65.0** 58.6*** 
Age (Years) 1,008 4.2 4.0 84,698 3.5 2.0 0.6*** 2.0*** 
Minimum Investment ($) 992 528,559 100,000 83,327 727,619 149,690 −199,060*** −49,960*** 
Management Fees (%) 992 1.31 1.25 84,349 1.35 1.50 −0.04** −0.25*** 
Incentive Fees (%) 420 8.45 10.00 84,943 7.48 10.00 0.97*** 0.00*** 
Number of Holdings 1,008 23.2 21.0 − − − − − 
Number of Styles 1,008 4.5 5.0 − − − − − 
Panel B: Comparing registered FoFs to database FoFs (FoF/Qtr)
 
  Registered FoF
 
Database FoF
 
Differences
 
Mean Median Mean Median Means Medians 
Quarterly Net Return (%) 996 0.89 1.71 84,698 0.82 1.73 0.08 −0.01% 
AUM ($MM) 1,008 273.0 113.0 84,698 208.0 54.4 65.0** 58.6*** 
Age (Years) 1,008 4.2 4.0 84,698 3.5 2.0 0.6*** 2.0*** 
Minimum Investment ($) 992 528,559 100,000 83,327 727,619 149,690 −199,060*** −49,960*** 
Management Fees (%) 992 1.31 1.25 84,349 1.35 1.50 −0.04** −0.25*** 
Incentive Fees (%) 420 8.45 10.00 84,943 7.48 10.00 0.97*** 0.00*** 
Number of Holdings 1,008 23.2 21.0 − − − − − 
Number of Styles 1,008 4.5 5.0 − − − − − 

Panel A provides the number of registered fund-of-funds (FoF) that began reporting in a given calendar year. We define initial reporting as the year the initial N-2 registration was filed with the SEC. Panel B compares our sample of registered FoFs to the universe of FoFs that report to the union of commercial databases. All observations are at the FoF/Quarter level. Information about the registered FoFs has been collected from their registration filings. Registered FoF quarterly returns are calculated using the underlying hedge fund positions reported in their SEC filings. We only use individual hedge fund returns where the cost field has not changed from quarter to quarter, that is, no capital was added to or subtracted from the underlying fund by the FoF. Number of holdings is the average number of underlying hedge funds that a registered FoF holds in a particular quarter. $$***p {\lt } 0.01$$, $$**p {\lt } 0.05$$, $$*p {\lt } 0.1$$.

Table 1, Panel B, reports summary statistics for our set of FoFs and compares these funds to nonregistered FoFs found in the Lipper TASS, HFR, and BarclayHedge databases. The average FoF in our sample earns a mean (median) net of fee return of 0.89% (1.71%) a quarter, whereas the mean (median) database FoF earns 0.82% (1.73%) a quarter. Differences in means and medians are not statistically significant. The average FoF in our sample is slightly older (4.2 vs. 3.5 years), manages more money ($273.0 vs. $208.0 million in assets), and requires a lower minimum investment ($528,559 vs. $727,619). Registered FoFs charge slightly lower management fees but appear to offset this with higher performance fees. The average registered FoF holds over twenty-three unique hedge funds at any point in time, representing an average of 4.5 unique hedge fund styles (these data are not available for the database FoFs). Interestingly, however, differences in contract features between registered and nonregistered FoFs do not contribute to differences in performance on average.

1.2 Sample construction

Registered FoFs disclose their quarterly holdings in N-CSRS, N-CSR, and N-Q filings. The filings disclose the name of the underlying hedge fund, the FoF's cost basis in the position, and the current value of the position.8 The time-series combination of these forms allows us to utilize the underlying holdings to create a quarterly panel of hedge fund returns.

One of the primary innovations in our paper is that our hedge fund returns are not supplied by a commercial data provider (e.g., Lipper TASS). Instead, we calculate the returns of the hedge funds realized by the FoFs and implied through the change in cost and value of their portfolio hedge funds. Our methodology is perhaps best illustrated through an example. We provide an example of our return calculation using data provided by the March 2007 and June 2007 filings of the UBS Equity Opportunity Fund II, LLC (one of the sample FoFs).

UBS held a position in the Cobalt Partners, L.P., fund (an underlying hedge fund) in March 2007. The cost basis of this holding was $29,000,000, whereas the value of the holding was $48,873,238. UBS maintained this position in June 2007, reporting an increased value of $51,754,258. The cost basis remains $29,000,000. We use the following formula to generate our quarterly return:  

(1)
$${\rm{Fund}}\,{\rm{Retur}}{{\rm{n}}_{i,t}} = \frac{{{\rm{Valu}}{{\rm{e}}_{i,t}} - {\rm{Change}}\,{\rm{in}}\,{\rm{Cos}}{{\rm{t}}_{i,(t - 1,t)}}}}{{{\rm{Valu}}{{\rm{e}}_{i,t - 1}}}} - 1.$$

In the case of the Cobalt Partners, L.P., fund, its second-quarter return for 2007 is 5.89%. We repeat this process for every hedge fund holding in each FoF's quarterly filings and generate a sample of quarterly hedge fund returns from 2004 to 2009.

For a small sample of our funds, we find discrepancies in how they report changes in cost. Although most FoFs report the actual dollar change in cost, some report the change as a percentage of the fund's value.9 Additionally, changes in cost require placing strong assumptions on the timing of cost changes throughout the quarter. Given our inability to be certain of the regime used by each fund, the remainder of the paper focuses exclusively on those returns in which cost basis does not change. Our results are robust to their inclusion and are discussed more fully in Section 3.

Multiple FoFs may hold the same underlying hedge fund during the quarter. The DE Shaw Oculus fund, for example, was held by three distinct FoFs during the first quarter of 2008. However, variation in FoF investment timing could lead to differences in highwater marks or fees, resulting in small differences in net quarterly returns for the same hedge fund. In the results that follow, we take the median return for any fund-quarter with multiple return observations. Our results are robust to using means as well. Additionally, we trim our returns at the 0.5% and 99.5% levels to reduce the outliers that accompany hand-collected data. Our results are robust to the inclusion of the trimmed data. See Section 3 for a discussion of these and additional robustness tests. After forming median returns across FoFs for every quarter, our final sample consists of 1,445 unique hedge funds and 10,126 unique fund-quarter returns.

Of these 10,126 fund-quarters, 4,925 fund-quarters (49%) have a corresponding fund-quarter in either the Lipper TASS, HFR, BarclayHedge, Morningstar, or EurekaHedge databases.10 The remaining 5,201 (51%) fund-quarter observations comprise the first large set of hedge fund returns from funds that declined to report to a commercial database. The goal of this paper is to compare these two separate distributions of hedge fund returns to estimate the potential selection bias inherent in voluntarily reported hedge fund returns.11

We perform several robustness checks to ensure that our returns are calculated accurately. For the fund-quarters that also match to either Lipper TASS, HFR, or BarclayHedge, we can compare the fund's actual returns reported to the database to the returns we calculate from FoF holdings. The average discrepancy between the samples is near zero: Both the mean and median sample differences are less than 0.01 bp/quarter. Additionally, the correlations between our calculated returns and those of the reported database returns are over 0.98 across each of the three databases. Any discrepancies likely come from small differences in fees/highwater marks between the FoFs, differences in taxes, or errors in our name matching. Importantly, we note that for the 49% of our sample for which returns are available in a database, we instead use our calculated returns from Equation (1) for our tests. This is done to ensure that our return methodology and/or data errors in the inputs to calculate returns do not drive our results. That is, any errors from our name matching or return generation procedure are equally likely to affect both database and nondatabase funds.

1.3 Database and nondatabase returns

Table 2 details the distribution of quarterly returns in our sample and reports summary statistics of returns for database and nondatabase funds. The average (median) return for a database fund is 1.68% (2.03%) a quarter, whereas the average (median) return for a nondatabase fund is 0.60% (1.76%). The equality of both means and medians between the two groups is rejected at the 1% level. However, the difference in mean returns (1.08%) is significantly larger than the difference in medians (0.27%), suggesting that the distribution of nondatabase returns may have a thicker left tail than the distribution of database returns. We also note that the nondatabase sample is more volatile than the database sample (standard deviation of 8.58% vs. 6.85%). A variance ratio test reveals these standard deviations are significantly different at the 1% level. In Figure 1, we present a kernel density estimation of the distribution of returns for database versus nondatabase funds. The nondatabase distribution appears to have a thicker left tail than that of the database distribution, indicating that nondatabase funds have a higher frequency of poor performance. This suggests that the performance differences between database and nondatabase funds could be driven by hedge funds who wish to conceal extreme poor performance. We investigate this issue more fully in Section 2.3.

Figure 1

Kernel density estimations

Figure 1 compares the distribution of returns found in the union of commercial databases to those returns not reported to a database.

Figure 1

Kernel density estimations

Figure 1 compares the distribution of returns found in the union of commercial databases to those returns not reported to a database.

Table 2

Hedge fund quarterly returns calculated using fund-of-fund holdings

  Mean $${\sigma }$$ Median 5th 95th 
All Returns 10,126 1.13% 7.80% 1.91% −12.96% 11.93% 
Database Returns 4,925 1.68 6.85 2.03 −10.11 11.90 
Nondatabase Returns 5,201 0.60 8.58 1.76 −15.22 11.98 
  Differences 1.07***   0.27***     
Database Returns of Funds that Leave 829 1.59 7.09 2.16 −11.74 11.59 
Dead Returns 1,083 −0.16 9.84 1.43 −20.66 12.42 
  Differences 1.75***   0.73***     
  Mean $${\sigma }$$ Median 5th 95th 
All Returns 10,126 1.13% 7.80% 1.91% −12.96% 11.93% 
Database Returns 4,925 1.68 6.85 2.03 −10.11 11.90 
Nondatabase Returns 5,201 0.60 8.58 1.76 −15.22 11.98 
  Differences 1.07***   0.27***     
Database Returns of Funds that Leave 829 1.59 7.09 2.16 −11.74 11.59 
Dead Returns 1,083 −0.16 9.84 1.43 −20.66 12.42 
  Differences 1.75***   0.73***     

Table 2 provides descriptives for the quarterly hedge fund returns used in this study. Only returns calculated from the underlying holdings of Funds-of-Funds (FoF) collected from SEC filings are used. Database Returns describes all fund-quarter returns that are also found in the union of hedge fund databases. NonDatabase Returns describes all fund-quarter returns that are not found in the union of hedge fund databases. Database Returns of Funds that Leave are the pre-delisting returns of funds that subsequently leave a database. Dead Returns are returns that appear after a fund last reports to the databases. Quarterly underlying fund returns are calculated using the value and cost fields from hand-collected FoF SEC filings (see Equation 1). The value of FoF's investments used to calculate returns are net of all fees. We report returns calculated from a sample where the cost field is not allowed to change from quarter to quarter, that is, no capital was added to or subtracted from the underlying fund by the FoF. Different FoFs may hold the same underlying hedge fund, so multiple reported returns for the same hedge fund are possible in a given quarter. We report the median return across all FoFs that report a given hedge fund in a quarter. Returns have been trimmed at the 0.5% and 99.5% levels. We test for differences in means using a two-sample $$t$$-test with unequal variances and for differences in medians using a Wilcoxon rank-sum test. $$***p {\lt } 0.01$$, $$**p {\lt } 0.05$$, $$*p {\lt } 0.1$$.

In addition to comparing the returns that are reported to a database to those that are not, we also explore the returns of funds that reported to a commercial database at some point in their life cycle but subsequently chose to stop reporting. As mentioned previously, the nature of our data allows us to track the performance of these funds even after they stop reporting, and as noted the average performance of dead funds is dramatically lower than their live counterparts. For funds that delist, their average (median) quarterly return prior to delisting is 1.59% (2.16%); however, following delisting their average (median) quarterly return falls to −0.16% (1.43%). Differences between means and medians are both statistically significant at the 1% level. This is consistent with the findings of Agarwal, Fos, and Jiang (forthcoming), who also show that hedge fund performance declines following delisting.

2. Results

The univariate results in Table 2 suggest that hedge fund returns missing from commercial databases are on average lower than those found in the databases, potentially creating a selection bias that distorts the overall performance picture of the hedge fund industry. In this section, we examine whether these performance differences could lead to differences in estimates of managerial skill in a multivariate framework. We first test for an overall selection bias by testing differences in abnormal performance (alpha) between database and nondatabase funds. We then test for a delisting bias and discuss the contribution this bias may have toward an overall selection bias in hedge fund data. Finally, we formally test an implication found in both Table 2 and Figure 1, which hinted that the performance differences between database and nondatabase funds could be driven by extreme poor performance in the left tail of the nondatabase returns distribution.

2.1 Hedge fund manager skill of database versus nondatabase funds

We estimate hedge fund manager skill using a pooled ordinary least squares (OLS) factor pricing methodology similar to Aragon (2007). Specifically, we estimate abnormal performance as the intercept from two common factor pricing models and a benchmark-based model using our pooled sample of quarterly hedge fund returns:  

(2)
$${{\rm{r}}_{i,t}} = {\rm{Intercept}} + \sum\limits_{j = 1}^J {{\beta _j}*{\rm{facto}}{{\rm{r}}_{j,t}} + { \epsilon _{i,t}},} $$
where $$r_{i,t}$$ is a fund's quarterly return in excess of the three-month risk-free rate and the controls come from the Carhart (1997) 4-factor model, the Fung and Hsieh (2004) 7-factor model,12 or a modified version of the Jagannathan, Malakhov, and Novikov (2010) hedge fund (HF) benchmark model.13 Ideally we would like to estimate fund-level alphas to allow for variation in factor exposure and to correct for return smoothing.14 However, the dimensionality of our data does not allow this; we simply have too few quarterly return observations per fund. In subsequent analysis, we control for differences in risk exposure between database and nondatabase funds. Additionally, the quarterly nature of our hand-collected returns mitigates the smoothing problem, as it is unlikely that quarterly returns exhibit much autocorrelation (Bollen and Pool 2009). Standard errors are robust to heteroscedasticity and are clustered at the fund level. We suppress our estimates of the factor exposures for expositional ease.

Table 3 reports the results from these regressions. We begin in Panel A by estimating alpha using only the subsample of database funds. Consistent with past literature, we find that the average database fund has significant alpha (Intercept) in each of the three pricing models, with estimates ranging from 72 to 120 bps/quarter. These magnitudes are consistent with those found in prior studies.15 However, it is important to note that the estimates of alpha in Panel A do not reflect the performance of funds that opted not to report to commercial databases.

Table 3

Pooled OLS results: Database returns versus nondatabase returns

Panel A: Database funds only 
  4-Factor 7-Factor HF Benchmark 
Intercept 0.0087*** (8.56) 0.0120*** (7.82) 0.0072*** (6.26) 
Observations 4,925 4,925 4,925 
Adjusted $$R^{2}$$ 0.19 0.18 0.19 
Panel A: Database funds only 
  4-Factor 7-Factor HF Benchmark 
Intercept 0.0087*** (8.56) 0.0120*** (7.82) 0.0072*** (6.26) 
Observations 4,925 4,925 4,925 
Adjusted $$R^{2}$$ 0.19 0.18 0.19 
Panel B: Full sample 
  4-Factor 7-Factor HF Benchmark 
Intercept 0.0040*** (4.77) 0.0045*** (4.03) 0.0026*** (2.94) 
Observations 10,126 10,126 10,126 
Adjusted $$R^{2}$$ 0.21 0.21 0.21 
Panel B: Full sample 
  4-Factor 7-Factor HF Benchmark 
Intercept 0.0040*** (4.77) 0.0045*** (4.03) 0.0026*** (2.94) 
Observations 10,126 10,126 10,126 
Adjusted $$R^{2}$$ 0.21 0.21 0.21 
Panel C: Full sample, including database indicator
 
  4-Factor 7-Factor HF Benchmark 
Database 0.0085*** (5.65) 0.0087*** (5.75) 0.0087*** (5.73) 
Intercept −0.0002 (−0.14) 0.0005 (0.39) −0.0017 (−1.33) 
Observations 10,126 10,126 10,126 
Adjusted $$R^{2}$$ 0.21 0.21 0.22 
Panel C: Full sample, including database indicator
 
  4-Factor 7-Factor HF Benchmark 
Database 0.0085*** (5.65) 0.0087*** (5.75) 0.0087*** (5.73) 
Intercept −0.0002 (−0.14) 0.0005 (0.39) −0.0017 (−1.33) 
Observations 10,126 10,126 10,126 
Adjusted $$R^{2}$$ 0.21 0.21 0.22 
Panel D: Full sample, including interactions between database indicator and factor returns 
  4-Factor 7-Factor HF Benchmark 
Database 0.0088*** (5.65) 0.0133*** (6.09) 0.0088*** (5.21) 
Intercept −0.0001 (−0.08) −0.0013 (−0.84) −0.0017 (−1.30) 
Observations 10,126 10,126 10,126 
Adjusted $$R^{2}$$ 0.22 0.22 0.22 
Panel D: Full sample, including interactions between database indicator and factor returns 
  4-Factor 7-Factor HF Benchmark 
Database 0.0088*** (5.65) 0.0133*** (6.09) 0.0088*** (5.21) 
Intercept −0.0001 (−0.08) −0.0013 (−0.84) −0.0017 (−1.30) 
Observations 10,126 10,126 10,126 
Adjusted $$R^{2}$$ 0.22 0.22 0.22 

Table 3 describes the results from a pooled OLS regression of quarterly hedge fund returns using three different factor models: the Carhart (1997) 4-factor model, the Fung and Hsieh (2004) 7-factor model, and a modified version of the Jagannathan, Malakhov, and Novikov (2010) hedge fund–style benchmark model. Only returns calculated from the underlying holdings of Funds-of-Funds (FoF) collected from SEC filings are used. Panel A uses just those returns of funds held by a registered FoF that report to a commercial database, whereas Panel B includes all of the returns in our sample. Panel C includes all of the returns in our sample, but introduces a database indicator that equals one if a fund-quarter return from the hand-collected sample matches to a fund-quarter return in the union of hedge fund databases. Database is the coefficient for that indicator. Therefore, in these models, the excess returns for funds that do not report is simply Intercept, whereas the excess returns for reporting funds can be interpreted as Intercept plus Database. Finally, the regression specifications in Panel D include interaction terms between the database indicator and the appropriate factor returns to capture potential risk differences between the database and nondatabase return samples. Returns are calculated from a sample where the cost field is not allowed to change from quarter to quarter, that is, no capital was added to or subtracted from the underlying fund by the FoF. We suppress the individual factor loadings and interaction terms. The fund values used to calculate returns are net of all fees. Fund returns are defined as the median return across all FoF advisors who report that underlying fund in that quarter, are net of the risk-free rate, and have been trimmed at the 0.5% and 99.5% levels. Standard errors are robust to heteroscedasticity and are clustered at the hedge fund level. We report $$t$$-statistics in parentheses. $$***p {\lt } 0.01$$, $$**p {\lt } 0.05$$, $$*p {\lt } 0.1$$.

In Panel B, we estimate alpha for the combined sample of database and nondatabase funds. Including the nondatabase funds in the sample significantly reduces our alpha estimates. In the case of the 7-factor model, the alpha estimate drops over 60%, from 120 bps/quarter for the database-only sample to 45 bps/quarter for the full sample. Though we still find positive and statistically significant alpha on average, estimates of skill from the database funds are more than double those of the full sample. This evidence is suggestive of a considerable positive selection bias in commercial hedge fund databases.

The pattern of results in Panels A and B suggests that, on average, nondatabase funds significantly underperform the funds that choose to report to the database. We seek to estimate this difference in performance directly in Panels C and D by including a dummy variable in our factor regressions indicating whether a fund's return was reported to a database. Specifically, in Panel C, we estimate16 

(3)
$${{\rm{r}}_{i,t}} = {\rm{Intercept}} + Database*{\gamma _{i,t}} + \sum\limits_{j = 1}^J {{\beta _j}*{\rm{facto}}{{\rm{r}}_{j,t}} + { \epsilon _{i,t}}.} $$

Our indicator variable $$({\gamma })$$ shifts the intercept of the factor regression for database funds and estimates the difference in alphas between database and nondatabase funds. Under this framework, one can interpret Intercept as an estimate of the average alpha of nondatabase funds and the sum of Intercept and Database as an estimate of the average alpha of database funds.

In Panel C, Database is positive and statistically different from zero for each of the models. In the case of the 7-factor model, the alpha of database funds is 87 bps/quarter larger than the alpha of nondatabase funds. The results are similar with the 4-factor model and the hedge fund benchmark model.

In Panel D, we allow for the possibility of differences in risk exposures between the database and nondatabase funds by including an interaction of each of the factors in the model with the database indicator:17 

(4)
$${{\rm{r}}_{i,t}} = {\rm{Intercept}} + Database*{\gamma _{i,t}} + \sum\limits_{j = 1}^J {{\beta _j}*{\rm{facto}}{{\rm{r}}_{j,t}} + \sum\limits_{j = 1}^J {{\phi _j}*{\rm{facto}}{{\rm{r}}_{j,t}}*{\gamma _{i,t}}} + { \epsilon _{i,t}}.} $$

With the exception of the market factor, sensitivities to the factors are similar between the database and nondatabase funds. Database funds have lower sensitivities to the market factor. In each of the models, Database remains positive and statistically different from zero. In the case of the 7-factor model, the estimate of alpha for database funds is 133 bps/quarter ($$t$$-statistic = 6.09) higher than for nondatabase funds. For both the 4-factor and HF benchmark model, estimates of alpha for database funds are 88 bps/quarter higher than for nondatabase hedge funds. Interestingly, in each of the factor models in Panels C and D, Intercept is not statistically different from zero, indicating that nondatabase funds do not have alpha over our sample period. In other words, the alpha we find for the full sample in Panel B is driven by the superior performance of database funds.

To summarize, we find evidence of some skill in aggregate hedge fund performance; however, these estimates are less than half of those that only consider the performance of database funds. We interpret this as evidence in support of a positive selection bias in commercial hedge fund databases, meaning that estimates of managerial skill from commercial databases likely exaggerate the true ability of the hedge fund universe.

2.2 How does delisting affect hedge fund returns?

Funds that report to commercial databases have considerable discretion over the timing of their disclosure and can choose to delist from the database at any time. Additionally, a fund can delay reporting over six months before being considered delisted by a commercial database company.18 When a hedge fund delists, its performance record is typically retained by the database company in its “Graveyard” database and is commonly referred to as a “dead” fund. However, a dead fund has not necessarily stopped operating; the fund has merely stopped reporting its performance to a database. Understanding the returns of dead funds is important because hedge fund investments are illiquid, and traditional investors cannot simply liquidate their positions immediately after a fund delists. Thus, the true aggregate hedge fund investor experience necessarily comprises the returns of dead funds.

Although no study examines the returns of dead funds directly, there have been two primary explanations put forth explaining why a fund would stop reporting its returns to a database.19 Most researchers suggest that funds choose to stop reporting because they realized or anticipated to realize poor performance (Malkiel and Saha 2005). Fund managers who underperform their peers do not have an incentive to continue advertising that fact in a commercial database. These managers may benefit from delisting and waiting until their track record improves before they advertise to new investors. Additionally, very poor performers may stop reporting because they have ceased operations and begun the liquidation process. In this case, dead funds should exhibit very poor returns on average.

A competing hypothesis suggests that funds may choose to stop reporting because they have been very successful (Ackermann, McEnally, and Ravenscraft 1999). Asuccessful fund may not need to advertise its performance in a database because it has strong word-of-mouth reputation or has reached critical mass and closed to new investors. As the marginal benefit from advertising falls for these funds, they may choose to stop reporting returns in fear that their competitors would replicate their investment strategy. If this were the case, dead funds would likely exhibit higher than average returns.

Because of our unique sample, we are able to observe the returns of delisted hedge funds that continue to be held by FoFs after they delist. To study the performance of the dead funds, we identify the funds that matched to any commercial database and compare their last database return date to the return observation date generated from our sample. We classify return observations that occur after the last date the fund stopped reporting to all databases as dead returns. We employ a factor model regression methodology similar to the one used above, only now we are estimating the differential performance of dead funds:  

(5)
$${{\rm{r}}_{i,t}} = {\rm{Intercept}} + {\rm{Dead*}}{\theta _{i,t}} + \sum\limits_{j = 1}^J {{\beta _j}*{\rm{facto}}{{\rm{r}}_{j,t}} + { \epsilon _{i,t}},} $$
 
(6)
$${{\rm{r}}_{i,t}} = {\rm{Intercept}} + {\rm{Dead*}}{\theta _{i,t}} + \sum\limits_{j = 1}^J {{\beta _j}*{\rm{facto}}{{\rm{r}}_{j,t}}} + \sum\limits_{j = 1}^J {{\phi _j}*{\rm{facto}}{{\rm{r}}_{j,t}}*{\theta _{i,t}}} + { \epsilon _{i,t}}.$$

In Table 4, we report a series of tests using Equations (5) and (6) with the Carhart (1997) 4-factor model, the Fung and Hsieh (2004) 7-factor model, and the modified version of the Jagannathan, Malakhov, and Novikov (2010) benchmark model. The indicator variable $${\theta }$$ allows for a test of abnormal performance following delisting. The first three columns of Panel A contain both funds that matched to a database and those that did not and provide a test of whether dead funds underperform the general population of hedge funds in our sample. In each of the models, Dead is negative and statistically significant. In the 7-factor model, Dead is −109 bps/quarter ($$t$$-statistic = −3.65), indicating that the average fund that delists from a database subsequently experiences an economically large reduction in performance. The results are similar using the 4-factor and hedge fund benchmark model.

Table 4

Pooled OLS results: Dead returns

Panel A: Factor model regressions 
  Full sample
 
Database and dead funds only
 
4-Factor 7-Factor HF Bench 4-Factor 7-Factor HF Bench 
Dead −0.0108*** (−3.73) −0.0109*** (−3.65) −0.0111*** (−3.81) −0.0141*** (−4.89) −0.0146*** (−4.85) −0.0144*** (−4.90) 
Intercept
 
0.0051*** (6.08)
 
0.0059*** (5.12)
 
0.0038*** (4.16)
 
0.0082*** (8.02)
 
0.0098** * (6.63)
 
0.0074*** (6.71)
 
Observations 10,126 10,126 10,126 6,026 6,026 6,026 
Adjusted $$R^{2}$$ 0.21 0.21 0.21 0.22 0.21 0.21 
Panel A: Factor model regressions 
  Full sample
 
Database and dead funds only
 
4-Factor 7-Factor HF Bench 4-Factor 7-Factor HF Bench 
Dead −0.0108*** (−3.73) −0.0109*** (−3.65) −0.0111*** (−3.81) −0.0141*** (−4.89) −0.0146*** (−4.85) −0.0144*** (−4.90) 
Intercept
 
0.0051*** (6.08)
 
0.0059*** (5.12)
 
0.0038*** (4.16)
 
0.0082*** (8.02)
 
0.0098** * (6.63)
 
0.0074*** (6.71)
 
Observations 10,126 10,126 10,126 6,026 6,026 6,026 
Adjusted $$R^{2}$$ 0.21 0.21 0.21 0.22 0.21 0.21 
Panel B: Include interactions between dead indicator and factor returns 
  Full Sample
 
Database and dead funds only
 
4-Factor 7-Factor HF Bench 4-Factor 7-Factor HF Bench 
Dead −0.0103*** (−3.86) −0.0166*** (−4.67) −0.0087*** (−3.13) −0.0137*** (−5.06) −0.0217*** (−5.88) −0.0134*** (−4.79) 
Intercept 0.0053*** (6.28) 0.0068*** (5.87) 0.0035*** (3.87) 0.0087*** (8.54) 0.0120*** (7.80) 0.0062*** (2.76) 
Observations 10,126 10,126 10,126 6,026 6,026 6,026 
Adjusted $$R^{2}$$ 0.22 0.22 0.22 0.23 0.22 0.22 
Panel B: Include interactions between dead indicator and factor returns 
  Full Sample
 
Database and dead funds only
 
4-Factor 7-Factor HF Bench 4-Factor 7-Factor HF Bench 
Dead −0.0103*** (−3.86) −0.0166*** (−4.67) −0.0087*** (−3.13) −0.0137*** (−5.06) −0.0217*** (−5.88) −0.0134*** (−4.79) 
Intercept 0.0053*** (6.28) 0.0068*** (5.87) 0.0035*** (3.87) 0.0087*** (8.54) 0.0120*** (7.80) 0.0062*** (2.76) 
Observations 10,126 10,126 10,126 6,026 6,026 6,026 
Adjusted $$R^{2}$$ 0.22 0.22 0.22 0.23 0.22 0.22 

Table 4 describes the performance of dead funds from pooled OLS regressions of quarterly hedge fund returns using three different factor models: the Carhart (1997) 4-factor model, the Fung and Hsieh (2004) 7-factor model, and a modified version of the Jagannathan, Malakhov, and Novikov (2010) hedge fund–style benchmark model. Only returns calculated from the underlying holdings of Funds-of-Funds (FoF) collected from SEC filings are used. These models also include a dead indicator that equals one if a fund's hand-collected return that quarter appears after the fund has last reported to the union of databases, that is, is no longer in a database. Dead is the coefficient for that indicator. Therefore, in these models, the excess returns for funds that do not report is simply Intercept, whereas the excess returns for reporting funds can be interpreted as Intercept plus Dead. The regression specifications in Panel B include interaction terms between the dead indicator and the appropriate factor returns. We suppress the individual factor loadings and interaction terms. Each set of factor model regressions is run twice: first using all hedge fund returns from the hand-collected sample and then only using returns from funds that reported to a database at some point during their life. Returns are calculated from a sample where the cost field is not allowed to change from quarter to quarter, that is, no capital was added to or subtracted from the underlying fund by the FoF. Hedge fund returns are defined as the median return across all FoF advisors who report that underlying fund in that quarter, are net of the risk-free rate, and have been trimmed at the 0.5% and 99.5% levels. Standard errors are robust to heteroscedasticity and are clustered at the hedge fund level. We report $$t$$-statistics in parentheses. $$***p {\lt } 0.01$$, $$**p {\lt } 0.05$$, $$*p {\lt } 0.1$$.

In the last three columns of Panel A, we limit the sample to only those funds that reported to a commercial database at some point. As before, Dead is negative and statistically significant in each of the models. Using the 7-factor model, dead funds underperform database funds by an average of 146 bps/quarter ($$t$$-statistic = −4.85). In Panel B, we repeat the analysis while allowing for differences in risk exposure (Equation 6) and find similar results. The results in Table 4 suggest that the average performance of dead funds is nontrivially poor after they delist, which supports the hypothesis that commercial databases suffer from a positive delisting bias.

A positive delisting bias should be of particular concern to hedge fund researchers and investors given the relatively high annual attrition rate of commercial hedge fund databases. Over our sample period, roughly one of every ten database hedge funds delists from the union of commercial databases each year. Given the considerable reporting leeway afforded to hedge funds by commercial databases, it could take investors several months before they realize a particular fund has delisted from the database. Redemption restrictions imposed by the fund could further delay an investor's ability to divest from a dead fund. These factors suggest that even if one restricts their analysis to database funds only, the poor performance of dead funds may be unavoidable and could have a nontrivial negative impact on the actual performance of hedge fund portfolios. As such, we recommend researchers follow the lead of recent studies, such as Titman and Tiu (2011) and Ang and Bollen (2011), who check the robustness of their results by attributing a large delisting return to delisted funds ranging from −25% to −100%.20

2.2.1 Does delisting fully explain the selection bias?

The results in the previous section suggest the overall selection bias in commercial databases (i.e., significantly different performance between database and nondatabase funds) is in part driven by the delisting bias, because dead funds are nondatabase funds by definition. This leaves open the question as to whether funds that have never reported to databases perform differently from those that have. We examine this issue in Table 5 by simultaneously controlling for both database and dead funds in our regressions. That is, we estimate both Database and Dead in a series of factor pricing models. In Panel A, we estimate the models without any interaction terms, whereas in Panel B, we allow for different risk exposures for database and dead funds by including interaction terms with each of the factors. The results are similar. Focusing on Panel B, Database is positive and statistically different from zero in each of the models. In the 7-factor model, database alpha is 105 bps/quarter ($$t$$-statistic = 4.56) larger than the alpha of nondatabase (nondead) funds, even when controlling for dead funds. Dead remains negative and significant (−113 bps/quarter). By comparing these results to Table 3, one can see that although the performance of dead funds contributes to the overall positive selection bias (Database drops from 133 bps/quarter to 105 bps/quarter), it does not fully explain it. Rather, funds that have never reported to a database have worse performance than those that do report.

Table 5

Pooled OLS results: Database and dead indicators together

Panel A: Factor model regressions 
  4-Factor 7-Factor HF Bench 
Database 0.0071*** (4.35) 0.0072*** (4.49) 0.0072*** (4.43) 
Dead −0.0069** (−2.24) −0.0070** (−2.20) −0.0071** (−2.29) 
Intercept 0.0013 (0.96) 0.0021 (1.37) −0.0002 (−0.12) 
Observations 10,126 10,126 10,126 
Adjusted $$R^{2}$$ 0.21 0.21 0.22 
Panel A: Factor model regressions 
  4-Factor 7-Factor HF Bench 
Database 0.0071*** (4.35) 0.0072*** (4.49) 0.0072*** (4.43) 
Dead −0.0069** (−2.24) −0.0070** (−2.20) −0.0071** (−2.29) 
Intercept 0.0013 (0.96) 0.0021 (1.37) −0.0002 (−0.12) 
Observations 10,126 10,126 10,126 
Adjusted $$R^{2}$$ 0.21 0.21 0.22 
Panel B: Include interactions between both indicators and factor returns 
  4-Factor 7-Factor HF Bench 
Database 0.0072*** (4.29) 0.0105*** (4.56) 0.0078*** (4.23) 
Dead −0.0065** (−2.23) −0.0113*** (−2.29) −0.0045 (−1.50) 
Intercept 0.0015 (1.08) 0.0015 (0.89) −0.0006 (−0.44) 
Observations 10,126 10,126 10,126 
Adjusted $$R^{2}$$ 0.22 0.22 0.22 
Panel B: Include interactions between both indicators and factor returns 
  4-Factor 7-Factor HF Bench 
Database 0.0072*** (4.29) 0.0105*** (4.56) 0.0078*** (4.23) 
Dead −0.0065** (−2.23) −0.0113*** (−2.29) −0.0045 (−1.50) 
Intercept 0.0015 (1.08) 0.0015 (0.89) −0.0006 (−0.44) 
Observations 10,126 10,126 10,126 
Adjusted $$R^{2}$$ 0.22 0.22 0.22 

Table 5 describes the results from a pooled OLS regression of quarterly hedge fund returns on two different indicators (Database and Dead) using three different factor models: the Carhart (1997) 4-factor model, the Fung and Hsieh (2004) 7-factor model, and a modified version of the Jagannathan, Malakhov, and Novikov (2010) hedge fund–style benchmark model. Only returns calculated from the underlying holdings of Funds-of-Funds (FoF) collected from SEC filings are used. All models include a database indicator that equals one if a fund-quarter return from the hand-collected sample matches to a fund-quarter return in the union of hedge fund databases and a dead indicator that equals one if a fund's hand-collected return that quarter appears after the fund has last reported to the union of databases, that is, is no longer in a database. Database and Dead are the coefficients for each indicator, respectively. The regression specifications in Panel B include interaction terms between both indicators and the appropriate factor returns. We suppress the individual factor loadings and interaction terms. Returns are calculated from a sample where the cost field is not allowed to change from quarter to quarter, that is, no capital was added to or subtracted from the underlying fund by the FoF. Hedge fund returns are defined as the median return across all FoF advisors who report that underlying fund in that quarter, are net of the risk-free rate, and have been trimmed at the 0.5% and 99.5% levels. Standard errors are robust to heteroscedasticity and are clustered at the hedge fund level. We report $$t$$-statistics in parentheses. $$***p {\lt } 0.01$$, $$**p {\lt } 0.05$$, $$*p {\lt } 0.1$$.

2.2.2 Dead returns and prior-fund performance.

The results in Tables 4 and 5 show that dead funds underperform the universe of hedge funds on average. However, Ackermann et al. (1999) and others have argued that some funds might delist out of strength, suggesting that some dead funds may have positive abnormal performance. We test this possibility here. Although the fund's stated intent for delisting is available in certain commercial databases, this variable is often labeled as “no reason given” and is in many ways arbitrary, as the fund manager has no reason to provide an answer. As such, we use a more objective measure to proxy for delisting intent by examining the fund's style-adjusted performance in the months leading to delisting. In Table 6, we estimate the pooled factor models as before and additionally include two interactions with the dead indicator. The first interaction takes a value of one if the fund's style-adjusted performance prior to delisting was positive (Dead*Positive), whereas the second interaction takes a value of one if the fund's style-adjusted performance was negative (Dead*Negative). Funds that perform worse (better) than their matched hedge fund–style benchmark prior to delisting are considered negative (positive) delisters. We estimate the prior performance measure over different horizons (three, six, and twelve months) and find similar results. On average, funds with negative relative performance prior to delisting perform poorly after exiting the database, whereas funds with positive relative performance do not suffer as much.

Table 6

Pooled OLS results: Conditional performance of dead funds

Panel A: Full sample 
  4-Factor 4-Factor 4-Factor 7-Factor 7-Factor 7-Factor HF Bench HF Bench HF Bench 
Database 0.0071*** (4.50) 0.0071*** (4.54) 0.0073*** (4.63) 0.0073*** (4.64) 0.0073*** (4.69) 0.0075*** (4.78) 0.0073*** (4.61) 0.0073*** (4.66) 0.0075*** (4.72) 
Dead*Negative3 −0.0160*** (−3.23)     −0.0164*** (−3.24)     −0.0158*** (−3.19)     
Dead*Positive3 −0.0021 (−0.67)     −0.0012 (−0.36)     −0.0026 (−0.81)     
Dead*Negative6   −0.0159*** (−2.93)     −0.0163*** (−2.91)     −0.0155*** (−2.83)   
Dead*Positive6   −0.0028 (−0.90)     −0.0019 (−0.59)     −0.0033 (−1.06)   
Dead*Negative12     −0.0118*** (−2.71)     −0.0117*** (−2.61)     −0.0118*** (−2.70) 
Dead*Positive12     −0.0036 (−1.08)     −0.0029 (−0.83)     −0.0041 (−1.19) 
Intercept
 
0.0014 (1.05)
 
0.0013 (1.03)
 
0.0012 (0.92)
 
0.0021 (1.42)
 
0.0021 (1.41)
 
0.0019 (1.29)
 
−0.0001 (−0.07)
 
−0.0001 (−0.09)
 
−0.0002 (−0.19)
 
Observations 10,126 10,126 10,126 10,126 10,126 10,126 10,126 10,126 10,126 
Adjusted $$R^{2}$$ 0.21 0.21 0.21 0.21 0.21 0.21 0.22 0.22 0.22 
Panel A: Full sample 
  4-Factor 4-Factor 4-Factor 7-Factor 7-Factor 7-Factor HF Bench HF Bench HF Bench 
Database 0.0071*** (4.50) 0.0071*** (4.54) 0.0073*** (4.63) 0.0073*** (4.64) 0.0073*** (4.69) 0.0075*** (4.78) 0.0073*** (4.61) 0.0073*** (4.66) 0.0075*** (4.72) 
Dead*Negative3 −0.0160*** (−3.23)     −0.0164*** (−3.24)     −0.0158*** (−3.19)     
Dead*Positive3 −0.0021 (−0.67)     −0.0012 (−0.36)     −0.0026 (−0.81)     
Dead*Negative6   −0.0159*** (−2.93)     −0.0163*** (−2.91)     −0.0155*** (−2.83)   
Dead*Positive6   −0.0028 (−0.90)     −0.0019 (−0.59)     −0.0033 (−1.06)   
Dead*Negative12     −0.0118*** (−2.71)     −0.0117*** (−2.61)     −0.0118*** (−2.70) 
Dead*Positive12     −0.0036 (−1.08)     −0.0029 (−0.83)     −0.0041 (−1.19) 
Intercept
 
0.0014 (1.05)
 
0.0013 (1.03)
 
0.0012 (0.92)
 
0.0021 (1.42)
 
0.0021 (1.41)
 
0.0019 (1.29)
 
−0.0001 (−0.07)
 
−0.0001 (−0.09)
 
−0.0002 (−0.19)
 
Observations 10,126 10,126 10,126 10,126 10,126 10,126 10,126 10,126 10,126 
Adjusted $$R^{2}$$ 0.21 0.21 0.21 0.21 0.21 0.21 0.22 0.22 0.22 
Panel B: Database and dead funds only 
  4-Factor 4-Factor 4-Factor 7-Factor 7-Factor 7-Factor HF Bench HF Bench HF Bench 
Dead*Negative3 −0.0235*** (−4.92)     −0.0243*** (−4.93)     −0.0235*** (−4.86)     
Dead*Positive3 −0.0067** (−2.14)     −0.0062* (−1.91)     −0.0072** (−2.26)     
Dead*Negative6   −0.0236*** (−4.52)     −0.0245*** (−4.50)     −0.0234*** (−4.38)   
Dead*Positive6   −0.0075** (−2.43)     −0.0070** (−2.20)     −0.0080** (−2.58)   
Dead*Negative12     −0.0192*** (−4.66)     −0.0195*** (−4.55)     −0.0193*** (−4.61) 
Dead*Positive12     −0.0082** (−2.43)     −0.0078** (−2.26)     −0.0086** (−2.53) 
Intercept
 
0.0082*** (7.89)
 
0.0082*** (7.79)
 
0.0081*** (7.67)
 
0.0097*** (6.36)
 
0.0097*** (6.41)
 
0.0096*** (6.29)
 
0.0074*** (6.55)
 
0.0074*** (6.56)
 
0.0073*** (6.46)
 
Observations 6,026 6,026 6,026 6,026 6,026 6,026 6,026 6,026 6,026 
Adjusted $$R^{2}$$ 0.22 0.22 0.22 0.21 0.21 0.21 0.22 0.22 0.22 
Panel B: Database and dead funds only 
  4-Factor 4-Factor 4-Factor 7-Factor 7-Factor 7-Factor HF Bench HF Bench HF Bench 
Dead*Negative3 −0.0235*** (−4.92)     −0.0243*** (−4.93)     −0.0235*** (−4.86)     
Dead*Positive3 −0.0067** (−2.14)     −0.0062* (−1.91)     −0.0072** (−2.26)     
Dead*Negative6   −0.0236*** (−4.52)     −0.0245*** (−4.50)     −0.0234*** (−4.38)   
Dead*Positive6   −0.0075** (−2.43)     −0.0070** (−2.20)     −0.0080** (−2.58)   
Dead*Negative12     −0.0192*** (−4.66)     −0.0195*** (−4.55)     −0.0193*** (−4.61) 
Dead*Positive12     −0.0082** (−2.43)     −0.0078** (−2.26)     −0.0086** (−2.53) 
Intercept
 
0.0082*** (7.89)
 
0.0082*** (7.79)
 
0.0081*** (7.67)
 
0.0097*** (6.36)
 
0.0097*** (6.41)
 
0.0096*** (6.29)
 
0.0074*** (6.55)
 
0.0074*** (6.56)
 
0.0073*** (6.46)
 
Observations 6,026 6,026 6,026 6,026 6,026 6,026 6,026 6,026 6,026 
Adjusted $$R^{2}$$ 0.22 0.22 0.22 0.21 0.21 0.21 0.22 0.22 0.22 

Table 6 describes how the performance of dead funds after they stop reporting to a database is related to their performance prior to leaving the database. We use pooled OLS and the three different factor models: the Carhart (1997) 4-factor model, the Fung and Hsieh (2004) 7-factor model, and a modified version of the Jagannathan, Malakhov, and Novikov (2010) hedge fund–style benchmark model. Only returns calculated from the underlying holdings of Funds-of-Funds (FoF) collected from SEC filings are used. All models include a database indicator that equals one if a fund-quarter return from the hand-collected sample matches to a fund-quarter return in the union of hedge fund databases and a dead indicator that equals one if a fund's hand-collected return that quarter appears after the fund has last reported to the union of databases, that is, is no longer in a database. Database and Dead are the coefficients for each indicator, respectively. Dead*Negative3 is the interaction between the dead indicator and an indicator that equals one if a fund underperformed its Credit Suisse Tremont style index over the three-month period prior to leaving a database. The other interactions are defined similarly, but for positive performance and different periods. In Panel A, we use all hedge fund returns from the hand-collected sample, whereas in Panel B, we use only returns from funds that reported to a database at some point during their life. Returns are calculated from a sample where the cost field is not allowed to change from quarter to quarter, that is, no capital was added to or subtracted from the underlying fund by the FoF. Hedge fund returns are defined as the median return across all FoF advisors who report that underlying fund in that quarter, are net of the risk-free rate, and have been trimmed at the 0.5% and 99.5% levels. Standard errors are robust to heteroscedasticity and are clustered at the hedge fund level. We report $$t$$-statistics in parentheses. $$***p {\lt } 0.01$$, $$**p {\lt } 0.05$$, $$*p {\lt } 0.1$$.

Panel A uses the full sample, whereas Panel B restricts the sample to just those funds that report to a database at some point in their lives. Focusing on those funds that reported to a database (Panel B), we find that hedge funds with negative performance in the three months prior to delisting have an alpha that is between 235 bps and 243 bps lower than the alpha of funds that continue to report. The Dead*Positive interaction is also negative and significant in all models, but the coefficient is always less than half the size of the Dead*Negative interaction. Positive performers prior to delisting also perform worse, but their drop in performance is less severe.

Overall, our results from Table 6 provide evidence that conditioning on prior delisting performance can be a useful tool for inferring the future performance of delisted funds. However, we also find evidence that funds that delist following positive performance underperform database funds as well. This suggests that the delisting effect is detrimental across the population of dead funds.

2.3 Are the worst (and best) hedge funds missing from commercial databases?

The univariate results in Table 2 reveal that, though both are statistically significant, the difference in average performance between database and nondatabase funds (107 bps/quarter) is much larger than the difference in median performance (27 bps/quarter). This implies performance differences are nonconstant across the two returns distributions. Further, the fifth percentile of nondatabase returns is 511 bps lower per quarter, whereas the ninety-fifth percentile is actually 8 bps/quarter higher than database returns. There is a common assumption in the hedge fund literature that both the worst and the best funds are missing from the databases. This hypothesis would imply that the observable distribution of hedge fund performance is truncated on both the left and right side of the distribution. Though the kernel density estimates in Figure 1 suggest the database returns are truncated only on the left, we test this hypothesis more formally below.

Table 7 tests the truncation hypothesis in a univariate setting. We sort database and nondatabase hedge fund returns independently into deciles. This allows us to compare differences between the samples across the distribution of returns. If the selection bias truncates the observable returns distribution, we should find significant differences between the tails of the database and nondatabase distributions. For each decile, we test whether the means are different across the two distributions. Average database returns are significantly larger than average nondatabase returns for deciles 1 through 6. These differences are economically large. In the first decile, the average nondatabase return is 648 bps/quarter lower than the average database return. The performance differences shrink monotonically from deciles 1 to 10, yet remain meaningful (43 bps/quarter) in the fifth decile. In the right tails of the distributions (deciles 9 and 10), nondatabase returns are actually higher than database returns on average. For instance, in the tenth decile, nondatabase returns are 53 bps/quarter higher than database returns. In a univariate setting, these findings support the hypothesis that databases are missing some of the best- and worst-performing funds. Moreover, the significant disparity in performance in the bottom 30% of the two distributions suggests that the average selection bias is driven by the extreme poor performance found in the left tail of the nondatabase returns distribution. This suggests that the voluntary nature of hedge fund performance reporting not only shifts the distribution of observable returns but also truncates its left tail, hiding the worst hedge fund performance.

Table 7

Hedge fund returns by deciles

  Nondatabase Returns Database Returns Difference 
1 (Worst) −18.73% −12.25% −6.48%*** 
−6.41 −3.66 −2.75*** 
−2.30 −0.89 −1.41*** 
−0.20 0.52 −0.71*** 
1.11 1.54 −0.43*** 
2.34 2.50 −0.16*** 
3.52 3.59 −0.07*** 
5.00 5.04 −0.04 
7.32 7.23 0.09* 
10 (Best) 13.79 13.26 0.53* 
  Nondatabase Returns Database Returns Difference 
1 (Worst) −18.73% −12.25% −6.48%*** 
−6.41 −3.66 −2.75*** 
−2.30 −0.89 −1.41*** 
−0.20 0.52 −0.71*** 
1.11 1.54 −0.43*** 
2.34 2.50 −0.16*** 
3.52 3.59 −0.07*** 
5.00 5.04 −0.04 
7.32 7.23 0.09* 
10 (Best) 13.79 13.26 0.53* 

Table 7 reports the hand-collected quarterly hedge fund returns for each sample (nondatabase and database) sorted into deciles. We create deciles for each sample separately and then compare the means of each decile. Only returns calculated from the underlying holdings of Funds-of-Funds (FoF) collected from SEC filings are used. Database Returns describes all fund-quarter returns that are also found in the union of hedge fund databases. Hand-collected returns are calculated using the value and cost fields collected from hand-collected FoF filings. Returns are calculated from a sample where the cost field is not allowed to change from quarter to quarter, that is, no capital was added to or subtracted from the underlying fund by the FoF. The fund values used to calculate returns are net of all fees. Fund returns are defined as the median hedge fund return across all FoF advisors who report that underlying hedge fund in that quarter and have been trimmed at the 0.5% and 99.5% levels. For each decile, we perform a $$t$$-test for differences in means between the two samples. $$***p {\lt } 0.01$$, $$**p {\lt } 0.05$$, $$*p {\lt } 0.1$$.

We now examine the distribution of returns while controlling for risk using a series of factor pricing models. In Table 8, we use a simultaneous quantile regression framework (Koenker and Bassett 1978) with a focus on the tails of the returns distribution. The most common application of quantile regressions is a median regression whereby one minimizes the sum of the absolute deviations of the errors to estimate a median effect of the response variable. However, the same framework can be used to estimate the effect on any quantile in the distribution and proves particularly useful in estimating the impact of an explanatory variable, in our case the Database indicator, at different points of the returns distribution.

Table 8

Quantile regression results: Database returns versus nondatabase returns

Panel A: 7-Factor model Return Quantile 
  10th 25th 50th 75th 90th 
Database 0.0142*** (4.84) 0.0063*** (3.92) 0.0020** (1.98) 0.0018 (1.21) 0.0003 (0.11) 
Intercept
 
−0.0777*** (−24.86)
 
−0.0326*** (−19.26)
 
0.0044*** (3.62)
 
0.0402*** (26.31)
 
0.0803*** (31.51)
 
Pseudo $$R^{2}$$ 0.23 0.17 0.10 0.08 0.07 
Panel A: 7-Factor model Return Quantile 
  10th 25th 50th 75th 90th 
Database 0.0142*** (4.84) 0.0063*** (3.92) 0.0020** (1.98) 0.0018 (1.21) 0.0003 (0.11) 
Intercept
 
−0.0777*** (−24.86)
 
−0.0326*** (−19.26)
 
0.0044*** (3.62)
 
0.0402*** (26.31)
 
0.0803*** (31.51)
 
Pseudo $$R^{2}$$ 0.23 0.17 0.10 0.08 0.07 
Panel B: 4-Factor model Return Quantile 
  10th 25th 50th 75th 90th 
Database 0.0173*** (6.07) 0.0074*** (4.78) 0.0025** (2.53) 0.0022* (1.74) 0.0016 (0.65) 
Intercept
 
−0.0768*** (−27.83)
 
−0.0301*** (−22.74)
 
0.0049*** (6.29)
 
0.0364*** (33.46)
 
0.0732*** (48.97)
 
Pseudo $$R^{2}$$ 0.22 0.17 0.10 0.08 0.07 
Panel B: 4-Factor model Return Quantile 
  10th 25th 50th 75th 90th 
Database 0.0173*** (6.07) 0.0074*** (4.78) 0.0025** (2.53) 0.0022* (1.74) 0.0016 (0.65) 
Intercept
 
−0.0768*** (−27.83)
 
−0.0301*** (−22.74)
 
0.0049*** (6.29)
 
0.0364*** (33.46)
 
0.0732*** (48.97)
 
Pseudo $$R^{2}$$ 0.22 0.17 0.10 0.08 0.07 
Panel C: HF benchmark model Return Quantile 
  10th 25th 50th 75th 90th 
Database 0.0204*** (7.65) 0.0076*** (5.88) 0.0026*** (2.77) 0.0013 (0.98) 0.0004 (0.16) 
Intercept
 
−0.0775*** (−29.82)
 
−0.0316*** (−27.80)
 
0.0012 (1.39)
 
0.0320*** (31.40)
 
0.0731*** (38.22)
 
Pseudo $$R^{2}$$ 0.22 0.19 0.13 0.09 0.05 
Panel C: HF benchmark model Return Quantile 
  10th 25th 50th 75th 90th 
Database 0.0204*** (7.65) 0.0076*** (5.88) 0.0026*** (2.77) 0.0013 (0.98) 0.0004 (0.16) 
Intercept
 
−0.0775*** (−29.82)
 
−0.0316*** (−27.80)
 
0.0012 (1.39)
 
0.0320*** (31.40)
 
0.0731*** (38.22)
 
Pseudo $$R^{2}$$ 0.22 0.19 0.13 0.09 0.05 

Table 8 describes the results from a quantile regression of quarterly hedge fund returns on the database indicator and three different factor models: the Fung and Hsieh (2004) 7-factor model, the Carhart (1997) 4-factor model, and a modified version of the Jagannathan, Malakhov, and Novikov (2010) hedge fund–style benchmark model. Only returns calculated from the underlying holdings of Funds-of-Funds (FoF) collected from SEC filings are used. These models also include a database indicator that equals one if a fund-quarter return from the hand-collected sample matches to a fund-quarter return in the union of hedge fund databases. Database is the coefficient for that indicator. Therefore, in these models, the excess returns for funds that do not report is simply Intercept, whereas the excess returns for reporting funds can be interpreted as Intercept plus Database. We suppress the individual factor loadings. Returns are calculated from a sample where the cost field is not allowed to change from quarter to quarter, that is, no capital was added to or subtracted from the underlying fund by the FoF. The fund values used to calculate returns are net of all fees. Fund returns are defined as the median hedge fund return across all FoF advisors who report that underlying hedge fund in that quarter and are net of the risk-free rate. Returns have been trimmed at the 0.5% and 99.5% levels. Standard errors are bootstrapped (400 repetitions). We report $$t$$-statistics in parentheses. $$***p {\lt } 0.01$$, $$**p {\lt } 0.05$$, $$*p {\lt } 0.1$$.

We estimate five quantiles (10%, 25%, 50%, 75%, and 90%) using the 7-factor, 4-factor, and hedge fund benchmark models, in Panels A, B, and C, respectively, of Table 8. Standard errors are bootstrapped (400 repetitions). In each model, our variable of interest is the database indicator, which measures the difference in alphas between database and nondatabase funds.

If the selection bias is constant across the returns distribution, we would expect Database to remain constant and equal to that of our pooled OLS result from Table 3. Instead, we find substantial variation in the magnitude and statistical significance of Database across the returns distribution.

Similar to the univariate results of Table 7, the disparity in abnormal performance of database and nondatabase funds shrinks monotonically across the conditional returns distribution. For example, turning to the Fung and Hsieh 7-factor model (Panel A), database funds have a 142 bps/quarter ($$t$$-statistic = 4.84) higher alpha at the 10th percentile of returns but an insignificant 3 bps/quarter higher alpha ($$t$$-statistic = 0.11) at the 90th percentile of returns. We still find a significant and meaningful estimate of performance differences at the 50th percentile (20 bps, $$t$$-statistic = 1.98). However, when compared to Database at the median, the relative magnitudes of Database at the 10th and 25th percentiles imply the selection bias is driven by the worst-performing funds missing from commercial databases. This result is robust to each of the three factor models. In Table 9, we repeat the analysis by allowing for differences in risk as in Table 3, Panel D. Our conclusions are similar.

Table 9

Quantile regression results, including factor interactions

Panel A: 7-Factor model with interactions Return Quantile 
  10th 25th 50th 75th 90th 
Database 0.0249*** (5.52) 0.0152*** (6.17) 0.0080*** (3.99) 0.0059** (2.43) 0.0011 (0.26) 
Intercept
 
−0.0809*** (−20.70)
 
−0.0366*** (−16.89)
 
0.0012 (0.83)
 
0.0384** * (24.88)
 
0.0792*** (29.07)
 
Pseudo $$R^{2}$$ 0.23 0.17 0.11 0.08 0.07 
Panel A: 7-Factor model with interactions Return Quantile 
  10th 25th 50th 75th 90th 
Database 0.0249*** (5.52) 0.0152*** (6.17) 0.0080*** (3.99) 0.0059** (2.43) 0.0011 (0.26) 
Intercept
 
−0.0809*** (−20.70)
 
−0.0366*** (−16.89)
 
0.0012 (0.83)
 
0.0384** * (24.88)
 
0.0792*** (29.07)
 
Pseudo $$R^{2}$$ 0.23 0.17 0.11 0.08 0.07 
Panel B: 4-Factor model with interactions Return Quantile 
  10th 25th 50th 75th 90th 
Database 0.0207*** (6.03) 0.0099*** (5.54) 0.0032*** (2.85) 0.0017 (1.15) 0.0005 (0.22) 
Intercept
 
−0.0786*** (−28.62)
 
−0.0308*** (−21.79)
 
0.0046*** (5.20)
 
0.0366*** (36.20)
 
0.0737*** (50.77)
 
Pseudo $$R^{2}$$ 0.22 0.17 0.10 0.08 0.07 
Panel B: 4-Factor model with interactions Return Quantile 
  10th 25th 50th 75th 90th 
Database 0.0207*** (6.03) 0.0099*** (5.54) 0.0032*** (2.85) 0.0017 (1.15) 0.0005 (0.22) 
Intercept
 
−0.0786*** (−28.62)
 
−0.0308*** (−21.79)
 
0.0046*** (5.20)
 
0.0366*** (36.20)
 
0.0737*** (50.77)
 
Pseudo $$R^{2}$$ 0.22 0.17 0.10 0.08 0.07 
Panel C: HF benchmark model with interactions Return Quantile 
  10th 25th 50th 75th 90th 
Database 0.0238*** (6.54) 0.0087*** (4.74) 0.0030** (2.15) 0.0007 (0.31) 0.0027 (0.80) 
Intercept
 
−0.0802*** (−25.37)
 
−0.0321*** (−21.45)
 
0.0010 (1.06)
 
0.0323*** (26.24)
 
0.0722*** (35.90)
 
Pseudo $$R^{2}$$ 0.22 0.19 0.13 0.09 0.05 
Panel C: HF benchmark model with interactions Return Quantile 
  10th 25th 50th 75th 90th 
Database 0.0238*** (6.54) 0.0087*** (4.74) 0.0030** (2.15) 0.0007 (0.31) 0.0027 (0.80) 
Intercept
 
−0.0802*** (−25.37)
 
−0.0321*** (−21.45)
 
0.0010 (1.06)
 
0.0323*** (26.24)
 
0.0722*** (35.90)
 
Pseudo $$R^{2}$$ 0.22 0.19 0.13 0.09 0.05 

Table 9 describes the results from a quantile regression of quarterly hedge fund returns on the database indicator and three different factor models with the indicator interacted with the appropriate factor returns: the Fung and Hsieh (2004) 7-factor model, the Carhart (1997) 4-factor model, and a modified version of the Jagannathan, Malakhov, and Novikov (2010) hedge fund–style benchmark model. Only returns calculated from the underlying holdings of Funds-of-Funds (FoF) collected from SEC filings are used. These models also include a database indicator that equals one if a fund-quarter return from the hand-collected sample matches to a fund-quarter return in the union of hedge fund databases. Database is the coefficient for that indicator. Therefore, in these models, the excess returns for funds that do not report is simply Intercept, whereas the excess returns for reporting funds can be interpreted as Intercept plus Database. We suppress the individual factor loadings and interaction terms. Returns are calculated from a sample where the cost field is not allowed to change from quarter to quarter, that is, no capital was added to or subtracted from the underlying fund by the FoF. The fund values used to calculate returns are net of all fees. Fund returns are defined as the median hedge fund return across all FoF advisors who report that underlying hedge fund in that quarter and are net of the risk-free rate. Returns have been trimmed at the 0.5% and 99.5% levels. Standard errors are bootstrapped (400 repetitions). We report $$t$$-statistics in parentheses. $$***p {\lt } 0.01$$, $$**p {\lt } 0.05$$, $$*p {\lt } 0.1$$.

These findings are perhaps best seen graphically. Figure 2 provides the estimated Database coefficients for the 5th to 95th percentiles using the 7-factor model with interactions. For comparison, we provide the estimate from the OLS regression (Table 3, Panel D) and the 5th and 95th confidence intervals. When returns are low, Database has a large, positive value, implying database alphas are greater than nondatabase alphas. The value of Database falls as returns increase. The pattern is similar for the other factor models.

Figure 2

Estimates of database indicator by return quantile

Figure 2 reports different estimates of the database coefficient from a quantile regression using the Fung and Hsieh (2004) 7-factor model and interactions with the risk factors. The confidence interval (5% and 95%) for the coefficient is also shown. The coefficient estimate from the corresponding OLS regression is given for comparison purposes.

Figure 2

Estimates of database indicator by return quantile

Figure 2 reports different estimates of the database coefficient from a quantile regression using the Fung and Hsieh (2004) 7-factor model and interactions with the risk factors. The confidence interval (5% and 95%) for the coefficient is also shown. The coefficient estimate from the corresponding OLS regression is given for comparison purposes.

The extreme disparity between database and nondatabase performance in the left tail of the returns distribution suggests the commercial hedge fund databases are missing the truly worst performance of the universe of hedge funds. Further, in our sample, we find little evidence that nondatabase funds in the right tail of the distribution perform better than database funds. The economic magnitude of this difference is modest when compared to the difference in the left tail of the distribution, indicating that “good” nondatabase funds do little to cancel out the poor performance of “bad” nondatabase funds.21

3. Robustness

In the following section, we test whether our conclusions are driven by selection in the data and ensure that our results are robust to varying certain methodological assumptions. Additionally, we consider the impact of the financial crisis of 2008 on the selection bias. We have included all tables for this section in an appendix available online at The Review of Financial Studies Web site.

3.1 Sample selection and generalizability of results

The hedge funds in our sample are all held by registered FoFs. As such, a potential concern is that the returns for this group of funds may introduce a different form of selection bias. Fortunately, for the 1,445 underlying hedge funds in our data, we have returns for both funds in a database and funds not in a database. We therefore control for this selection bias by focusing only on the differences in performance between the database and nondatabase funds contained in our sample. That is, both the test and control groups are drawn from the same sample of funds selected by an FoF. In doing so, we assume that fund characteristics of the database and nondatabase funds are similar, which ensures that any selection bias will net out in our analyses. Also, for our results to be generalizable, we assume that hedge funds selected by an FoF in our sample perform similarly to the population of hedge funds. We address the validity of these claims below.

3.1.1 Comparing database funds and nondatabase funds.

In Table A1 in the appendix, we examine whether differences in nonperformance characteristics of database funds and nondatabase funds could be driving the aforementioned performance documented. Other than returns, the only information available in the quarterly holdings data is style, fund size, and liquidity (withdrawal frequency).22 In Panel A, we find that the distributions of styles are economically similar. For example, long/short equity is the most common style in both samples, whereas the short-bias style is one of the least common.

In Panel B, we address the issue of fund size. Because of diseconomies of scale, fund size can have a negative effect on returns (Chen et al. 2004; Berk and Green 2004). If smaller hedge funds outperform larger funds, differences in size between the database and nondatabase funds may bias our findings. However, we find that database funds are larger than nondatabase funds. The mean (median) fund size is $751.6 million ($335.9 million) for database funds and $584.7 million ($267.1 million) for nondatabase funds. Although these differences are both economically and statistically meaningful, they bias against our findings. We instead find that these smaller, nondatabase funds underperform their larger, database peers.

Finally, in Panel C, we address liquidity. Liquidity can impact the expected returns of hedge funds. For instance, Aragon (2007) attributes positive alpha to a liquidity premium earned by the fund's limited partners. If it were the case that database funds impose greater liquidity restrictions on their investors, then differences in liquidity may drive our results. Using withdrawal frequency to proxy for the liquidity of the underlying hedge funds, we find similar liquidity for both database and nondatabase funds. The median withdrawal frequency for both database and nondatabase funds is 90 days. Average liquidity is slightly lower for database funds, implying that database funds would actually have lower expected returns than nondatabase funds, the opposite of our findings.

3.1.2 Comparing sample database funds to the population of database funds.

Because FoF managers do not select hedge funds at random, it is likely the case that the selected hedge funds differ from the population across certain dimensions, such as size, age, or redemption frequency.23 However, we argue that the selection bias we document in our sample of hedge funds is generalizable to the population if the performance characteristics of database funds in our sample do not markedly differ from those in the hedge fund population.

We explore this issue in Table A2 by comparing many of the characteristics of our sample of database funds to the universe of database funds found in the union of the Lipper TASS, BarclayHedge, and HFR databases over our sample period 2004–2009. In Panel A, we compare the style distributions of funds selected by our sample FoFs to the universe of database funds. We find that they are very similar, with the exception of a preference for event-driven managers (19.9% of sample vs. 8.7% of the overall universe).

Panel B compares the two samples across many fund-level characteristics. We find evidence that FoFs hire hedge funds that are older and have significantly greater assets under management than the average fund. The average investment size made by an FoF in our sample is $10.2 million, indicating these FoFs are seeking out hedge funds that are large enough to be both willing and able to accept relatively large allocations. Investing with many small funds would cause an FoF to spend more resources performing due diligence and force the FoF to become a large percentage of each individual hedge fund's asset base. We also find evidence that FoFs hire hedge funds that have higher minimum investments, longer lock-up periods, longer redemption notice periods, and less frequent redemptions than the average hedge fund. This indicates FoFs may create value as an intermediary by offering their investors exposure to funds with relatively more restrictions.

Importantly, however, we find no material differences in performance between our sample of database hedge funds and the population of database hedge funds. We test for differences in performance between our sample and the population in three ways. First, we examine raw, monthly fund performance and find that the mean and median monthly returns of our sample of funds are virtually identical to the population (less than 1 bp/month difference). Second, to adjust for risk, we estimate fund-level alphas for every fund in the population (including those in our sample) using the monthly returns reported in the database over our sample period.24 Mean and median fund-level alphas are not statistically different between sample funds and population funds in any of the three different factor models. Finally, we form calendar-time portfolios over our sample period at the monthly level and estimate aggregate alphas for our sample of funds as compared to the population using a long-short portfolio approach (Panel C). We find no significant differences in either the 4-factor or 7-factor calendar-time alphas.

Collectively, the results indicate that the performance characteristics of our sample of database funds provide a similar representation of the performance characteristics of the population of funds reporting to a database over our sample period. Although our funds differ from the universe in many observable dimensions, such as age, size, and redemption flexibility, we do not find evidence that they differ across the most important dimension for our study: risk-adjusted (or unadjusted) performance. This is consistent with our argument that the difference in performance between our sample of database funds and nondatabase funds is an unbiased estimate of the difference between nondatabase funds and the population of database hedge funds. However, we point out that our sample is devoid of any hedge funds that are both missing from a database and not held by a registered FoF. The size and composition of this sample are unknown. For example, these unobserved funds could consist of talented managers who are closed to new investment or who do not want to appear in the filings made by registered FoFs. If true, these funds would close the gap between database and nondatabase returns.25 Alternatively, it is possible that these funds have poor returns. If registered FoFs avoid or divest from struggling managers, we will never observe the complete return history of these poorly performing funds. This would cause us to underestimate the gap between database and nondatabase funds. It is unclear how these missing funds may affect the inferences from our tests.

3.2 Sensitivity of methodological assumptions

3.2.1 Changing cost basis.

Using Equation (1), we generate our return series by comparing changes in the value of a hedge fund from two consecutive SEC filings. FoFs can of course add or subtract capital to their hedge fund investments over time. To capture the true return for the hedge fund holding, one must account for this change in cost basis. Unfortunately, FoFs in our sample appear to account for capital withdrawals differently. Whereas most funds appear to account for changes in cost basis in dollars, some appear to present percent changes in cost basis. For example, if an FoF invested $1 million in a hedge fund that subsequently doubled in value, its cost basis would be $1 million, whereas the value of its holdings would be $2 million. If the FoF then decided to withdraw half of the value of its investment ($1 million), it could account for its reduced cost basis using a total dollar value approach resulting in a new cost basis of $0. Alternatively, the fund could apply a proportion of initial capital approach whereby the fund reduces its cost basis in proportion to the total value of the fund. This would result in a new cost basis of $0.50 million. An inability to accurately capture the reporting regime of an FoF will create measurement error in our return variable. Although this is unlikely to affect returns when FoFs add capital to a fund, we previously omitted these observations to exclude any bias this asymmetry may cause. This omission excluded 1,448 quarterly hedge fund returns (12.5% of the sample). Our final sample consists of 11,574 unique fund-quarter returns when we relax the cost change constraint. As before, 49% of these fund-quarters have a corresponding fund-quarter in one or more of the five commercial hedge fund databases, whereas 51% do not, indicating that the propensity for cost basis to change is of equal probability for both database and nondatabase funds. In Table A3, we report summary statistics for this expanded sample. The difference in performance between the database and nondatabase samples is similar to the one we document in Table 2.

In Table A4, we repeat the regression analysis of Table 3, by using a series of common factor models and the addition of our database indicator variable, while allowing for additions in capital. Our conclusions are unchanged. As before, Database remains positive and significant in each of the pricing models. Ultimately, the conclusions for our tests are largely unaffected by our cost change restriction.

3.2.2 Long/short equity funds only.

We use a pooled OLS approach throughout the paper because of the short time series and quarterly frequency of our data. The pooled nature of our analysis forces the average exposures of the factor model to be homogeneous across different fund styles. To assess how this constraint affects the inferences from our tests, we restrict our analysis to a subsample of the most frequently observed style, Long/Short Equity managers. These managers are potentially following similar strategies with similar risk characteristics, which means that our results will be less influenced by model error. Additionally, these funds follow relatively more liquid, equity-based strategies that should be less affected by return smoothing, as described in Getmansky, Lo, and Makarov (2004). This restriction leaves us with only 4,815 fund-quarter observations. In Table A5, Database is still positive and statistically significant, ranging from 44 bps/quarter to 98 bps/quarter.

3.2.3 Return outliers.

As stated previously, our returns are generated from hand-collected data. To mitigate any data errors and reduce the impact of outliers, we trimmed our return observations at the 0.5% and 99.5% levels. In Table A6, we recreate Table 3 and include these trimmed returns. Our results are similar, as Database remains positive and significant for all factor model regressions.

3.2.4 Multiple hedge fund observations.

It is possible for multiple FoFs to hold the same hedge fund in a given quarter. Based on differences of when the FoF invested in the hedge fund, it is possible for each of these FoFs to have slightly different highwater marks, fees, or tax status that could lead to slight discrepancies in returns for the same hedge fund. Previously, we took the median quarterly return for any hedge fund held by multiple FoFs in an effort to reduce any outliers. For robustness, in Table A7, we instead take the mean return and again replicate Table 3. Our results are similar. In Table A8, we also examine the subset of hedge funds that are held by two or more FoFs. The potential advantage of this subset is that the funds may be more homogeneous in terms of size because a hedge fund held by multiple FoFs is likely to be larger. The disadvantage here is that we are reducing the power of our tests to identify performance differences because it is likely that nondatabase funds held by multiple FoFs at the same time represent the best-performing nondatabase funds. Despite this issue, Database remains positive and significant.

3.2.5 Backfill bias.

Another important database bias that has been documented in the hedge fund literature is the backfill bias. This bias is caused by successful managers choosing to enter a database and include their past returns. Accordingly, some of the hand-collected returns that we are matching to a database could be backfilled returns, which could potentially cause our positive results. We address this issue by marking a return as backfilled if it appeared during the first two years that a fund reported to a database. We then treat these backfilled returns as missing from the database (i.e., nondatabase returns). In Table A9, we recreate Table 3 using this backfill-adjusted data and find similar results.

3.2.6 Calendar-time regressions.

The pooled regression approach exhausts the time variation of fund returns and characteristics in the full sample of hedge funds, yet it imposes restrictions on individual fund loadings on the factor indices. Throughout the paper, we have included a database interaction term with each of the factors to allow factor sensitivities to vary between groups, yet the pooled approach restricts factor sensitivities within groups. As a robustness check, we form portfolios of funds and utilize a calendar-time approach to test for differences in database and nondatabase funds in Table A10. We separately form equal-weighted portfolios of database funds and nondatabase funds. To test for the difference in alphas between the groups, we form a long-short portfolio that is long database funds and short nondatabase funds. For the 7-factor model, the alpha for the database portfolio is 123 bps/quarter, while the alpha for the nondatabase portfolio is −10 bps/quarter. The long-short portfolio has an alpha of 133 bps/quarter ($$t$$-statistic = 4.29).26

3.3 Impact of the financial crisis

Our results in Section 2.3 suggest that the selection bias is driven by the large left tail of the nondatabase returns distribution. Because of the financial crisis, aggregate hedge fund performance was extremely low in the last two quarters of 2008.27 In this section, we examine whether the selection bias we document was driven by this period of extreme poor performance. In Table A11, we estimate factor regressions as in Table 3 but include an indicator variable for the crisis period as well as an interaction between our database indicator and the crisis indicator to test for differences in the bias during this crisis period. As in Table 3, we estimate our factor models both with and without a database interaction with the risk factors. In each model, the coefficient on the database variable remains positive and significant, albeit with lower magnitudes than those found in Table 3. Thus, the selection bias is not entirely driven by the financial crisis. Not surprisingly, the crisis indicator variable is negative and significant in each model, consistent with the poor aggregate performance of hedge funds in that time period. Interestingly, the interaction of the crisis and database indicator variables is positive and significant, with coefficients ranging from 247 to 377 bps/quarter. This indicates that though the selection bias was significant throughout the sample period, its magnitude was larger during the crisis period. These results are similar if we change our definition of the crisis period to include the entire years of 2008 and 2009.

One possible explanation for this finding is that hedge fund managers with less skill (i.e., nondatabase managers) were less able to mitigate performance shocks during the crisis. An alternative explanation could be that poor aggregate performance changed the overall composition of the nondatabase hedge fund sample. For instance, poor performance could have induced more hedge funds to delist from databases, resulting in the nondatabase sample containing more recently failed funds. Additionally, because of concerns regarding liquidity and potential “fire-sales,” many hedge funds enacted gates and other liquidity restrictions during the crisis that limited the ability of their investors to exit their positions. This meant that FoFs were forced to hold positions in poorly performing funds they likely would have divested from their portfolios otherwise. Because an underlying hedge fund needs to be held by an FoF in consecutive quarters for us to observe its performance, we naturally cannot observe the last returns of a nondatabase hedge fund that is divested by an FoF. This implies that in normal times without liquidity restrictions, a nondatabase hedge fund that delivers extreme poor performance within a quarter would likely be divested by the FoF and never show as a return in our sample. However, liquidity restrictions initiated in response to the crisis means fewer bad funds could be divested and would thus remain present in our sample.

4. Conclusion

This paper uses a new data set of hedge fund returns from SEC filings to test for a selection bias in commercial hedge fund databases. Comparing reported and nonreported returns allows us to estimate the direction and magnitude of the selection bias. We find that the performance of database funds is significantly greater than the performance of nondatabase funds. We find that this difference in performance is largely driven by the thick left tail of the nondatabase returns distribution, suggesting that commercial databases are missing the worst returns of hedge funds. We conclude that, on average, there is a positive selection bias in voluntarily reported hedge fund performance data, suggesting that managerial skill estimates from commercial databases likely overstate the true managerial skill of the hedge fund universe. Some of the worst performance is attributable to funds that at one time reported to a database but subsequently decided to delist. These “dead” funds tend to have poor performance prior to delisting and have dramatically lower performance after they delist. However, the poor returns of delisted hedge funds do not solely create the selection bias that we document.

Our results have important implications for hedge fund investors, the hedge fund industry, and financial market regulators. Investors using observed historical performance and risk metrics may mistakenly view hedge funds as a disproportionally attractive investment vehicle, potentially leading to an inefficient allocation of capital. Additionally, investors may gauge the relative performance of hedge fund managers against upwardly biased indices of hedge fund performance generated by commercial databases. This could create unfavorable performance comparisons and potentially exacerbate the flight-to-quality problem during crises, as investors withdraw their money after observing what they infer to be inferior relative fund performance. Moreover, funds that trail their benchmarks may have an incentive to increase portfolio risk to the potential detriment of fund investors (e.g., see Brown, Harlow, and Starks 1996; Chevalier and Ellison 1999). Finally, because available data may not accurately capture the size, performance, or risk of the hedge fund industry, regulators may be unable to properly gauge the potential systemic risks that hedge funds pose for the financial system.

One potential interpretation of our results is that investors should simply avoid investing in nondatabase hedge funds and treat the fund's decision to report to a database as a certification mechanism. This is certainly plausible given that managers who start reporting may have strong incentives to continue reporting to satisfy an implicit commitment of continued disclosure to investors. However, we caution that simply selecting funds that report to a database is not a panacea for hedge fund investors. The problem remains that the reporting decision by hedge fund managers is at all times voluntary and returns can be reported with a considerable lag. Thus, fund managers have the powerful advantage of hindsight combined with discretion, which allows them to strategically delay or stop reporting when performance is poor. Because hedge funds are more likely to restrict investor liquidity when they underperform (Ang and Bollen 2011), hedge fund investors may be unable to completely avoid the poor returns of nondatabase funds.

For their useful comments, we thank an anonymous referee, Laura Starks (the Editor), Anup Agrawal, George Aragon, Jeffrey Coles, Michael Hertzel, Brad Jordan, Laura Lindsey, Shawn Thomas, Shane Underwood, several hedge fund managers who wish to remain anonymous, and seminar participants at Arizona State University, the University of Cincinnati, the University of Kentucky, the Securities and Exchange Commission, the FIRS Finance Conference (2010) in Florence, Italy; the FMA Conference (2010) in New York; the FMA Europe Conference (2012) in Istanbul, Turkey; and Quinnipiac University. We thank Monika Rabarison and Xin Hong for their research assistance. Supplementary materials for this article are available on The Review of Financial Studies Web site.

References

Ackermann
C.
McEnally
R.
Ravenscraft
D.
The performance of hedge funds: Risk, return, and incentives
Journal of Finance
 , 
1999
, vol. 
54
 (pg. 
833
-
74
)
Agarwal
V.
Daniel
N. D.
Naik
N. Y.
Do hedge funds manage their reported returns?
Review of Financial Studies
 , 
2011
, vol. 
24
 (pg. 
3281
-
320
)
Agarwal
V.
Fos
V.
Jiang
W.
Inferring reporting-related biases in hedge fund databases from hedge fund equity holdings
Management Science
  
Forthcoming
Agarwal
V.
Jiang
W.
Tang
Y.
Yang
B.
Uncovering hedge fund skill from the portfolio holdings they hide
Journal of Finance
  
Forthcoming
Aggarwal
R. K.
Jorion
P.
The performance of emerging hedge funds and managers
Journal of Financial Economics
 , 
2010
, vol. 
96
 (pg. 
238
-
56
)
Ang
A.
Rhodes-Kropf
M.
Zhao
R.
Do funds-of-funds deserve their fees-on-fees?
Journal of Investment Management
 , 
2008
, vol. 
6
 (pg. 
34
-
58
)
Ang
A.
Bollen
N. P. B.
When hedge funds block the exits
2011
 
SSRN Working Paper
Aragon
G. O.
Share restrictions and asset pricing: Evidence from the hedge fund industry
Journal of Financial Economics
 , 
2007
, vol. 
83
 (pg. 
33
-
58
)
Aragon
G. O.
Nanda
V.
Strategic delays and clustering in hedge fund reported returns
2011
 
SSRN Working Paper.
Aragon
G. O.
Martin
J. Spencer
A unique view of hedge fund derivatives usage: Safeguard or speculation?
Journal of Financial Economics
 , 
2012
, vol. 
105
 (pg. 
436
-
56
)
Berk
J. B.
Green
R. C.
Mutual fund flows and performance in rational markets
Journal of Political Economy
 , 
2004
, vol. 
112
 (pg. 
1269
-
95
)
Bollen
N. P. B.
Pool
V. K.
Do hedge fund managers misreport returns? Evidence from the pooled distribution
Journal of Finance
 , 
2009
, vol. 
64
 (pg. 
2257
-
88
)
Brown
K. C.
Harlow
W. V.
Starks
L. T.
Of tournaments and temptations: An analysis of managerial incentives in the mutual fund industry
Journal of Finance
 , 
1996
, vol. 
51
 (pg. 
85
-
110
)
Brown
S. J.
Goetzmann
W. N.
Liang
B.
Fees on fees in funds of funds
Journal of Investment Management
 , 
2004
, vol. 
2
 (pg. 
39
-
56
)
Brown
S. J.
Gregoriou
G. N.
Pascalau
R. C.
Diversification in funds of hedge funds: Is it possible to overdiversify?
Review of Asset Pricing Studies
 , 
2012
, vol. 
2
 (pg. 
89
-
110
)
Carhart
M. M.
On persistence in mutual fund performance
Journal of Finance
 , 
1997
, vol. 
52
 (pg. 
57
-
82
)
Chen
J.
Hong
H.
Huang
M.
Kubik
J.
Does fund size erode mutual fund performance? The role of liquidity and organization
American Economic Review
 , 
2004
, vol. 
94
 (pg. 
1276
-
302
)
Chevalier
J.
Ellison
G.
Career concerns of mutual fund managers
Quarterly Journal of Economics
 , 
1999
, vol. 
114
 (pg. 
389
-
432
)
Cici
G.
Kempf
A.
Puetz
A.
The valuation of hedge funds' equity positions
2011
 
SSRN Working Paper
Fung
W.
Hsieh
D. A.
Empirical characteristics of dynamic trading strategies: The case of hedge funds
Review of Financial Studies
 , 
1997
, vol. 
10
 (pg. 
275
-
302
)
Fung
W.
Hsieh
D. A.
Performance characteristics of hedge funds and commodity funds: Natural vs. spurious biases
Journal of Financial and Quantitative Analysis
 , 
2000
, vol. 
35
 (pg. 
291
-
307
)
Fung
W.
Hsieh
D. A.
Hedge fund benchmarks: A risk-based approach
Financial Analysts Journal
 , 
2004
, vol. 
60
 (pg. 
65
-
80
)
Fung
W.
Hsieh
D. A.
Measurement biases in hedge fund performance data: An update
Financial Analysts Journal
 , 
2009
, vol. 
65
 (pg. 
36
-
38
)
Fung
W.
Hsieh
D. A.
Naik
N. Y.
Ramadorai
T.
Hedge funds: Performance, risk, and capital formation
Journal of Finance
 , 
2008
, vol. 
63
 (pg. 
1777
-
803
)
Getmansky
M.
Lo
A. W.
Makarov
I.
An econometric model of serial correlation and illiquidity in hedge fund returns
Journal of Financial Economics
 , 
2004
, vol. 
74
 (pg. 
529
-
609
)
Griffin
J. M.
Xu
J.
How smart are the smart guys? A unique view from hedge fund stock holdings
Review of Financial Studies
 , 
2009
, vol. 
22
 (pg. 
2331
-
70
)
Hodder
J. E.
Jackwerth
J. C.
Kolokolova
O.
Recovering delisting returns of hedge funds
2009
 
SSRN Working Paper
Ibbotson
R. G.
Chen
P.
Zhu
K. X.
The ABCs of hedge funds: Alphas, betas, & costs
Financial Analysts Journal
 , 
2011
, vol. 
67
 (pg. 
15
-
25
)
Jagannathan
R.
Malakhov
A.
Novikov
D.
Do hot hands exist among hedge fund managers? An empirical evaluation
Journal of Finance
 , 
2010
, vol. 
65
 (pg. 
217
-
55
)
Koenker
R. W.
Bassett
G.
Regression quantiles
Econometrica
 , 
1978
, vol. 
46
 (pg. 
33
-
50
)
Kosowski
R.
Naik
N. Y.
Teo
M.
Do hedge funds deliver alpha? A Bayesian and bootstrap analysis
Journal of Financial Economics
 , 
2007
, vol. 
84
 (pg. 
229
-
64
)
Malkiel
B. G.
Saha
A.
Hedge funds: Risk and return
Financial Analysts Journal
 , 
2005
, vol. 
61
 (pg. 
80
-
88
)
Patton
A. J.
Ramadorai
T.
Streatfield
M.
The reliability of voluntary disclosures: Evidence from hedge funds
2012
 
SSRN Working Paper
Stulz
R. M.
Hedge funds: Past, present, and future
Journal of Economic Perspectives
 , 
2007
, vol. 
21
 (pg. 
175
-
94
)
2 13F filings are quarterly snapshots of long equity-based positions disclosed at the advisor, rather than the fund, level. The filings are mandatory for institutional investors managing over $100 million in “equity-like” securities referred to by the SEC as 13F securities. Besides common equity, these include securities, such as options and convertible bonds.
3 In Section 3, we address whether our approach suffers from a different type of selection bias.
4 In the hedge fund literature, delisting bias is distinct from survivorship bias. Survivorship bias refers to a bias that may occur from studying only those hedge funds that currently report to databases, that is, “live” funds. This issue can be mitigated by including the previously reported performance of dead funds. On the other hand, delisting bias refers to the notion that delisted funds may have abnormal nonreported performance following delisting. By definition, these data are not available in commercial databases.
5 For the subsample of our funds that report to a database, our calculated return measure has a 0.98 correlation with the corresponding returns reported to the commercial databases. In comparison, Agarwal, Fos, and Jiang (forthcoming) report a correlation of 0.54 between their set of implicit returns and the corresponding database returns, similar to the correlation found in Griffin and Xu (2009).
6 Q76 on the NSAR lists the closing price for closed-end funds. Registered funds do not have a closing price, as they are not listed on any exchange. As such, they report “0.00” to this question. We further confirm that our sample is a registered fund of funds using additional regulatory filings (e.g., N-2).
7 Appendix available online at The Review of Financial Studies Web site.
8 In less than 5% of our filings, the FoF did not disclose the cost basis for the position. Because a return series cannot be generated without the cost basis, we remove these observations from our data.
9 Conversations with managers at registered funds revealed this discrepancy. Mutual fund holdings largely avoid this issue, as they list the number of shares they hold in a registered security, not their cost basis.
10 Morningstar has acquired both the CISDM and MSCI hedge fund databases and incorporated their funds into the Morningstar global hedge fund database. Thus, we have matched our sample of funds across a grand total of seven of the most widely used hedge fund databases in the academic literature. We were able to access the full monthly return series for Lipper TASS, BarclayHedge, and HFR databases. Morningstar and EurekaHedge were kind enough to provide us the names and dates for funds in their databases. This was sufficient for identification purposes. We arrive at this sample by hand-matching the fund names found in our SEC filings to the names found in each of the five commercial databases. This process is inherently imperfect and adds an element of noise to our results. However, any errors from name matching are likely random and unlikely to bias our results, as the difficulty of text matching should be unrelated to performance.
11 We note that these proportions should not be interpreted as 51% of the hedge fund universe is missing from a database. Hedge fund databases are comprised of many small funds. Because FoFs typically select from larger hedge funds, this skews our proportion of database funds downward. Finally, note that these are fund-quarter observations. At the fund level, roughly 53% of our funds report a live database return at some point during our sample period. About 60% of the funds in our sample reported to a database at some point in their history.
12 The Fung and Hsieh (2004) model includes the following factors: a market factor (the S&P 500 total return), a size spread factor (the return of the Wilshire Small Cap 1750 less the return of the Wilshire Large Cap 750), a bond market factor (the quarterly change in the ten-year constant maturity treasury yield), a credit spread factor (the quarterly change in the Moody's Baa yield less the ten-year treasury constant maturity yield), and three trend-following factors for the bond market, the currency market, and the commodities market. We thank David Hsieh for providing the risk factors on his Web site (http://faculty.fuqua.duke.edu/%7Edah7/HFRFData.htm).
13 The Jagannathan, Malakhov, and Novikov (2010) model includes the fund's style index return and a market factor. We use the return on the S&P 500 and the return on the Credit Suisse/Tremont Hedge Fund Index associated with the fund (each less the risk-free rate) as our factors for this modified model.
14 Hedge funds may strategically distort their reported returns (as suggested in Getmansky, Lo, and Makarov 2004 and Bollen and Pool 2009), leading to return smoothing that may underestimate betas and overestimate alphas.
15 In unreported analysis, we perform a similar test on the entire pooled sample of database funds from the union of Lipper TASS, HFR, and BarclayHedge and find similar alpha estimates.
16 This pooled OLS method with additional fund characteristics is similar to the methodology used in Aragon (2007).
17 This test allows for a direct test of the difference in alphas between the database subsample and the full sample as reported in Panels A and B above. The interpretation of Intercept and Database remain the same as in Panel C.
18 For example, Lipper TASS allows managers to delay reporting performance for six months before they send them a warning of their plans to consider the fund to be delisted and officially transferred to their “Graveyard” database. The hedge fund then has one month to respond (Lipper TASS Database FAQ).
19 However, some studies have attempted to infer the returns of these funds from other data sources. As mentioned before, Agarwal, Fos, and Jiang (forthcoming) estimate the returns of dead funds from equity positions. Additionally, Hodder, Jackwerth, and Kolokolova (2009) attempt to infer the delisting returns of individual hedge funds using the aggregate performance reported by FoFs.
20 Given the nature of our data, it is difficult to prescribe a specific delisting return from our study. Because our returns come from changes in quarterly fund values with cost remaining constant, we are unable to accurately assess the performance of dead funds in periods in which FoFs have withdrawn capital, which is likely to happen when performance is poor. Further, we cannot identify the performance of the last period an FoF holds a fund, as it will not be present in its end-of-quarter holdings statement. Thus, we likely miss the final liquidation returns of failed funds. For this reason, we believe the poor returns of dead funds that we document represent an upper bound on the actual performance of these funds. Also, as we indicate below, the actual delisting return will likely be conditional on several factors, such as performance prior to delisting.
21 However, our sample is missing any hedge funds that are both not held by a registered FoF and absent from a database. If it were the case that registered FoFs did not have access to the best-performing nondatabase hedge funds, our conclusions may be different.
22 Fund size and liquidity data are available for 21% and 68% of our sample, respectively.
23 See Brown, Goetzmann, and Liang (2004), Ang, Rhodes-Kropf, and Zhao (2008), and Brown, Gregoriou, and Pascalau (2012), among others, for a discussion of the constraints and incentives faced by FoF managers.
24 We require a fund to have a minimum of twenty-four months of returns to estimate alphas with greater precision. However, our results are not affected by loosening this restriction to twelve months.
25 Anecdotal evidence regarding the funds selected by registered FoFs indicates that they are not precluded from investing in large, well-known funds. Our sample includes AQR, Citadel, DE Shaw, and SAC, among others.
26 Because calendar-time portfolios aggregate funds across all styles, we are unable to construct portfolio alphas that control for style returns analogous to the HF benchmark model.
27 The Dow Jones Credit Suisse Hedge Fund index return was −19.0% in 2008, the only year of negative returns for hedge funds in our sample. The poor performance in 2008 was driven by the −19.5% return in the last half of 2008.

Supplementary data