Bias properties of logarithmic and asinh magnitudes at very low flux levels

Abstract

The observation of flux sources near the limit of detection requires a careful evaluation of possible biases in magnitude determination. Both the traditional logarithmic magnitudes and the recently proposed inverse hyperbolic sine (asinh) magnitudes are considered. Formulae are derived for three different biasing mechanisms: the statistical spread of the observed flux values arising from e.g. measurement error; the dependence of these errors on the true flux; and the dependence of the observing probability on the true flux. As an example of the results, it is noted that biases at large signal-to-noise ratios R, at which the two types of magnitude are similar, are of the order of — (p+1)/R², where the exponent p parametrizes a power-law dependence of the probability of observation on the true flux.

methods: data analysis, methods: statistical

1 Introduction

This paper is concerned with unifying, and extending, the work in four other papers. The incentive for the present research was the study of some of the properties of a new magnitude scale, asinh (or ‘inverse hyperbolic sine’) magnitudes, proposed by Lupton, Gunn & Szalay (1999), and used in the Sloan Digital Sky Survey (see e.g. Fan et al. 2000a,b). There have been a few recent studies dealing with two different biases in ordinary logarithmic magnitudes (Clarke 1996; Koen & Menzies 1996; Hogg & Turner 1998), and it was thought a useful enterprise to apply these ideas also to the asinh magnitude scale.

The situation considered below is as follows: a single brightness measurement of a star with a true mean flux S₀ is obtained, the measurements being subject to scatter. There may be several sources of scatter, both intrinsic (e.g. photon statistics) and extrinsic (e.g. measurement errors). The measured flux S is transformed into some other quantity x, i.e. (e.g. logarithmic or asinh magnitude). There are two conditional probability density functions (PDFs) which may then be of interest: first, the distribution of x, given S₀ [or equivalently to S₀, It is of interest to consider the difference between the most likely observed value ◯ of x (i.e. the mode of the conditional PDF) and the ‘true’ value The second PDF examined below is p(x₀∼x), i.e. the probability distribution of the true parameter value, given the observed quantity x. Once again the mode of the distribution (the most likely value of the true flux) is of prime interest.

The two conditional PDFs are related by Bayes's theorem:

 

(1)

where p(S₀) and p(x) are the (marginal) PDFs of the true flux and the observed quantity x, respectively. The distribution p(S₀) is of course determined by the intrinsic distribution of source brightnesses, and their distance (and extinction) distributions, as well as by the properties of the observing equipment. The simplifying assumption that S₀ is uniformly distributed is made in 2 and 3, and more general cases dealt with in Section 4.

Three effects are considered below.

• (1)

  Measurements of physical quantities are always contaminated by noise. Often the measured quantity none the less provides an unbiased estimate of the true value. However, Clarke (1996) and Koen & Menzies (1996) have shown that the presence of scatter in the observed flux can lead to biased estimates of ordinary logarithmic magnitudes. The quoted papers concentrated on the measurement of relatively high flux levels; in Section 2 we deal with very low flux levels. Similar biases in asinh magnitudes are also studied.

• (2)

  Another important source of bias, induced by the dependence of the size of the measurement errors on the source brightness, is considered in Section 3.

• (3)

  Hogg & Turner (1998) showed that the combination of scatter in the flux and source counts which increase with distance leads to a bias in magnitude determinations which is analogous to the Lutz–Kelker bias in parallax measurements. This is dealt with in Section 4.

The paper is concluded in Section 5.

2 Noise-induced bias

It is assumed throughout that the measured flux S has a Gaussian distribution about its true mean value S₀:

 

(2)

The assumption (2) is, of course, made in order to illustrate the general procedures, although it is hoped that it may apply generally. The offset of the observed magnitude m from its zero-point m∗ is

 

(3)

and its distribution corresponding to (2) is

 

(4)

where S∗ is the flux level corresponding to the magnitude zero-point, and

 

The quantity ψ∗ has the dimensions of flux, and hence its introduction allows an additional check on the correctness of the equations below.

Lupton et al. (1999) define the asinh magnitude corresponding to flux S as

 

(5)

where b is a constant, which they choose to be

 

(6)

The relation (5) can be inverted to give an equation for S, namely

 

(7)

from which the distribution function corresponding to (2) may be derived:

 

(8)

Clarke (1996) pointed out that a spread in the values of the measured flux would give rise to a bias in the corresponding magnitude. This is easily seen from (4): for a single determination, the most likely measured value of m is given by the mode m̂ of (4), which is the solution of

 

(9)

The answer is

 

(10)

where is the true signal-to-noise ratio and is the true magnitude. The observer is thus more likely to measure m̂ than the true magnitude m₀: the last term in (10) constitutes a noise-induced bias. Note that the bias approaches zero for large signal-to-noise ratios, but may be important near the limit of detection of sources.

The mode of (8) is given by the solution of

 

(11)

From (5) and (7),

 

(12)

It is shown in the Appendix that

 

(13)

for moderately large signal-to-noise ratios. If then

 

The exact solution of (11) and (12), and the approximation (13) are both shown in Fig. 1 for b as in (6). The approximation is evidently very good for signal-to-noise ratios in excess of about 4, and also for or so.

Figure 1.

The bias in the measured asinh magnitude owing to a Gaussian distribution in the measured flux. The points correspond to the exact solution of (11) and (12), while the line shows the approximation (13).

For a uniform source brightness distribution and [see equation (1)], and it follows from (4) that

 

or

 

and from (8) that

 

Of more practical interest are conditional PDFs such as

 

(14)

and

 

(15)

The mode of (14) is at

 

(16)

where is the observed signal-to-noise ratio. The mode of (15) is given by the solution of

 

(17)

where and Noting that the observed signal-to-noise ratio is and proceeding as in the derivation of (13),

 

where, for large R,

 

(18)

Ultimately, the similarity between the expressions for m̂ and m̂₀ (and between those for μ^{^} and μ^{^}₀) can be traced to the symmetry between S and S₀ in (2). The situation below differs from this, and PDFs conditioned on observed values are emphasized, as these are the cases of greater practical importance.

3 Flux-dependent errors

In order to provide a simple demonstration of the existence of another type of bias, consider the idealized situation where there are two types of sources only, with respective true mean fluxes S₁ and S₂ The measurement errors associated with the two types of sources are σ₁ and In order to demonstrate clearly the origin of the bias, assume also that there are equal numbers of source types 1 and 2. The set-up is illustrated schematically in Fig. 2.

Figure 2.

A schematic illustration of the bias arising from different noise levels. There are equal numbers of sources with true fluxes of 3 and 5.5 respectively, and with measurement error standard deviations of 0.5 and 1.4. The likelihood of classifying a source as having flux 3 is enhanced.

A little reflection makes it clear that the larger measurement errors are more likely to scatter type 2 sources into a small interval around than vice versa. The consequence is a bias towards measuring source brightnesses as being S₁.

Consider now the more general situation where the observed flux S has a Gaussian distribution around the true flux S₀, and where the scatter σ depends on the true flux, i.e.

 

(19)

and The most likely observed value of S is

 

(20)

while m̂ and μ^{^} are again given by (10), and by the solution of (11).

The functional dependence of σ on S₀ has the result that equation (19) is no longer symmetrical in S and S₀. By Bayes's theorem (1),

 

(21)

for uniform measurement probability of all sources [i.e. p(S₀) constant]. The most likely value of S₀, given an observation S, is then the solution of

 

(22)

The solution of (22) is, of course, dependent on the functional dependence of σ on S₀. The power-law dependence

 

(23)

with K a constant, is a simple and useful example for consideration. For this law, (22) reduces to

 

(24)

where and the observed signal-to-noise ratio is

 

For large R, y will approach unity, i.e. with Substitution into (24) then gives

 

(25)

Gilliland & Brown (1988) identified three ‘error’ regimes when measuring brightnesses from CCD exposures of sparsely populated fields. In terms of magnitude errors

 

these are (scintillation noise), (photon noise) and (sky background noise at very low signal levels). It is tempting to deduce the corresponding behaviour of the flux errors from the approximation

 

[see equation (3)]. However, at the very low flux levels considered here, with σ(S) comparable to or even larger than S, this is incorrect. Common sense tells us that σ(S) will increase rapidly with decreasing S, owing to the increasingly important role of sky background errors and contamination of measurements by the brightness wings of stars that are nearby in the exposures. This is borne out by a study of the actual errors that can be accurately estimated from time-series photometry. For purposes of illustration, we will therefore use in what follows, but the reader should be aware that in practice in crowded fields.

For i.e. and there is no bias. Solutions of (24) for and −1 are shown in Fig. 3.

Figure 3.

The flux ratio for the power-law relation (23) with (open circles, bottom curve) and (filled circles, top curve). The circles show the exact numerical solutions of (24), and the solid lines the approximation (25).

The difference between (20) and (22) can be appreciated by referring to the respective conditional distributions: in the case of (20) a single value of S₀, and hence of σ, is involved (the distribution is conditioned on the specific value of S₀). In the case of (22), a range of values of S₀, each with an associated σ(S₀), may give rise to the same observed S, and (22) specifies only the most likely value of S₀.

The conditional PDFs p(m₀∼m) and p(μ₀∼μ) again have the forms (14) and (15), but their modes are shifted: in the case of (14), the most likely value of m₀ is given by the solution of

 

(26)

where and σ_m is the derivative of σ with respect to m₀. For the power law (23)

 

(27)

and hence (26) becomes

 

(28)

For (16) is again obtained. For it can be shown that

 

(29)

The exact numerical solution of (28) and the approximation (29) are plotted against the signal-to-noise ratio in Fig. 4.

Figure 4.

The (logarithmic) magnitude bias for the power-law relation (22) with (open circles, top curve) and (filled circles, bottom curve). The circles show the exact numerical solutions of (28), and the solid lines the approximation (29).

We now turn to the asinh magnitudes. Equating the derivative of (15) to zero gives

 

(30)

where and Noting that

 

(30) can be written as

 

(31)

where and

For the power law (23), and

 

(32)

where has the form of a signal-to-noise ratio, and the convenient choice has been made. Proceeding as in Appendix A,

 

for large R∗. It follows that

 

(33)

For the result (17) of Section 2 is recovered; for and −1 solutions of (32), and the corresponding approximations (33), are plotted in Fig. 5.

Figure 5.

The asinh magnitude bias for the power-law relation (23) with (open circles, top curve) and (filled circles, bottom curve). The circles show the exact numerical solutions of (32), and the solid lines the approximation (33).

4 Incorporation of observing probabilities

Up to this point in the paper it has been assumed that p(S₀) in (1) is a constant. In reality this function will have two components: p₁(S₀) which describes the solid-angular density of sources with flux S₀, and p₂(S₀) which gives the instrumental measurement probability.

We first consider p₁(S₀). Generally speaking there are more faint stars than bright, so that p₁(S₀) will increase with decreasing S₀. The implications of the power law

 

(34)

have been explored by Hogg & Turner (1998) (note that their definition of q is slightly different from that used here). They found that

 

(35)

For logarithmic magnitudes

 

by (1), with p(m∼m₀) given by (4) (with S₀ replaced by The marginal PDF of m₀ corresponding to (34) is

 

and thus

 

It follows that

 

(36)

the approximation being valid for large signal-to-noise ratios. This result is different from that quoted by Hogg & Turner (1998, their equation 7).

The case of the asinh magnitudes can be treated similarly:

 

(37)

The mode of (37) is at the solution of

 

(38)

with as before. For large R, following the procedure in Appendix A gives

 

with It is noteworthy that this does not reduce to (A2) for the reason is that the order of (38) is one higher than that of (17). The generalization of (18) is then

 

(39)

which further reduces to

 

(40)

as Comparison of (40) and (36) shows that the biases in the logarithmic and asinh magnitudes are the same for large signal-to-noise ratios; this is hardly surprising, as the asinh magnitude scale was designed so that for large R. Fig. 6 contains plots of the bias for The approximation (39) is very poor for for and is not shown.

Figure 6.

The asinh magnitude bias for different source space densities, as described by the power law (34). The values of the exponent q are 0 (dashed line), 1 (solid line), 2 (open dots) and 3 (filled dots).

An explicit expression for the bias at zero signal-to-noise ratio is easily derived: the solution of (38) is

 

and

 

The second component of p(S₀), namely p₂(S₀), is determined by the instrumental response: at intermediate magnitudes, the probability that brightnesses of stars will be measured successfully will be high, whereas very bright and very faint stars will have lower probabilities (respectively because of saturation and under-exposure). At low flux levels the effects arising from p₁(S₀) and p₂(S₀) will compete, and at high flux levels they will reinforce each other. Of course, the specific properties of p₂(S) will also depend on observing conditions.

The reader may feel that the material in the preceding sections of the paper is somewhat academic, as the situation does not arise in practice. However, the situation is akin to that of Lutz–Kelker bias: the context determines whether the probability density of the sources is relevant or not (e.g. Koen & Laney 1998). Thus, for example, when dealing with individual objects, the form of p(S₀) is not relevant. On the other hand, when the interest is in objects selected on the basis of brightness, the solid-angular densities of the various true fluxes are important.

5 Closing remarks

General expressions for biases that incorporate all three of the effects discussed in 2–4 are easily written down. Bayes's theorem states that

 

where x and x₀ are the observed and true values of the quantity of interest (flux, logarithmic magnitude or asinh magnitude). Subject to (2),

 

(41)

where it is understood that S₀ is replaced by in expressions for σ, p(S₀) and the Jacobian and S is replaced by The most likely value ◯₀ of x₀ is then given by the solution of

 

(42)

where σ_x is the derivative of σ with respect to x₀.

We end by noting an interesting interpretation of the asinh magnitude scale associated with equation (16) for m̂₀: the latter can also be written as

 

Comparison with the definition (5) of the asinh magnitude scale shows that, for the choice

 

the asinh magnitude corresponding to an observed flux is equal to the most likely logarithmic magnitude. The choice (6) for b differs very little from so that the asinh magnitudes as defined by Lupton et al. (1999) are effectively equal to m̂₀ in the case of constant σ, and flux-independent measurement probability. In essence, therefore, asinh magnitudes are equivalent to bias-corrected ordinary logarithmic magnitudes, at least for cases where the effects discussed in 3 and 4 can be neglected.

Acknowledgments

The author thanks Dr John Caldwell for suggesting the study of the asinh magnitude scale, and Dr John Menzies for commenting on version 1 of the paper.

Appendix

Appendix A: The bias in μ for large signal-to-noise ratios

Clearly Δ in (12) will approach zero as R increases, i.e. z can be written as

 

where as Using a first-order Taylor expansion,

 

where the derivative is to be evaluated in the point Since

 

it follows that

 

and thus

 

(A1)

Substitution of in (11) gives

 

to first order in δ. Consequently,

 

(A2)

and so

 

(A3)