Von Neumann Normalisation of a Quantum Random Number Generator Alastair A. Abbott∗ Cristian S. Calude† Department of Computer Science University of Auckland Private Bag 92019, Auckland, New Zealand 1 1 www.cs.auckland.ac.nz/~{aabb009,cristian} 0 2 January 26, 2011 n a J 5 2 Abstract ] InthispaperwestudyvonNeumannun-biasingnormalisationforidealandrealquan- T tum random number generators, operating on finite strings or infinite bit sequences. In I . theidealcasesonecanobtainthedesiredun-biasing. Thisreliescriticallyontheindepen- s dence of the source, a notion we rigorously define for our model. In real cases, affected by c [ imperfectionsinmeasurementandhardware,onecannotachieveatrueun-biasing,but,if the bias “drifts sufficiently slowly”, the result can be arbitrarily close to un-biasing. For 1 v infinite sequences, normalisation can both increase or decrease the (algorithmic) random- 1 ness of the generated sequences. 1 A successful application of von Neumann normalisation—in fact, any un-biasing 7 transformation—does exactly what it promises, un-biasing, one (among infinitely many) 4 symptoms of randomness; it will not produce “true” randomness. . 1 0 1 1 Introduction 1 : v The outcome of some individual quantum-mechanical events cannot in principle be predicted, i X so they are thought as ideal sources of random numbers. An incomplete list of quantum phe- r nomena used for random number generation include nuclear decay radiation sources [26], the a quantum mechanical noise in electronic circuits known as shot noise [27] or photons travelling through a semi-transparent mirror [21, 25, 29, 30, 32]. Our methods are primarily developed to address these latter photon-based quantum random number generators (QRNGs), one of the most direct and popular ways to generate QRNs, but many of our mathematical results will be applicable to other QRNGs. Duetoimperfectionsinmeasurementandhardware,theflowofbitsgeneratedbyaQRNG contains bias and correlation, two symptoms of non-randomness [8].1 The first and simplest ∗AA was in part supported by the CDMTCS. †CC was in part supported by the CDMTCS and UoA R&SL grant. 1As discussed in [1], “true randomness” does not mathematically exist. Various forms of algorithmic ran- domness[13]areeachdefinedbyaninfinityofconditions,some“statistical”(likebias),some“non-statistical” (like lack of computable correlations). technique for reducing bias was invented by von Neumann [34]. It considers pairs of bits, and takes one of three actions: a) pairs of equal bits are discarded; b) the pair 01 becomes 0; c) the pair 10 becomes 1. Contrary to wide spread claims, the technique works for some sources of bits, but not for all. The output produced by an independent source of constantly biased bits is transformed (after reducing the number of bits produced significantly) into a flow of bits in which the frequencies of 0’s and 1’s are equal: 50% for each. As we shall show, a stronger property is true: the un-biasing works not only for bits but for all reasonable long bit-strings. However, if the bias is not constant the procedure does not work. Finally, von Neumann procedure cannot assure “true randomness” in its output. To understand the behaviour of QRNGs we need to study the un-biasing transformations on both (finite) strings and (infinite) sequences of bits produced by the source. In this paper we will focus on von Neumann normalisation2 because it is very simple, easy to implement, and (along with the more efficient iterated version due to Peres [24] for which the results will also apply) is widely used by current proposals for QRNGs [21, 22, 16, 29]. The main results of this paper are the following. In the “ideal case”, the von Neumann normalised output of an independent constantly biased QRNG is the probability space of the uniform distribution (un-biasing). This result is true for both for finite strings and for the infinite sequences produced by QRNGs (the QRNG runs indefinitely in the latter case). It is important to note that independence in the mathematical sense of multiplicity of probabilities is a model intended to correspond to the physical notion of independence of outcomes [18]. In order to study the theoretical behaviour of QRNGs, which are based on the assumption of physical independence of measurements, we must translate this appropriately into our formal model. We carefully define independence of QRNGs to achieve this aim. As explained above, QRNGs do not operate in ideal conditions. We develop a model for a real-world QRNG in which the bias, rather than holding steady, drifts slowly (within some bounds). In this framework we evaluate the speed of drift required to be maintained by the source distribution to guarantee that the output distribution is as close as one wishes to the uniform distribution. We have also examined the effect von Neumann normalisation has on various properties of infinite sequences. In particular, Borel normality and (algorithmic) randomness are invariant under normalisation, but for ε-random sequences with 0 < ε < 1, normalisation can both decrease or increase the randomness of the source. 2 Notation We present the main notation used throughout the paper. By 2X we denote the power set of X. By |X| we denote the cardinality of the set of X. Let B = {0,1} and denote by B∗ the set of all bit-strings (λ is the empty string). If x ∈ B∗ and i ∈ B then |x| is the length of x and # (x) represents the number of i’s in x. By i Bn we denote the finite set {x ∈ B∗ | n = |x|}. The concatenation product of two subsets X,Y of B∗ is defined by XY = {xy | x ∈ X,y ∈ Y}. If X = {x} then we write xY instead of {x}Y. By Bω we denote the set of all infinite binary sequences. For x ∈ Bω and natural n we denote by x(n) the prefix of x of length n. We write w (cid:60) v or w (cid:60) x in case w is a prefix of the string v or the sequence x. 2Many improvements of the scheme have been proposed [14, 24]. 2 A prefix-free (Turing) machine is a Turing machine whose domain is a prefix-free set of strings [8]. The prefix complexity of a string, H (σ), induced by a prefix-free machine W is W H (σ) = min{|p| : W(p) = σ}. Fix a computable ε with 0 < ε ≤ 1. An ε–universal prefix- W free machine U is a machine such that for every machine W there is a constant c (depending on U and W) such that ε·H (σ) ≤ H (σ)+c, for all σ ∈ B∗. If ε = 1 then U is simply U W called a universal prefix-free machine. A sequence x ∈ Bω is called ε–random if there exists a constant c such that H (x(n)) ≥ ε·n−c, for all n ≥ 1. Sequences that are 1–random are U simply called random. A sequence x is called Borel m–normal (m ≥ 1) if for every 1 ≤ i ≤ 2m one has: lim Nm(x(n))/(cid:98)n(cid:99) = 2−m; here Nm(y) counts the number of non-overlapping occur- n→∞ i m i rences of the ith (in lexicographical order) binary string of length m in the string y. The sequence x is called Borel normal if it is Borel m–normal, for every natural m ≥ 1. A probability space is a measure space such that the measure of the whole space is equal to one [5]. More precisely, a (Kolmogorov) probability space is a triple consisting of a sample space Ω, a σ–algebra F on Ω, and a probability measure P, i.e. a countably additive function defined on F with values in [0,1] such that P(Ω) = 1. 3 The finite case 3.1 Source probability space and independence In this section we define the QRNG source probability space and the independence property. Consider a string of n independent bits produced by a (biased) QRNG. Let p ,p be the 0 1 probability that a bit is 0 or 1, respectively, with p +p = 1, p ,p ≤ 1. 0 1 0 1 The probability space of bit-strings produced by the QRNG is (Bn,2Bn,P ) where P : n n 2Bn → [0,1] is defined by P (X) = (cid:88)p#0(x)p#1(x), (1) n 0 1 x∈X for all X ⊆ Bn. It is easy to verify that the Kolmogorov axioms are satisfied for the space (Bn,2Bn,P ), n so we have: Fact 1. The space (Bn,2Bn,P ) with P defined in (1) is a probability space. n n The space (Bn,2Bn,P ) is just the n-fold product of the single bit probability space n (B,2B,P ). For this reason this space is often called an “independent identically-distributed 1 bit source”. The resulting space is “independent” because each bit is independent of previous ones. But what is “an independent probability space”? Physically the independence of a QRNG is usually expressed as the impossibility of ex- tracting any information from the flow of bits x ,...,x to improve chances of predict- 1 k−1 ing the value of x , other than what one would have from knowing the probability space. k The fact that photon-based QRNGs obey this physical independence between photons (and thus generated bits) rather well [2, 29] is the primary motivation for our modelling of these devices. These sources (where the condition of independence still holds) are often termed 3 “independent-bit sources” [33]. In a real device we cannot, of course, expect each bit to be identically distributed, so we study this more general case more thoroughly in Section 3.5. Formally, two events A,B ⊆ Bn are independent (in a probability space) if the probability oftheirintersectioncoincideswiththeproductoftheirprobabilities[7](acomplexity-theoretic approach was developed in [12]). This motivates the definition of independence of a general sourceprobabilityspacegiveninDefinition3. Butfirstweneedthefollowingsimpleproperty: Fact 2. For every bit-string x and non-negative integers n,k such that 0 ≤ k +|x| ≤ n we have: (cid:16) (cid:17) P BkxBn−k−|x| = p#0(x)p#1(x) = P ({x}). (2) n 0 1 |x| Definition 3. The probability space (Bn,2Bn,Prob ) is independent if for all 1 ≤ k ≤ n and n all x ...x ∈ Bk the events x x ...x Bn−k+1 and Bk−1x Bn−k are independent, i.e. 1 k 1 2 k−1 k (cid:16) (cid:17) (cid:16) (cid:17) (cid:16) (cid:17) Prob x x ...x x Bn−k = Prob x x ...x Bn−k+1 ·Prob Bk−1x Bn−k . n 1 2 k−1 k n 1 2 k−1 n k Fact 4. The probability space (Bn,2Bn,P ) with P defined in (1) is independent. n n Proof. Using (2) we have: (cid:16) (cid:17) P x x ...x x Bn−k = p#0(x1...xk)p#1(x1...xk) n 1 2 k−1 k 0 1 = p#0(x1...xk−1)p#1(x1...xk−1)p#0(xk)pxk) 0 1 0 1 (cid:16) (cid:17) (cid:16) (cid:17) = P x x ...x Bn−k+1 ·P Bk−1x Bn−k . n 1 2 k−1 n k As we will see later, there are other relevant independent probability spaces. 3.2 Von Neumann normalisation function Here we present formally the von Neumann normalisation procedure. We define the mapping F : B2 → B∪{λ} as (cid:40) λ if x = x , 1 2 F(x x ) = 1 2 x if x (cid:54)= x , 1 1 2 and f : B → B2 as f(x) = xx¯, where x¯ = 1−x. Note that for all x ∈ B we have F(f(x)) = x and, for all x ,x ∈ B with 1 2 x (cid:54)= x , f(F(x x )) = x x . 1 2 1 2 1 2 (cid:16) (cid:17) For m ≤ (cid:98)n/2(cid:99) we define the normalisation function VN : Bn → (cid:83) Bk ∪{λ} as n,m k≤m (cid:16) (cid:17) VN (x ...x ) = F(x x )F(x x )···F x x . n,m 1 n 1 2 3 4 (2(cid:98)m(cid:99)−1) 2(cid:98)m(cid:99) 2 2 Fact 5. For all 1 < m ≤ (cid:98)n/2(cid:99) and y ∈ Bm there exists an x ∈ Bn such that y = VN (x). n,m 4 Proof. Take x = f(y )f(y )···f(y )0n−2m. 1 2 m In fact we can define the “inverse” normalisation VN−1 : 2Bm → 2Bn as n,m (cid:40) VN−1 (Y) = u f(y )u f(y )···u f(y )u v | y = y ...y ∈ Y, n,m 1 1 2 2 m m m+1 1 m m+1 (cid:41) (cid:88) u ∈ {00,11}∗,v ∈ B∪{λ},|v|+2m+ |u | = n . i i i=1 While this isn’t a “true” inverse, for every y ∈ Bm we have: VN (cid:0)VN−1 (y)(cid:1) = {y}. n,n n,m 3.3 Target probability space and normalisation We now construct the target probability space of the normalised bit-strings over Bm for m ≤ (cid:98)n/2(cid:99), i.e. the probability space of the output bit-strings produced by the application of the von Neumann function on the output bit-strings generated by the QRNG. The von Neumann normalisation function VN transforms the source probability space n,m (Bm,2Bm,P ) into the target probability space (Bm,2Bm,P ). The target space of nor- n n→m malised bit-strings of length 1 < m ≤ (cid:98)n/2(cid:99) associated to the source probability space (Bm,2Bm,P ) is the space (Bm,2Bm,P ), where P : 2Bm → [0,1] is defined for all n n→m n→m Y ⊆ Bm by the formula: P (cid:0)VN−1 (Y)(cid:1) P (Y) = n n,m . n→m P (cid:0)VN−1 (Bm)(cid:1) n n,m Proposition 6. The target space (Bm,2Bm,P ) of normalised bit-strings of length 1 < n→m m ≤ (cid:98)n/2(cid:99) associated to the source probability space (Bm,2Bm,P ) is a probability space. n Proof. We need to check only additivity: For X,Y ⊆ Bm, X ∩Y = ∅ =⇒ P (X ∪Y) = n→m P (X)+P (Y). This equality is valid since VN−1 (X∪Y) = VN−1 (X)∪VN−1 (Y) n→m n→m n,m n,m n,m and P (cid:0)VN−1 (Y)∪VN−1 (X)(cid:1) = P (cid:0)VN−1 (Y)(cid:1) + P (cid:0)VN−1 (X)(cid:1), as VN−1 (X) ∩ n n,m n,m n n,m n n,m n,m VN−1 (Y) = ∅ because X and Y are disjoint. n,m 3.4 Normalisation of the output of a source with constant bias We now show that von Neumann procedure transforms the source probability space with constant bias into the probability space with the uniform distribution over Bm, i.e. the target probability space (Bm,2Bm,P ) has P = U , the uniform distribution. Independence n→m n→m m and the constant bias of P play a crucial role. n Theorem 7 (von Neumann). Assume that 1 < m ≤ (cid:98)n/2(cid:99). In the target probability space (Bm,2Bm,P ) associated to the source probability space (Bm,2Bm,P ) we have n→m n P (Y) = U (Y) = |Y|·2−m, for every Y ⊆ Bm. n→m m Proof. Since P is additive it suffices to show that for any y ∈ Bm, P ({y}) = 2−m. n→m n→m Let Z = P (cid:0)VN−1 (Bm)(cid:1). n n,m 5 We have (the sums are over all u ∈ {00,11}∗, v ∈ B ∪{λ} such that |v|+(cid:80)m+1|u | = i i=1 i n−2m): P ({y}) = 1 (cid:88)p#0(u1f(y1)...umf(ym)um+1v)p#1(u1f(y1)...umf(ym)um+1v) n→m Z 0 1 ui,v = p#00(f(y1)...f(ym))p#11(f(y1)...f(ym)) (cid:88)p#0(u1...um+1v)p#1(u1...um+1v) Z 0 1 ui,v = pm0 pm1 (cid:88)p#0(u1...um+1v)p#1(u1...um+1v), Z 0 1 ui,v which is independent of y. Since P (Bm) = 1 and for all x ,x ∈ Bm we have n→m 1 2 P ({x }) = P ({x }) it follows that P ({y}) = 2−m = U ({y}); by additivity, n→m 1 n→m 2 n→m m for every Y ⊆ 2m we have P (Y) = U (Y) = |Y|·2−m. n→m m Itisnaturaltocheckwhethertheindependenceandconstantbiasofthesourceprobability space are essential for the validity of the von Neumann normalisation procedure. Example 8. Thesourceprobabilityspace(B2,2B2,Prob )whereProb (00) = 0,Prob (01) = 2 2 2 Prob (10) = Prob (11) = 1/3 is independent and Prob = U . 2 2 2→1 1 Example 9. The source probability space (B2,2B2,Prob ) where Prob (00) = Prob (11) = 2 2 2 0,Prob (01) = 1/3,Prob (10) = 2/3 is independent but Prob (cid:54)= U . 2 2 2→1 1 Comment. One could present the above examples in the more general framework of Theo- rem 7. Theorem 10. Let m ≥ 1 and n = 2m. Consider the source probability space (Bn,2Bn,Prob ) = Πm (B2,2B2,Pi), where Pi(01) = Pi(10), for all 1 ≤ i ≤ m. Then, in the n i=1 2 2 2 target probability space (Bm,2Bm,Prob ), where Prob = Πm Pi, we have Prob = n→m n i=1 2 n→m U . m Proof. It is easy to check that for every y = y ...y ∈ Bm we have Prob ({y ...y }) = 1 m n→m 1 m (cid:81)m Pi(y y¯)/Prob (VN−1 (Bm)), so Prob ({y ...y }) does not depend on y (because i=1 2 i i n n,m n→m 1 m Pi(aa¯) = Pi(a¯a), for every a ∈ B). Hence, Prob = U . 2 2 n→m m The source probability space (Bm,2Bm,Prob ) in Theorem 10 is not constantly biased n and may be independent or not, but von Neumann normalisation still produces the uniform distribution under these conditions. Example11. Thesourceprobabilityspace(B4,2B4,Prob )asinTheorem10whereP1(00) = 4 2 P1(01) = 1/3,P1(10) = 1/4,P1(11) = 1/12 and P2(00) = 1/12,P2(01) = 1/4,P2(10) = 2 2 2 2 2 2 P2(11) = 1/3 is not independent and Prob = U . 2 4→2 2 6 The outcome of successive context preparations and measurements, such as is the case for thetypeofQRNGusuallyenvisioned,arepostulatedtobeindependentofpreviousandfuture outcomes [17]. This means there must be no causal link between one measurement and the next within the system (preparation and measurement devices included) so that the system has no memory of previous or future events. For QRNGs this translates into the condition that the probability that each successive bit is either 0 or 1 is independent of the previous bit measured. We will only consider such independent probability spaces, as this is a necessary property of a good RNG, so most QRNGs are designed to conform to this requirement. The above assumption needs to be made clear as in high bit-rate experimental configura- tions to generate QRNs with, e.g., photons, its validity may not always be clear. If the wave- functions of successive photons “overlap” the assumption no longer holds and (anti)bunching phenomena may play a role. This is an issue that needs to be more seriously considered in QRNG design and will only become more relevant as the bit-rate of QRNGs is pushed higher and higher. While we leave study of the nature of these temporal correlations (and any non- independence they may cause) to future research [2], we pose the following open question which may help to quantify any possible effect they may have. Open Question. Fix an integer k ≥ 0 and small positive real κ. Consider the probability space (Bn,2Bn,P†) where P† is a modification of the probability P satisfying the conditions n n n that for all i ≤ n and x ∈ B we have P (Bi−1x Bn−i) = P†(Bi−1x Bn−i), i n i n i (cid:12) (cid:12) (cid:12)P†(Bi−1x Bn−i)−P†(Bi−1x Bn−i | Bi−k−1x ...x Bn−i−1)(cid:12) ≤ κ, (cid:12) n i n i i−k i−1 (cid:12) and for all l > k P†(Bi−1x Bn−i | Bi−l−1x ...x Bn−i−1) n i i−l i−1 = P†(Bi−1x Bn−i | Bi−k−1x ...x Bn−i−1). n i i−k i−1 In other words, the probability of each bit depends on no more than the previous k bits, and the difference in probabilities for a bit between that given by P† conditioned on the previous n k bits and P is no more than κ. If the output of such a source is normalised with the von n Neumann procedure, how close is the resulting probability space of strings of length m to the uniform distribution (see Definition 17 for a definition of the closeness of probability spaces)? 3.5 Normalisation of the output of a source with non-constant bias NowweconsidertheprobabilitydistributionobtainedifvonNeumannnormalisationisapplied toastringgeneratedfromanindependentsourcewithanon-constantbias—an“independent- bitsource”. Weconsideronlyabiaswhichvariessmoothly; thisexcludestheeffectsofsudden noise which could make the bias jump significantly from one bit to the next.Such a source corresponds to a QRNG in which the bias varies slowly (drifts) from bit to bit over time, but never too far from its average point. We choose this to model photon-based QRNGs since the primary cause of variation in the bias will be of this nature. For example, the detector efficiencies may vary as a result of slow changes in temperature or power supply. While abrupt changes—which this model does not account for—are plausible, their relatively rare occurrence (in comparison with the bit generation rate in the order of MHz) will mean they have little effect on the resultant distribution. 7 Let p ,p < 1 and p +p = 1 be constant. Let x = x x ...x ∈ Bn be the generated 0 1 0 1 1 2 n string. Then define the probability of an individual bit x being either zero or one as i (cid:40) p −ε if x = 0, qxi = 0 i i (3) i p +ε if x = 1. 1 i i The variation in the bias is bounded, so we require that for all i, |ε | ≤ β, with β < min(p ,p ). i 0 1 Let γ = ε −ε . Furthermore, we assume that the “speed” of variation be bounded, i.e. i i+1 i there exists a positive δ such that |γ | ≤ δ, (4) i for all i. Evidently we have δ ≤ β (presumably in any real situation δ (cid:28) β); however, we introducetwoseparateconstantssincetheycorrespondtotwophysicallydifferent(butrelated) concepts. Note that we will discuss in more detail the importance of these two parameters for the approximation of the uniform distribution and their relevance to calibration of the QRNG lateroncetheanalysisiscompleted. Indeed,therateofchange,γ ,ismoreimportant;theneed i forβ stemsfromtheneedtorealisethat, eventhoughtheprobabilitiescanfluctuate, theycan (cid:80) only fluctuate in one direction for so long (since q ∈ [0,1]), hence | γ | = |ε −ε | ≤ 2β. i i i n 1 For a string y = y y ...y ∈ Bk and positive integer i we introduce, for convenience, the 1 k k following notation: q (y) = qy1qy2 ···qyk . i i i+1 i+k−1 The difference in probability between 01 and 10 depends only on γ , and this allows us to i evaluate the effect of normalisation on such a string: q (01)−q (10) = (p −ε )(p +ε )−(p +ε )(p −ε ) i i 0 i 1 i+1 1 i 0 i+1 = (p +p )(ε −ε ) 0 1 i+1 i = γ . (5) i Let us first formally define the probability space generated by this QRNG. Proposition 12. The probability space of bit-strings produced by the QRNG is (Bn,2Bn,R ) n where R : 2Bn → [0,1] is defined for all X ⊆ Bn as follows: n (cid:88) R (X) = q (x). (6) n 1 x∈X Proof. WeverifyonlythatR (Bn) = 1,whichiseasilyshownsinceq0+q1 = 1,andR (Bn) = n i i n (q0+q1)···(q0 +q1). 1 1 n n Fact 13. For all i ≥ 1 and x,y ∈ {0,1}∗ we have: q (xy) = q (x)q (y). i i i+|x| Fact 14. For all k,n ≥ 1, x ∈ {0,1}∗ with 0 ≤ k+|x| ≤ n we have: (cid:16) (cid:17) R Bn−kxBn−k−|x| = q (x). (7) n n−k+1 8 Proof. Using Fact 13 we get: (cid:16) (cid:17) (cid:88) (cid:88) R Bn−kxBn−k−|x| = q (yxz) n 1 y∈Bn−kz∈Bn−k−|x| (cid:88) (cid:88) = q (y)q (x)q (z) 1 |y|+1 |y|+|x|+1 y∈Bn−kz∈Bn−k−|x| (cid:88) (cid:88) = q (x) q (y)q (z) n−k+1 1 |y|+|x|+1 y∈Bn−kz∈Bn−k−|x|   (cid:88) (cid:88) = qn−k+1(x) q1(y) q|y|+|x|+1(z) y∈Bn−k z∈Bn−k−|x| = q (x). n−k+1 Fact 15. The probability space (Bn,2Bn,R ) with R defined in (6) is independent. n n Proof. Using (7) we have: (cid:16) (cid:17) R x x ...x x Bn−k = q (x x ...x x ) n 1 2 k−1 k 1 1 2 k−1 k = q (x x ...x )q (x ) 1 1 2 k−1 k k (cid:16) (cid:17) (cid:16) (cid:17) = R x x ...x Bn−k+1 ·R Bk−1x Bn−k . n 1 2 k−1 n k As with the constantly biased source, we consider the probability space R . We first n→m investigatethesimplestcasen = 2m. Inthissituation,foranyy ∈ Bm wehaveVN−1 ({y}) = n,m {f(y )f(y )···f(y )} and VN−1 (Bm) = {f(z )f(z )···f(z ) | z = z ...z ∈ Bm}. 1 2 m n,m 1 2 m 1 m Fact16. Theprobabilityspaceofnormalisedbit-stringsoflengthm = n/2is(Bn,2Bn,R ) n→m where R : 2Bn → [0,1] is defined for all Y ⊆ Bm as follows: n→m R (Y) = Rn(VNn−,m1 (Y)) = (cid:88)(cid:89)m q2i−1(f(yi)) . (8) n→m Rn(VNn−,m1 (Bm)) y∈Y i=1 q2i−1(01)+q2i−1(10) 3.6 Approximating of the uniform distribution Unlike the case for a constantly biased source, we no longer have q (01) = q (10); in fact i i by (5) we have q (01) = q (10) + γ . As a result the normalised equation is no longer the i i i uniform distribution, but only an approximation thereof. We now explore how closely R n→m approximates U . m We first need to define what we mean by approximating U . m Definition 17. The total variation distance between two probability measures P and Q over the space Ω is ∆(P,Q) = max |P(A)−Q(A)|. We say that P and Q are ρ-close if A⊆Ω ∆(P,Q) ≤ ρ. 9 It is well known (see for example [33]) that Lemma 18. For finite Ω we have ∆(P,Q) = 1 (cid:80) |P({x})−Q({x})|. 2 x∈Ω The variation ∆(R ,U ) depends on each γ and q (thus on p , p and each ε ), but n→m m i i 0 1 i wewishtocalculatetheworstcaseintermsoftheboundsδ,β andp ,p , i.e.usingLemma18, 0 1 1 (cid:88) max∆(R ,U ) = max |R ({y})−2−m|. n→m m n→m γi,qi 2 γi,qi y∈Bm Let us first note that we can write q (f(y )) q (f(y )) 2i−1 i 2i−1 i = q2i−1(01)+q2i−1(10) 2q2i−1(f(yi))−(−1)yiγ2i−1 = 1 (cid:18)1+ (−1)yiγ2i−1 (cid:19), 2 2q2i−1(f(yi))−(−1)yiγ2i−1 and hence we have R ({y}) = 2−m(cid:89)m (cid:18)1+ (−1)yiγ2i−1 (cid:19). n→m q (01)+q (10) 2i−1 2i−1 i=1 Wehaverewrittenthedenominatorinitsoriginalformtoemphasisethatonlythesigns(−1)yi depend on y. Thus, we want to find the values of q and γ which maximise 2i−1 2i−1 (cid:12) (cid:12) (cid:88) (cid:12)(cid:12)1−(cid:89)m (cid:18)1+ (−1)yiγ2i−1 (cid:19)(cid:12)(cid:12), (9) (cid:12) q (01)+q (10) (cid:12) (cid:12) 2i−1 2i−1 (cid:12) y∈Bm i=1 subject to the constraints that |γ | ≤ δ and |ε | ≤ β for 1 ≤ (cid:96) ≤ n. (cid:96) (cid:96) Lemma 19. The function (cid:12) (cid:12) n (cid:88) (cid:12)(cid:89) (cid:12) g(c ,...,c ) = (cid:12) (1+(−1)yic )−1(cid:12) 1 n (cid:12) i (cid:12) (cid:12) (cid:12) y∈Bn i=1 is strictly increasing for 0 ≤ c < 1, i = 1,...,n (note that for 1 ≤ i ≤ n, i g(c ,...,c ,...,c ) = g(c ,...,−c ,...,c )). 1 i n 1 i n Proof. We take 0 ≤ c < 1 for 1 ≤ i ≤ n. For y = y ...y ∈ Bn define p(y,j) = (cid:81)n (1+ i 1 n i=1,i(cid:54)=j (−1)yici). Without loss of generality pick a j ≤ n and let ε > 0 be an (arbitrarily small) positive real with c +ε ≤ 1. Note that j (cid:88) g(c ,...,c ) = |(1+(−1)yjc )p(y,j)−1|. 1 n j y∈Bn We partition Bn as follows: Y = {y | (1−c −ε)p(y,j)−1 ≥ 0}, 1 j Y = {y | (1−c −ε)p(y,j)−1 < 0 and (1−c )p(y,j)−1 ≥ 0}, 2 j j Y = {y | (1−c )p(y,j)−1 < 0 and (1+c )p(y,j)−1 ≥ 0}, 3 j j Y = {y | (1+c )p(y,j)−1 < 0 and (1+c +ε)p(y,j)−1 ≥ 0}, 4 j j Y = {y | (1+c +ε)p(y,j)−1 < 0}. 5 j 10