автореферат диссертации по информатике, вычислительной технике и управлению, 05.13.18, диссертация на тему:Численные методы изучения ошибок в некоторых статистических задачах

MINISTRY OF HIGHER EDUCATION - RUSSIAN FEDERATION

SAINT PETERSBURG STATE UNIVERSITY FACULTY OF MATHEMATICS AND MECHANICS

Numerical Methods for Studying Errors in Some Statistical Problems

BY

MAHMOUD SAIF ABDEL-RAHMAN EID

SUPERVISOR Prof. ERMAKOV S. M.

SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF PHILOSOPHY OF DOCTORATE (Ph.D.)

SPECIALITY: 05.13.18

MATHEMATICAL MODELLING, NUMERAL METHODS AND COMPLEX PROGRAMMING

FACULTY OF MATHEMATICS AND MECHANICS

SAINT-PETERSBURG 2002

r

п - / Г

' *у СУ 1 / ' 7 «-«'

/ ' / / / > 7' иХ

X _

Министерство образования Российской Федерации

САНКТ-ПЕТЕРБУРГСКИЙ ГОСУДАРСТВЕННЫЙ

УНИВЕРСИТЕТ

На правах рукописи

МАХМУД САИФ АБДЕЛЬ-РАХМАН ЭЙД

ЧИСЛЕННЫЕ МЕТОДЫ ИЗУЧЕНИЯ ОШИБОК В НЕКОТОРЫХ СТАТИСТИЧЕСКИХ ЗАДАЧАХ

Специальность 05.13.18 — Математическое моделирование, численные методы и комплексы программ.

диссертация

на соискание ученой степени кандидата физико-математических наук

научный руководитель, доктор физико-математических наук,

профессор Ермаков С.М.

Санкт-Петербург 2002

DEDICATION

To my wife Khitam

To my sister Maha

To my daughters Siham, Hia, Lama and Falasteen

To My son Muhammad and my brother's son Hisham

To my parents9 sons, daughters, brothers, and sisters

To my teachers, neighbors, friends and relatives

Table of Contents

Tables v

Abstract vi Acknowledgements viii

1 Regression Estimation and A2-Distribution in the Case of Uncorrelated Observations or Errors 1

1.1 Introduction........................................................1

1.2 Definitions: ........................................................2

1.3 Some Properties of A2 -Distribution..............................3

1.4 A2-Distribution in the Discrete Measure Case.........12

1.5 The Implementation of the A2-Distribution and Examples . . 14

1.6 A simulation to An Experiment.................21

1.7 Experimental Design and Dividing Components of Error ... 24

2 Regression Estimation and A2-Distribution in the Case of Correlated Observations or Errors 29

2.1 Introduction............................29

2.2 A summary from Linear Algebra and Numerical Analysis ... 30

2.3 The Method of Least Squares..................32

2.4 The Least Squares Estimators in the Dependent Errors Case . 34

2.5 A2-Distribution in the Discrete Measure Case.........37

2.6 The Implementation of the A2-Distribution and Examples . . 42

2.7 A simulation to An Experiment.................51

2.8 Generalizing A2-Distribution to the Continuous Measure (In

the Case of Correlated Errors)..................54

3 An Introduction to the Tolerance Intervals 57

3.1 Introduction............................57

3.2 The Concepts of Prediction and Tolerance Intervals......58

3.3 Types of Tolerance Intervals...................60

3.4 Examples on Tolerance Intervals.................61

3.5 Notations and Definitions ....................62

3.5.1 The Cover of Tolerance Interval.............63

3.5.2 Coverage of T.I. and it's Distribution...........63

3.6 Tolerance Intervals for the Normal Distribution........64

3.6.1 /^-Expectation T.I. for Normal Distribution.......65

3.6.2 ^-Content T.I for Normal Distribution..........67

3.7 A short Review of the Literature ................69

3.7.1 Two-Sided Prediction Intervals to Contain at Least k Out of m Future Observation from A Normal Distribution............................70

3.7.2 Simultaneous T.I. for Normal Populations with Common Variance.......................71

3.8 Gamma Distribution T.I. Problem................72

4 Tolerance Intervals for Gamma Distribution 73

4.1 Introduction............................73

4.2 Tolerance Intervals for Exponential Distribution........74

4.3 /^-Expectation T.I. for Gamma Distribution..........78

4.3.1 One-Sided ^-Expectation T.I...............79

4.3.2 Two-Sided /?-Expectaion T.I...............91

4.4 /3-Content T.I. for Gamma Distribution.............103

4.4.1 One-Sided ^-Content T.I.................103

4.4.2 Two-Sided /3-Content T.I.................119

4.5 Tolerance Intervals Simulation and Conclusions........140

4.5.1 Tolerance Intervals Simulation..............140

4.5.2 Discussions and Conclusions...............146

4.6 A comparison between T.I. for the Normal and T.I. for the Gamma Distributions.......................147

A references 151

B programs 155

B.l progl.pas .............................156

B.2 prog2.for..............................176

B.3 prog3.pas .............................181

Tables

Title Page No

1- Tables(4.1-4.6): Values of Q for upper-bound /^-expectation T.I.

with /3=0.9,0.95 and «1=1,2,3,4 and a3=2(l)4................82

2- Tables(4.7-4.12): Values of Q for lower-bound /^-expectation T.I.

with ¡3=0.9,0.95 and «1=1,2,3,4 and a2=2(l)4..............88

3- Tables(4.13-4.24): Values of Qi and Q2 for two-sided /^-expectation T.I. with /3=0.9,0.95 and 0^=1,2,3,4 and <*2=2(1)4..............97

4- Tables(4.25-4.36): Values of Q for upper-bound /^-content T.I.

with /3=0.9,0.95 and ^=1,2,3,4 and a2=2(l)4 ; 7=0.90 , 0.95 .....105

5- Tables(4.37-4.48): Values of Q for lower-bound ^-content T.I.

with /3=0.9,0.95 and «1=1,2,3,4 and a2=2(l)4 ; 7=0.90 , 0.95 .....113

6- Tables(4.49-4.72): Values of Q, and Q2 for twosided /3-content T.I. with /3=0.9,0.95 and ^=1,2,3,4 and a2=2(l)4 ; 7=0.90 , 0.95 .....128

7- Tables(4.73-4.74): Simulation for upper-bound ^-expectation T.I. . . .142

8- Tables(4.75-4.76): Simulation for lower-bound /^-expectation T.I. . . . 143

9- Tables(4.77-4.78): Simulation for two-sided /^-expectation T.I......143

10- Tables(4.79-4.80): Simulation for upper-bound /3-content T.I......144

11- Tables(4.81-4.82): Simulation for lower-bound /3-content T.I......145

12- Tables(4.83-4.84): Simulation for two-sided /3-content T.I........145

Abstract

Some Numerical Methods for Studying errors in Statistical Methods

Prepared by MAHMOUD SAIF ABDUL-RAHMAN EID Supervised by Prof. S.M. Ermakov

There is a special distribution called A2-distribution which was introduced by Ermakov and Zoltukhin(1960) for decreasing the variance in the Monte Carlo calculation integrals. This distribution used in making randomization in the analysis of linear regression . Subsequently, Ermakov(1963,1989) studied interpolation procedures where the nodes were chosen at random according to the A2-distribution, that allowed for extending its use in the field of the experimental design. Up to the previous year, A2-distribution was introduced and used under the condition that random errors in regression models are independent or uncorrelated .

One of the aims of this work is to implement the A2 -distribution and to discuss some of it's properties and to make a simulation study for these properties like the possibility of dividing the error components. Another aim of this work is to generalize this distribution to the case when errors or observations are dependent or correlated in the regression model for estimation.

Tolerance intervals are frequently used when it is necessary to use past data to

make statistical statement about future observations. Many researchers discussed tolerance intervals for normal distribution for their importance [1].

Gamma distribution is a very important distribution since it is frequently used as a distribution for the life time of some systems or for the lifetime of its components.

Due to the importance of the Gamma distribution, we will discuss its tolerance intervals. We will find tolerance intervals for a population having the Gamma distribution.

In chapter one we will discuss A2-distribution in the case of independent errors , we will implement this distribution in the two cases 1- when it exists a density of this distribution with respect to lebesgue measure 2- with respect to a discrete measure, and we will give examples and made a simulation study for randomizing designs using this distribution, also we will discuss the procedure of dividing the systematic and the random components of the errors of the parameters in the linear regression.

In chapter two we introduce a new A2-distribution in the case of correlated errors or observations in the regression models of an experiment. We will study this distribution in the case of discrete probability measure and we will numerically implement this distribution and give examples, also we will make randomization of experimental designs using this distribution and we will simulate a statistical experiment in the case of correlated random errors , and finally we will generalize a similar distribution to the case of continuous probability measure.

In chapter three we will contrast the concepts of prediction and tolerance intervals, types of tolerance intervals, a brief review to tolerance intervals for the Normal distribution and reviewing some works concerning tolerance and prediction intervals for the Normal distribution.

In chapter four we will discuss in details tolerance intervals for the gamma distribution. A simulation study was conducted for gamma tolerance intervals and the results of this study will be included, also some kind of comparison between tolerance intervals of Normal and Gamma distributions in this dissertation was made.

Acknowledgements

I would like to thank (ALLAH ) for every thing and I would like to express, with sincere gratitude and thanks, my respect to all those who helped in putting this dissertation in to existence. I deeply appreciate with sincere gratitude the great encouragement given by my honorable and respect supervisor Dr. S.M.Ermakov. I would like to thank him for his wise supervision and guidance which made this thesis fruitful. His devotion and wide knowledge gave me the confidence I needed to overcome all problems and difficulties.

Same respect and thanks are due to Prof. Nevzorov V.B. and to Prof. Sedunov E.V. for their sharing in testing and examining this dissertations and to Dr. Neck-rotkin , Dr.Golandina(Nina) and Prof. Yuri Sushkov for their friendly encouragement.

Of course, I am grateful to my parents, to my wife and my children for their patience and love. Without them this work would never have come into existence (literally). Also, I would like to express respect and gratitude to my brothers, sisters, uncles,neighbors, friends specially Abdulkhaleq, all my family and my wife's family.

I would like to thank Al-Najah university and the university of Jordan for their helps in letting me use their libraries , also I would like to thank all who helps me in the faculty mathematics and mechanics in Saint Petersburg State university.

I would like to thank Saint Petersburg State University and Russia for giving me this chance and for good questing. Finally, I wish to thank the following: Maya, Nadia, Samer, Mohamad, Mueen, husni, Taher, Wa'el, Hisham, Jamal, Ebrahim, Ahmad, Fahmi, Maher, Dr.Taher and my uncle abo sa'ed, all my cosines ,my wife's family (for their helps and encouragement).

Chapter 1

Regression Estimation and A2 -Distribution in the Case of Uncorrelated Observations or Errors

1.1 Introduction

The study of approximation theory involves two general types of problems. One problem arises when a function given explicitly but we wish to find a "simpler" type of function, such as polynomial, that can be used to determine approximate value of the given function. The other problem in approximation theory is concerned with fitting functions to a given data and finding the "best" function in a certain class that can be used to represent the data.

There are many methods can be used to make the required approximation but each method has advantages and disadvantages , as an example, the Taylor polynomial of degree n about the number Xo is an excellent approximation for an (n + 1 )-times differentiable function / in a small neighborhood of xq another method is called the

least-squares approach to this problem involves determining the best approximating line or curve when the error involved is the sum of the squares of the differences between the values on the approximating line and the given values.

In this chapter and on the coming chapter we will use least-squares method in estimating a function or in fitting functions to a given data and finding the best function in a certain class that can be used to represent the data, we will use a linear independent class of functions, orthogonal and orthonormal class of functions in estimation and we will discuss the A2-distribution which was first introduced by S.M.Ermakov and V.G.Zolotukin(1960) for decreasing the variance in the Monte Carlo calculation of integral. This distribution appeared to be efficient in choosing nodes randomly by using its distribution in interpolation procedures when we have in independent errors

In this chapter we will discuss properties of A2-distribution when we have a linear independent system of functions used in making the least-squares estimation in estimating the parameters of the regression model. Also we will give procedures and algorithms for calculating A2-distribution in the discrete case, we will implement this distribution, also we will give many examples, also we will discuss the experiment simulation and the procedures to divide the systematic and random errors.

1.2 Definitions:

we will use some definitions in the coming discussion, we see that its important to remind the readers about these important definitions. Definition]..1

The set functions {<£i,..., cf>n} is said to be linearly independent on [a,b] where b > a if, whenever

Ci4>i{x) + c24>2{%) + • • - + cn(j)n(x) — 0 for all xe[a,6],

then cj = c2 = ... = cn = 0. Otherwise the set of functions is said to be linearly dependent.

Definition 1.2: A integrable function w is called a weight function on [a,b] if w{x) > 0 for xe[a,6], but w(x) ^ 0 on any subinterval of [a,b] Definition 1.3

The set functions {4> 1,..., 4>n} is said to be an orthogonal set of function for the interval [a,b] with respect to the weight function w if

If, in addition, Qfc = 1 for each k=0,l,... ,n, the set is said to be orthonormal. Definitionl.4.

A system of functions is called regular if

Let x be a set, a probability measure on x and / Instead of / a function

— 6?i = 1? • • • jm may be calculated or observed at m points xi,...,xm from X■ Moreover, we can assume that £ is a random perturbation of /:

0 , whenever j ^ k otk > 0 , whenever j = k

nm({(xu ...xm): det |№(*;)II£=1 = 0}) = 0

1.3 Some Properties of A2 -Distribution

& = f(xi) + £i

E(e¿) = 0 and

Cov(£i,£j) —

, i # j

Let {(f>i}£Lj be a set of functions defined on x and assume that this set is an orthonormal set with respect to the measure /¿, let us approximate the function / by a linear combination of this set i.e ~ Y^T-i

We will choose the points xi, ■ • •, xm at random according to the density of A2 -distribution which is introduced for the case of uncorrected errors by Ermakov and Zoltoukin(1960)

PÍx1,xa>...,®m) = (m!r1(det||^(a:i)||^asl)a (1.3.1)

Then if we solve the problem of interpolation for the ■ ■ ■, with the

observations £,\h\ ..., where k is the replication number. If we calculate the value of cf from the system of equations k^4>¡(x^) = <fjfc' , By Crammer rule the the value of equal to

(it) _ det\\<j>i(xj),Mxj), • • • • • • >#m(gj)HjLi . .

Cl - detwM^m . ( j

under the assumption that the denominator of equation in (1.3.7) not equal 0 .

Theorem 1.1: Assume we have two sets of linear independent functions and {t/>j}" which are numeric and second integrable functions i.e eL2(/j) where fi is a measure function on x > ^ Q = {xiix2i ■ ■ • then:

I = J fin(dQ). det det ||^Ar(®jOII(fc,¿:=i) = det ||(<?!>/, '0¿)||(lJ-Ijt=i)

(1.3.3)

where (<f>i:ipk) — f (j>i(x).ipk{x)fidx. and fin(dQ) = fi(dxi).fi(dx2). ■. f¿(dxn).

proof: let us denote

by A and by B. i.e

and

i-i i-i -ei ... <f>n(xi)

A = Mx 2) (¡>2{X2) • • <f>n{x 2)

_ <t>l(Xn) <h{xn) • • <f>n{xn)

lpl(xl) ^2{xx) ... ^n(xi)

B = i>l{x2) ip2{x2) ... i>n{x2)

•

i>2(xn) ... i>n{xn)

Then by substitution det(A) = Y^j-ii*l)J'+1-aji-lAjil where ¡Aji| denotes the determinant of the matrix A after deleting column 1 and row j, which implies that we use column number 1 as a minor, then I will be as follows

1= f ^¿(^¿(-l)>+1.«il.|^-1|.det(B)

J j=i

then by inserting the integration sign in side the summation sign, I will be as follows

I==T<JKdxi)---J tifci-i) J ^(xjJ.CrfxiJdetiB)

(1.3.4)

Now for any row j of B, we can multiply its elements by <f>i{xj) and inserting the integral sign to the row j, this row of matrix B will be as the following

J Mxj)Mxj)v{dxj), j Mxj^iixj^idxj),..., J <k{xj)'ij>n{xj)n(dxj)

for example row 1 of the matrix B will be as follows

J J ..., J (f>1(xi).tpn(xi)(j.(dxi)

In equation 1.3.4 we have n terms which are equals since term number 2 can be like term number 1 by the following procedure:

Step 1: Exchange the places of the row of integration which is row2 and row 1, by this step we deleted the minus sign from the term since (—1)2+1 = —1.

Step 2: Change the name of the variable which is inside the integration from x2 to xi since the the integration doesn't depend on the names of variables.

step 3: Change the name of variable xj by x2 in the term.

After these three steps term 2 will be like term 1.

Do the above procedure to any row j which has the integration, take care in in stepl, you exchange row j with row 1, in step 2 you made j-2 exchanges other than the first exchange which is in stepl( i.e, exchange of rows 3 and 2 ... up to j-1 and row j-2 and so on. In step 3 you change names of variables xi by x2 , x2 by ..., i by Xj , notice that you made totaly j-1 exchanges which is enough to delete the minus sign if it exist in the term. After this, equation (1.3.4) will be as follows

/ = n J fi(dx2) ...J fi(dXj-0 J (i{dxn). det 11^(^)11^2). det(B') (1.3.5)

where the matrix B' is ||V'»(:c.;)ll(i=i) but the first row is

J «MzOV^iM^i)) J <l>i(xi)ih{xi)Kdxi)> • • • > J

By running the same procedure on det ||<fo(®j)||"jt-=2) by finding its determinant also by making the integral row in the matrix B' row 2 then equation (1.3.5) will be equal

I — n.(n — 1) J n(dx3)...J p{dxj-1) J n(dxn). det ||<M*i)ll(U)- det(B") (1.3.6)

where (B") have two integral rows 1 and 2, and so on until we reach required result which is

I = n!

f 4>i(x)x^i(x)fidx J ^>1(x).i^2(x)fidx

f fa(x)rf>i(x){idx f <fa(x).xf>2(x)fidx

. J fa(x)ij>n(x)iidx ■ $ <t>-z{x)4>n(x)ndx

f <j>n{x)i}>i(x)ndx f 4>n(x).^2{x)[idx ... / 4>n{x)tpn{x)ndx

which is the right hand side of equation 1.3.3, which ends the proof.

Theorem 1.2 The expectation of c^ is equal to the Ith Fourier coefficient of the function f with respect to the orthonormal system

E(c!fc)) = f f{x)<j)i{x)fi(dx) Jx

proof:

E(c\k^) = Jcjk\p(xi,x2,..., xn).fimd(Q) by substituting the values of cj^ from equation (1.3.2) and the value of p from equation 1.3.1 which the A2 -distribution then

(k) _ Jx jid(Q).det ■ ■ ■ 4>(i-i)(xj), f(xj), <f>(i+i)(xj),..., Maj)ll£rdet llfrfojOllE

ml

Now by applying theorem (1.1) on the above numerator, the above equation will be as follows

E(c^) =

(4>2,4>\)... (<f>2,<f>[-l) (<&,/) (<fa,<f>l+l)

{4>i,<t>i) i&Ai-i) (<f>t,f) ((h,<f>i+i)

(4>l,4>m)

($2, 4>m)

(<f>h<f>m)

(1.3.7)

(4>m,<i> l)--- (0m,0f-l) (<l>m,f) (<t>m, (f>l+l) ••• (<t>m,<t>m)

where (cf>i,tpk) ~ f <f>i(x).t(>k(x)ndx. and (<) = f<pi(x).f./j,(dx). and fin(dQ) = /j,(dxi).fi(dx2),.. .¡j.{dxm). But we assumed that the set {<fri, • • •, <f>m} is an or~ thonormal set of functions, then equation ( 1.3.7 ) will be as follows

10 0 0 1 0 0 0

0 0 ...

0 0 ...

0 0 ...

...(&,/) 0

■••(&,/) 0

: 0

(<f>,,f) 0

: 0

(4>m,f) o

(1.3.8)

which implies that E(c\k^) = (/,</>/) = f f(x).<f>i(x)n(dx) which is ends the proof. Theorem 1.3 Under the assumption that the system of functions {(^¡(x)}^ are orthonormal and regular then

a- The systematic component of the variance of c/ is equal to

/m

i=l

fi(dx)

where (/,<£/) = f f(x).<f>i(x)fi(dx)

b-The covariance between any two different parameters equals to 0. c-The equal sign in (a) will replaced by < if the system of functions is not regular. For the proof of parts (b) and (c), see [25]. Proof: a-

Var(ci) = E(%) - (£(c,))2 = E(c]) - (<f>hf)2 since from theorem 1.2, E(ci) = (</>/, /)

(1.3.9)

E(cf) = j /,m(dQ)(c]).P(xl,x2,...,xm).

we make the integration on the region where det ll'&i^i)!!^;/=i) ^ 0. By the assumption, the set of system of functions is regular, then the above equation will be as follows

r,,^ JxM<?).(det IIMxj), ■ ■ ■ 0(/-l)(xj),f{Xj), . . -, <f>m{xj)H7=1)2

E{ci) =-

ml

After applying theorem 1.1 on the numerator, we have

{<t>\,<t>i)--' {<fa,<f>i-i) (4>i,f) (<f>i,<f>i+i) (fafa)... {<h,<l>i-i) (<f>2,f) {<h,4>i+i)

E(cf) =

{f,4> 0-.. (/,<M (/,/) (f,<f>l+l)

i,<f>l)-- (<f>m,<f>l-l) (<f>m,f) {<f>m,<f>l+1)

(<£ 2 Am) VAm)

(<f>m,<f>m)

(1.3.10)

but the of functions </>i, </>2,..., </>mare orthonormal then, equation 1.3.10 will be as follows ,

E(%) =

1 0 0

0 1 0

0 ...(&,/) 0 ...(&,/)

(/> l) (/,^2) 0 0

0 0

0 0

0 0 0

(/,/) 1) Ofr+1,/) 0

0 0

0 0 0

(fM 0

0 1

after finding the above determinant equation 1.3.11 will be as follows

m

(1.3.11)

(1.3.12)

by substituting the value of E(cf) in equation (1.3.9) the variance of q will be as follows

m

Var{ci) = £(c?) - (£(c,))2 = JS?(cf) - {</>,, /)2 = (/, /) - £((*(*), /(*)))2 (I-3-13)

>'=i

the right hand side of equation 1.3.13 is equal to the following

m » r re 12

(/»/) - £«*•(*). = / - /)*(*)) -/«w (1.3.14)

we need to prove equation (1.3.14) and the proof of the theorem will finish. We will use the induction method in proving the above equation.

Stepl: Let m=2

The left hand side (L.H.S) of 1.3.14 is equal to (/,/) - (/,</>i)2 - (/,<£2)2 and the R.H.S of 1.3.14 after analyzing the square sign and inserting the integral sign will be equals the following

(/, /) - (/, <M2 - (/, faf - (/, faf + (/, 4>i)2+

which is equal to (/,/) — (/, (f>i)2 — {f,<fa)2 which is equal to the L.H.S of equation (1.3.14) which implies that it is true for n=2

Step 2: Assume its true for m i.e

m « m

(/>/)-£(om*),/(*)))2== / /00-£(№./)*(*))

i=1 ^ L ¿=1

.fi(dx) (1.3.15).

we will prove that is true for m+1 , assume that we have m+1 orthonormal system functions i.e the L.H.S in case m+1 of equation 1.3.14 will be equal

T 2

.li(dx) =

/

which is equivalent to

2 •/

/(*) - E.™ i ((<&> f)Mx)) ~ (/, <?Wi)<W

,fi(dx)—

.fj,(dx)

(f(X) ~ 127= i((&> /)&(*)))(/» 0m+l)-0m+l +

fa+MJ*) (1-3.16)

The first term of equation (1.3.16) by the assumption equal the L.H.S of equation (1.3.15) , the second term of equation (1.3,16) is equal to — 2((/, <£m+i))2 since /Efci((^»>/M"(a:)-(/>^m+i)*^»n+i)/l(<ia?) = 0 because the / = 0

for i — 1 ,...,m , from orthonormality of the system. The third term of equation (1.3.16) = (/,^>m+1)2.l since f<%,+1(a:).fi(dx) = 1. which implies that

m

i - (/, /) - £(/, -+tt

then I = (/,/) — EUtH/i ^i)2 which is required since it true for m+1 this implies that it is true for m and this ends the proof of the theorem .

2

Assume that be a set of function defined on x and this set is linear inde-

pendent with respect to the measure fi, let us approximate the function / by a linear combination of this set i.e f(xi) ~ ]CI=i ^et V* = v(xi) = f(x*) + £i

The least squares estimator of c = (ci,..., c„) is that value of the parameters which minimize the squares of errors i.e.

/m

[y(*)-£ci<fc ]2fi{dx). (1.3.17)

¿=1

then

det <f>k),..., (</>;_!, <f>k), (y, 4>k), (<fa+1, 4>k), . . . , (<ftm, <f>k)\\k=1 /iqiQX

where,here, (<f>k,(f>r) = fx (f>k(xj)<f)r(xj)dxj and (y,<j>k) = f Vy4>k{xj)dxj. The A2-distribution defined on x with the condition that the set functions i

are linearly independent with respect to fi is as the following

( (det ||<fc(xj)llij)2

^.....Ib) ~ ^Mm^m-MW2 1 j

Lemma 1.1 Let (xi,..., xm)~Q are a random points with probability density of A2 and a..., am determined from the following system of equations

m

^ a/0/(i,) = y(x,) t = l,...,m (1.3.20)

then the expected value ai is the least squares estimator ¿i for I = 1,... , m. proof:-

From equation (1.3.17) we have the system of normal equations

Til

£ ct{<h, 4k) = {yAk) k = 1,..., m (1.3.21)

¿=1

and from equation (1.3.20) and by Crammer's rule we have

det H Mxi),..-,<j>k-i{xi),y{xi),<l>k+i(xi),...,<f>m{xi)\\^1 .

= detllfefx,)!!^, (L3'22)

and

E(ak) = j MdQW.p(xi,...,xm) (1.3.23)

where f ¿tmd(Q) = J jxdxi,..., J fidxm then by substituting the values of a^ which is in equation 1.3.22 and the value of A2 distribution which is in equation 1.3.19 in equation 1.3.23 and by applying theorem 1.1 on the denominator and numerator of 1.3.23 we will reach the following

Fi - det IK^1' &)> • • • > (^1-1» &)> (y, fa), (<k+l,fa), (4>m, 0k)HkLi

^ det ||(<£k,<£r)ll™r=i

which is the least squares estimator of cjt defined in equation 1.3.18.

Also we can see that 1.3.24 is coincided with the solution of the normal equations in 1.3.21 obtained by Crammer's rule.

This result was obtained without the assumption of orthogonality or orthonor-mality of the system of function ,..., <f>m.

1.4 A2-Distribution in the Discrete Measure Case

Ermakov and Schwabe [26] discussed the least square estimating of the coefficients in linear regression in the simplest formulation corresponding to the previous section in the discrete measure case.

Only a finite set of points 2/j ^ • • • 5 tTfi IS of interest and we use the uniform measure H which concentrated on the points x\,..., xn with equal weights 1 /n. assume that we have the following predicted values or responses ..., yn corresponding to Xj,..., xn and suppose that y,- = /(xt) + £,• , with e^s are independents with 0 expected value and constant variance a2.

If we approximate / by the linear combination YIJL^ c/0j where fy are linearly independent functions on the set xi,...,xn the the least squares estimators c/ of the coefficient c/ are expressed by

1.3.24)

. _ det 11(01, <f>k),..., (01-1,4>k), (y, fa), (<t>i+1,• • •, (0m, <MHfcLi n 4 n

where,here, (<£*,<&.) = ^ ^jLi Mxj)Mxi) and (y,0fc) = ^ Ejli yjMxj)

Lemma 1.2 if the functions 0i,..., 4>m are orthonormal on the set points xi,..., xn then the estimators of the coefficients c\ in equation (1.4-1) simplified to

1 N

« = Âf (1-4-2)

3=1

a/so ¿y using the Cauchy-Binet formula ci in equation (1-4.1) will be as follows

- <...<im<N MÛ, im)&(il, • • • , *m)

c, ----=—-—-—--(1.4.3)

Zul<H<-<im<N ° V'l? • • • 5 «m/

where A2(î'x, ..., im) = det ||(0fc(z,\,)||£j=1 and

Ai(jr, ii,..., im) = det ||0i(^tj),..., 4*i—i y{xij), (f>l+i(xi3), -■-, «M^OIIjLi for the proof see S.Ermakov[25] and S.Ermakov and R.Schwabe[26] .

If we introduce a probability distribution of sets of points Xit,..., X{m by the A2-distribution in the discrete case which is as follows

1<1] <—<im<N

then the random variable defined by

Xf(2"ii •••>*"*) = Af(^,21,...,27n)/A(2i,...,im) (1.4.5)

which represent the estimator of the coefficient c/ by using only m from n points and these m points are one combination from the m from n combinations , this (x\() has expectation (conditionally on y fixed) equal to the value of the least squares estimators q defined in (1.4.1) which depends on n points Xi,..., xn.

Notice that at the same time according to Crammer's rule the value xi{i • • ■■> 4) are the solution of the system of equations

1=1

In section 1.6 we will make a simulation experiment to prove that the expected value of Xi defined on (1.4.5) equals the least squares estimator cj by the probability density of the A2-distribution defined in (1.4.4).

1.5 The Implementation of the A2-Distribution and Examples

A2-distribution was introduced for decreasing the variance in the Monte Carlo calculation of integrals, also this distribution was introduced for the nodes in the interpolation formulae. In this section we will give some examples on A2-distribution in the discrete case and independent errors.

A computer program called Prog3.pas in the appendix implement this p.d.f and can find the A2-distribution to any linear independent system of functions which may used to estimate any type functions or can be used in fitting any data to any function, polynomials or trigonometric functions or any kinds of linear independent functions at one dimension or at two dimension by calling the procedure which called finddelta-file, this procedure calls other procedures which solves problems like finding the determinant up to 15 x 15 matrix of two dimension also it calls procedures which finds the required combinations of points. It can easily be modified to be solve bigger sizes of problems.

In this section, we will give examples on A2-distribution defined in equation

As an example to a linear combination polynomials, we will take the Legendere polynomials which are orthogonal system of functions with respect to the weight

TO

(1.4.6)

(1.4.4).

function w(x)=l and defined on the interval [-1,1], and a trigonometric system of orthogonal functions with respect to the weight function w(x)=l defined on the interval

[—7T,7T].

Let us define the Legendere polynomials (f>%(a;,-),..., «^(z.) as follows

3 1 5 3

<f>i(z) = 1, <fct{x) = x, =-x2 -4>a(x) = -x3 - -x

w x 35 4 15 2 3 , , , 63 5 35 3 15

M*) = - -j + s' M*) = Yx " T + ~8X

Let xi,x2,...,xn n > m be n equidistant poin