Канонические представления Текст научной статьи по специальности «Математика»

CANONICAL REPRESENTATIONS

G. VAN DIJK

Department of Mathematics, Leiden University P.O Box 9512. 2300 RA Leiden, The Netherlands e-mail: [email protected]

These lecture notes provide an introduction to the theory of so-called canonical representations, a special type of (reducible) unitary representations. The simplest way to define them and to see their importance is done in the context of the group SL(2, S.), the group of 2 x 2 matrices of determinant one, see [16]. We have chosen a class of groups, namely G = SU(l,n), n > 1. Notice that SL(2, ¡R) is isomorphic to SU(1, 1). Canonical representations can be seen, generally speaking, as the completion of L2(C/K) with respect to a new G-invariant innner product, in the same spirit as the “complementary series" is obtained from the “principal series” for G. Here K = SU(n). But this is only one (but important) point of view, see section 5. Canonical representations occur also when studying tensor products of holomorphic and anti-holomorphic discrete series representations. This is explained in section 4. The connection with quantization in the sense of Berezin is not treated in these notes because W'e will emphasize the representation theory. On the other hand, Berezin has made a large contribution to the understanding of canonical representations.

The main problem is to decompose the canonical representations into irreducible constituents. This is not an easy task. It has been done by Berezin [1] and, later, by Upmeier and Unterberger [15]. There are however, in both treatments, conditions on the set of parameters of the representations: only large parameters are allowed. For small parameters (see [3]) an interesting new phenomenon occurs: finitely many complementary series representations take part in the decomposition. We shall treat the case G = SU(1, n) in detail in these notes and try to illustrate all aspects of the theory of canonical representations we have mentioned.

1. SPHERICAL FOURIER ANALYSIS ON COMPLEX HYPERBOLIC SPACES

The main reference for this section is [9].

1.1. Complex hyperbolic spaces and their bounded realizations

Let n > 1. Consider on Cn + 1 the Hermitian form

[x,y] = y0x0----------------ynx,i- 0-1)

Let G — Sl'(l. Ti) be the group of (n + 1) x (n + 1) complex matrices which preserve this form and have determinant equal to 1. The group G acts on the projective space P„(C) and the stabilizer of the line generated by the vector (1, 0,. .., 0) is the compact subgroup K = S(U(1) x U(n)). We call X = G/I\ a complex hyperbolic space. X is, in addition, a Riemannian symmetric space of the non-compact type, of rank one, and carries a complex structure, as we will see.

Let 7r denote the natural projection map

7T : Cn+1\{0}^Pn(C),

sending each vector to the line generated by it.

The hyperbolic space X is the image under tt of the open set

{x 6 C,l+1 : [a,-, x] > 0}.

On Cn we have the ususal inner product

(x,y) = V\Xi +------VVnxn

with norm |jx-1j = (x,x')1,/2. Let

‘ B(Cn) = [i 6 C” : IHI < 1}

(1.2)

(1.3)

(1,1.)

(1.5)

the unit ball m Cn! The space X can be realized as the unit ball in C”: the map from (1.3) to Cn, given

x^y with yp = xpxa1 (16)

defines, by passing to the quotient space, a real analytic bijection of X onto B(£n). G acts on B(Cn) transitively by fractional linear transformations. If g £ G is of the form g G f° M , with matrices a(l x 1), 6(1 x n), c(n x 1) and d(n x n), then

9 ■ V = (dy + c) ((6, y) + a)-1 qj)

where y and c are regarded as column vectors and

(b,y) = biyi + • • ■ + bnyn. (]8)

Clearly K = Stab (o).

An easy computation shows that

and

1 {g ' V, 9 ■ z) - ((6, z) + a) ■ [1 - (y, z)] ■ {{b,y) + a) \ (1.9)

1 - llff ■ y\\2 = [1 - llz/!l2] ' 1(6, V) + °|~2 (1.10)

(b,y) +a = «6, g -y) +2)-1 if g-1 = Vj . (in)

On the other hand, the absolute value of the Jacobian of the real analytic transformation y -* g ■ v (v G B{Cn)) is easily seen to be equal to

|(6, y) + a|-2(n + 1). (1.12)

If dy is the Euclidean measure on Cn , then clearly

Ml/) = (1 - ll2/||2)_(r!+1)aiZ/ (1.13)

is a G-invariant measure on B(£n).

1.2. Fine structure of SU( 1, n)

Let J be the (n + 1) x (h -f ]) matrix diag {1, — 1, . .., — 1}. For any complex matrix A" of type

(n + 1) x (n + 1) we set A* = J X* J. '

The Lie algebra g of G consists of the matrices A” that verify the relation

X + A" = 0, trace A' = 0. (114)

These are the matrices of the form

'Zj Z,

z\ Z3) (L15)

with Zi and Z3 anti-Herrnitian and Z2 arbitrary, trace (Z, + Z3) = 0. Let 6 be the involutive automorphism of g defined by

6X = JXJ. (M6)

1 hen 9 is a Cartan involution with the usual decomposition g = t + p. Here t is the Lie algebra of K. Let L be the following element of @:

/0 0 1\

1=0 O 0 . (1.17)

\1 0 0 j

W e have L e p and a = 1 L is a maximal Abelian subspace of p. We are going to diagonalize the operator adL. The centralizer of L in t is

(u 0 0\ .

m={|0 v Oj : u + u = 0, v £ u(n — 1), 2 u + trace v = 0). (1 18)

\00 u

Let a = 1. The nonzero eigenvalues of ad L are ±a, ±2a. The space ga consists of the matrices

/0 2* 0

X = [ z O -z | (1.19)

\0 z* 0

where z is a matrix of type (n - 1,1) and 2* = ~z‘.

The dimension of @a is equal to ma — 2(n — 1). The space 02« consists of the matrices of the form

w 0 — w'

X = { 0 0 0 I (1.20)

w 0 — w t

with w + w = 0. The dimension of g2a is equal to m2a — 1. We have g = 0-2« + 8-a + a + m + 0Q + 02a-

Let A be the subgroup exp a. This is the subgroup of the matrices

(cosh t 0 sinh t

0 / 0 | (1.21)

sinhi 0 cosh^

where t is a real number. The centralizer of A in K is the subgroup M of the matrices

'u 0 0\

0 v 0 (1-22)

0 0 u)

with |u| = 1 and v £ U(n - 1), u2det v = 1. The Lie algebra of M is m. The subspace n = 0a + g2a is a

nilpotent subalgebra. Set N = exp n. This is the subgroup of the matrices

1 + w-^[z,z] z* -w+^[z,z] \

(z, w) = | 2 I -Z \ (1.23)

W-±[z,z] Z* l-W+^[z,z)J

w

.’ith w + w — 0 and with z a matrix of type (n - 1, 1), z* = -z , and if

z2 \ i z

z

~zn

( Z'A

UJ

then [z, z'} = —z'2Z2 — ■ ■ ■ z'nzn.

The composition law in N is the following:

n(w, z) ■ n(w', z') = n(w + w' + Im [z, z'],z + z'). (1 -24)

The subgroup A normalizes N:

at n(z, w) a_t = n(e2iw, etz). (1-25)

Let 2p be the trace of the restriction of ad L to n:

P — ^(ma + 2m2o) = n- O'26)

We have the I was aw a decomposition G - KAN = NAK. Each g £ G can uniquely be written as

g = kat^n accordingly. One has the corresponding integral formula:

f f{g)dg= f f(katn) e2pt dkdtdn (1-27)

Ja JKAN

for / £ 'D(G'). This is also equal to

f(natk) e~2pt dndidk. (1-28)

Jnak

Here dn = dzdw (n = n(z,u>)) and dk is the normalized Haar measure on K. Observe that NA

parametrizes X = G/K. Moreover, we have the Cartan decomposition G = KA+I\ where

A+ = {at : t > 0}, (1.29)

and, after dg is normalized according to (1.27), the corresponding integral formula

f f(o) dg - iff f(katk') S(t) dkdtdk1. (1.30)

Jg Jk Jo Jk

Here .

i-i \ ^ 7rn / ■ t \m /Sinh2t.ryl , ,

fi(0 = 2|^y(sinh<)m"(—2~)ma“. (1.31)

1.3. Spherical functions, inversion and Plancherel formula

For s £ C let

<ps(g) = f ¿-"M'-'Vdk (geG) (1.32)

Jk

be the zonal spherical function with parameter s, in integral form, according to Harish-Chandra. It is known that <fs(g) — <p-,(g) — <ps(g~1)- Furthermore let c(s) denote Harish-Chandra’s c-function:

For / £ V(G//K), the space of bi-A'-invariant, compactly supported C°°-functions on G, we define its spherical Fourier transform as

f(s)~ f f(g)f-s(g)dg (s € C). (1.34)

Jg '

f is a function of Paley-Wiener class, and / is even in the argument s. One has:

Inversion formula:

dfi

|c(z»|

f(g) = co f(iv)viy.(g) — (g£G), (1.35)

and

Plancherel formula:

,2 dV

f \f(g)\2dg = c0 f |/(z»|2 Jg jo

>)l2’

where Co = 22n-2r(72)/7Tr,+1.

The function <p(t,s) := v?s(a0 is the unique solution of the ordinary differential equation

(1.36)

d2y . cosh t cosh2i.dy , ,

J + + (137)

that satisfies <^(0,s) = 1. So

v>(t,s)= 2F1 (^A s2+P;p;-sinh2f). (1.38)

There is another solution for t > 0, 4>(£,s), which has the asymptotic behaviour e(s as t —> oc and is given explicitly by

*(f, s) = 2s-p (sinh tyiFl(-‘-2r±l,=!±£. 1 - 5; -sinh-2 t) (1.39)

for s 1, 2, 3, .... If s is not an integer, then

(p(t, s) = c(s) <E>(2, s) + c(-s) $(¿, -s) (t > 0). (1.40)

Moreover, as t approaches 0, and have the following asymptotic behaviour:

x / C(s) t2~2n if n 1 , n

( ,S) ~ ((7(5) log i if n = 1 ’ ( ^

d*(s,t) ((2-2n)C(s)t^ ifn#l

dt ~\C(s)l/t ifn = l, 1 J

for a certain function C of s. Notice that $ and are integrable with respect to the measure 6(t)dt

on (0,oo) whenever Res < —p.

It easily follows from (1.36) that for t > 0:

/00 ^ J

(1.43)

■oo cvW

Since / is of Paley-Wiener class and c(s)-1 of polynomial growth (see (1.33)) for Res > —1, one has in addition, by Cauchy’s theorem:

f(at) — c0 f f{<r + in)m,-cr-i{j.)-.............^ . (1.44)

J-oo Cl17 + W

for a > —1, t > 0 and f £ V(G//K).

2. CANONICAL REPRESENTATIONS

2.1. Definition of canonical representations

For A £ R and g £ G we set

iMff) = (i - IMlV (2-1)

where ¡j £ B(£n), y = g ■ o. Clearly ip\ is a bi-A'-invariant continuous function on G. Observe that ipx(at) — (cosh¿)_2A (t £l). An easy computation shows that

, , -! , rO-M’Ki-ll-inf ,,,,

if z,y £ B(Cn), z = gi • o, y = g2 ■ o.

Let us denote this expression by B\(y,z). B\ is called a Berezin kernel of X. Since products

and (uniform) limits of positive-definite kernels are again positive-definite, we easily get, by expanding

[1 — (2,2/)]-A into a power series:

[i-(;,»)]-* = £f“„V»r<-‘r (2-3>

v Tn

m-Q v

with ("¡^) — (~AX~A~1,^y(~A~m+1) t that B\ is a positive-definite kernel for A > 0. Or, otherwise said, 1px is a positive-definite function for A > 0.

Let 7r\ denote the unitary representation of G naturally associated with ip\ or Bx-

We call the tt\ (A > 0) canonical representations after Vershik, Gel’fand and Graev [16] and we shall study in this section their spectral decomposition in detail.

2.2. Spectral decomposition

The function ipx is the reproducing distribution of ttx in the sense of L. Schwartz, see [2]. We shall determine the integral decomposition of ipx into (elementary) positive-definite spherical functions. It is

well-known (see [11]) that the funct.ons y>„ defined in (1.32), are positive-definite if and only if s is purely

im gBy (1 13), the function 4>x is actually a function in Ll{G) for A > p, or even for Re A > p, since V'a is well-defined for complex A too. So in this case (Re A > p), it suffices to determine the spherical Fourier

transform ax(n) of ip\:

a\(t*)= V'A (g)<P-iMd9- ^2'4^

Jg ■

The computation of ax{n) is surprisingly simple. Applying theCartan decomposition G = KA+K, (1.30)

and (1.39), we get by making the change of variable x = sinh t,

axM = £ 2fl(Z^, _,) (1 + ,) - (2-5)

This expression is by [7], 20.2 (9) equal to

' ^r(A + ^)r(A+^P) (26)

ax(N — T(A)2

We may, in particular, reconclude that tp\ is a positive-definite function for A > p.

Moreover y°o dfJ

(V>A,/) = V'a(s) fid) d9 — co a\in)fiw) |c(z-ii)|2 ^ ’

for all f G V(G!/K). Here c0 is as in (1.35). j

We will now describe the decomposition (2.7) in another way, m order to gam insight how to proceed

in the case 0 < A < p, where xp\ is still positive-definite. .

We apply (1.44). The function <!> satisfies:

m, -a - iriwumt) < c0e^+2ReX-p)t c2-8)

for some positive constant C0 and t large. Thus for <r and A such that <r + 2 Re A > p:

Awx.f) = co + + (2-9)

where ,co . ,

6a(s) := ${t,s)i>x{t)t{t)dL ( }

Jo

For Re A > p we have:

axiri = J°° M*) [c(in№, "»>)№di

= c(i/z)6a(—¿A1) + c(—¿/z)6a(v)- ^

We will now take a closer look at the function 6A. Since $(f,-s) is analytic m s for^ Re-.(s) > -1, it: is

immediately seen that 6A(s) is analytic in s on VA = {s I Re(s) > max(p 2Re , )}. e w

the problem of analytic continuation of b\(s). anrl

Fix A > 0 and let C{s) = /T(n). Using (1.39) and making the substitutions r - smh t and

X = y/(l - y), we get ■

6xis) = Cis) /°° 1 + s;-x)r^+A-1(l + x')-Arfx-

y

1 - V

(2.12)

Applying the relation z , „x

2Fl(a,b;c-,z) = {l-z)-a2Fiia,c-b-,c-j—j) iL )

(cf. [8], 2.1.4 (p. 64)) yields

bx(s) = C(s)£ 2Fl(S-P2 + 2!S~P2 + ‘2-,l + s-y)y^^dy. (2.14)

By substituting the series expansion

2Fi(a, 6; c; z) = -TV yp (M < 1) (2.15)

l=o {C)l

and taking care of the possible singularity in y = 1 (in case n — 1) we obtain

(^ + 1)? _ 1 ,=o(s+1)'/! *1

since

This series is absolutely convergent for s ^ —1, —2,... and s ^ s/(A) = p — 2 A — 21 (I = 0,1,2,...), sir the terms are majorated by Z-1-5 for some 6 between 0 and n, for / large. This can easily be seen by using Euler’s limit formula for the gamma function:

T(z) = lim (2.17)

n — co (2)n + i

Thus bx has a meromorphic extension to C with poles in s — s;(A) (/ = 0,1,2,...) and s — —1, -2, —3,.... The residues in s/(A) are equal to

22X + 21-2p+l wn (1-A-/),2

r(n) (p — 2\ — 21 + 1); /! ’ 1 j

if s/(A) ^ —1, —2, —3,.... An easy observation shows that these residues are strictly positive for A > 0 for all values of I such that sj > 0.

Consider the relation (2.11) again. The explicit expression for aA shows that for any fixed / 0 in IK., A —ax(fi) depends analytically on A for A in some strip around the positive real axis. So does the right-hand side of (2.11). Thus this relation actually holds for all p. ^ 0 in R and A > 0.

Let <3>_j be the bi-A-invariant function on G defined by $_s(at) := $(i, — s) for t > 0. The definition of b\ can then be reformulated as

bx(s) = f ipx(x)$-s(x)dx (2.19)

Jx

for s £ V'a. Formula (1.30) together with (1.41), (1-42) shows that the integral exists for those s. Let us define the differential operator A a on X — G/K by

Ax ■■= cx(A +dx) (2.20)

with cx — —1/(4A2), dx = —4A(A — p) and A the Laplace-Beltrami operator of X.

A direct computation using the explicit form for tpx yields

Axipx = ipx+i (2-21;

for all A > 0. Fix A > 0. One has

J Aipx{z)$-s(x)dx

= C0(s) + / rpx(x)A^^s(x)dx = C0(s) + (s2 - p2)bx{s) (2 22;

Jx

for all s 6 C such that Re (s) > p — 2A with

<9<i>

C0(s) :=ljmipx(at)(2.23;

This limit can easily be computed for Re (s) > 0 (cf. [9], Ch. IV, section V, §2, p. 415-416) and is equal to

o2-2p n

£<>(*) =—nvT'sc(s)' (2-24)

For A > 0 and s £ V\ n {Re(s) > 0} we obtain

b\+i (s) = Ax i>\(x)$-s(x)dx

Jx

= ca{Co(s) + (s2 - (p - 2A)2) bx(s)}. (2.25)

Because Co(s) is analytic for Re(s) > —1, this relation can be used to extend bx to Re(s) > — 1 by

iteration. Since c(s) and b\+k(s) remain bounded as |s| —► oo in the strip 0 < Re(s) < p for sufficiently large k £ N, it is now easily seen that bx{s) remains bounded too when |s| -+”oo in that strip. We thus have:

Proposition 2.1. Let A > 0. The function bx(s), defined for s £ Vx, has a meromorphic extension to C with poles in si(A) = p - 2A - 21 (I = 0,1, 2,...) and -1, -2, -3,..., given by (2.16). The residues in «¡(A) are equal to

22A + 2i-2p+l7rn (1 - A - l)f

r(n) (p - 2A - 2/ + 1); /! ’

provided si(A) ^ —1, —2, —3,.... Moreover, bx remains bounded as |s| —> oo in the strip 0 < Re(s) < p.

Fix A > 0. Let / £ V(G//k) and consider the function

f(s)b a(s)

gx : s c0-----——.

c(s)

The function gx is meromorphic for Re(s) > 0 with simple poles in s; = s;(A), I such that s; > 0. Let jr be the contour determined by the rectangle given by the points ±iR and p ± iR. Since / is of Paley-Wiener type and bx remains bounded as |s| —► oo in the strip 0 < Re(s) < p, integrating gx over jr and letting R tend to infinity yields

{r!>x,f) = 2x rKA)(^n/) + co / -7(2.26)

l,s,>0 J-x

for A > 0. Here we used relation (2.9) with a — p and

r'(^) :=------Ress=J, bx(s). (2.27)

C0 C(6'/)

Thus we finally obtain, by using (2.11),

__ y* OO J

(^a,/) = 2tt^ r,(A )(<pSl,f) + c0 ax(ip.) {‘fiifi 1 /) . .. . 2 (2.28)

/,3i >0 lCW)l

for all A > 0. So, in particular, we pick up complementary series representations in s — s/(A).

Theorem 2.2. Let A > 0. If A > p/2, nx decomposes into a direct integral of principal series representations. If 0 < A < p/2 the spectrum of tc\ has a discrete part consisting of finitely many complementary series representations. The continuous part consists of principal series representations.

3. ASYMPTOTIC BEHAVIOUR OF THE CANONICAL REPRESENTATIONS

In this section we consider the asymptotic behaviour of 7Ta (or ipx) as A tends to infinity. Therefore we apply an alternative meaning of ipx- We refer to [10].

Consider on {.r £ Cn + 1 : [x, z] > 0} the Riemannian metric

This metric is invariant under x —> Ax (A G C, A ^ 0) and thus gives a Riemannian metric on X which is invariant under G. Corresponding to this metric we have a G-invariant second order differential operator, the Laplacian A, which we already met in section 2. Let ir be the map defined in (1.2). If / is a function of class C2 on X, we set / = / o ir, so that / is defined on the open set

{x G Cn+1 : [x,a;]>0} *

and satisfies /(Ax) — f(x) (A G C, A ^ 0). We have

A/ = [x, x]Af (3.2)

where A is the pseudo-Laplacian associated with the pseudo-Euclidean metric ds2 = — [dx,dx] on Cri+1. Consider also on the set {x G <Cn+1 : [x, x] > 0} the function Q defined by

«*> = {& (3-3)

Q satisfies Q(tx) = Q(x) (f G C, i / 0) and therefore Q = Q o 7r for some function Q on X. Q has the following properties:

• Q is invariant under K,

• Q is real analytic,

• Q(x) > i.

• Q has a non-degenerate critical point x° = eK; the Hessian of Q at x° has signature (2n, 0),

• Q(x) — t (t > 1) is a A'-orbit on X.

Let F be a complex-valued function on E of class C2. Then

A(FoQ) = {LF)oQ (3,1)

w'here L is the ordinary differential operator

L = a(t)-^ + b(t)jt

with a(t) = 4.1 (t — 1), b(t) = 4[(n + 1)< — 1]. This follows easily from (3.2).

Recall that V(X) is the space of complex-valued C°°-functions on X with compact support. Fix an invariant measure dx on X, corresponding to the Riemannian metric. If t is not a critical value of Q (so

t ^ 1), we can define the average M/(t) of a function / G T>(X) over the surface {Q(x) = t} by means of

the formula

' J F(Q(x))f(x)dx = J F(t) Mj(t) dt (3.5)

for any continuous function F on 1.

The function Mf has a singularity at the critical value t = 1 of Q. More precisely

Mj(t) = Y{t-l)(t-l)n-lip(t) (3.6)

with ip G X)(M). Here Y is the Heaviside function: Y(t) = 1 for t > 0, Y(t) = 0 for t < 0. Moreover

cf{x°) - <p( 1)

where c — 7rn/r(ri). Since Q is a A'-invariant function, we can associate with Q a G-invariant kernel I\q on X x X, with

r, _ x [a;i, a?2] [a?2, ^i] ^

I\q{xi,x2) - -r-------^------r (•$■<)

1*1, ®lj [X2,X2\

(zi> X2 G {x € Cn + 1 : [i,ar]>0}).

One easily verifies that this kernel corresponds on B(Crl) x B{Cn) to the kernel

[1 ~ (y,z)] [1 ~ jz,y)} q\

[i-IMI2][i-INI2] '

if aTl _> y, x2 — z'(y,z G B(Cn)). Therefore, V'a(s) = <2(<7)-A for all JGG. This interpretation of t/>a will be of great help in finding the asymptotic behaviour of ipx as A -> oo.

Consider the distribution

f J Qix)~X f(x)dx

(/ G V(X)). By (3.6) we get

j Q(x)-X f(x) dx = ^ t-x (t - I)"“1 p(i) dt.

Observe that this expression is an entire analytic function of A. For A: = 0,1,2,... one has

r°° \ <u 1 T(A — n — k) F(n + k)

/ ,-(«-!)”«-* = -!------------------^------------i,

for example for Re A > n + k. Write

<p(t) = y?(l) + {i ~ ^^(l) + "—2_^_^,(i) with \ip(t)\ < maxs |yj(2)(s)|, and consider the distribution T\ given by

r(A) c(x)-x _ r(A) >

r(A - n) r(n)c “ 7T- T(A - n) Q{) ’

for A —► oc. We get

c(Tx,f) = f(1) + + ¿W) + \v"W> n(n + + (3'9)

as A —> oo.

Let L' denote the transpose of L with respect to dt. We have MA* j = L' M} for all s G N. A computation yields _

L'[(t - i)"-i *,(*)] = (t- I)"'1 {[(4» + 4)t - 4]^'(<) + 4t(t - 1) *>"(/)}

and from this equation one can derive that

c&f(x°) = 4n<p\l), c&'2f(x°) = 16 n(n + 1) [^"(1) + ip'(l)].

It is an easy exercise to arrive at

V?'(l) = ^-A/(x-°), (3-10)

An

/'(1) = „ ,i.{¿V(J°) - 4(n + l)A/(*“)}. (3.11)

16n(n + 1)

Substituting (3.10) and (3.11) into (3.9) yields:

(Ta,/) = /(x0)+^A/(x°)

+ ^{A2/(x0) + 4(n + 1)A/(2'0)} + O(^) (A —> oc'). (3.12)

So Tx -* 6 as A — oc. In terms of Berezin quantization (cf. [1], where A = l/h with h denoting Planck’s constant) it means in particular that the correspondence principle is true.

It is clear that there is no obstruction in determining higher order terms of the asymptotic expansion,

due to our method.

4. TENSOR PRODUCTS OF HOLOMORPHIC AND ANTI-HOLOMORPHIC DISCRETE SERIES

4.1. The space L2(G/K,l)

Denote by xi (I G the character of K given by

' (4-1)

where [a | = 1, d G U(n), a det d — 1. Let pi = Ind^jG Xi and V\ the space of/?;. So / £ Vj if

(i) / : G —+ C is measurable,

(ii) f(gk) = Xi{k~l)f{g),

GO li/ll2 = IG/K\f(g)\Ml) < °°< where g = gK.

Here dji(]j) is the invariant measure on G/K ~ B(Cn), see (1.13). Instead of Vj one also uses the notation L2(G/K,l). We shall identify Vi with a space of functions on the unit ball B = B(Cn) in Cn.

Recall that G acts on B, by (1-7), K = Stab (o) and gK G G/K corresponds to g ■ o. Now' define

Af(g) = a1 f(g) (4.2)

for / G L2(G/K, I), g — . Then Af(gk) — Af(g) for all k G K. So Af is defined on B and one

has

ll/H2 = f \Af(z)\*(l-\\ztf)'dn(z),

JB

with dfi(z) as in (1.13). Let Hi denote the Hilbert space of all measurable functions v? on B such that

f l^(2)|2(l - \\z\\2y dfi{z) < oc. (4.3)

J B

7ii is a G-space; G acts unitarily in Hi by iri, given by

*i(gM2) = ^(¡T1 ■z) i(b’z) + a)~!

if g_1 = . A is a unitary intertwining operator between pi and 7r;.

4.2. The holbmorphic discrete series; Fock spaces

For A £ E consider the Fock space T\ of holomorphic functions on B satisfying

II/IIa := / l/(‘)l2(i-IWI2)Ad/i(z)<°°- (44)

Jb

This space is non-trivial for A > p (p = n), since contains the function which is identically 1 in this

CaS6' 0116 ^ in „2 _ »" (4.5)

'I llA - 2(A - 1) • ■ -(A - n)'

Moreover, Tx is a closed subspace of L2(B, dpx), hence a Hilbert space, where dpx(z) = (1 - |M|2)A dp(z). It also has a reproducing kernel, namely

R(.,.) = ^-1)^(A^li-(..-)l->- <«>

It is also a unitary module for the action of the universal covering group G of G; for integer A (A > p) it

is even a G-module: a holomorphic discrete series representation of scalar type. The group G c y

TO,(g)f{z) = f{^z^+—)i(b’z) + a) X’ ^4'7^

•T1

fa b Y Let us denote by Tx the space of complex conjugates of elements in TX- It consists of - \cdj ~

anti-holomorphic functions and gives rise to an obvious unitary action « of G as well. So

if g-i _ b^j f £ Jx. A € Z. For A G N (A > p) we get part of the anti-holomorphic discrete series.

4.3. Tensor products ^

Consider the Hilbert space tensor product Tx®2?\, with A > p. Ihe group 6 acts diagonally It

turns out that we actually have a G-action, which for integer A is given by

go ■ (/(*) £ ff(u0) = f(fhl ■ z) ® 9(90'1 • w) (a + (6- Z))-A (a + (b'wV (4-9)

li9*1= (c d)' ^ _

Let denote the bilinear map, defined on tensors in ^a®2^a by

f(z) S g(w) — f(z)g(z) (1 - ||z||2)A. (4-10)

Then, restricting ylA to polynomial functions, Aa is densely defined with image in HQ. Furthermore, according to J. Repka ([14], Proposition 4.1):

• Ax has trivial kernel and dense image,_____

• Aa intertwines the G-actions on Tx®2^x and ^o,

• Aa is closed.

Let = |Aa41|1/2 Ux be the polar decomposition of Ax. Then U\ is a unitary equivalence between

the G-spaces Tx®2^x and 'Ho ~ L2(G/A). IMJ|2 < i/c.

Actually Aa can be extended to a bounded operator from Tx®2^x mto H0 with \\M / a,

where cA = ||1||2a. Indeed, functions F(z, w), holomorphic in z, anti-holomorphic m w, such tha

í f \F(z,w)\2 dpx(z)djj.x(w)

Jb Jb

< oo,

are general elements in the Hilbert space Fx^Fx- Clearly for such functions F,

F(z,t)= f Ex(w,t)F(z,w)dfix(w), (4.11)

Jb

therefore,

Hence

So

AxF(z) = f Ex(w,z)F(z, w)dpx(w) ■ (1 - ||z||2)A.

Jb

\AxF(z)\2< f \Ex(w,z)\2 dfix(w) ■ i \F(z, w)\2 dpx(vj) (1 - ||z|| JB JB

\\AxF\\2 = / \AxF(z)\2d^z)<-\\F\\l Jb ca

4.4. The adjoint of A\. The Berezin kernel

Let F(z,w) be holomorphic in z, anti-holomorphic in w, and belonging to T\®2T\, or L2{B x B, dux <S> dfix)- Let h belong to L2(B, dp). We shall determine an explicit expression for A*x h. It is clear that Ax h is in Tx'&ïJ' x, so Ax h(z, w) is holomorphic in z and anti-holomorphic in w. One has

(A“x h, F) = (h, AXF) = [ f h{z)Ex{z,w)F(z,w){\ - \\z\\2)x dpx(w)dp(z).

Jb Jb

So A\ h is the projection of the function

(z, w) -+ h(z) Ex(z, w)

onto Tx®2?x- The above function is in L2(B x B,dp\ <g> dpx). The orthogonal projection, call it E, is given by

EF(z, w) = f [ Extw^^Exiz^^Fiz^w^dtixiz'Wxiw'). (4.12)

Jb Jb

Hence

A\h(z,w) = [ Ex{z,z') Ex(z',w)h(z')dfix{z').

Jb

Define for A > p and f,g£ "Ho = L2(B, dp):

(f,g)x = (AxA‘J,g). (4.13)

This form is clearly (strictly) positive-definite. More explicitly:

Ax A*xf(z) =

f Ex(z,z')Ex(z',z)f(z')dpx{z'){l - IM|2)A.

Jb

So Ax Ax is a kernel operator with kernel

5a(2 ^ _ c- j(i-iMia)(i-im\A. (4.H)

W>z) l[l-(z,^)][l-(^^)]J ^ J

This is again the Berezin kernel (up to a factor); it is G-invariant, positive-definite, and defines a bounded

Hermitian form on L2(G/K) for A > p. Notice that the Berezin kernel is given by

Ex{z,z')Ex{z',z) (415)

Ex(z,z)Ex{z',z')’

5. MAXIMAL DEGENERATE REPRESENTATIONS OF

SL(n + 1, C)

5.1. Definition of the representations

Let G = SU(l,n) and Gc = SL(n + 1,C), a complexification. Denote by Kc the subgroup

I\c = S(GL(1,C) x GL(n, C))

of Gc and set U = SU(n + 1), K = S(U(1) x U(n)). Let P± be the two (standard) maximal parabolic subgroups of Gc consisting of upper and lower block matrices respectively:

P+:(ê ») <51)

with a E C, ^ j £ A'c, i a row (column) vector in Cn. For /j £ C, define the character ui^ of by the formula:

wî*(p) = M".

where p G has one of the forms (5.1). Consider the representations of Gc induced from P±:

7rJ = Indu^. (5.2)

Let us describe these representations in the “compact picture”. One has the following decompositions:

G = UP+ = UP-, (5.3)

which we call Iwasawa type decompositions. For the corresponding decompositions g = up of an clement g G Gc, the factors p and u are defined up to an element of the subgroup U fli3"1" = UPiP~ = U H I\c ~ K. The coset spaces Gc/P^ can be identified with the coset space U/K. Set

S’ = {z G Cn+I : ||z||2 = 1},

which clearly can be identified with SU(n + l)/SU(n) via u —> ueo (u G SU(n + 1)).

Let us denote by V the vector space of C°°-functions ¡p on S satisfying

ip(Xs) = if(s) (5.4)

for all A G C with |A| = 1.

V can be seen as the representation space of both ?r+ and ir~. In fact tt+ = 7r~ o r where r is the Cartan involution of Gc: r(g) = (ÿ‘)

The group Gc acts on 5; denote by g ■ s (g G Gc, $ 6 S) the action of g on s:

d(s) /-

g - s = .. , .... (o.o)

IlffOOII

W e have for ip G V:

V (sMs) = v{g~l ■s) Ilff-'Wir- (5-6)

In a similar way we have:

rf(g)<p(s) = <Kr(g_1) •s) IKff-1)5!!''- (5J)

Let ( | ) denote the standard inner product on L2(S),

(ip)ii>) = J ip(s)yj(s) ds. (5.8)

Here ds is the normalized [/-invariant measure on S. This measure ds is transformed by the action of

g G Gc as follows:

ds = ||ÿ(s)||_2(n + 1)tfs, s = gs. (5.9)

It implies that the Hermitian form (5.8) is invariant with respect to the pairs

K.’r:?r-2(n + l)) and ’rijr-2(n + l))-

Therefore, if Re/i = — (n+1), then the representations 7rJ are unitarizable, the inner product being (5.8).

5.2. Intertwining operators and irreducibility

It is an interesting problem to determine the complex numbers ¡x such that 7r* is irreducible, and in case it is reducible, to obtain the composition series. We will not persue this problem here. We refer to [6] where a similar method is used. It turns out that is at least irreducible if fj, £ 7L. Let us turn to intertwining operators.

Define the operator A^ on V by the formula

AM*) = js\(s,t)\-»~2{n + l)v{t)dt. (5.10)

This integral converges absolutely for Re/i < —‘in — 1 and can be analytically extended to the whole /i-plane as a mereomorphic function. It is easily checked that AM is an intertwining operator

= ^,(g)Afi, g eGc, (5.11)

with y! = — (i — 2(n -f 1).

The operator A_/J_2(n+i) ° A^ intertwines with itself and is therefore a scalar c(/z), independent of the ±-sign. In general c(/z) will be a meromorphic function of fi. It can be computed using A'-types, see e.g. [6], §1. It turns out that

c(n) = c(-n - 2(n + 1)).

It. turns out, in addition, that only the tt* with Re/i = — (n + 1) are unitarizable (see again [6], §1).

5.3. Restriction to G

Consider the diagonal matrix J = diag {1, —1, ..., —1}. Then

G={geGc : g* =Jg-lJ}. (5.12)

So the Cartan involution r of Gc restricted to G is given by r(g) = JgJ (g 6 G). Consequently, 7r+ is

equivalent to tt~ on G: the equivalence is given by ip —>■ E<p with

E<p{s) = tp(Js) (5.13)

for if £ V.

Now consider the action of G on S given by (5.5). There are 3 orbits, given by

[s,s]>0, [s, s] = 0 and [s,s]<0. (5-14)

All three orbits are invariant under s —> A s with A £ C, |A| = 1. Call O1.Ov.O3 the corresponding

G-orbits on S/ ~ where s ~ s' if and only if s = A s' for some A £ C, |Aj = 1. Then we have:

Oi ~ G/K via g —* g ■ e0, (5.15)

Oi^G/MAN via g —> g ■ (e0 + e„), (-5.16)

03 ~ G/S(U(l,n- 1) x U(l)) via g-^g-en. (5.17)

For the subgroup MAN (a minimal parabolic subgroup of 6') see section 1.2.

Let ip be a C^-function with compact support on [s,s] ^ 0, satisfying (5.4). Set

li>(S) = *=>(*) Mr1/2/1- (5.18)

Then ip satifies the same condition (5.4). Moreover,

Tj)(g~l ■ s) = <p(g-1 ■ s) Hff-'OOII'1 |[s, s]|-1/2/1

= K (g)<p(s) ¡[s. s]|“1/2/J. -

So the linear map tp —► i/> intertwines the restriction of 7r~ to G with the left regular representation of G on V(G/K) and V(G/H) respectively where H = S(U(1, n — 1) x U(l)).

A G-invariant measure on 0\ U O3 is given by

MA = (51S) So, if we provide V{S) with the inner product on [s, s] 0 given by

(¥>1-^2)= / iPi(s) ^2(s)|[s,s]rRefI-(n+1)ds, (5.20)

Js

then 7T~ becomes unitary, if we restrict it to G. From now on we shall only consider the restriction of to G and call it R^. Clearly R^ is equivalent to R^. The intertwinig operator becomes

AMS) = i \[s,t]\-^n+l^(t)dt. (5.21)

Js

Observe that AM is defined on [s, s] > 0 for all fi, provided ip has compact support in this open set. Then A^p is a C°°-function on this set (non-necessarily with compact support). On [s,s] < 0 one still has to deal with analytic continuation in ¡1. since convergence of the integral is not garantueed for all ¡1.

For <£>1, ip2 Ç. V and /i G M, consider the Hermitian form

{iPuAilf2)= J J |[s,i]|_'"'2(n+1Vi(s)^2(<)dsdi- (5.22)

This form is clearly invariant with respect to R^. Applying the linear transformation (5.18) on the open set [.s, s] ^ 0, we get the following:

(01, ^2) — Vi(s)ip2(t) Js Js

s,s] [t,t]

¡M] [t,s\

f +(n + l)

du(s)du(t). (5.23)

Now restrict to [s, s] > 0. Then

[s.slW

MM

is the Berezin kernel on V(G/K), see (3.7).

NOTES

• Canonical representations have been introduced for classical Hermitian symmetric spaces by Berezin

and later, in a different context, by Vershik, Gel’fand and Graev for SL(2,IR) (see [1], [16]).

• A more conceptual treatment for Hermitian symmetric spaces in the context of Jordan algebras has recently been given by Upmeier and Unterberger [15].

• An extension to hyperbolic spaces, also for small values of the parameters, and for line bundles

over these spaces, is due to Ilille and van Dijk [3], [4], [12].

• Canonical representations for para-Hermitian spaces were proposed and introduced by Molchanov

[13].

• A thorough treatment of the rank one para-IIermitian space SL(n,lR)/GL(n — 1,K) is due to van Dijk and Molchanov [5], [6].

The generalization to para-Hermitian spaces follows the scheme of section 5., which was proposed by Molchanov.

REFERENCES

[ 1] Berezin, F.A. (1973): Quantization in complex bounded domains. Dokl. Akad. Nauk SSSR 211, 1263-1266. Engl, transl.: Sov. Math., Dokl. 14, 1209-1213 (1973)

[ 2] Dijk, G. van (1994): Group representations on spaces of distributions. Russian J. Math. Physics

2, No. 1, 57-68

[ 3] Dijk, G. van, Hille, S.C. (1995): Canonical representations related to hyperbolic spaces. Report no. 3, Institut Mittag Leffler 1995/96

[ 4] Dijk, G. van, Hille, S.C. (1996): Maximal degenerate representations, Berezin kernels and canonical representations. Report W 96-04, Leiden University

[ 5] Dijk, G. van, Molchanov, V.F. (1997): The Berezin form for the space SL(n,M)/ GL(n - 1,E). Preprint Leiden University

[ 6] Dijk, G. van, Molchanov, V.F. (1997): Tensor products of maximal degenerate series representations of the group SL(n, R). Preprint Leiden University

[ 7] Erdelyi, A. et al. (1954): Tables of Integral Transforms, volume II. McGraw-Hill, New York [ 8] Erdelyi, A. et al. (1953): Higher Transcental Functions, volume I. McGraw-Hill, New York [ 9] Faraut, J. (1982): Analyse harmonique sur les paires de Guelfand et les espaces hyperboliqu- -In: “Analyse Harmonique”, Les Cours de CIMPA, 315-446

[10] Faraut, J. (1979): Distributions sphériques sur les espaces hyperboliques. J. Math. Pures • : Appl. 58, 369-444

[11] Flensted-Jensen, M., Koornwinder, T.H. (1979): Positive-definite spherical functions on a nc:.-compact rank one symmetric space. Springer Lecture Notes in Math. 739, 249-282

[12] Hille S.C. (1996): Canonical representations associated to a character on a Hermitian symmetr: space of rank one. Preprint Leiden University

[13] Molchanov, V.F. (1996): Quantization on para-Hermitian symmetric spaces. Adv. in Mali. Sci., AMS Transi., Ser. 2, 175, 81-96

[14] Repka, J. (1979): Tensor products of holomorphic discrete series representations. Can. J. Mali. 31, 836-844

[15] Unterberger A., Upmeier, H. (1994): The Berezin transform and invariant differential operators Comm. Math. Physics 16/,, No. 3, 563-597

[16] Vershik, A.M., Gel’fand, I.M., Graev, M.I. (1973): Representations of the group SL(2, R) when. R is a ring of functions. Usp. Mat. Nauk 28, No. 5, 82-128. Engl, transi.: Russ. Math. Surv. 28. .Y_ 5, 87-132 (1973)