Note on non-discrete complex hyperbolic triangle groups of type (n, n, [infinity]; k) II.

1. Introduction. Let n and k be integers greater than 2. Let [I.sub.1],[I.sub.2],[I.sub.3] be the following matrices:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

Assume that [rho], [sigma], [tau] satisfy the conditions [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] (the identity matrix) and [I.sub.2][I.sub.3] is a unipotent element. We call the group generated by [I.sub.1], [I.sub.2] and [I.sub.3] a complex hyperbolic triangle group of type (n,n, [infinity]; k) and denote it by [GAMMA](n,n, [infinity]; k). Up to conjugation, there is a one-parameter family of these groups parametrized by k.

It is interesting to ask which values of the parameter correspond to discrete groups as mentioned in [12].

The purpose of this paper is to show the following theorem, which improves our previous result in [5] and gives a new list of non-discrete groups of type (n, n, [infinity]; k).

Theorem 1. Let [GAMMA] = ([I.sub.1],[I.sub.2],[I.sub.3]) be a complex hyperbolic triangle group of type (n, n, [infinity]; k) with k [greater than or equal to] [n/2] + 1. The following groups are non-discrete:

(1) [GAMMA](5,5, [infinity]; 3).

(2) [GAMMA](6,6, [infinity]; 5).

(3) [GAMMA](7,7, [infinity]; 4), [GAMMA] (7,7, [infinity]; 5), [GAMMA](7,7, [infinity]; 6).

(4) [GAMMA](8, 8, [infinity]; 5), [GAMMA](8, 8,1;7).

(5) [GAMMA](9, 9, [infinity]; k) for 5 [less than or equal to] k [less than or equal to] 8.

(6) [GAMMA](10,10, [infinity]; k) for 6 [less than or equal to] k [less than or equal to] 9.

(7) [GAMMA](11,11, [infinity]; k) for 6 [less than or equal to] k [less than or equal to] 11.

(8) [GAMMA](12,12, [infinity]; k) for 7 [less than or equal to] k [less than or equal to] 16.

(9) [GAMMA](13,13, [infinity]; k) for 7 [less than or equal to] k [less than or equal to] 38.

(10) [GAMMA](14,14, [infinity]; k) for k [greater than or equal to] 8.

(11) [GAMMA](n, n, [infinity]; k) for any n (> 15).

In [12] Schwartz classified complex hyperbolic triangle groups into two types. It is said that [GAMMA](n, n, 1) has type B if there is a positive number [k.sub.0] such that [I.sub.1][I.sub.2][I.sub.3] becomes regular elliptic for k > [k.sub.0]. If n [greater than or equal to] 14, then [GAMMA](n, n, [infinity]) has type B. Thus we have:

Corollary 2. If [GAMMA](n,n, [infinity]) has type B, then [GAMMA](n, n, [infinity]; k) is not discrete.

Details for background material on complex hyperbolic space will be found in [2]. For material on complex hyperbolic triangle groups see [3], [6], [7], [9], [12] and [13].

2. Proof of Theorem 1. To show a group of type (n,n, [infinity]; k) to be non-discrete, we find regular elliptic elements of infinite order.

Lemma 1. Let g be an element of [GAMMA](n, n, [infinity]; k). If trace(g) is real and contained in (-1,3), then g is regular elliptic and trace(g) = 1 + 2 cos [phi][pi]. Moreover, g has finite order if and only if [phi] is a rational number.

In our previous papers [5,8] we used the result by Conway and Jones in [1]. Parker extended their results as follows (see [9] and [10]):

Lemma 2 ([10; Theorem A.1.1]). Suppose that we have at most six distinct rational multiples of [pi] lying strictly between 0 and [pi]/2, for which some rational linear combination of their cosines is zero but no proper subset has this property, then the appropriate linear combination is propositional to one of the following:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]

We consider the element [I.sub.3][I.sub.1][I.sub.3][I.sub.1][I.sub.2][I.sub.1] in [GAMMA](n, n, [infinity]; k), which is denoted by [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]. It is seen that

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

We note that trace [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] is real. It can be shown that the element [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] is unipotent in [GAMMA](3,3, [infinity]; k) for any k [greater than or equal to] 4. Also we see that for n [greater than or equal to] 4 the element [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] is regular elliptic in [GAMMA](n, n, [infinity]; k) with k [less than or equal to] n - 1 and it is unipotent in [GAMMA](n, n, [infinity]; n).

We are ready to prove our theorem.

Proof of Theorem 1. First we consider the group [GAMMA](21, 21, [infinity]; 18). From the above, the element [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] is regular elliptic in [GAMMA](21, 21, [infinity]; 18). Lemma 1 implies that [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] has finite order if there is a rational number [phi] satisfying

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII],

that is,

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

Lemma 2 lists all possible trigonometric Diophantine equations with up to six. We use this result to conclude that there are no rational numbers [phi] satisfying the equation above. It follows from Lemma 1 that [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] has infinite order in the group [GAMMA](21,21,[infinity];18), which implies that [GAMMA](21, 21,[infinity]; 18) is not discrete.

In the same manner as above, we show that there are no rational numbers [phi] satisfying the following each equation:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

These correspond to the equations trace [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] in the following groups, respectively:

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

It follows that in each group above, [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] is a regular elliptic element with infinite order. Thus the groups above are not discrete. Together with Theorem 2.3 in [5], we obtain our theorem.

Remark 3. (1) In [11] Parker, Wang and Xie showed that [GAMMA](3, 3,[infinity]; k) is discrete for k [greater than or equal to] 4. (2) In [GAMMA](13,13,[infinity];8), [GAMMA](11,11,[infinity];7), [GAMMA](7, 7,[infinity];5) among the groups above, [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] is also regular elliptic.

(3) In [GAMMA](8, 8,[infinity];6), [MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII] is a regular elliptic element of order 6.

References

[1] J. H. Conway and A. J. Jones, Trigonometric Diophantine equations (On vanishing sums of roots of unity), Acta Arith. 30 (1976), no. 3, 229-240.

[2] W. M. Goldman, Complex hyperbolic geometry, Oxford Mathematical Monographs, The Clarendon Press, Oxford University Press, New York, 1999.

[3] W. M. Goldman and J. R. Parker, Complex hyperbolic ideal triangle groups, J. Reine Angew. Math. 425 (1992), 71-86.

[4] S. Kamiya, Remarks on complex hyperbolic triangle groups, in Complex analysis and its applications (Osaka, 2007), 219 223, OCAMI Stud., 2, Osaka Munic. Univ. Press, Osaka, 2007.

[5] S. Kamiya, Note on non-discrete complex hyperbolic triangle groups of type (n,n, [infinity]; k), Proc. Japan Acad. Ser. A Math. Sci. 89 (2013), no. 8, 100-102.

[6] S. Kamiya, Complex hyperbolic triangle groups of type (n, n, 1), Math. Newsl. 24 (2014), no. 4, 97-103.

[7] S. Kamiya, J. R. Parker and J. M. Thompson, Notes on complex hyperbolic triangle groups, Conform. Geom. Dyn. 14 (2010), 202-218.

[8] S. Kamiya, J. R. Parker and J. M. Thompson, Non-discrete complex hyperbolic triangle groups of type (n, n, [infinity]; k), Canad. Math. Bull. 55 (2012), no. 2, 329-338.

[9] A. Monaghan, Complex hyperbolic triangle groups, ProQuest LLC, Ann Arbor, MI, 2013.

[10] J. R. Parker, 2-Generator Mobius Groups, Ph.D. thesis, University of Cambridge, 1989.

[11] J. R. Parker, J. Wang and B. Xie, Complex hyperbolic (3,3, n) triangle groups, Pacific J. Math. 280 (2016), no. 2, 433-453.

[12] R. E. Schwartz, Complex hyperbolic triangle groups, in Proceedings of the International Congress of Mathematicians, Vol. II (Beijing, 2002), 339-349, Higher Ed. Press, Beijing, 2002.

[13] J. Wyss-Gallifent, Complex Hyperbolic Triangle Groups, Ph.D. thesis, University of Maryland, 2000.

By Shigeyasu KAMIYA

Osaka City University Advanced Mathematical Institute, 3-3-138 Sugimoto, Sumiyoshi-ku, Osaka 558-8585, Japan

(Communicated by Kenji FUKAYA, M.J.A., June 13, 2017)

2010 Mathematics Subject Classification. Primary 22E40, 32Q45, 51M10.

doi: 10.3792/pjaa.93.67