Trends in Mathematics

Trends in Mathematics is a series devoted to the publication of volumes arising from con-
ferences and lecture series focusing on a particular topic from any area of mathematics.
Its aim is to make current developments available to the community as rapidly as possible
without compromise to quality and to archive these for reference.

Proposals for volumes can be sent to the Mathematics Editor at either

Springer Basel AG
Birkhäuser
P.O. Box 133
CH-4010 Basel
Switzerland

or

Birkhauser Boston
233 Spring Street
New York, NY 10013
USA

Material submitted for publication must be screened and prepared as follows:

All contributions should undergo a reviewing process similar to that carried out by journals
and be checked for correct use of language which, as a rule, is English. Articles without
proofs, or which do not contain any significantly new results, should be rejected. High
quality survey papers, however, are welcome.

We expect the organizers to deliver manuscripts in a form that is essentially ready for
direct reproduction. Any version of TeX is acceptable, but the entire collection of files must
be in one particular dialect of TeX and unified according to simple instructions available
from Birkhäuser.

Furthermore, in order to guarantee the timely appearance of the proceedings it is essential

that the final version of the entire material be submitted no later than one year after the
conference. The total number of pages should not exceed 350. The first-mentioned author
of each article will receive 25 free offprints. To the participants of the congress the book
will be offered at a special rate.
Complex Analysis
Several Complex Variables and
Connections with PDE Theory
and Geometry

Peter Ebenfelt
Norbert Hungerbühler
Joseph J. Kohn
Ngaiming Mok
Emil J. Straube
Editors

Birkhäuser
Editors:

Peter Ebenfelt Ngaiming Mok

Department of Mathematics Department of Mathematics
University of California, San Diego (UCSD) The University of Hong Kong
9500 Gilman Drive # 0112 Pokfulam Road
La Jolla, CA 92093-0112, USA Hong Kong SAR, China
e-mail: [email protected] e-mail: [email protected]

Norbert Hungerbühler Emil J. Straube

Department of Mathematics Department of Mathematics
University of Fribourg Texas A&M University
Chemin du musée 23 College Station, TX 77843, USA
1700 Fribourg, Switzerland e-mail: [email protected]
e-mail: [email protected]

Joseph J. Kohn
Department of Mathematics
Fine Hall, Washington Road
Princeton, NJ 08544-1000, USA
[email protected]

2000 Mathematics Subject Classification 32-06

Library of Congress Control Number: 2010926412

Bibliographic information published by Die Deutsche Bibliothek. Die Deutsche Bibliothek lists
this publication in the Deutsche Nationalbibliografie; detailed bibliographic data is available in
the Internet at http://dnb.ddb.de

ISBN 978-3-0346-0008-8

This work is subject to copyright. All rights are reserved, whether the whole or part of the
material is concerned, specifically the rights of translation, reprinting, re-use of illustrations,
recitation, broadcasting, reproduction on microfilms or in other ways, and storage in data
banks. For any kind of use permission of the copyright owner must be obtained.

© 2010 Springer Basel AG

P.O. Box 133, CH-4010 Basel, Switzerland
Part of Springer Science+Business Media
Printed on acid-free paper produced from chlorine-free pulp. TCF ∞
Cover Design: Alexander Faust, Basel, Switzerland
Printed in Germany

ISBN 978-3-0346-0008-8 e-ISBN 978-3-0346-0009-5

987654321 www.birkhauser.ch
Contents
N. Hungerbühler
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Extended Curriculum Vitae of Linda Preiss Rothschild . . . . . . . . . . . . . . . . . . . . xi

Publication List of Linda Preiss Rothschild . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

D. Barlet and H.-M. Maire 

Oblique Polar Lines of X |f |2λ |g|2μ  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

S. Berhanu
On Involutive Systems of First-order Nonlinear Partial
Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

A. Bove, M. Mughetti and D.S. Tartakoff

Gevrey Hypoellipticity for an Interesting Variant
of Kohn’s Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

D.W. Catlin and J.P. D’Angelo

Subelliptic Estimates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

J.P. D’Angelo
Invariant CR Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

M. Derridj and B. Helffer

On the Subellipticity of Some Hypoelliptic Quasihomogeneous
Systems of Complex Vector Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

F. Forstnerič
Invariance of the Parametric Oka Property . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

S. Fu
¯
Positivity of the ∂-Neumann Laplacian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

K. Gansberger and F. Haslinger

Compactness Estimates for the ∂-Neumann Problem
in Weighted L2 -spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
vi Contents

P. Guan
Remarks on the Homogeneous Complex Monge-Ampère Equation . . . . 175

J. Hounie
A Radó Theorem for Locally Solvable Structures of Co-rank One . . . . 187

F. Lárusson
Applications of a Parametric Oka Principle for Liftings . . . . . . . . . . . . . . 205

Ch. Laurent-Thiébaut
Stability of the Vanishing of the ∂ b -cohomology Under
Small Horizontal Perturbations of the CR Structure
in Compact Abstract q-concave CR Manifolds . . . . . . . . . . . . . . . . . . . . . . . 213

L. Lempert
Coherent Sheaves and Cohesive Sheaves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

G.A. Mendoza
Characteristic Classes of the Boundary of a Complex b-manifold . . . . . 245

A. Meziani
Solvability of Planar Complex Vector Fields
with Applications to Deformation of Surfaces . . . . . . . . . . . . . . . . . . . . . . . . 263

N. Mok
On the Zariski Closure of a Germ of Totally Geodesic
Complex Submanifold on a Subvariety of a Complex
Hyperbolic Space Form of Finite Volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279

L. Ni
The Large Time Asymptotics of the Entropy . . . . . . . . . . . . . . . . . . . . . . . . . 301

M.-C. Shaw
The Closed Range Property for ∂ on Domains
with Pseudoconcave Boundary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307

D. Zaitsev
New Normal Forms for Levi-nondegenerate Hypersurfaces . . . . . . . . . . . . 321
Complex Analysis
Trends in Mathematics, vii–x

c 2010 Springer Basel AG

Preface
Norbert Hungerbühler

The idea to organize a conference in honour of Linda Rothschild emerged in 2006.

This idea began to substantiate in 2007 when the Swiss Mathematical Society as-
signed the traditional Spring Meeting to the University of Fribourg. An organizing
committee was quickly formed:

Organizing committee
Norbert Hungerbühler University of Fribourg, Switzerland
Frank Kutzschebauch University of Berne, Switzerland
Bernhard Lamel University of Vienna, Austria
Francine Meylan University of Fribourg, Switzerland
Nordine Mir Université de Rouen, France

In order to ensure a high-quality conference program, the search for a scien-

tific committee began. Soon after, a distinguished group was found who started
working right away:

Scientific committee
Peter Ebenfelt University of California, San Diego, USA
Franc Forstnerič University of Ljubljana, Slovenia
Joseph J. Kohn Princeton University, USA
Emil J. Straube Texas A&M University, USA

Spring Meeting of the Swiss Mathematical Society

Conference on Complex Analysis 2008
Several Complex Variables and Connections with PDEs and Geometry
In honour of Linda Rothschild, Fribourg, July 7–11
viii N. Hungerbühler

Only a little while later it became clear that the subject and the top-class
speakers who agreed to participate in the conference called for a proceedings vol-
ume to make the presented results available shortly after the conference. This
project was carried out under the direction of the editorial board:

Editorial board
Peter Ebenfelt University of California, San Diego, USA
Norbert Hungerbühler University of Fribourg, Switzerland
Joseph J. Kohn Princeton University, USA
Ngaiming Mok The University of Hong Kong
Emil J. Straube Texas A&M University, USA

Focus on youth
The aim of the conference was to gather worldwide leading scientists, and to offer
the occasion to PhD students and postdocs to come into contact with them. The
committees explicitly encouraged young scientists, doctoral students and postdocs
to initiate scientific contact and to aim at an academic career. The topic of the
conference was apparently very attractive for young scientists, and the event an
ideal platform to promote national and international doctoral students and post-
docs. This aspect became manifest in a poster session where junior researchers
presented their results.
The conference was intended to have a strong component in instruction of
PhD students: Three mini courses with introductory character were held by Pengfei
Guan, Mei-Chi Shaw and Ngaiming Mok. These three mini courses have been very
well received by a large audience and were framed by the series of plenary lectures
presenting newest results and techniques.
The participation of junior female researchers, PhD students and mathemati-
cians from developing countries has been encouraged in addition by offering grants
for traveling and accommodation.

The subject
The conference Complex Analysis 2008 has been devoted to the subject of Several
Complex Variables and Connections with PDEs and Geometry. These three main
subject areas of the conference have shown their deep relations, and how techniques
from each of these fields can influence the others. The conference has stimulated
further interaction between these areas.
The conference was held in honor of Prof. Linda Rothschild who is one of the
most influential contributors of the subject during the last decades. A particular
aim was to encourage female students to pursue an academic career. In fact, female
mathematicians have been well represented among the speakers, in the organizing
committee and in the poster sessions.
 Preface ix

Several Complex Variables is a beautiful example of a field requiring a wide

range of techniques coming from diverse areas in Mathematics. In the last decades,
many major breakthroughs depended in particular on methods coming from Par-
tial Differential Equations and Differential and Algebraic Geometry. In turn, Sev-
eral Complex Variables provided results and insights which have been of fundamen-
tal importance to these fields. This is in particular exemplified by the subject of
Cauchy-Riemann geometry, which concerns itself both with the tangential Cauchy-
Riemann equations and the unique mixture of real and complex geometry that real
objects in a complex space enjoy. CR geometry blends techniques from algebraic
geometry, contact geometry, complex analysis and PDEs; as a unique meeting
point for some of these subjects, it shows evidence of the possible synergies of a
fusion of the techniques from these fields.
The interplay between PDE and Complex Analysis has its roots in Hans
Lewy’s famous example of a locally non solvable PDE. More recent work on PDE
has been similarly inspired by examples from CR geometry. The application of
analytic techniques in algebraic geometry has a long history; especially in recent
¯
years, the analysis of the ∂-operator has been a crucial tool in this field. The
¯
∂-operator remains one of the most important examples of a partial differential
operator for which regularity of solutions under boundary constraints have been
extensively studied. In that respect, CR geometry as well as algebraic geometry
have helped to understand the subtle aspects of the problem, which is still at the
heart of current research.
Summarizing, our conference has brought together leading researchers at the
intersection of these fields, and offered a platform to discuss the most recent de-
velopments and to encourage further interactions between these mathematicians.
It was also a unique opportunity for younger people to get acquainted with the
current research problems of these areas.

Organization
The conference was at the same time the 2008 Spring Meeting of the Swiss Math-
ematical Society. The event has profited from the organizational structures of the
SMS and the embedding in the mathematical community of Switzerland. The Uni-
versity of Fribourg has proven to be the appropriate place for this international
event because of its tradition in Complex Analysis, the central geographic location,
and its adequate infrastructure. In turn, its reputation and that of the region has
benefited from this conference.
The conference has been announced internationally in the most important
conference calendars and in several journals. Moreover, the event has been adver-
tised by posters in numerous mathematics institutes worldwide, by e-mails and in
the regular announcements of the Swiss Mathematical Society.
x N. Hungerbühler

Acknowledgment
It becomes increasingly difficult to find sponsors for conferences of the given size,
in particular in mathematics. We are all the more grateful to our sponsors who
have generously supported the conference, and the proceedings volume in hand:

List of Sponsors

• Ciba
• Roche
• Merck Serono
• Novartis
• Syngenta
• Swiss Mathematical Society
• Troisième cycle romand de mathématiques
• Swiss Academy of Sciences
• Stiftung zur Förderung der mathematischen Wissenschaften in der Schweiz
• Centre Interfacultaire Bernoulli CIB, EPFL
• Department of Mathematics, University of Fribourg
• Faculty of Sciences, University of Fribourg
• Rectorate, University of Fribourg
• Swiss National Science Foundation
• Walter Haefner Stiftung
• Swiss Doctoral Program in Mathematics

In the name of the conference committees and of all participants, we would

like to thank all sponsors – foundations, institutions and companies – very cordially
for their contributions and the shown appreciation for our work as mathematicians:
Thank you!
We also thank the team of Dr. Thomas Hempfling of the Birkhäuser pub-
lishing company for their help and professional expertise during the production
process of these proceedings.
Finally, we would like to thank Elisabeth François and Claudia Kolly who
assumed the secretariat of the conference.

Fribourg, August 2009 Norbert Hungerbühler

Complex Analysis
Trends in Mathematics, xi–xx

c 2010 Springer Basel AG

Extended Curriculum Vitae of

Linda Preiss Rothschild

Linda Rothschild was born February 28, 1945, in Philadel-

phia, PA. She received her undergraduate degree, magna
cum laude, from the University of Pennsylvania in 1966 and
her PhD in mathematics from MIT in 1970. Her PhD the-
sis was “On the Adjoint Action of a Real Semisimple Lie
Group”. She held positions at Tufts University, Columbia
University, the Institute for Advanced Study, and Prince-
ton University before being appointed an associate professor
of mathematics at the University of Wisconsin-Madison in
1976. She was promoted to full professor in 1979. Since 1983 she has been professor
of mathematics at the University of California at San Diego, where she is now a
Distinguished Professor.
Rothschild has worked in the areas of Lie groups, partial differential equa-
tions and harmonic analysis, and the analytic and geometric aspects of several
complex variables. She has published over 80 papers in these areas. Rothschild
was awarded an Alfred P. Sloan Fellowship in 1976. In 2003 she won the Ste-
fan Bergman Prize from the American Mathematical Society (jointly with Salah
Baouendi). The citation read in part:
“The Bergman Prize was awarded to Professors Salah Baouendi and
Linda Rothschild for their joint and individual work in complex anal-
ysis. In addition to many important contributions to complex analysis
they have also done first rate work in the theory of partial differential
equations. Their recent work is centered on the study of CR manifolds to
which they and their collaborators have made fundamental contributions.
Rothschild, in a joint paper with E. Stein, introduced Lie group meth-
ods to prove Lp and Hölder estimates for the sum of squares operators
as well as the boundary Kohn Laplacian for real hypersurfaces. In later
joint work with L. Corwin and B. Helfer, she proved analytic hypoellip-
ticity for a class of first-order systems. She also proved the existence of
a family of weakly pseudoconvex hypersurfaces for which the boundary
Kohn Laplacian is hypoelliptic but does not satisfy maximal L2 esti-
mates.”
xii Extended Curriculum Vitae of Linda Preiss Rothschild

In 2005, Rothschild was elected a Fellow of the American Academy of Arts

and Sciences, and in 2006 she was an invited speaker at the International Congress
of Mathematics in Madrid.
Rothschild served as President of the Association for Women in Mathematics
from 1983 to 1985 and as Vice-President of the American Mathematical Society
from 1985 to 1987. She served on the editorial committees of the Transactions of
the AMS and Contemporary Mathematics. She is also an editorial board member
of Communications in Partial Differential Equations and co-founder and co-editor-
in-chief of Mathematical Research Letters. She has served on many professional
committees, including several AMS committees, NSF panels, and an organization
committee for the Special Year in Several Complex Variables at the Mathematical
Sciences Research Institute. She presented the 1997 Emmy Noether Lecture for
the AWM. Rothschild has a keen interest in encouraging young women who want
to study mathematics. A few years ago she helped establish a scholarship for
unusually talented junior high school girls to accelerate their mathematical training
by participating in a summer program.

Educational Background
B.A. University of Pennsylvania, 1966
Ph.D. in mathematics, Massachusetts Institute of Technology, 1970
Dissertation: On the Adjoint Action of a Real Semisimple Lie Group
Advisor: Isadore Manual Singer

Professional Employment
1982– Professor, University of California, San Diego
2001–05 Vice Chair for Graduate Affairs, Mathematics Dept., UCSD
1979–82 Professor, University of Wisconsin
1981–82 Member, Institute for Advanced Study
1978 Member, Institute for Advanced Study
1976–77 Associate Professor, University of Wisconsin
1975–76 Visiting Assistant Professor, Princeton University
1974–75 Member, Institute for Advanced Study
1972–74 Ritt Assistant Professor, Columbia University
1970–72 Assistant Professor, Tufts University
1970–72 Research Staff, Artificial Intelligence Laboratory, M.I.T.

Honors and Fellowships

2005 Fellow, American Academy of Arts and Sciences
2003 Stefan Bergman Prize
1976–80 Alfred P. Sloan Foundation Fellow
1966–70 National Science Foundation Graduate Fellow
 Extended Curriculum Vitae of Linda Preiss Rothschild xiii

Selected Invited Lectures

• Invited address, International Congress of Mathematicians, Madrid, August
2006
• “Frontiers in Mathematics” Lecturer, Texas A&M University, September
1999
• Invited hour speaker, Sectional joint meeting of American Mathematical So-
ciety and Mathematical Association of America, Claremont, October 1997
• Emmy Noether Lecturer (Association for Women in Mathematics), Annual
Joint Mathematics Meetings, San Diego January 1997
• Invited hour lecturer, Annual Joint Mathematics Meetings, Orlando, January
1996
• Invited hour speaker, Annual Summer meeting of American Mathematics
Society, Pittsburgh, August1981

Students
Mark Marson University of California, San Diego, 1990
Joseph Nowak University of California, San Diego, 1994
John Eggers University of California, San Diego, 1995
Bernhard Lamel University of California, San Diego, 2000
Slobodan Kojcinovic University of California, San Diego, 2001
Robert Kowalski University of California, San Diego, 2002

Selected National Committees and Offices

National Science Foundation, Mathematics Division

• Advisory Panel, 1984–87 and other panels 1997–99, 2004

American Mathematical Society (AMS)

• Bocher Prize Committee 2001–04

• National Program Committee 1997–2000
Chair 1998–1999
• Nominating Committee, 1982–84, 1994–96
• Committee on Science Policy, 1979–82, 92–9
• AMS Vice President, 1985–87
• Committee on Committees, 1977–79, 1979–81
• Executive Committee, 1978–80
• Council of the AMS, 1977–80
xiv Extended Curriculum Vitae of Linda Preiss Rothschild

Association for Women in Mathematics (AWM)

• Noether Lecture Committee 1988–90, 1994–1997
Chair 1989–90
• Schafer Prize Committee 1993–94
• AWM President, 1983–85.
Mathematical Association of America
• Chauvenet Prize Committee, 1998–2000
Mathematical Sciences Research Institute
• Board of Trustees, 1996–1999
• Budget Committee 1996–1998
California Science Museum
• Jury to select California Scientist of the Year Award, 1995–1999
Institute for Pure and Applied Mathematics (IPAM)
• Board of Trustees, 2002–2005

Editorial Positions
• Co-Editor-in-Chief, Mathematical Research Letters, 1994–
• Editorial Board, Journal of Mathematical Analysis and Applications, 2001–
• Editorial Board, Communications in Partial Differential Equations, 1984–
• Editorial Board, Contemporary Mathematics, 1990–1994
• Editor for complex and harmonic analysis, Transactions of the American
Mathematical Society, 1983–1986
Publication List of Linda Preiss Rothschild
[1] Peter Ebenfelt and Linda P. Rothschild. New invariants of CR manifolds and a
criterion for finite mappings to be diffeomorphic. Complex Var. Elliptic Equ., 54(3-
4):409–423, 2009. ISSN 1747-6933.
[2] M.S. Baouendi, Peter Ebenfelt, and Linda P. Rothschild. Transversality of holo-
morphic mappings between real hypersurfaces in different dimensions. Comm. Anal.
Geom., 15(3):589–611, 2007. ISSN 1019-8385.
[3] Peter Ebenfelt and Linda P. Rothschild. Analyticity of smooth CR mappings of
generic submanifolds. Asian J. Math., 11(2):305–318, 2007. ISSN 1093-6106.
[4] Peter Ebenfelt and Linda P. Rothschild. Images of real submanifolds under finite
holomorphic mappings. Comm. Anal. Geom., 15(3):491–507, 2007. ISSN 1019-8385.
[5] M.S. Baouendi, Peter Ebenfelt, and Linda P. Rothschild. Projection on Segre vari-
eties and determination of holomorphic mappings between real submanifolds. Sci.
China Ser. A, 49(11):1611–1624, 2006. ISSN 1006-9283.
[6] Peter Ebenfelt and Linda P. Rothschild. Transversality of CR mappings. Amer. J.
Math., 128(5):1313–1343, 2006. ISSN 0002-9327.
[7] Linda Preiss Rothschild. Iterated Segre mappings of real submanifolds in complex
space and applications. In International Congress of Mathematicians. Vol. II, pages
1405–1419. Eur. Math. Soc., Zürich, 2006.
[8] M. Salah Baouendi, Linda Preiss Rothschild, Jörg Winkelmann, and Dmitri Zaitsev.
Lie group structures on groups of diffeomorphisms and applications to CR manifolds.
Ann. Inst. Fourier (Grenoble), 54(5):1279–1303, xiv, xx, 2004. ISSN 0373-0956.
[9] M.S. Baouendi, P. Ebenfelt, and Linda Preiss Rothschild. Dynamics of the Segre
varieties of a real submanifold in complex space. J. Algebraic Geom., 12(1):81–106,
2003. ISSN 1056-3911.
[10] Linda Preiss Rothschild. Mappings between real submanifolds in complex space. In
Explorations in complex and Riemannian geometry, volume 332 of Contemp. Math.,
pages 253–266. Amer. Math. Soc., Providence, RI, 2003.
[11] M.S. Baouendi, Nordine Mir, and Linda Preiss Rothschild. Reflection ideals and map-
pings between generic submanifolds in complex space. J. Geom. Anal., 12(4):543–
580, 2002. ISSN 1050-6926.
[12] M.S. Baouendi, Linda Preiss Rothschild, and Dmitri Zaitsev. Equivalences of real
submanifolds in complex space. J. Differential Geom., 59(2):301–351, 2001. ISSN
0022-040X.
[13] M.S. Baouendi, Linda Preiss Rothschild, and Dmitri Zaitsev. Points in general po-
sition in real-analytic submanifolds in CN and applications. In Complex analysis
and geometry (Columbus, OH, 1999), volume 9 of Ohio State Univ. Math. Res. Inst.
Publ., pages 1–20. de Gruyter, Berlin, 2001.
[14] M.S. Baouendi, P. Ebenfelt, and Linda Preiss Rothschild. Local geometric properties
of real submanifolds in complex space. Bull. Amer. Math. Soc. (N.S.), 37(3):309–336
(electronic), 2000. ISSN 0273-0979.
[15] M.S. Baouendi, P. Ebenfelt, and Linda Preiss Rothschild. Convergence and finite
determination of formal CR mappings. J. Amer. Math. Soc., 13(4):697–723 (elec-
tronic), 2000. ISSN 0894-0347.
xvi Extended Curriculum Vitae of Linda Preiss Rothschild

[16] M.S. Baouendi, P. Ebenfelt, and Linda Preiss Rothschild. Rational dependence of
smooth and analytic CR mappings on their jets. Math. Ann., 315(2):205–249, 1999.
ISSN 0025-5831.
[17] M. Salah Baouendi, Peter Ebenfelt, and Linda Preiss Rothschild. Real submanifolds
in complex space and their mappings, volume 47 of Princeton Mathematical Series.
Princeton University Press, Princeton, NJ, 1999. ISBN 0-691-00498-6.
[18] M. Salah Baouendi and Linda Preiss Rothschild. Local holomorphic equivalence of
real analytic submanifolds in CN . In Several complex variables (Berkeley, CA, 1995–
1996), volume 37 of Math. Sci. Res. Inst. Publ., pages 1–24. Cambridge Univ. Press,
Cambridge, 1999.
[19] M.S. Baouendi, P. Ebenfelt, and Linda Preiss Rothschild. CR automorphisms of real
analytic manifolds in complex space. Comm. Anal. Geom., 6(2):291–315, 1998. ISSN
1019-8385.
[20] M.S. Baouendi, P. Ebenfelt, and Linda Preiss Rothschild. Parametrization of local
biholomorphisms of real analytic hypersurfaces. Asian J. Math., 1(1):1–16, 1997.
ISSN 1093-6106.
[21] Linda Preiss Rothschild. Holomorphically nondegenerate algebraic hypersurfaces and
their mappings. In Multidimensional complex analysis and partial differential equa-
tions (São Carlos, 1995), volume 205 of Contemp. Math., pages 247–252. Amer.
Math. Soc., Providence, RI, 1997.
[22] M.S. Baouendi, Xiaojun Huang, and Linda Preiss Rothschild. Regularity of CR
mappings between algebraic hypersurfaces. Invent. Math., 125(1):13–36, 1996. ISSN
0020-9910.
[23] M.S. Baouendi and Linda Preiss Rothschild. Unique continuation of harmonic func-
tions at boundary points and applications to problems in complex analysis. In Partial
differential equations and functional analysis, volume 22 of Progr. Nonlinear Differ-
ential Equations Appl., pages 35–40. Birkhäuser Boston, Boston, MA, 1996.
[24] M.S. Baouendi, Xiao Jun Huang, and Linda Preiss Rothschild. Nonvanishing of the
differential of holomorphic mappings at boundary points. Math. Res. Lett., 2(6):737–
750, 1995. ISSN 1073-2780.
[25] M.S. Baouendi and Linda Preiss Rothschild. Mappings of real algebraic hypersur-
faces. J. Amer. Math. Soc., 8(4):997–1015, 1995. ISSN 0894-0347.
[26] Serge Alinhac, M.S. Baouendi, and Linda Preiss Rothschild. Flat analytic discs at-
tached to real hypersurfaces of finite type. Math. Res. Lett., 1(3):359–367, 1994. ISSN
1073-2780.
[27] M.S. Baouendi and Linda Preiss Rothschild. Directions of analytic discs attached to
generic manifolds of finite type. J. Funct. Anal., 125(1):149–171, 1994. ISSN 0022-
1236.
[28] M.S. Baouendi and Linda Preiss Rothschild. A generalized complex Hopf lemma
and its applications to CR mappings. Invent. Math., 111(2):331–348, 1993. ISSN
0020-9910.
[29] M.S. Baouendi and Linda Preiss Rothschild. A local Hopf lemma and unique contin-
uation for harmonic functions. Internat. Math. Res. Notices, 8:245–251, 1993. ISSN
1073-7928.
 Publication List of Linda Preiss Rothschild xvii

[30] M.S. Baouendi and Linda Preiss Rothschild. Unique continuation and a Schwarz
reflection principle for analytic sets. Comm. Partial Differential Equations, 18(11):
1961–1970, 1993. ISSN 0360-5302.
[31] M.S. Baouendi and Linda Preiss Rothschild. Images of real hypersurfaces under
holomorphic mappings. J. Differential Geom., 36(1):75–88, 1992. ISSN 0022-040X.
[32] M.S. Baouendi and Linda Preiss Rothschild. Remarks on the generic rank of a CR
mapping. J. Geom. Anal., 2(1):1–9, 1992. ISSN 1050-6926.
[33] M.S. Baouendi and Linda Preiss Rothschild. A general reflection principle in C2 . J.
Funct. Anal., 99(2):409–442, 1991. ISSN 0022-1236.
[34] M.S. Baouendi and Linda Preiss Rothschild. Holomorphic mappings of real analytic
hypersurfaces. In Several complex variables and complex geometry, Part 3 (Santa
Cruz, CA, 1989), volume 52 of Proc. Sympos. Pure Math., pages 15–26. Amer. Math.
Soc., Providence, RI, 1991.
[35] M.S. Baouendi and Linda Preiss Rothschild. Minimality and the extension of func-
tions from generic manifolds. In Several complex variables and complex geometry,
Part 3 (Santa Cruz, CA, 1989), volume 52 of Proc. Sympos. Pure Math., pages
1–13. Amer. Math. Soc., Providence, RI, 1991.
[36] Serge Alinhac, M.S. Baouendi, and Linda Preiss Rothschild. Unique continuation and
regularity at the boundary for holomorphic functions. Duke Math. J., 61(2):635–653,
1990. ISSN 0012-7094.
[37] M.S. Baouendi and Linda Preiss Rothschild. Cauchy-Riemann functions on manifolds
of higher codimension in complex space. Invent. Math., 101(1):45–56, 1990. ISSN
0020-9910.
[38] M.S. Baouendi and Linda Preiss Rothschild. Geometric properties of mappings be-
tween hypersurfaces in complex space. J. Differential Geom., 31(2):473–499, 1990.
ISSN 0022-040X.
[39] M.S. Baouendi and Linda Preiss Rothschild. Transversal Lie group actions on ab-
stract CR manifolds. Math. Ann., 287(1):19–33, 1990. ISSN 0025-5831.
[40] M.S. Baouendi, S.R. Bell, and Linda Preiss Rothschild. Mappings of three-dimen-
sional CR manifolds and their holomorphic extension. Duke Math. J., 56(3):503–530,
1988. ISSN 0012-7094.
[41] M.S. Baouendi and Linda Preiss Rothschild. Extension of holomorphic functions
in generic wedges and their wave front sets. Comm. Partial Differential Equations,
13(11):1441–1466, 1988. ISSN 0360-5302.
[42] M.S. Baouendi and Linda Preiss Rothschild. Germs of CR maps between real analytic
hypersurfaces. Invent. Math., 93(3):481–500, 1988. ISSN 0020-9910.
[43] Alain Grigis and Linda Preiss Rothschild. L2 estimates for the boundary Laplacian
operator on hypersurfaces. Amer. J. Math., 110(4):577–593, 1988. ISSN 0002-9327.
[44] Gerardo A. Mendoza and Linda Preiss Rothschild. Analytic approximability of so-
lutions of partial differential equations. Ark. Mat., 26(2):289–303, 1988. ISSN 0004-
2080.
[45] M.S. Baouendi, S.R. Bell, and Linda Preiss Rothschild. CR mappings of finite mul-
tiplicity and extension of proper holomorphic mappings. Bull. Amer. Math. Soc.
(N.S.), 16(2):265–270, 1987. ISSN 0273-0979.
xviii Extended Curriculum Vitae of Linda Preiss Rothschild

[46] M.S. Baouendi and Linda Preiss Rothschild. CR mappings and their holomorphic
extension. In Journées “Équations aux derivées partielles” (Saint Jean de Monts,
1987), pages Exp. No. XXIII, 6. École Polytech., Palaiseau, 1987.
[47] M.S. Baouendi and Linda Preiss Rothschild. Normal forms for generic manifolds and
holomorphic extension of CR functions. J. Differential Geom., 25(3):431–467, 1987.
ISSN 0022-040X.
[48] Lawrence Corwin and Linda Preiss Rothschild. Solvability of transversally elliptic
differential operators on nilpotent Lie groups. Amer. J. Math., 108(3):589–613, 1986.
ISSN 0002-9327.
[49] M.S. Baouendi and Linda Preiss Rothschild. Analytic approximation for homoge-
neous solutions of invariant differential operators on Lie groups. Astérisque, 131:189–
199, 1985. ISSN 0303-1179. Colloquium in honor of Laurent Schwartz, Vol. 1
(Palaiseau, 1983).
[50] M.S. Baouendi and Linda Preiss Rothschild. Semirigid CR structures and holomor-
phic extendability. In Proceedings of the conference on partial differential equations,
Vol. 1, 2 (Saint Jean de Monts, 1985), pages Exp. No. 1, 4. Soc. Math. France, Paris,
1985.
[51] M.S. Baouendi, Linda Preiss Rothschild, and F. Trèves. CR structures with group
action and extendability of CR functions. Invent. Math., 82(2):359–396, 1985. ISSN
0020-9910.
[52] Lawrence Corwin, Bernard Helffer, and Linda Preiss Rothschild. Smoothness and
analyticity for solutions of first-order systems of partial differential equations on
nilpotent Lie groups. Invent. Math., 81(2):205–216, 1985. ISSN 0020-9910.
[53] Linda Preiss Rothschild. Integrability and holomorphic extendibility for rigid CR
structures. In Pseudodifferential operators and applications (Notre Dame, Ind.,
1984), volume 43 of Proc. Sympos. Pure Math., pages 237–240. Amer. Math. Soc.,
Providence, RI, 1985.
[54] Linda Preiss Rothschild. Analyticity of solutions of partial differential equations
on nilpotent Lie groups. In Lie group representations, III (College Park, Md.,
1982/1983), volume 1077 of Lecture Notes in Math., pages 389–395. Springer, Berlin,
1984.
[55] Alain Grigis and Linda Preiss Rothschild. A criterion for analytic hypoellipticity
of a class of differential operators with polynomial coefficients. Ann. of Math. (2),
118(3):443–460, 1983. ISSN 0003-486X.
[56] Linda Preiss Rothschild. A remark on hypoellipticity of homogeneous invariant dif-
ferential operators on nilpotent Lie groups. Comm. Partial Differential Equations,
8(15):1679–1682, 1983. ISSN 0360-5302.
[57] Linda Preiss Rothschild and David S. Tartakoff. Analyticity of relative fundamental
solutions and projections for left invariant operators on the Heisenberg group. Ann.
Sci. École Norm. Sup. (4), 15(3):419–440, 1982. ISSN 0012-9593.
[58] Linda Preiss Rothschild. Local solvability of second-order differential operators on
nilpotent Lie groups. Ark. Mat., 19(2):145–175, 1981. ISSN 0004-2080.
[59] Linda Preiss Rothschild and David S. Tartakoff. Inversion of analytic matrices and
local solvability of some invariant differential operators on nilpotent Lie groups.
Comm. Partial Differential Equations, 6(6):625–650, 1981. ISSN 0360-5302.
 Publication List of Linda Preiss Rothschild xix

[60] Linda Preiss Rothschild. Nonexistence of optimal L2 estimates for the boundary
Laplacian operator on certain weakly pseudoconvex domains. Comm. Partial Dif-
ferential Equations, 5(8):897–912, 1980. ISSN 0360-5302.
[61] Linda Preiss Rothschild. A criterion for hypoellipticity of operators constructed from
vector fields. Comm. Partial Differential Equations, 4(6):645–699, 1979. ISSN 0360-
5302.
[62] Linda Preiss Rothschild. Local solvability of left invariant differential operators on
the Heisenberg group. Proc. Amer. Math. Soc., 74(2):383–388, 1979. ISSN 0002-9939.
[63] Linda Preiss Rothschild. Smoothness of solutions of certain partial differential equa-
tions constructed from vector fields. In Harmonic analysis in Euclidean spaces (Proc.
Sympos. Pure Math., Williams Coll., Williamstown, Mass., 1978), Part 2, Proc.
Sympos. Pure Math., XXXV, Part, pages 227–230. Amer. Math. Soc., Providence,
R.I., 1979.
[64] Linda Preiss Rothschild and David Tartakoff. Parametrices with C ∞ error for cmb
and operators of Hörmander type. In Partial differential equations and geometry
(Proc. Conf., Park City, Utah, 1977), volume 48 of Lecture Notes in Pure and Appl.
Math., pages 255–271. Dekker, New York, 1979.
[65] Linda Preiss Rothschild. Book Review: Estimates for the ∂-Neumann problem. Bull.
Amer. Math. Soc., 84(2):266–270, 1978. ISSN 0002-9904.
[66] Linda Preiss Rothschild. Parametrices for the boundary Laplacian and related hy-
poelliptic differential operators. In Several complex variables (Proc. Sympos. Pure
Math., Vol. XXX, Part 1, Williams Coll., Williamstown, Mass., 1975), pages 195–
203. Amer. Math. Soc., Providence, R. I., 1977.
[67] Linda Preiss Rothschild and E.M. Stein. Hypoelliptic differential operators and nilpo-
tent groups. Acta Math., 137(3-4):247–320, 1976. ISSN 0001-5962.
[68] Linda Preiss Rothschild and Joseph A. Wolf. Eigendistribution expansions on Heisen-
berg groups. Indiana Univ. Math. J., 25(8):753–762, 1976. ISSN 0022-2518.
[69] Frederick P. Greenleaf, Martin Moskowitz, and Linda Preiss Rothschild. Compact-
ness of certain homogeneous spaces of finite volume. Amer. J. Math, 97:248–259,
1975. ISSN 0002-9327.
[70] Paul S. Wang and Linda Preiss Rothschild. Factoring multivariate polynomials over
the integers. Math. Comput., 29:935–950, 1975. ISSN 0378-4754.
[71] Frederick P. Greenleaf, Martin Moskowitz, and Linda Preiss-Rothschild. Automor-
phisms, orbits, and homogeneous spaces of non-connected Lie groups. Math. Ann.,
212:145–155, 1974/75. ISSN 0025-5831.
[72] Frederick P. Greenleaf, Martin Moskowitz, and Linda Preiss Rothschild. Central
idempotent measures on connected locally compact groups. J. Functional Analysis,
15:22–32, 1974.
[73] Frederick P. Greenleaf, Martin Moskowitz, and Linda Preiss Rothschild. Unbounded
conjugacy classes in Lie groups and location of central measures. Acta Math.,
132:225–243, 1974. ISSN 0001-5962.
[74] Linda Preiss Rothschild. A distribution theoretic proof of Kirillov’s character formula
for nilpotent Lie groups. Math. Z., 140:63–65, 1974. ISSN 0025-5874.
xx Extended Curriculum Vitae of Linda Preiss Rothschild

[75] Linda Preiss Rothschild and Joseph A. Wolf. Representations of semisimple groups
associated to nilpotent orbits. Ann. Sci. École Norm. Sup. (4), 7:155–173 (1975),
1974. ISSN 0012-9593.
[76] David L. Ragozin and Linda Preiss Rothschild. Central measures on semisimple Lie
groups have essentially compact support. Proc. Amer. Math. Soc., 32:585–589, 1972.
ISSN 0002-9939.
[77] Linda Preiss Rothschild. On uniqueness of quasi-split real semisimple Lie algebras.
Proc. Amer. Math. Soc., 24:6–8, 1970. ISSN 0002-9939.
Complex Analysis
Trends in Mathematics, 1–23

c 2010 Springer Basel AG


Oblique Polar Lines of X |f |2λ |g|2μ
D. Barlet and H.-M. Maire

Abstract. Existence of oblique polar lines for the meromorphic extension of

the current valued function |f |2λ |g|2μ  is given under the following hy-
potheses: f and g are holomorphic function germs in Cn+1 such that g is
non-singular, the germ Σ := { df ∧ dg = 0} is one dimensional, and g is
proper and finite on S := {df = 0} . The main tools we use are interaction
of strata for f (see [4]), monodromy of the local system H n−1 (u) on S for
a given eigenvalue exp(−2iπu) of the monodromy of f , and the monodromy
of the cover g|S . Two non-trivial examples are completely worked out.

Mathematics Subject Classification (2000). 32S40,58K55.

Keywords. Asymptotic expansion, fibre integrals, meromorphic extension.

Introduction
Given an open subset Y in Cm , two holomorphic functions  f, g on Y and a C ∞
compactly support (m, m)-form φ in Y , the integral Y |f |2λ |g|2μ φ, for (λ, μ)
in C2 with λ and μ > 0 , defines a holomorphic function in that region. As
a direct consequence of the resolution of singularities, this holomorphic function
extends meromorphically to C2 , see Theorem 1.1. The polar locus of this extension
is contained in a union of straight lines with rational slopes (see [8] for other results
on this integral). In this paper we look for geometric conditions that guarantee a
true polar line of this extension for at least one φ ∈ Λm,m Cc∞ (Y ), in other words
a true polar line of the meromorphic extension of the holomorphic current valued
function 
(λ, μ) → |f |2λ |g|2μ .
Y
Since existence
 of horizontal
 or vertical polar lines follows directly from existence
of poles of Y |g|2μ  or Y |f |2λ  that have been extensively studied in [1], [2] and
[3], we will concentrate on oblique polar lines. Because desingularization is quite
hard to compute, it is not clear how to determine these polar lines. Moreover,
2 D. Barlet and H.-M. Maire

only a few of the so obtained candidates are effectively polar and no geometric
conditions are known to decide it in general.
In Section 2, we expose elementary properties of meromorphic functions of
two variables that are used later for detecting oblique polar lines. Four examples
of couples (f, g) for which these results apply are given.
In Sections 3 and 4 we give sufficient criteria to obtain oblique polar lines in
rather special cases, but with a method promised to a large generalization. They
rely on results which give realization in term of holomorphic differential forms of
suitable multivalued sections of the sheaf of vanishing cycles along the smooth part
of the singular set S (assumed to be a curve) of the function f . The second function
g being smooth and transversal to S at the origin. The sufficient condition is then
given in term of the monodromy on S ∗ := S \ {0} on the sheaf of vanishing cycles
of f for the eigenvalue exp(−2iπu) assuming that the meromorphic extension of
|f |2λ
 has only simple poles at −u − q for all q ∈ N (see Corollary 4.3).
X 
To be more explicit, recall the study of |f |2λ  started in [4] and com-
pleted in [5], for a holomorphic function f defined in an open neighbourhood of
0 ∈ Cn+1 with one-dimensional critical locus S . The main tool was to restrict
f to hyperplane sections transverse to S ∗ and examine, for a given eigenvalue
exp(−2iπu) of the monodromy of f , the local system H n−1 (u) on S ∗ formed
by the corresponding spectral
 subspaces. Higher-order poles of the current valued
meromorphic function |f |2λ  at −u − m, some m ∈ N, are detected using the
existence of a uniform section of the sheaf H n−1 (u) on S ∗ which is not extendable
at the origin. So an important part of this local system remained unexplored in
[4] and [5] because only the eigenvalue 1 of the monodromy Θ of the local system
H n−1 (u) on S ∗ is involved in the exact sequence
0 → H 0 (S, H n−1 (u)) → H 0 (S ∗ , H n−1 (u)) → H{0}
1
(S, H n−1 (u)) → 0.
In this paper, we will focus on the other eigenvalues of Θ .
Let us assume the following properties:
(1) the function g is non-singular near 0 ;
(2) the set Σ := { df ∧ dg = 0} is a curve;
(3) the restriction g|S : S → D is proper and finite;
(4) g|−1 ∗
S (0) = {0} and g|S ∗ is a finite cover of D := D \ {0} .
Condition (2) implies that the singular set S := { df = 0} of f has dimension
 1 . We are interested in the case where S is a curve.
Remark that condition (4) may always be achieved by localization near 0
when conditions (1), (2) and (3) are satisfied. These conditions hold in a neigh-
bourhood of the origin if (f, g) forms an isolated complete intersection singularity
(icis) with one-dimensional critical locus, assuming g smooth. But we allow also
the case where Σ has branches in {f = 0} not contained in S .
The direct image by g of the constructible sheaf H n−1 (u) supported in S
will be denoted by H ; it is a local system on D∗ . Let H0 be the fibre of H
at t0 ∈ D∗ and Θ0 its monodromy which is an automorphism of H0 . In case
 
Oblique Polar Lines of X
|f |2λ |g|2μ  3

where S is smooth, it is possible to choose the function g in order that g|S is an

isomorphism and Θ0 may be identified with the monodromy Θ of H n−1 (u) on
S ∗ . In general, Θ0 combines Θ and the monodromy of the cover g|S ∗ .
Take an eigenvalue1 exp(−2iπl/k) = 1 of Θ , with l ∈ [1, k−1] and (l, k) = 1 .
We define an analogue of the interaction of strata in this new context. The auxiliary
non singular function g is used to realize analytically the rank one local system
on S ∗ with monodromy exp(−2iπl/k). To perform this we shall assume that the
degree of g on the irreducible branch of S we are interested in, is relatively prime
to k . Of course this is the case when S is smooth and g transversal to S at the
origin. Using then a k th root of g we can lift our situation to the case where
we consider an invariant section of the complex of vanishing cycles of the lifted
function f˜ (see Theorem 4.2) and then use already known results from [4]. The
existence of true oblique polar lines follows now from results of Section 2.
The paper ends with a complete computation of two non-trivial examples
that illustrate the above constructions.


1. Polar structure of X
|f |2λ 
Theorem 1.1. Bernstein & Gelfand. For m and p ∈ N∗ , let Y be an open
subset in Cm , f : Y → Cp a holomorphic map and X a relatively compact open
set in Y . Then there exists a finite set P (f ) ⊂ Np \ {0} such that, for any form
φ ∈ Λm,m Cc∞ (X) with compact support, the holomorphic map in the open set
{λ1 > 0} × · · · × {λp > 0} given by

(λ1 , . . . , λp ) → |f1 |2λ1 . . . |fp |2λp φ (1.1)
X
has a meromorphic extension to Cp with poles contained in the set

{ a | λ
+ l = 0}.
a∈P (f ),l∈N∗

Proof. For sake of completeness we recall the arguments of [10].

Using desingularization of the product f1 . . . fp , we know [12] that there
exists a holomorphic manifold Ỹ of dimension m and a holomorphic proper map
π : Ỹ → Y such that the composite functions f˜j := fj ◦π are locally expressible as
k k
a a
f˜k (y) = y1 1 . . . ymm uk (y), 1  k  p, (1.2)
where ∈ N and uk is a holomorphic nowhere vanishing function. Because
akj
π −1 (X) is relatively compact, it may be covered by a finite number of open set
where (1.2) is valid.
For ϕ ∈ Λm,m Cc∞ (X) and λ1 , . . . , λp positive, we have
 
|f1 | . . . |fp | φ =
2λ1 2λp
|f˜1 |2λ1 . . . |f˜p |2λp π ∗ φ.
X π −1 (X)

1 Note that the eigenvalues of Θ are roots of unity.

4 D. Barlet and H.-M. Maire

Using partition of unity and setting μk := a1k λ1 + · · · + apk λp , 1  k  m, we are

reduced to give a meromorphic extension to

(μ1 , . . . , μm ) → |y1 |2μ1 . . . |ym |2μm ω(μ, y), (1.3)
Cm
∞
where ω is a C form of type (m, m) with compact support in Cm valued in
the space of entire functions on Cm . Of course, (1.3) is holomorphic in the set
{μ1 > −1, . . . , μm > −1} .
The relation
(μ1 + 1).|y1 |2μ1 = ∂1 (|y1 |2μ1 .y1 )
implies by partial integration in y1
 
−1
|y1 |2μ1 . . . |ym |2μm ω(μ, y) = |y1 |2μ1 .y1 .|y2 |2μ2 . . . |ym |2μm ∂1 ω(μ, y).
Cm μ1 + 1 Cm
Because ∂1 ω is again a C ∞ form of type (m, m) with compact support in Cm
valued in the space of entire functions on Cm , we may repeat this argument for
each coordinate y2 , . . . , ym and obtain

|y1 |2μ1 . . . |ym |2μm ω(μ, y) =
Cm

(−1)m
= |y1 |2μ1 .y1 .|y2 |2μ2 .y2 . . . |ym |2μm .ym .∂1 . . . ∂m ω(μ, y).
(μ1 + 1) . . . (μm + 1) Cm
The integral on the RHS is holomorphic for μ1 > −3/2, . . . , μm > −3/2 . There-
fore the function (1.3) is meromorphic in this domain with only possible poles in
the union of the hyperplanes {μ1 + 1 = 0}, . . . , {μm + 1 = 0} .
Iteration of these arguments concludes the proof. 

Remark 1.2. An alternate proof of Theorem 1.1 has been given for p = 1 by
Bernstein [9], Björk [11], Barlet-Maire [6], [7]. See also Loeser [13] and Sabbah [14]
for the general case.

In case where f1 , . . . , fp define an isolated complete intersection singularity

(icis), Loeser and Sabbah gave moreover the following information on the set P (f )
of the “slopes” of the polar hyperplanes in the meromorphic extension of the
function (1.1): it is contained in the set of slopes of the discriminant locus Δ of
f , which in this case is an hypersurface in Cp . More precisely, take the (p − 1)-
skeleton of the fan associated to the Newton polyhedron of Δ at 0 and denote by
P (Δ) the set of directions associated to this (p − 1)-skeleton union with the set
{(a1 , . . . , ap ) ∈ Np | a1 . . . ap = 0} . Then
P (f ) ⊆ P (Δ).
In particular, if the discriminant locus is contained in the hyperplanes of coordi-
nates, then there are no polar hyperplanes with direction in (N∗ )p .
 
Oblique Polar Lines of X
|f |2λ |g|2μ  5

The results of Loeser and Sabbah above have the following consequence for
an icis which is proved below directly by elementary arguments, after we have
introduced some terminology.
Definition 1.3. Let f1 , . . . , fp be holomorphic functions on an open neighbour-
hood X of the origin in Cm . We shall say that a polar hyperplane H ⊂ Cp for
the meromorphic extension of X |f1 |2λ1 . . . |fp |2λp  is supported by the closed set
 ⊂ X , 2λ
F when H is not a polar hyperplane for the meromorphic extension of
X\F
|f 1 | 1
. . . |fp |2λp  . We shall say that a polar direction is supported in F if
any polar hyperplane with this direction is supported by F .
Proposition 1.4. Assume f1 , . . . , fp are quasi-homogeneous functions for the
weights w1 , . . . , wp , of degree a1 , . . . , ap . Then if there exists a polar hyperplane
direction supported by the origin for (1.1) in (N∗ )p it is (a1 , . . . , ap ) and the
corresponding poles are at most simple.
In particular, for p = 2 , and if (f1 , f2 ) is an icis, all oblique polar lines have
direction (a1 , a2 ).
Proof. Quasi-homogeneity gives fk (tw1 x1 , . . . , twm xm ) = tak fk (x), k = 1, . . . , p.
m 
Let Ω := 1 (−1)j−1 wj xj dx0 ∧· · ·∧ 
dxj ∧· · ·∧ dxm so that dΩ = ( wj ) dx.
From Euler’s relation, because fkλk is quasi-homogeneous of degree ak λk :
dfkλk ∧ Ω = ak λk fkλk dx,
dxδ ∧ Ω = w | δ
xδ dx, ∀δ ∈ Nm .
Take ρ ∈ Cc∞ (Cm ); then, with 1 = (1, . . . , 1) ∈ Np and ε ∈ Nm :

d(|f |2λ xδ x̄ε ρΩ ∧ dx̄) =

= ( a | λ
+ w | δ + 1
)|f |2λ xδ x̄ε ρ dx ∧ dx̄ + |f |2λ xδ x̄ε dρ ∧ Ω ∧ dx̄.
Using Stokes’ formula we get
 
( a | λ
+ w | δ + 1
) |f |2λ xδ x̄ε ρ dx ∧ dx̄ = − |f |2λ xδ x̄ε dρ ∧ Ω ∧ dx̄.

For ρ = 1 near 0 , dρ = 0 , near 0 . Therefore the right-hand side has no poles

supported by the origin. Now the conclusion comes from the Taylor expansion at
0 of the test function. 

2. Existence of polar oblique lines

In this section, we consider two holomorphic functions f, g : Y → C, where Y is
an open subset in Cm and fix a relatively compact open subset X of Y . Without
loss of generality, we assume 0 ∈ X . We study the possible oblique polar lines of
the meromorphic extension of the current valued function

(λ, μ) → |f |2λ |g|2μ . (2.1)
X
6 D. Barlet and H.-M. Maire

The following elementary lemma is basic.

Lemma 2.1. Let M be a meromorphic function in C2 with poles along a family of
lines with directions in N2 . For (λ0 , μ0 ) ∈ C2 , assume
(i) {λ = λ0 } is a polar line of order  k0 of M ,
(ii) {μ = μ0 } is not a polar line of M ,
(iii) λ → M (λ, μ0 ) has a pole of order at least k0 + 1 at λ0 .
Then there exists (a, b) ∈ (N∗ )2 such that the function M has a pole along the
(oblique) line {aλ + bμ = aλ0 + bμ0 } .
Proof. If M does not have an oblique polar line through (λ0 , μ0 ), then the function
(λ, μ) → (λ−λ0 )k0 M (λ, μ) is holomorphic near (λ0 , μ0 ). Therefore, λ → M (λ, μ0 )
has at most a pole of order k0 at λ0 . Contradiction. 

It turns out that to check the first condition in the above lemma for the
function (2.1), a sufficient condition is that the poles of the meromorphic extension
of the current valued function

λ → |f |2λ  (2.2)
X
are of order  k0 . Such a simplification does not hold for general meromorphic
functions. For example,
λ+μ
(λ, μ) →
λ2
has a double pole along {λ = 0} but its restriction to {μ = 0} has only a simple
pole at 0 .
Proposition 2.2. If the meromorphic extension of the current valued function (2.2)
has a pole of order k at λ0 ∈ R− , i.e., it has a principal part of the form
Tk T1
+ ···+ ,
(λ − λ0 )k λ − λ0
at λ0 , then the meromorphic extension of the function (2.1) has a pole of order
k0 := max{0  l  k | supp Tl ⊆ {g = 0}} (2.3)
along the line {λ = λ0 } .
Proof. As a consequence ofthe Bernstein identity (see [11]), there exists N ∈ N
such that the extension of X |f |2λ φ in {λ > λ0 − 1} can be achieved for φ ∈
Λm,m CcN (X). Our hypothesis implies that this function has a pole of order  k
at λ0 . Because |g|2μ φ is of class C N for μ large enough, the function

λ → |f |2λ |g|2μ φ
X

has a meromorphic extension in {λ > λ0 − 1} with a pole of order  k at λ0 .

We have proved that (2.1) has a pole of order  k along the line {λ = λ0 } .
 
Oblique Polar Lines of X
|f |2λ |g|2μ  7

Near λ0 , the extension of X
|f |2λ φ writes
Tk , φ
T1 , φ

+ ···+ + ··· .
(λ − λ0 )k λ − λ0

Hence that of X
|f |2λ |g|2μ φ looks
Tk |g|2μ , φ
T1 |g|2μ , φ

+ · · · + + ··· .
(λ − λ0 )k λ − λ0
If supp Tk ⊆ {g = 0} , then the first term vanishes for μ large enough, because
Tk is of finite order (see the beginning of the proof). So the order of the pole along
the line {λ = λ0 } is  k0 .
Take x0 ∈ supp Tk0 such that g(x0 ) = 0 and V a neighborhood of x0 in
which g does not vanish. From the definition of the support, there exists ψ ∈
Λm,m Cc∞ (V ) such that Tk0 , ψ
= 0 . With φ := ψ|g|−2μ ∈ Λm,m Cc∞ (V ), we get
Tk0 |g|2μ , φ
= Tk0 , ψ
= 0.
Therefore, the extension of (2.1) has a pole of order k0 along the line {λ = λ0 } .


Corollary 2.3. For (λ0 , μ0 ) ∈ (R− )2 , assume

(i) the extension of the current valued function (2.2) has a pole of order k at
λ0 ,
(ii) μ0 is not an integer
 translate of a root of the Bernstein polynomial of g ,
(iii) λ → Pf(μ = μ0 , X |f |2λ |g|2μ ) has a pole of order l0 > k0 where k0 is
defined in (2.3) at λ0 .
Then the meromorphic extension of the current valued function (2.1) has at least
l0 − k0 oblique lines, counted with multiplicities, through (λ0 , μ0 ).

Proof. Use Proposition 2.2 and a version of Lemma 2.1 with multiplicities. 
8 D. Barlet and H.-M. Maire

Example 2.4. m = 3 , f (x, y, z) = x2 + y 2 + z 2 , g(x, y, z) = z .

Example 2.5. m = 4 , f (x, y, z) = x2 + y 2 + z 2 + t2 , g(x, y, z, t) = t2 .

 
Oblique Polar Lines of X
|f |2λ |g|2μ  9

Example 2.6. m = 3 , f (x, y, z) = x2 + y 2 , g(x, y, z) = y 2 + z 2 .

In this last example, Corollary 2.3 does not apply because for λ0 = −1 we have
k0 = l0 . Existence of an oblique polar
 line through
 (−1, 0)is obtained by compu-
tation of the extension of λ → Pf μ = 1/2, X |f |2λ |g|2μ  .

3. Pullback and interaction

In this section, we give by pullback a method to verify condition (iii) of Corollary
2.3 when g is a coordinate. As a matter of fact the function λ → X |f |2λ |g|2μ0 
is only known by meromorphic extension (via Bernstein identity) when μ0 is neg-
ative; it is in general difficult to exhibit some of its poles.
In Cn+1 , denote the coordinates by x1 , . . . , xn , t and take g(x, t) = t. We
consider therefore only one holomorphic function f : Y → C, where Y is an open
subset in Cn+1 and fix a relatively compact open subset X of Y . Let us introduce
also the finite map
p : Cn+1 → Cn+1 such that p(x1 , . . . , xn , τ ) = (x1 , . . . , xn , τ k ) (3.1)
−1
for some fixed integer k . Finally, put f˜ := f ◦p : X̃ → C where X̃ := p (X).
Proposition 3.1. With the above notations and λ0 ∈ R− suppose
(a) the extension of the current valued function (2.2) has a pole of order  1 at
λ0 , 
(b) λ → X̃ |f˜|2λ  has a double pole at λ0 .
Thenthere exists l ∈ [1, k−1] such that the extension of the current valued function
λ → X |f |2λ |t|−2l/k  has a double pole at λ0 .
10 D. Barlet and H.-M. Maire

Proof. Remark that the support of the polar part of order 2 of X̃ |f˜|2λ  at λ0 is
contained in {τ = 0} because we assume (a) and p is a local isomorphism outside
{τ = 0} .
After hypothesis (b) there exists ϕ ∈ Λn+1,n+1 Cc∞ (X̃) such that

 
A := P2 λ = λ0 , |f˜|2λ ϕ = 0.
X̃
Consider the Taylor expansion of ϕ along {τ = 0}

ϕ(x, τ ) = τ j τ̄ j ϕj,j  (x) ∧ dτ ∧ dτ̄ + o(|τ |N )
j+j  N

where N is larger than the order of the current defined by P2 on a compact set
K containing the support of ϕ. Therefore

  
A= P2 λ = λ0 , |f˜|2λ τ j τ̄ j ϕj,j (x)χ(|τ |2k ) ∧ dτ ∧ dτ̄
j+j  N X̃

where χ has support in K and is equal to 1 near 0 . Because A does not vanish
there exists (j, j  ) ∈ N2 with j + j   N such that

  
Aj,j := P2 λ = λ0 , |f˜|2λ τ j τ̄ j ϕj,j  (x)χ(|τ |2k ) ∧ dτ ∧ dτ̄ = 0.
X̃

The change of variable τ → exp(2iπ/k)τ that leaves f˜ invariant, shows that

Aj,j  = exp(2iπ(j − j  )/k)Aj,j . Hence Aj,j = 0 for j − j  ∈ kZ. We then get the
existence of (j, j  ) ∈ N2 verifying j  = j + kν with ν ∈ Z and Aj,j = 0 .
The change of variable t = τ k in the computation of Aj,j  gives

 
P2 λ = −λ0 , |f |2λ |t|2(j−k+1)/k t̄ν ϕj,j+kν (x)χ(|t|2 ) ∧ dt ∧ dt̄ = 0.
X
This ends the proof with −l = j −k+1 if ν  0 and with −l = j  −k+1 if ν < 0 . No-
tice that l < k in all cases. Necessarily l = 0 because from hypothesis (a), we know
that the extension of the function (2.2) does not have a double pole at λ0 . 
Theorem 3.2. For Y open in Cn+1 and X relatively compact open subset of Y ,
let f : Y → C be holomorphic and g(x, t) = t. Assume (f, g) satisfy properties
(1) to (4) of the Introduction. Moreover suppose

(a) |f |2λ  has a at most a simple pole at λ0 − ν , ∀ν ∈ N;
(b) e2πiλ0 is a eigenvalue of the monodromy of f acting on the H n−1 of the
Milnor fiber of f at the generic point of a connected component Sı∗ of S ∗ ,
and there exist a non zero eigenvector such its monodromy around 0 in Sı
is a primitive k -root of unity, with k  2 ;
(c) the degree dı of the covering t : Sı∗ → D∗ is prime to k ;
(d) e2πiλ0 is not an eigenvalue of the monodromy of f˜ acting on the H n−1 of
the Milnor fiber of f˜ at 0 , where f˜(x, τ ) = f (x1 , . . . , xn , τ k ).

Then there exists an oblique polar line of |f |2λ |g|2μ  through (λ0 − j, −l/k),
some j ∈ N, and some l ∈ [1, k − 1].
 
Oblique Polar Lines of X
|f |2λ |g|2μ  11

Remark that condition (b) implies that λ0 ∈ Z because of the result of [1].

Proof. Notice first that (λ, μ) → |f |2λ |g|2μ  has a simple pole along {λ = λ0 } .
Indeed the support of the residue current of λ → |f |2λ  at λ0 contains Sı∗
where t does not vanish and Proposition 2.2 applies.
Denote by z a local coordinate on the normalization of Sı . The function t
has a zero of order dı on this normalization and hence dı is the degree of the
cover Sı∗ → D∗ induced by t. Without loss of generality, we may suppose t = z dı
on the normalization of Sı .
Lemma 3.3. Let k, d ∈ N∗ and put
Yd := {(z, τ ) ∈ D2 /τ k = z d }, Yd∗ := Yd \ {0}.
If k and d are relatively prime, then the first projection pr1,d : Yd∗ → D∗ is a
cyclic cover of degree k .
Proof. Let us prove that the cover defined by pr1,d is isomorphic to the cover
defined by pr1,1 , that may be taken as definition of a cyclic cover of degree k .
After Bézout’s identity, there exist a, b ∈ Z such that
ak + bd = 1. (3.2)
Define ϕ : Yd → C by ϕ(z, τ ) = (z, z τ ). From (3.2), we have ϕ(Yd ) ⊆ Y1 and
2 a b

clearly pr1,1 ◦ϕ = pr1,d .

The map ϕ is injective because
ϕ(z, τ ) = ϕ(z, τ  ) =⇒ τ k = τ k and τ b = τ b ,
hence τ = τ  , after (3.2). It is also surjective: take (z, σ) ∈ Y1∗ ; the system
τ b = σz −a , τ k = z d , when σ k = z,
has a unique solution because the compatibility condition σk z −ak = z bd is satis-
fied. 
 −2iπl 
End of the proof of Theorem 3.2. Take the eigenvector with monodromy exp k
with (l, k) = 1 given by condition (b). Its pullback by p becomes invariant under
the monodromy of τ because of the condition (c) and the lemma given above.
After (d), this section does not extend
 through 0 . So we have interaction of strata
(see [4]) and a double pole for λ → |f˜|2λ  at λ0 −j with some j ∈ N. It remains
to use Proposition 3.1 and Corollary 2.3. 

4. Interaction of strata revised

The main result of this paragraph is Corollary 4.3 of Theorem 4.2 which guarantees
an oblique polar line for the meromorphic extension of

|f |2λ |g|2μ .
12 D. Barlet and H.-M. Maire

Notations and hypotheses are those of the introduction. As before, the function g
is the last coordinate denoted by t.
We consider the subsheaf for the eigenvalue exp(−2iπu) = 1 of the local
system of vanishing cycles of f along S ∗ . The main difference with the case of the
eigenvalue 1 is that the notion of extendable section at the origin has no a priori
meaning. So we shall lift the situation to a suitable branched covering in order to
kill this monodromy and to reach the situation studied in [4].
Using realization of vanishing cycles via holomorphic differential forms, we
give a precise meaning downstairs to the notion of extendable section at the origin
in term of the sheaf H n[S] (OX ). This will be achieved in Theorems 4.1 and 4.2
which explain the extendable case and the non extendable case for a given suitable
multivalued section on S ∗ .
We suppose that the eigenvalue exp(−2iπu) of the monodromy of f is simple
at each point of S ∗ . Therefore, this eigenvalue is also simple for the monodromy
acting on the group H n−1 of the Milnor fibre of f at 0 . In order to compute
the constructible sheaf H n−1 (u) on S we may use the complex (ΩX [f −1 ], δu ),
that is the complex of meromorphic forms with poles in f −1 (0) equipped with the
differential δu := d − u df
f ∧ along S . This corresponds to the case k0 = 1 in [4].
We use the isomorphisms
rn−1 : hn−1 → H n−1 (u) over S and τ1 : hn−1 → hn over S ∗ , (4.1)
where hn−1 [resp. hn ] denotes the (n − 1)th [resp. nth] cohomology sheaf of the
complex (ΩX [f −1 ], δu ).
In order to look at the eigenspace for the eigenvalue exp(−2iπl/k) of the
monodromy Θ of the local system H n−1 (u) on S ∗ , it will be convenient to consider
the complex of sheaves
 l dt 
Γl := Ω• [f −1 , t−1 ], δu − ∧
k t
which is locally isomorphic along S ∗ to (ΩX [f −1 ], δu ) via the choice of a local
branch of tl/k and the morphism of complexes (ΩX [f −1 ], δu ) → Γl which is given
by ω → t−l/k .ω and satisfies
l dt
δu (t−l/k ω) − ∧ t−l/k ω = t−l/k δu (ω).
k t
But notice that this complex Γl is also defined near the origin. Of course, a global
section σ ∈ H 0 (S ∗ , hn−1 (Γl )) gives, via the above local isomorphism, a multival-
ued global section on S ∗ of the local system H n−1 (u)  hn−1 with monodromy
exp(−2iπl/k) (as multivalued section).
So a global meromorphic differential (n − 1)-form ω with poles in {f.t = 0}
such that dω = u df f ∧ ω + k t ∧ ω defines such a σ , and an element in H0 with
l dt

monodromy exp(−2iπl/k).
We shall use also the morphism of complexes of degree +1
τ̌1 : Γl → Γl
 
Oblique Polar Lines of X
|f |2λ |g|2μ  13

given by τ̌1 (σ) = df

f
∧ σ . It is an easy consequence of [4] that in our situation τ̌1
induces an isomorphism τ̌1 : hn−1 (Γl ) → hn (Γl ) on S ∗ , because we have assumed
that the eigenvalue exp(−2iπu) for the monodromy of f is simple along S ∗ .

Our first objective is to build for each j ∈ N a morphism of sheaves on S ∗

rj : hn−1 (Γl ) → H n[S] (OX ), (4.2)

via the meromorphic extension of X |f | |t|  . Here H [S] (OX ) denotes the
2λ 2μ n

subsheaf of the moderate cohomology with support S of the sheaf H nS (OX ). It is

given by the nth cohomology sheaf of the Dolbeault-Grothendieck complex with
support S :

H n[S] (OX )  Hn (H 0S (Db0,•
X ), d ).
Let w be a (n − 1)-meromorphic form with poles in {f = 0} , satisfying δu (w) =
∗
k t ∧ w on an open neighbourhood U ⊂ X \ {t = 0} of a point in S . Put for
l dt

j ∈ N:

 
 2λ 2μ ¯−j df

rj (w) := Res λ = −u, Pf μ = −l/k, |f | |t| f ∧w∧ .
U f
 
Here Res λ = −u, . . . denotes the residue at λ = −u and P f (μ = −l/k, . . . ) is
the constant term in the Laurent expansion at μ = −l/k .
These formulae define d -closed currents of type (n, 0) with support in
S∗ ∩ U .
Indeed it is easy to check that the following formula holds in the sense of
currents on U :

 
  
d Pf λ = −u, Pf μ = −l/k, |f |2λ |t|2μ f¯−j w ∧ 
U

 
 df 
= Res λ = −u, Pf μ = −l/k, |f |2λ |t|2μ f¯−j ∧w∧ .
U f
On the other hand, if w = δu (v) − kl dt
t
∧ v for v ∈ Γ(U, Ωn−2 [f −1 ]), then

 
  df 
d Res λ = −u, Pf μ = −l/k, |f |2λ |t2μ f¯−j ∧v∧
U f

 
 df 
= Res λ = −u, Pf μ = −l/k, |f |2λ |t|2μ f¯−j ∧w∧
U f

because the meromorphic extension of X |f |  has no double poles at λ ∈ −u−N
2λ

along S ∗ , since exp(−2iπu) is a simple eigenvalue of the monodromy of f along

S ∗ . It follows that the morphism of sheaves (4.2) is well defined on S ∗ .
By direct computation we show the following equality between sections on
S ∗ of the sheaf H n[S] (Ω1X ):
d rj (w) = −(u + j) df ∧ rj+1 (w)
where d : H n[S] (OX ) → H n[S] (Ω1X ) is the morphism induced by the de Rham
differential d : OX → Ω1X .
14 D. Barlet and H.-M. Maire

Because H n[S] (OX ) is a sheaf of OX -modules, it is possible to define the

product g.rj for g holomorphic near a point of S ∗ and the usual rule holds
d (g.rj ) = dg ∧ rj + g. d ρj .
Now we shall define, for each irreducible component Sı of S such that the
local system H n−1 (u)ı has exp(−2iπl/k) as eigenvalue for its monodromy Θı ,
linear maps


ρıj : Ker Θı − exp(−2iπl/k) → H 0 (Sı∗ , H n[S] (OX ))

as follows:
Let sı ∈ Sı∗ be a base point and let γ ∈ H n−1 (u)sı be such that Θı (γ) =
exp(2iπl/k).γ . Denote by σ(γ) the multivalued section of the local system
H n−1 (u)ı on Sı∗ defined by γ . Near each point of s ∈ Sı∗ we can induce σ by a
meromorphic (n − 1)-form w0 which is δu -closed. Choose a local branch of t1/k
near the point s and put w := tl/k w0 . Then it is easy to check that we define in
this way a global section Σ(γ) on Sı∗ of the sheaf hn−1 (Γl ) which is independent
of our choices. Now set
ρıj (γ) := rj (Σ(γ)).

Like in Section 3, define f˜ : X̃ → C by f˜ := f ◦p with p of (3.1). The

singular locus S̃ of f˜ is again a curve, but it may have components contained in
{τ = 0} (see for instance Example 5.1). Let S̃ ∗ := τ −1 (D∗ ) (so in S̃ we forget
about the components that are in τ −1 (0)) and define the local system H̃ on D∗ as
τ∗ (H̃ n−1 (u)|S̃ ∗ ). Denote its fiber H̃0 at some τ0 with τ0k = t0 and the monodromy
Θ̃0 of H̃0 .
We have
Θ̃0 = (π∗ )−1 ◦Θ0 ◦π∗ , where π(τ ) := τ k ,
and π∗ : H̃0 → H0 is the isomorphism induced by π .

Choose now the base points sı of the connected components Sı∗ of S ∗ in

{t = t0 } where t0 is the base point of D∗ . Moreover choose the base point τ0 ∈ C∗
such that τ0k = t0 .
In order to use the results of [4], we need to guarantee that for the component
Sı∗ of S ∗ , the map p−1 (Sı∗ ) → Sı∗ is the cyclic cover of degree k .
Fix a base point s̃ı ∈ p−1 (Sı∗ ) such that p(s̃ı ) = sı . The local system H̃ n−1 (u) on
the component S̃ı∗ of S̃ ∗ containing s̃ı is given by H̃ n−1 (u)s̃ı which is isomorphic
to H n−1 (u)sı , and the monodromy automorphism Θ̃ı . In case p : S̃ı∗ → Sı∗ is the
cyclic cover of degree k , we have Θ̃ı = (Θı )k .
After Lemma 3.3, this equality is true if k is prime to the degree dı of the
covering t : Sı∗ → D∗ .
Let γ̃ be the element in (H̃ n−1 (u)ı )s˜ı whose image by p is γ . Let σ(γ̃) the
multivalued section of the local system H̃ n−1 (u)ı on S̃ı∗ given by γ̃ on S̃ı∗ . By
 
Oblique Polar Lines of X
|f |2λ |g|2μ  15

construction, if (k, dı ) = 1 , we get Θ̃ı γ̃ = γ̃ . Therefore σ(γ̃) is in fact a global

(singlevalued) section of the local system H̃ n−1 (u)ı over S̃ı∗ .
Theorem 4.1. Notations and hypotheses are those introduced above. Take γ in
H n−1 (u)sı such that Θı (γ) = exp(−2iπl/k)γ where Θı is the monodromy of
H n−1 (u)sı and l is an integer prime to k , between 1 and k − 1 .
Assume that k is relatively prime to the degree of the cover t|Sı∗ of D∗ .
If the section σ(γ̃) of H̃ n−1 (u)ı on S̃ı∗ defined by γ is the restriction to S̃ı∗ of
a global section W on S̃ of the constructible sheaf H̃ n−1 (u) then there exists
ω ∈ Γ(X, Ωn−1
X ) such that the following properties are satisfied:
df l dt
(1) dω = (m + u) ∧ω+ ∧ ω , for some m ∈ N;
f k t
(2) The (n − 1)-meromorphic δu -closed form t−l/k ω/f m induces a section on S
of the sheaf hn−1 (Γl ) whose restriction to Sı∗ is given by Σ(γ);
(3) the current

 
 2λ ¯m−j 2μ df

Tj := Res λ = −m − u, P f μ = −l/k, |f | f |t| ∧ω∧
X f
satisfies d Tj = d Kj for some current Kj supported in the origin and
Tj − Kj is a (n, 0)-current supported in S whose conjugate induces a global
section on S of the sheaf H n[S] (OX ) which is equal to rj (γ) on Sı∗ .
Proof. After [2] and [4], there exist an integer m  0 and a (n − 1)-holomorphic
form ω̃ on X verifying the following properties:
df˜
(i) dω̃ = (m + u) ∧ ω̃ ;
f˜
ω̃
(ii) along S̃ the meromorphic δu -closed form induces the section W ;
˜
fm
(iii) the current

 ˜ 
˜ 2λ ¯
˜m−j df
T̃j := Res λ = −m − u, |f | f ∧ ω̃ ∧ 
X̃ f˜
satisfies d T̃j = d K̃j for some current K̃j supported in the origin and
T̃j − K̃j is a (n, 0)-current supported in S̃ whose conjugate induces the
element rj (W ) of HS̃n (X̃, OX̃ ).
On X̃ we have an action of the group Gk of k th roots of unity that is given
by Z.(x, τ ) := (x, ζτ ) where ζ := exp(2iπ/k). Then X identifies to the complex
smooth quotient of X̃ by this action. In particular every Gk -invariant holomorphic
form on X̃ is the pullback of a holomorphic form on X . For the holomorphic form
ω̃ ∈ Γ(X̃, ΩX̃
n−1
) above, we may write

k−1
ω̃ = ω̃ , with Z ∗ ω̃ = ζ  ω̃ .
l=0
16 D. Barlet and H.-M. Maire

 −j
Indeed, ω̃ = k1 k−1
j,=0 ζ (Z j )∗ ω̃ does the job. Because τ k− ω̃ is Gk -invariant,
there exist holomorphic forms ω0 , . . . , ωk−1 on X such that

k−1
ω̃ = τ −k p∗ ω . (4.3)
=0
Put ω := ωk−l . Because property (i) above is Z -invariant, each ω verifies it and
hence ω , whose pullback by p is τ k−l ω̃k−l , will satisfy the first condition of the
theorem, after the injectivity of p∗ .
The action of Z on γ̃ is Z γ̃ = ζ −l γ̃ ; therefore ω̃k−l verifies (ii) above and
hence for  = k − l , the form ω̃ induces 0 in H̃ n−1 (u) along S̃ .
Let us prove property (3) of the theorem. When ω̃ is replaced by ω̃k−l in
the definition of T̃j , the section it defines on S̃ does not change. On the other
hand, the action of Z on this section is given by multiplication by ζ −l . Because
this section extends through 0 , the same is true for τ k−l T̃j whose conjugate will
define a Gk -invariant section of H n[S] (OX̃ ) extendable through 0 . Condition (3)
follows from the isomorphism of the subsheaf of Gk -invariant sections of H n[S̃] (OX̃ )
and H n[S] (OX ). 

Our next result treats the case where there is a section W̃ of H̃ n−1 (u) on
S̃ ∗ which is not extendable at the origin and induces γ̃ on S̃ı∗ . Remark that there
always exists a global section on S̃ ∗ inducing γ̃ on S̃ı∗ : just put 0 on the branches
S̃ı∗ for each ı = ı.
The next theorem shows that in the case where our section  on S̃ ∗ is not
extendable at the origin, we obtain an oblique polar line for X |f | |g|2μ  .
2λ

Remark that in any case we may apply the previous theorem or the next one.
When S ∗ is not connected, it is possible that both apply, because it may exist at
the same time a global section on S̃ ∗ of the sheaf H̃ n−1 (u) inducing γ̃ on S̃ı∗ which
is extendable at the origin, and another one which is not extendable at the origin.
Theorem 4.2. Under the hypotheses of Theorem 4.1, assume that we have a global
section W̃ on S̃ ∗ of the local system H̃ n−1 (u) inducing γ̃ on S̃ı∗ which is not
extendable at the origin. Then there exists Ω ∈ Γ(X, ΩnX ) and l ∈ [1, k] with the
following properties:
df l dt
(1) dΩ = (m + u) ∧Ω+ ∧ Ω;
f k t

(2) along S ∗ the n-meromorphic (δu − lk dt t
∧)-closed form Ω/f m induces
τ̌1 (σ) = df
f
∧ σ in the sheaf hn (Γl ), for some global section σ on S ∗ of
n−1
the sheaf h (Γl );
(3) the current on X of type (n + 1, 0) with support {0} :

 
  2λ ¯m−j 2μ df

P2 λ = −m − u, P f μ = −(k − l )/k, |f | f |t| ∧Ω∧
X f
n+1
defines a non zero class in H[0] (X, OX ) for j large enough in N.
 
Oblique Polar Lines of X
|f |2λ |g|2μ  17

As a direct consequence, with the help of Corollary 2.3, we obtain:

Corollary
 4.3. Under the hypotheses of Theorem 4.2 , we get an oblique polar line
of X |f |2λ
 |g| 2λ
2μ
 through (−u − j, −l /k) for j  1 and some l ∈ [1, k − 1],
provided X |f |  has only simple poles at −u − q , for all q ∈ N.
Proof. As our assumption implies that H01 (S̃, H̃ n−1 (u)) = 0 , Proposition 10 and
Theorem 13 of [4] imply the existence of Ω̃ ∈ Γ(X̃, ΩnX̃ ) verifying
df˜
(i) dΩ̃ = (m + u) ∧ Ω̃, for some m ∈ N;
f
˜ 1 ([Ω̃]) = 0 , that is Ω̃/f˜m induces, via the isomorphisms (4.1), an element
(ii) ob
in H 0 (S̃ ∗ , H̃ n−1 (u)) which is not extendable at the origin;

(iii) Z Ω̃ = ζ −l Ω, for some l ∈ [1, k] and ζ := exp(−2iπ/k).
Define then γ  ∈ H0 by the following condition: (π∗ )−1 γ  is the value at τ0
1 ([Ω̃]). After condition (iii) we have Θ0 (γ  ) = ζ −l γ  .
of ob
As we did in (4.3), we may write

k−1
Ω̃ = τ −k p∗ Ω .
=0
∗ k−l ˜
Put Ω := Ωk−l . Because p Ω = τ Ω̃k−l and Ω̃ satisfies dΩ̃ = (m+u) dff ∧ Ω̃
for any  , property (1) of the theorem is satisfied thanks to injectivity of p∗ .
Relation (iii) implies that Ω̃k−l induces γ̃  := (π∗ )−1 γ  and Ω̃ induces 0
for  = k − l . Hence condition (2) of the theorem is satisfied.
In order to check condition (3), observe that the image of rj (γ̃  ) in
n+1 n+1
H[0] (X̃, ΩX̃ ) is equal to the conjugate of


 ¯ 
d Res λ = −m − u, |f˜|2λ f˜−j Ω̃k−l ∧ 
X̃

 ¯ df˜ ∧ Ω̃  
= P2 λ = −m − u, |f˜|2λ f˜−j k−l ∧  .
X̃
˜
f
After [4], this current is an analytic nonzero functional supported in the origin in
X̃ . There exists therefore w̃ ∈ Γ(X̃, Ω̃X̃
n+1
) such that

¯ df˜
P2 (λ = −m − u, |f˜|2λ f˜−j ¯ = 0,
∧ Ω̃k−l ∧ χw̃)
X̃ f˜
for any cutoff χ equal to 1 near 0. The change of variable τ → ζτ shows that
w̃ may be replaced by its component w̃k−l in the above relation. With w ∈
∗ k−l
Γ(X, Ωn+1
X ) such that p w = τ w̃k−l we get

  df 
P2 λ = −m − u, |f |2λ f¯−j |t|−2(k−l )/k ∧ Ωk−l ∧ χw̄ = 0. 
X f
18 D. Barlet and H.-M. Maire

Remark 4.4. The case l  = k is excluded because we assume that X |f |2λ  has
only simple poles at −u − q for all q ∈ N. Indeed, if l  = k , the last formula of
the proof above contradicts our assumption.

5. Examples
Example 5.1. n = 2 , f (x, y, t) = tx2 − y 3 . We shall show that both Theorems 4.1
and 4.2 may be applied
 in this example. Thanks to Corollary 4.3 we obtain that
the extension of X |f |2λ |t|2μ  presents an oblique polar line of direction (3, 1)
through (−5/6 − j, −1/2), for j  1 . In fact it follows from general facts that
j = 2 is large enough because here X is a neighborhood of 0 in C3 .
Proof. We verify directly that the standard generator of H 1 (5/6) (which is a local
system of rank 1) on S ∗ := {x = y = 0} ∩ {t = 0} has monodromy −1 =
exp(2iπ.1/2). We take therefore k = 2 and we have f˜(x, y, τ ) := τ 2 x2 − y 3 .
Put
S̃ ∗ = S̃1∗ ∪ S̃2∗ with S̃1∗ := {x = y = 0} ∩ {τ = 0}, S̃2∗ := {τ = y = 0} ∩ {x = 0}.
The form ω̃ := 3xτ dy − 2y d(xτ ) verifies
5 df˜
dω̃ = ∧ ω̃ (5.1)
6 f˜
and ω̃ induces a nonzero element in the H 1 of the Milnor fibre of f˜ at 0 because
it induces on S̃1∗ the pullback of the multivalued section of H 1 (5/6) we started
with. It follows that the form ω of Theorem 4.1 is
ω = 3xt dy − 2yt dx − xy dt.
∗
It verifies p ω = τ ω̃ and hence
5 df 1 dt
dω = ∧ω+ ∧ ω.
6 f 2 t
Theorem 4.2 may be used to see the existence of an oblique polar line as fol-
lows. Construct a section on S̃ ∗ of H̃ 1 (5/6) by setting 0 on S̃2∗ and the restriction
of ω̃ to S̃1∗ . The point is now to prove that this section is not extendable at the
origin. Using the quasi-homogeneity of f˜ it is not difficult to prove by a direct com-
putation that a holomorphic 1 -form near the origin satisfying (5.1) (it is enough
to look at quasi-homogeneous forms of weight 10 ) is proportional to ω̃ . As ω̃ is
invariant by the automorphism (τ, x, y) → (x, τ, y) which leaves f˜ invariant and
exchanges S̃1 and S̃2 , we see that ω̃ does not induce the zero section of H̃ 1 (5/6)
on S̃2∗ . This proves the non extendability of the section on S̃ ∗ defined above.
Another way to see the interaction of strata for f˜ for the eigenvalue
exp(−2iπ5/6) consists in looking at the holomorphic 2 -form Ω̃ := dτ τ ∧ ω̃ =
5 df˜
dτ ∧ (3x dy − 2y dx) that verifies dΩ̃ = 6 f˜ ∧ Ω̃.
 
Oblique Polar Lines of X
|f |2λ |g|2μ  19

Along S̃1∗ we have

5 df˜
Ω̃ = d(ω̃ log τ ) − ∧ ω̃ log τ = δ5/6 (ω̃ log τ ),
6 f˜
after (5.1). Hence the obstruction to write Ω̃ near the origin as δ5/6 (α), where
α is an holomorphic 1 -form near 0 is not zero. More precisely, using [4] we have
θ(Ω̃) = 0 in H 1 (S1∗ , H̃ 1 (5/6)), and also in
H 1 (S ∗ , H̃ 1 (5/6))  H 1 (S1∗ , H̃ 1 (5/6)) ⊕ H 1 (S1∗ , H̃ 1 (5/6)).
This implies that the interaction of strata occurs.
But this implies also from [4] that H{0} 1
(S̃, H̃ 1 (5/6)) = 0 , from which we can
conclude that dim H 0 (S̃, H̃ 1 (5/6)) is strictly less than the dimension of the space
H 0 (S̃ ∗ , H̃ 1 (5/6)) which is equal to 2 . This allows to avoid the direct computation
on the (quasi-homogeneous) holomorphic forms to evaluate the dimension of the
vector space H 0 (S̃, H̃ 1 (5/6)) via the complex with differential δ5/6 .

Notice also that the meromorphic extension of X |f |2λ  does not have a
double pole at −5/6 − j , for all j ∈ N, because interaction of strata is not present
for f at 0 for the eigenvalue exp(−2iπ5/6): the monodromies for H 1 and H 2 of
the Milnor fibre of f at 0 do not have the eigenvalue exp(−2iπ5/6) because they
are of order 3 thanks to homogeneity. 

Example 5.2. n = 2 , f (x, y, t) = x4 + y 4 + tx2 y . The extension of X |f |2λ |t|2μ 
presents an oblique polar line of direction (4, 1) through (−5/8, −1/2).
Proof. The Jacobian ideal of f relative to t, denoted by J/ (f ), is generated by
∂f ∂f
= 4x3 + 2txy and = 4y 3 + tx2 .
∂x ∂y
We have
∂f ∂f
t− 4x = 2(t2 − 8y 2 )xy. (5.2)
∂x ∂y
Put δ := t2 − 8y 2 and notice that for t = 0 the function δ is invertible at (t, 0, 0).
We use notations and results of [5]. Recall that E := Ω2/ d/ f ∧ d/ O is equipped
with two operations a and b defined by aξ = ξf , b( d/ ξ) := d/ f ∧ ξ and a
t-connection b−1 .∇ : P → E that commutes to a and b where
1. ∇ : E → E is given by ∇( d/ ξ) := d/ f ∧ ∂ξ
∂t
− ∂f
∂t
dξ ,
2. P := {α ∈ E | ∇(α) ∈ bE} .
Relation (5.2) gives
2xyδ = d/ f ∧ (t dy + 4x dx) = d/ f ∧ d/ (ty + 2x2 ); (5.3)
hence xyδ = 0 ∈ E, and xy ∈ J/ (f ) for t = 0 . As a consequence, for t0 = 0 fixed,
x3 and y 4 belong to J(ft0 ). Therefore the (a, b)-module Et0 := E/(t − t0 )E has
rank 5 over C[[b]]. The elements 1, x, y, x2 , y 2 form a basis of this module.
20 D. Barlet and H.-M. Maire

We compute now the structure of (a, b)-module of E over the set {t = 0} ,

i.e., compute the action of a on the basis. Let us start with

a(y 2 ) = x4 y 2 + y 6 + tx2 y 3 .

Relation (5.2) yields

2x2 yδ = d/ f ∧ (tx dy + 4x2 dx) = t.b(1) (5.4)

and also



2 tx dy + 4x2 dx 1 4
x y=b d = b(1) + 3 b(y 2 ) + b2 .E. (5.5)
2δ 2t t
From

b(1) = d/ f ∧ (x dy) = 4x4 + 2tx2 y = d/ f ∧ (−y dx) = 4y 4 + tx2 y

we get
8
4x4 = − b(y 2 ) + b2 E. (5.6)
t2
Therefore
1 4
4y 4 = b(1) − tx2 y = b(1) − 2 b(y 2 ) + b2 E. (5.7)
2 t
The relation
x3 y dy + 4x4 y dx
x4 y 2 = d/ f ∧
2δ
deduced from (5.3) shows x4 y 2 ∈ b2 E.
Relation (5.4) rewritten as 2t2 x2 y = tb(1) + 16x2 y 3 yields

t2 2 t
x2 y 3 = x y − b(1).
8 16
Moreover
∂f
y 3 (4y 3 + tx2 ) = y 3 = d/ f ∧ (−y 3 dx) = 3b(y 2 )
∂y
and hence
4y 6 = −tx2 y 3 + 3b(y 2 ).
On the other hand

b(y 2 ) = d/ f ∧ (xy 2 dy) = 4x4 y 2 + 2tx2 y 3 = 2tx2 y 3 + b2 E.

Finally
t 3 9
a(y 2 ) = x4 y 2 + y 6 + tx2 y 3 − x2 y 3 + b(y 2 ) + tx2 y 3 + b2 E = b(y 2 ) + b2 E.
4 4 8
 
Oblique Polar Lines of X
|f |2λ |g|2μ  21

Now, after (5.6), (5.7) and (5.5) we obtain successively

a(1) = x4 + y 4 + tx2 y
2 1 1 1 4
= − 2 b(y 2 ) + b(1) − 2 b(y 2 ) + b(1) + 2 b(y 2 ) + b2 E
t 8 t 2 t
5 1
= b(1) + 2 b(y ) + b E,
2 2

 8 t

2y 2 5 1 29 5 2y 2
a 1− 2 = b(1) + 2 b(y 2 ) − 2 b(y 2 ) + b2 E = b 1 − 2 + b2 E.
t 8 t t 8 8 t
Some more computations of the same type left to the reader give
a(x) = b(x) + b2 E,
7
a(y) = b(y) + b2 E,
8
11 2
a(x2 ) = b(x ) + b2 E.
8
2
Let us compute the monodromy M of t on the eigenvector v0 := 1− 2y
t2
+bE.
Because b−1 ∇ = ∂t
∂
it is given by
M = exp(2iπtb−1∇).
We have

1 4y 2
∇(1) = −x2 y = −b 1+ 3 + b2 E
2t t
and hence
∂ −1 4y 2
t (1) = 1 − 2 + bE.
∂t 2 t
Also


t2 2 t t2 1 4 t
∇(y 2 ) = −x2 y 3 = − x y + b(1) = − b(1) + 3 b(y 2 ) + b(1) + b2 E
8 16 8 2t t 16
gives
∂ 2 1
(y ) = − y 2 + bE.
t
∂t 2
Hence


∂ 2y 2 1 2y 2
t 1− 2 − 1 − 2 + bE
∂t t 2 t
from what we deduce M v = −v .
An analogous computation with τ 2 = t, shows that the eigenvector ṽ is
invariant under M̃ . On the other hand, the relation
2y 2
ṽ = 1 − + bẼ
τ4
where Ẽ is associated to the pair (f˜, τ ), with f˜(x, y, τ ) := x4 + y 4 + τ 2 x2 y , shows
that ṽ does not extend through 0 as a section of Ẽ .
22 D. Barlet and H.-M. Maire

This last assertion may be proved directly. It suffices to show that there does
not exist a holomorphic non-trival2 1 -form ω̃ near 0 such that
5 df˜
dω̃ = ∧ ω̃. (5.8)
8 f˜
Because f˜ is quasi-homogeneous of degree 8 with the weights (2, 2, 1) and because
ω̃/f˜5/8 is homogeneous of degree 0 , the form ω̃ must be homogeneous of degree
5 . So we may write
ω̃ = (α0 + α1 τ 2 + α2 τ 4 )dτ + τ β0 + τ 3 β1
where αj and βj are respectively 0 - and 1 -homogeneous forms of degree 2 − j
with respect to x, y . Setting β := β0 + τ 2 β1 , we get
df˜ ∧ ω̃ = d/ f˜ ∧ τ β and dω̃ = τ d/ β modulo dτ ∧ .
With (5.8) we deduce
8f˜ d/ β = 5 d/ f˜ ∧ β
and an easy computation shows that this can hold only if β = 0 . In that case
α = 0 also and the assertion follows. 
Remark 5.3. It is easy to check that in the previous example the holomorphic
2 -form
Ω̃ := 2x dy ∧ dτ + 2y dτ ∧ dx + τ dx ∧ dy
satisfies
5 df˜
dΩ̃ = ∧ Ω̃
8 f˜
but it is not as easy as in Example 5.1 to see that ob˜ 1 ([Ω̃]) is not an extendable
section of H̃ 1 (5/8) or that θ(Ω̃) is not 0 in H 1 (S̃ ∗ , H̃(5/8)). The reason is that
the transversal singularity is much more complicated in this example, and the
development above consists precisely to compute the section ob ˜ 1 ([Ω̃]) on S̃ ∗ of
the sheaf H̃(5/8) induced by [Ω̃] and to see that it does not extend at the origin
which means, with the notations of [4], that ob ˜ 1 ([Ω̃]) = 0 in H 1 (S̃, H̃(5/8)).
{0}

References
[1] D. Barlet, Contribution du cup-produit de la fibre de Milnor aux pôles de |f |2λ , Ann.
Inst. Fourier (Grenoble) 34 (1984), 75–107.
[2] , Contribution effective de la monodromie aux développements asymptotiques,
Ann. Sci. Ecole Norm. Sup. 17 (1984), 293–315.

[3] , Monodromie et pôles du prolongement méromorphe de X |f |2λ  , Bull. Soc.
Math. France (1986), 247–269.
[4] , Interaction de strates pour les cycles évanescents, Ann. Sci. Éc. Norm. Sup.
(4) 40 (1991), 401–505.

2 that is, not inducing 0 in the Milnor fibre of f˜ at 0

 
Oblique Polar Lines of X
|f |2λ |g|2μ  23

[5] , Sur certaines singularités d’hypersurfaces II, J. Algebraic Geometry 17

(2008), 199–254.
[6] D. Barlet and H.-M. Maire, Développements asymptotiques, transformation de Mellin
complexe et intégration sur les fibres, Springer Lecture Notes 1295, Springer, 1987,
pp. 11–23.
[7] , Asymptotic expansion of complex integrals via Mellin transform, J. Funct.
Analysis 83 (1989), 233–257.
[8] , Asymptotique des intégrales-fibres, Ann. Inst. Fourier (Grenoble) 43 (1993),
1267–1299.
[9] I.N. Bernstein, The analytic continuation of generalised functions with respect to a
parameter, Funkcional. Anal. i Prilozhen. 6 (1972), no. 4, 26–40.
[10] I.N. Bernstein and S.I. Gelfand, Meromorphic properties of the function P λ ,
Funkcional. Anal. i Prilozhen. 3 (1969), no. 1, 84–85.
[11] J.-E. Björk, Rings of differential operators, North-Holland Publishing Co., Amster-
dam-New York, 1979.
[12] H. Hironaka, Resolution of singularities of an algebraic variety over a field of char-
acteristic zero I, Ann. of Math. 79 (1964), 109–326.
[13] F. Loeser, Fonctions zêta locales d’Igusa à plusieurs variables, intégration dans les
fibres, et discriminants, Ann. Sci. Ecole Norm. Sup. 22 (1989), 435–471.
[14] C. Sabbah, Proximité évanescente II, Compositio Math. 64 (1987), 213–241.

D. Barlet
Institut Élie Cartan
Nancy-Université
BP 239
F-54506 Vandoeuvre-lès-Nancy
e-mail: [email protected]
H.-M. Maire
Université de Genève
2–4, rue du Lièvre
Case postale 64
CH-1211 Genève 4
e-mail: [email protected]
Complex Analysis
Trends in Mathematics, 25–50

c 2010 Springer Basel AG

On Involutive Systems of First-order Nonlinear

Partial Differential Equations
S. Berhanu
Dedicated to Linda Rothschild

Abstract. We study the local and microlocal analyticity of solutions u of a

system of nonlinear pdes of the form
Fj (x, u, ux ) = 0, 1≤j≤n
where the Fj (x, ζ0 , ζ) are complex-valued, real analytic in an open subset of
(x, ζ0 , ζ) ∈ RN ×C×CN and holomorphic in (ζ0 , ζ). The function u is assumed
to be a C 2 solution. The Fj satisfy an involution condition and dζ F1 ∧ · · · ∧
dζ Fn = 0 where dζ denotes the exterior derivative in ζ = (ζ1 , . . . , ζN ).
Mathematics Subject Classification (2000). 35A18, 35B65, 35F20.
Keywords. Analytic wave-front set, linearized operator.

0. Introduction
In this article we will study the local and microlocal regularity of solutions of
certain overdetermined systems of first-order pdes of the form
Fj (x, u, ux ) = 0, 1≤j≤n
which are involutive. For some of the results we will only assume that u is a solution
in some wedge W. As an application of our results, in Example 2.9 we will study
the analyticity of solutions u of the nonlinear system

m
∂u ∂φ ∂u ∂u
−i = ak + fk , 1 ≤ k ≤ n,
∂z k ∂z k ∂s ∂s
where m ≥ 2 is any integer, a = (a1 , . . . , an ) ∈ Cn , x = (x1 , . . . , xn ), y =
(y1 , . . . , yn ), and s are coordinates in R2n+1 and φ = φ(x, y), fk (x, y, s) are real
analytic functions.

Work supported in part by NSF DMS 0714696.

26 S. Berhanu

The nonlinear systems considered here are generalizations of the linear case where
one considers a pair (M, V) in which M is a manifold, and V is a subbundle of
the complexified tangent bundle CT M which is involutive, that is, the bracket of
two sections of V is also a section of V. We will refer to the pair (M, V) as an
involutive structure. The involutive structure (M, V) is called locally integrable if
the orthogonal of V in CT ∗ M is locally generated by exact forms. A function (or
distribution) u is called a solution of the involutive structure (M, V) if Lu = 0
for every smooth section L of V. When (M, V) is locally integrable, the local and
microlocal regularity of solutions u has been studied extensively. Some of these
results were extended in [EG] where regularity results were proved for boundary
values of solutions defined in a wedge W in M. Microlocal regularity of solutions
for a single nonlinear equation in the C ∞ or analytic cases was investigated in the
papers [A], [B], [Che], and [HT]. For results on the microlocal analytic regularity
of solutions of higher-order linear differential operators, we mention the works [H1]
and [H2].
When (M, V) is an involutive structure, near a point p ∈ M, one can always
choose local coordinates (x, t), x = (x1 , . . . , xm ), t = (t1 , . . . , tn ), m + n = dim M
and smooth vector fields
∂ m
∂
Lj = + ajk (x, t) , 1 ≤ j ≤ n,
∂tj ∂xk
k=1

such that {L1 , . . . , Ln } form a basis of V on some neighborhood U of p. A distri-

bution u ∈ D  (U ) is a solution if
Lj u = 0, 1 ≤ j ≤ n. (0.1)
Likewise, for the nonlinear system Fj (x, u, ux ) = 0, 1 ≤ j ≤ n studied here, we
can choose local coordinates (x, t) ∈ Rm × Rn so that the equations take the form
utj = fj (x, t, ux ), 1 ≤ j ≤ n.
Complex-valued solutions of first-order nonlinear pdes arise in several geo-
metric and physical situations (see [CCCF], [KO1], and [KO2]).
This article is organized as follows. In section 1 we introduce involutive systems
of first-order nonlinear pdes. The reader is referred to [T1] for a more detailed
treatment of the subject. In the same section we will recall some concepts from
[EG] that we will need to state our results. Section 2 contains the statements of
our results and some examples. In particular, Example 2.9 shows a class of systems
of involutive nonlinear pdes where the solutions are always real analytic. Sections
3 and 4 are devoted to the proofs of these results.

1. Preliminaries
The complex one-jet bundle of a smooth manifold M will be denoted by CJ 1 (M).
An element of CJ 1 (M) is a triple (x, a, ω) where x ∈ M, a ∈ C, and ω ∈ CTx∗ M.
We can therefore identify CJ 1 (M) with C × CT ∗ M. Let dim M = N and suppose
 On Involutive Systems of First-order Nonlinear Pdes 27

U is the domain of local coordinates x1 , . . . , xN . Let ζ1 , . . . , ζN denote the corre-

sponding complex coordinates in CTx∗ M at x ∈ U . Let O = C × (CT ∗ M|U ) ∼ =
U ×CN +1 be the open subset of the one-jet bundle that lies over U . The coordinate
in the first factor of CN +1 will be denoted by ζ0 .
Let F1 (x, ζ0 , ζ), . . ., Fn (x, ζ0 , ζ) be n C ∞ functions on O that are holomorphic
in (ζ0 , ζ). We will be studying systems of pdes of the form
Fj (x, u, ux ) = 0, 1≤j≤n (1.1)
generalizing the linear system (0.1). The linear independence of the system in (0.1)
is generalized by assuming that
dζ F1 ∧ · · · ∧ dζ Fn = 0 at every point of O. (1.2)
Here dζ denotes the exterior derivative with respect to (ζ1 , . . . , ζn ). Condition
(1.2) implies that n ≤ N . Moreover, if there is a point p ∈ O where F1 (p) = · · · =
Fn (p) = 0, then the set
Σ = {(x, ζ0 , ζ) ∈ O : Fj (x, ζ0 , ζ) = 0, 1 ≤ j ≤ n} (1.3)
is a smooth submanifold of O whose intersection with each fiber C N +1
is a holo-
morphic submanifold of complex dimension N + 1 − n.
The involutivity of the linear system (0.1) is generalized by using a notion of
Poisson brackets which we will now describe. If F = F (x, ζ0 , ζ) is a smooth function
on O which is holomorphic in (ζ0 , ζ), we define the holomorphic Hamiltonian of F
by
N N

∂F ∂ ∂F ∂F ∂
HF = − + ζi
i=1
∂ζi ∂xi i=1 ∂xi ∂ζ0 ∂ζi
N 
∂F ∂ ∂F
+ ζi −F + . (1.4)
i=1
∂ζi ∂ζ 0 ∂ζ 0

If G = G(x, ζ0 , ζ) is also a similar function, we define the Poisson bracket {F, G}

by
{F, G} = HF G = −HG F. (1.5)
Observe that for the class of functions being considered, the definition of the
Poisson bracket is independent of the choice of local coordinates x1 , . . . , xN since
this is true for each of the three vector fields
N
∂F ∂ N
∂F ∂ N
∂ ∂
− , ζi , and .
∂ζi ∂xi ∂xi ∂ζi ∂ζi ∂ζ0
i=1 i=1 i=1

Going back to the equations (1.1), the involutivity condition can be expressed
as:
{Fj , Fk } = 0 on the set (1.3) ∀ j, k = 1, . . . , n. (1.6)
Condition (1.6) is a formal integrability condition for the system of equations
Fj (x, u, ux ) = 0, j = 1, . . . , n. Indeed, suppose for every (x0 , ζ0 , ζ  ) in the set Σ
of (1.3), there is a C 2 solution u of the equations Fj (x, u, ux ) = 0, j = 1, . . . , n
28 S. Berhanu

near the point x0 satisfying u(x0 ) = ζ0 , and ux (x0 ) = ζ  . Let F = Fm , and
G = Fk for some m, k ∈ {1, . . . , n}. Differentiating the equations F (x, u, ux ) = 0
and G(x, u, ux ) = 0 we get:

N
Fxi + Fζ0 uxi + Fζj uxj xi = 0, 1 ≤ i ≤ n (1.7)
j=1

at the points (x, u(x), ux (x)), and likewise,

N
Gxi + Gζ0 uxi + Gζj uxj xi = 0, 1 ≤ i ≤ n. (1.8)
j=1

Multiply (1.7) by −Gζi , (1.8) by Fζi and add over i to get:

N
N
N
N
Fζi (Gxi + Gζ0 uxi ) − Gζi (Fxi + Fζ0 uxi ) = (Gζi Fζj − Gζj Fζi )uxj xi .
i=1 i=1 i=1 j=1

The right-hand side of the preceding equation is clearly zero and hence, at the
points (x, u(x), ux (x)), we get

N
N
Fζi (Gxi + Gζ0 uxi ) − Gζi (Fxi + Fζ0 uxi ) = 0.
i=1 i=1
Since (x, u(x), ux (x) ∈ Σ, the latter equation implies that at such points,

N
N
Fζi (Gxi + Gζ0 uxi ) − Gζi (Fxi + Fζ0 uxi ) + Fζ0 G − F Gζ0 = 0,
i=1 i=1
that is, HF G = {F, G} = 0 on Σ.
A global definition of involutive systems can now be stated as follows:
Definition 1.1. An involutive system of first-order partial differential equations of
rank n on M is a closed C ∞ submanifold Σ of CJ 1 (M) satisfying the following
properties:
(i) the projection CJ 1 (M) → M maps Σ onto M;
(ii) each point of Σ has a neighborhood O on which there are n C ∞ functions
F1 (x, ζ0 , ζ), . . . , Fn (x, ζ0 , ζ), holomorphic with respect to (ζ0 , ζ), such that
Σ ∩ O is the set (1.3), and that (1.2) and (1.6) hold.
Definition 1.2. Let Σ be as in Definition 1.1. A C 1 function u on M is called a
solution if its first jet lies in Σ.
Example 1.3. Let (M, V) be an involutive structure. Let V ⊥ ⊆ CT ∗ M denote
the one-forms ω such that ω, L
= 0 for every section L of V. Let Σ be the
preimage of V ⊥ under the projection CJ 1 (M) → CT ∗ M. Let {L1 , . . . , Ln } be a
smooth basis of V over an open set U such that [Li , Lj ] = 0 ∀ i, j = 1, . . . , n.
Then, over U , Σ is defined by the vanishing of the symbols Fj (x, ζ) (ζ ∈ CN ) of
the Lj regarded as functions on CJ 1 (M)|U . Property (1.2) follows from the linear
 On Involutive Systems of First-order Nonlinear Pdes 29

independence of the Lj and (1.6) is a consequence of the fact that the Lj commute.
Thus every involutive structure (M, V) defines an involutive system of first-order
partial differential equations on M.
We will next recall some notions on involutive structures that we will need
to state our results. The reader is referred to [EG] for more details.
Let (M, V) be a smooth involutive structure with fiber dimension of V over
C equal to n.
Definition 1.4. A smooth submanifold X of M is called maximally real if
CTp M = Vp ⊕ CTp X for each p ∈ X.
Note that if (M, V) is CR, then X is maximally real if and only if it is totally real
of maximal dimension.
If X is a maximally real submanifold and p ∈ X, define
VpX = {L ∈ Vp : L ∈ Tp X}.
We recall the following result from [EG]:
Proposition 1.5. (Lemma II.1 in [EG]) V X is a real subbundle of V|X of rank n.
The map
 : VX → T M
which takes the imaginary part induces an isomorphism
VX ∼
= T M|X /T X.
Proposition 1.5 shows that when X is maximally real, for p ∈ X,  defines
an isomorphism from VpX to an n-dimensional subspace Np of Tp M which is a
canonical complement to Tp X in the sense that
Tp M = T p X ⊕ Np .
Definition 1.6. Let E be a submanifold of M, dim E = k. We say an open set W
is a wedge in M at p ∈ E with edge E if the following holds: there exists a diffeo-
morphism F of a neighborhood V of 0 in RN (N = dim M) onto a neighborhood
U of p in M and a set B × Γ ⊂ V with B a ball centered at 0 ∈ Rk and Γ a
truncated, open convex cone in RN −k with vertex at 0 such that
F (B × Γ) = W and F (B × {0}) = E ∩ U.
Definition 1.7. Let E, W and p ∈ E be as in Definition 1.6. The direction wedge
Γp (W) ⊂ Tp M is defined as the interior of the set
{c (0)| c : [0, 1) → M is C ∞ , c(0) = p, c(t) ∈ W ∀ t > 0}.
Observe that Γp (W) is a linear wedge in Tp M with edge Tp E. Set

Γ(W) = Γp (W)
p∈E
30 S. Berhanu

Suppose W is a wedge in M with a maximally real edge X. As observed in

[EG], since Γp (W) is determined by its image in Tp M/Tp X, the isomorphism 
can be used to define a corresponding wedge in VpX by setting

ΓV
p (W) = {L ∈ Vp : L ∈ Γp (W)}.
X

ΓV
p (W) is a linear wedge in Vp with edge {0}, that is, it is a cone. Define
X

next
ΓTp (W) = {L : L ∈ ΓV
p (W)}.

Since the map  : VpX → Vp ∩ Tp X is onto, ΓTp (W) is an open cone in
Vp ∩ Tp X. Let

ΓT (W) = ΓTp (W).
p∈X

2. Main results and examples

In this section we will assume that M is a real analytic manifold and we will
consider a real analytic involutive system Σ by which we mean that the local
defining functions Fj in Definition 1.1 are real analytic. A real analytic wedge W
in M is one where the diffeomorphism F in Definition 1.6 is real analytic. Suppose
u is a C 2 solution of the involutive system Σ on an open set U ⊆ M which is a
domain of local coordinates x1 , . . . , xN . Let O be an open set in CJ 1 (M)|U and
F1 (x, ζ0 , ζ), . . . , Fn (x, ζ0 , ζ) functions in O satisfying the conditions of Definition
1.1. Consider the vector fields on U defined by

N
∂Fj ∂
Luj = (x, u(x), ux (x)) , 1 ≤ j ≤ n.
∂ζk ∂xk
k=1

If for v ∈ C 1 (U ) we define Fjv by

Fjv (x) = Fj (x, v(x), vx (x)),
then we see that Luj (v) is the principal part of the Fréchet derivative of the map
v → Fjv
at u. We will refer to the vector fields Luj as the linearized operators of Fj (x, u(x),
ux (x)) = 0 at u. Since the Fj satisfy (1.2), Lu1 , . . . , Lun are linearly independent and
span a bundle V u over U . If G1 (x, ζ0 , ζ), . . . , Gn (x, ζ0 , ζ) also satisfy Definition 1.1,
n
then each Gk = aij Fj for some aij ∈ C ∞ (O) and so the linearized operators
j=1
of Gk (x, u(x), ux (x)) = 0 at u are linear combinations of the Luj . In Section 3 we
will see that V u is closed under brackets.
In what follows, W Fa u denotes the analytic wave front set of a function u.
 On Involutive Systems of First-order Nonlinear Pdes 31

The main results of this article can be stated as follows:

Theorem 2.1. Let Σ be a real analytic involutive system of first-order partial differ-
ential equations of rank n on a real analytic manifold M. Let W be a real analytic
wedge in M with a real analytic edge E. Suppose u ∈ C 2 (W) and it is a solution
on W. Assume that
CTp M = CTp E ⊕ Vpu ∀ p ∈ E.
Then W Fa (u0 ) ⊆ (ΓT (W))◦ , where u0 = u|E , and polar refers to the duality
between T E and T ∗ E.

Theorem 2.1 will be proved in Section 3. In the linear case, Theorem 2.1 was
proved in [EG].
Given an involutive structure (M, V), for p ∈ M, let Tp◦ (M) = Vp⊥ ∩Tp∗ (M) which
is the characteristic set of V at p. If W is a wedge in M with edge E, recall that
ΓTp (W) is an open cone in Vp ∩ Tp E.
 ⊥
Since Vp ∩ Tp E = ι∗E (Tp◦ M) , where ι∗E : Tp∗ (M) → Tp∗ (E) is the pullback
map, we have:
 ◦
ι∗E (Tp◦ M) ⊆ ΓTp (W) .
As in the linear case (see [EG]), if in the nonlinear context of Theorem 2.1 u0 = u|E
is the trace of two solutions defined in opposite wedges, we get the following
stronger conclusion:

Corollary 2.2. Let Σ be a real analytic involutive system of first-order partial dif-
ferential equations of rank n on a real analytic manifold M. Let W + and W − be
real analytic wedges in M with a real analytic edge E having opposite directions
at p ∈ E, that is Γp (W + ) = −Γp (W − ). Let u+ ∈ C 2 (W + ), u− ∈ C 2 (W − ) be solu-
tions in W + and W − respectively such that u+ |E = u0 = u− |E , and assume that
+ −
CTp M = CTp E ⊕ Vp (where Vp = Vpu = Vpu ). Then, W Fa (u0 )|p ⊆ ι∗E (Tp◦ (M)).

Proof. By Theorem 2.1 we know that W Fa (u0 ) ⊆ (ΓTp (W + ))◦ ∩ (ΓTp (W − ))◦ . We
also always have
ι∗E (Tp◦ (M)) ⊆ (ΓTp (W + ))◦ ∩ (ΓTp (W − ))◦ .

If σ ∈ (ΓTp (W + ))◦ ∩ (ΓTp (W − ))◦ , then σ, v

= 0 ∀ v ∈ ΓTp (W + ). But then since
ΓTp (W + ) is open in (Re Vp ) ∩ Tp E, σ has to vanish on Vp ∩ Tp E = (ι∗E (Tp◦ M))⊥
and hence σ ∈ ι∗E (Tp◦ M). Thus,
W Fa (u0 ) ⊆ ι∗E (Tp◦ M) 

In Section 3 we will show that Corollary 2.2 implies a generalization of the

classical edge of the wedge theorem for nonlinear systems of pdes (see Corol-
lary 3.6).
32 S. Berhanu

When u is a solution in a full neighborhood, we get:

Corollary 2.3. Let Σ be a real analytic involutive system of first-order partial dif-
ferential equations of rank n on a real analytic manifold M. Let u be a C 2 solution.
Then
W Fa u ⊆ (V u )⊥ ∩ T ∗ M = T ◦ M.
Proof. We will use a trick from [HT]. We may assume that in a neighborhood of the
origin in RN , u satisfies the system of equations Fj (x, u, ux ) = 0 for j = 1, . . . , n
where the Fj are real analytic and satisfy the conditions in Definition 1.1. Introduce
additional variables y1 , . . . , yn with respective dual coordinates η1 , . . . , ηn and for
each θ ∈ [0, 2π), define
Gθj (x, y, ζ0 , ζ, η) = ηj − eiθ Fj (x, ζ0 , ζ), 1 ≤ j ≤ n.
Let w(x, y) = u(x). Note that Gθj (x, y, w, wx , wy ) = 0 for all j. It is easy to see
that the Gθj satisfy conditions (1.2) and (1.6). The linearized operators at w are:

∂ ∂Fj
N
∂
Lw
θ,j = − eiθ (x, u(x), ux (x)) , 1 ≤ j ≤ n.
∂yj ∂ζk ∂xk
k=1

Let M = RN × Rn and E = RN × {0}. Note that the characteristic set of the Lw

x y x θ,j
 is
at the origin in M
 = {(0; ξ, η) : ηj = eiθ Dζ Fj (0, u(0), ux(0)) · ξ, j = 1, . . . , n}.
Tθ0 M
It follows that
 = {(0; ξ) : eiθ Dζ Fj (0, u(0), ux (0)) · ξ is real for j = 1, . . . , n}.
ι∗E (Tθ0 M)
Observe that at the origin in M, the characteristic set of the operators

N
∂Fj ∂
Luj = (x, u(x), ux (x)) , 1≤j≤n
∂ζk ∂xk
k=1
  It is easy to see that CT0 M  = CT0 E ⊕ V w where
equals 0≤θ≤2π ι∗E (Tθ0 M). θ
θ,j at the origin. Let Γ be any open cone in Ry and set
Vθw is the span of the Lw + n

Γ− = −Γ+ . Then u(x) is the common boundary value of the solution w(x, y) of the
Gθj defined in opposite wedges and hence by Corollary 2.2, W Fa (u) ⊂ T 0 M. 

When n = 1, Corollary 2.3 was proved in [HT]. Again when n = 1 and the
pde is quasilinear, Theorem 2.1 was proved in [M].
We will next present a nonlinear wedge version of the classical Lewy extension
theorem.
Theorem 2.4. Let Σ be a real analytic involutive system of first-order partial differ-
ential equations of rank n on a real analytic manifold M. Let W be a real analytic
wedge in M with a real analytic edge E.
 On Involutive Systems of First-order Nonlinear Pdes 33

Suppose u ∈ C 2 (W) and it is a solution on W, and assume that

CTp M = CTp E ⊕ Vpu ∀ p ∈ E.

Let σ ∈ (Vpu )⊥ ∩ Tp∗ M = Tp◦ M for some p ∈ E. If L is a C 1 section of V u defined

near p in W such that L(p) ∈ ΓV p (W) and

1
σ, [L, L̄]
< 0, then i∗E σ ∈
/ W Fa u0 (u0 = u|E ).
i
When the solution u is defined in a full neighborhood of a point p, by adding
variables as in the proof of Corollary 2.3, we get:

Corollary 2.5. Let Σ be a real analytic involutive system of first-order partial dif-
ferential equations of rank n on a real analytic manifold M. Suppose u is a C 2
solution in a neighborhood of a point p. Let σ ∈ (Vpu )⊥ ∩ Tp∗ M = Tp◦ M. If L is a
C 1 section of V u defined near p such that 1i σ, [L, L̄]
< 0, then σ ∈
/ W Fa u.

 and E be as in
Proof. We assume the point p is the origin of RN . Let the Fj , M,
the proof of Corollary 2.3. For each 1 ≤ j ≤ n, let
Gj (x, y, ζ0 , ζ, η) = bj ηj − Fj (x, ζ0 , ζ)
where the bj are constants that will be chosen. If w(x, y) = u(x), then it satisfies
the equations Gj (x, y, w, wx , wy ) = 0 for all j, and the linearizations at w are

∂ ∂Fj
N
∂
Lw
j = bj − (x, u(x), ux (x)) , 1 ≤ j ≤ n.
∂yj ∂ζk ∂xk
k=1

Let

N
∂Fj ∂
Luj = (x, u(x), ux (x)) , 1 ≤ j ≤ n.
∂ζk ∂xk
k=1
n u 
n w
Let L = k=1 ak Lk and define L = k=1 ak Lk . We choose the bj so that
 
L (0) ∈ T0 E, and L (0) ∈ / T0 E. This is possible since L(0) = 0 because
by hypothesis, σ, [L, L]
= 0. Let W be a wedge in M  with edge E such that
w
L (0) ∈ ΓV0 (W) where V w
is the bundle generated by the Lw 
j . Let σ = (σ, 0). If
 → T ∗ E denotes the pullback map, ι∗ σ  = σ, and
ι∗E : T0∗ M 0

1    1
σ , [L , L̄ ]
= σ, [L, L̄]
< 0.
i i
Hence, since w|E = u, by Theorem 2.4, σ ∈
/ W Fa u. 

In the linear case, Theorem 2.4 was proved in [EG]. When n = 1 and the pde
is quasilinear, Theorem 2.4 was proved in [LMX] under a stronger assumption. For
n = 1 Corollary 2.5 was obtained in [B].
34 S. Berhanu

We will indicate next why Theorem 2.2 in [LMX] is a special case of Theorem 2.4.
We recall Theorem 2.2 in [LMX]. Consider the quasi-linear equation
∂u
N
∂u
+ aj (x, t, u) = b(x, t, u)
∂t j=1 ∂xj

on Ω×[0, T ) where Ω is an open subset of RN , and the functions aj , b, j = 1, . . . , N

are restrictions of holomorphic functions. Let
∂ N
∂ ∂
L= + aj (x, t, ζ0 ) + b(x, t, ζ0 ) .
∂t j=1 ∂xj ∂ζ0

Set ν0 = (a1 , . . . , aN ), ν1 = L(ν0 ) = (L(a1 ), . . . , L(aN )).

Theorem 2.6. (Theorem 2.2 in [LMX]). Let u be a C 2 solution of the quasi-linear
equation on Ω × [0, T ). If
∀x ∈ Ω, ν0 (x, 0, u(x, 0)) = 0, and ν1 (x0 , 0, u(x0 ), 0) · ξ > 0,
then (x0 , ξ) ∈
/ W Fa u(x, 0).
We will deduce this result from Theorem 2.4 by showing that whenever
ν0 (x0 , 0, u(x0 , 0)) = 0, then
(ξ, 0), [Lu , Lu ](x0 , 0)

ν1 (x0 , 0, u(x0 , 0)) · ξ = − ,

2i
where we recall that
∂ N
∂u
Lu = + aj (x, t, u) .
∂t j=1 ∂xj
Note that we will not require the vanishing of ν0 (x, 0, u(x, 0)) for all x. We have
⎛ ⎞
[Lu , Lu ] ∂aj ∂a ∂u ∂
= ⎝ ( ⎠
j
− + )
2i j
∂t ∂ζ 0 ∂t ∂xj
⎛ ⎞
∂aj ∂ ∂aj ∂u ∂
+ ⎝ ak + ak ⎠.
j
∂xk ∂xj j
∂ζ0 ∂xk ∂xj
k k

Using the assumption that ν0 (x0 , 0, u(x0 , 0)) = 0 and the equation that u satis-
fies, we get
⎛ ⎞ ⎛ ⎞
[Lu, Lu ] ∂aj ∂a ∂ ∂a ∂
− = ⎝ ( +b
j
) ⎠ + ⎝ ak
j ⎠
2i j
∂t ∂ζ 0 ∂xj
j
∂xk ∂xj
k

at the point (x0 , 0). Thus for any ξ ∈ R , N

(ξ, 0), [Lu , Lu ](x0 , 0)

ν1 (x0 , 0, u(x0 , 0)) · ξ = − ,

2i
which shows that Theorem 2.6 follows from Theorem 2.4.
 On Involutive Systems of First-order Nonlinear Pdes 35

The following result generalizes the analogous linear result in [EG] (see [C] also).
Theorem 2.7. Let Σ be a real analytic involutive system of first-order partial differ-
ential equations of rank n on a real analytic manifold M. Let W be a real analytic
wedge in M with a real analytic edge E. Suppose u ∈ C 3 (W) and it is a solution
on W, and assume that
CTp M = CTp E ⊕ Vpu ∀ p ∈ E.
Let σ ∈ (Vpu )⊥ ∩ Tp∗ M
= Tp◦ M
for some p ∈ E. If L is a C 2 section of V u defined
near p in W such that L(p) ∈ ΓVp (W), σ, [L, L̄]
= 0, and
√
3| σ, [L, [L, L̄]]
| <  σ, [L, [L, L̄]]
,
then i∗E σ ∈
/ W Fa u0 (u0 = u|E ).
When the solution u is defined in a full neighborhood of a point p, by adding
variables as in the proof of Corollary 2.5, we get:
Corollary 2.8. Let Σ be a real analytic involutive system of first-order partial dif-
ferential equations of rank n on a real analytic manifold M. Suppose u is a C 3
solution in a neighborhood of a point p. Let σ ∈ (Vpu )⊥ ∩ Tp∗ M = Tp◦ M. If L is a
C 2 section of V u defined near p such that σ, [L, L̄]
= 0, and
σ, [L, [L, L̄]]
= 0,
then σ ∈
/ W Fa u.
Proof. We use the notations of Corollary 2.5. We have
σ , [L , L̄ ]
= σ, [L, L̄]
= 0
and
σ , [L , [L , L̄ ]]
= σ, [L, [L, L̄]]
= 0.
We can therefore choose θ ∈ [0, 2π) such that if L = eiθ L , then
√
3| σ  , [L , [L , L¯ ]]
| <  σ, [L , [L , L¯ ]]
,
and this inequality holds for any choice of the bj . We now choose the bj so that
L (0) ∈ T0 E. Theorem 2.7 then implies that σ ∈/ W Fa u. 
Example 2.9. We consider the following nonlinear system in R2n+1 whose solv-
ability when m = 2 was studied in [T2]. Let xj , yj (j = 1, . . . , n) and s denote the
coordinates in R2n+1 and ξj , ηj , σ the respective dual coordinates. Let m ≥ 2 be
an integer. For k = 1, . . . , n, let
1
pk (x, y, s, ξ, η, σ) = (ξk + iηk ) − ak σ m − ibk (x, y)σ + fk (x, y, s)
2
where ak ∈ C, bk (x, y) and fk (x, y, s) are real analytic functions. It is clear that
dξ p1 ∧ · · · ∧ dξ pn = 0 where ξ = (ξ1 , . . . , ξn ). Assume that a = (a1 , . . . , an ) = 0.
For each k = 1, . . . , n, let
∂ ∂ ∂ ∂
Lk = − (ibk (x, y) + mak σ m−1 ) and L0k = − ibk (x, y) .
∂z k ∂s ∂z k ∂s
36 S. Berhanu

We have
∂ ∂ ∂pk ∂ n ∂pk ∂ n
∂fk ∂
Hpk = − (mak σ m−1 + ibk ) − − − ,
∂zk ∂s i=1 ∂xi ∂ξi i=1 ∂yi ∂ηi ∂s ∂σ
and therefore,

∂bk ∂bj
Hpk pj = iσ − + L k fj − L j fk .
∂z j ∂z k
Hence, the conditions:
∂bk ∂bj
(1) = and (2) Lk fj = Lj fk
∂z j ∂zk
imply the validity of the involution condition {pj , pk } = 0. To guarantee (1), we
∂φ
will assume that for some real analytic φ(x, y), ∂z j
= bj for j = 1, . . . , n. Condition
(2) is equivalent to the pair of equations
∂fj ∂fk
(3) L0k fj = L0j fk and (4) ak = aj .
∂s ∂s
Since a = 0, (4) is equivalent to the existence of a real analytic function W (x, y, s)
such that
(5) fj (x, y, s) = fj (x, y, 0) + aj W (x, y, s), j = 1, . . . , n.
Observe that once (5) holds, condition (3) can be satisfied if for example L0j W (x,
y, s) = 0 ∀j and fj (x, y, 0) = ∂z ∂ψ
j
(x, y) for some real analytic ψ(x, y). We thus
assume that
1 ∂φ
pk (x, y, s, ξ, η, σ) = (ξk + iηk ) − ak σ m − i (x, y)σ + fk (x, y, s)
2 ∂zk
∂ψ
where the fj are as in (5), with fj (x, y, 0) = ∂z j (x, y), and L0j W = 0 for all j. Let
u be a solution of the system
pk (x, y, s, ux , uy , us ) = 0, k = 1, . . . , n,
that is,
m
∂u ∂φ ∂u ∂u
−i = ak + fk .
∂z k ∂z k ∂s ∂s
The linearized operators are

m−1 
∂ ∂φ ∂u ∂
u
Lk = − i + mak , 1 ≤ k ≤ n.
∂z k ∂zk ∂s ∂s

Assume that ∂z ∂φ
k
(0) = 0 ∀k. Observe that at the origin, the characteristic set of
the linearized operators is given by
S = {(ξ, η, σ) : ξk = 2mσ(ak (us (0))m−1 ), ηk = 2mσ(ak (us (0))m−1 )}.
Suppose now a1 = a2 = 0, φ is real valued, φz1 z1 (0) > 0 and φz2 z2 (0) < 0. Then
any C 2 solution u of the nonlinear system is real analytic near the origin. To see
this, first note that by Corollary 2.3, at the origin, W Fa u ⊂ S. Next since S is one
 On Involutive Systems of First-order Nonlinear Pdes 37

dimensional, by applying Corollary 2.5 to the vector fields Lu1 and Lu2 , we conclude
that W Fa u is empty at the origin and so u is real analytic in a neighborhood of
the origin.
Example 2.10. Let u(x, t) be a C 3 function in a neighborhood of the origin in R2
that satisfies the equation
∂u ∂u
+ u2 = f (x, t)
∂t ∂x
where f (x, t) is a real analytic function. We have
∂
[Lu , Lu ] = 2(uut − uut + uux u2 − uux u2 )
∂x
∂
= −4i((f u − u ux ) + (u uux )) .
3 2
∂x
Suppose now u(0, 0) = 0. Then [Lu , Lu ](0, 0) = 0 and
∂
[Lu , [Lu , Lu ]](0, 0) = −4i(f (0)2 ) .
∂x
Therefore, if u(0, 0) = 0 and (f (0)2 ) = 0, then by Corollary 2.8, u is real analytic
at the origin.
Assume next that u is a C 2 solution in the region t ≥ 0, u(0, 0) = 0, and both
u(0, 0) and ux (0, 0) are real. Then [Lu , Lu ](0, 0) = −4iu(0, 0)f (0, 0) ∂x
∂
. Hence if
f (0, 0) = 0, by Theorem 2.4, u(x, 0) is microlocally real analytic either at (0; 1)
or at (0; −1).

3. Some lemmas and the proof of Theorem 2.1

Lemmas 3.2 and 3.4 are stated in [T1] without proofs. Lemma 3.1 was used in
[BGT] and [HT]. In the following lemma, for a C 1 function F (x, ζ0 , ζ) that is
holomorphic in (ζ0 , ζ), HF◦ will denote the principal part of the Hamiltonian HF
defined in (1.4).
Given such a function F and a C 1 function u(x), F u will denote the function
given by
F u (x) = F (x, u(x), ux (x)).
Lemma 3.1. Let u be a C 2 function on an open set U ⊆ M satisfying the system
of equations
Fj (x, u(x), ux (x)) = 0 ∀ j = 1, . . . , n
where the Fj (x, ζ0 , ζ) are C ∞ , holomorphic in (ζ0 , ζ). Let Luj be the linearized op-
erators of Fj (x, u(x), ux (x)) = 0 at u. If G(x, ζ0 , ζ) is a C 1 function, holomorphic
in (ζ0 , ζ), then
 u
Luj (Gu ) = HF◦j G ∀ j = 1, . . . , n.
38 S. Berhanu

Proof. Consider the C 1 mapping

Ψ(x) = (x, u(x), ux (x))

that maps U into (x, ζ0 , ζ) space. It is easy to see that the push forwards Ψ∗ (Luj )
agree with HF◦j on the class of C 1 functions that are holomorphic in (ζ0 , ζ). The
lemma follows. 

Lemma 3.2. The holomorphic Poisson bracket satisfies the Jacobi identity. That is,

{f, {g, h}} + {g, {h, f }} + {h, {f, g}} = 0.

Proof. Observe first that the sum {f, {g, h}} + {g, {h, f }} + {h, {f, g}} is a first-
order differential operator in each function since for example it equals Hf ({g, h})+
[Hg , Hh ](f ). Therefore, in the following computations, the notation p ∼ q will mean
that p − q is a sum of terms each of which has a second-order derivative of p or q.
We will use the notation

[F, G] = {F, G} − Fζ0 G + F Gζ0 .

We have

N
N
{f, {g, h}} = fζi ({g, h}xi + ζi {g, h}ζ0 ) − {g, h}ζi (fxi + ζi fζ0 )
i=1 i=1
+ fζ0 {g, h} − f {g, h}ζ0 .

{g, h}xi = [g, h]xi + (gζ0 h − ghζ0 )xi ∼ gζ0 hxi − gxi hζ0 .

{g, h}ζ0 = [g, h]ζ0 + (gζ0 h − ghζ0 )ζ0 ∼ [g, h]ζ0

∼ 0.

{g, h}ζi = [g, h]ζi + (gζ0 h − ghζ0 )ζi

∼ gζi hζ0 − gζ0 hζi + gζ0 hζi − gζi hζ0 ∼ 0.

It follows that

N
{f, {g, h}} ∼ fζi (gζ0 hxi − gxi hζ0 ) + fζ0 {g, h}
i=1

N
N
= fζi (gζ0 hxi − gxi hζ0 ) + fζ0 gζi (hxi + ζi hζ0 )
i=1 i=1

N
− fζ0 hζi (gxi + ζi gζ0 ) + fζ0 (gζ0 h − ghζ0 ),
i=1
 On Involutive Systems of First-order Nonlinear Pdes 39

N
{g, {h, f }} ∼ gζi (hζ0 fxi − hxi fζ0 ) + gζ0 {h, f }
i=1

N
N
= gζi (hζ0 fxi − hxi fζ0 ) + gζ0 hζi (fxi + ζi fζ0 )
i=1 i=1

N
− gζ0 fζi (hxi + ζi hζ0 ) + gζ0 (hζ0 f − hfζ0 ),
i=1

and

N
{h, {f, g}} ∼ hζi (fζ0 gxi − fxi gζ0 ) + hζ0 {f, g}
i=1

N
N
= hζi (fζ0 gxi − fxi gζ0 ) + hζ0 fζi (gxi + ζi gζ0 )
i=1 i=1

N
− hζ0 gζi (fxi + ζi fζ0 ) + hζ0 (fζ0 g − f gζ0 ).
i=1
Hence the Jacobi identity holds. 
Lemma 3.3. Let u be a C 2 function on an open set U ⊆ M satisfying the system
of equations
Fj (x, u(x), ux (x)) = 0 ∀ j = 1, . . . , n
where the Fj (x, ζ0 , ζ) are C ∞ , holomorphic in (ζ0 , ζ). Assume that the Fj sat-
isfy (1.2) and (1.6). Then there are smooth functions aljk (x, ζ0 , ζ), holomorphic in
(ζ0 , ζ) on the set Σ = {(x, ζ0 , ζ) : Fj (x, ζ0 , ζ) = 0, 1 ≤ j ≤ n}, such that

n
[HFj , HFk ] = aljk (x, ζ0 , ζ)HFl .
l=1

Proof. By Lemma 3.2, [HFj , HFk ] = H{Fj ,Fk } and by hypothesis, {Fj , Fk } = 0 on
Σ. The lemma follows from these equations. 
Lemma 3.4. Let u and the Fj be as in Lemma 3.3. Let V u be the bundle generated
by Lu1 , . . . , Lun . Then V u is involutive.
Proof. First observe that on the set Σ, Lemma 3.3 implies for the principal parts
HF◦j that

n
◦ ◦
[HFi , HFj ] = alij HF◦l (3.1)
l=1
For each i, j = 1, . . . , n, write

m
∂
[Lui , Luj] = crij (x) .
r=1
∂xr
40 S. Berhanu

Since the coefficients of the Hamiltonians HFi are holomorphic in (ζ0 , ζ), by
Lemma 3.1 we have:
    u
Lui Luj (xr ) = HF◦i HF◦j (xr )
and so
   u
crij (x) = Lui , Luj (xr ) = HF◦i , HF◦j (xr )
 n u

l ◦
= aij HFl (xr ) (by (3.1))
l=1

n
= alij (x, u(x), ux (x))Lul (xr ).
l=1

Hence [Lui , Luj ] is a linear combination of the Luk . 

Lemma 3.5. Let Σ, u, and E be as in Theorem 2.1. Then near each point p ∈ E, we
can get real analytic coordinates (x, t), x = (x1 , . . . , xm ), t = (t1 , . . . , tn ) vanishing
at p such that in the new coordinates,
utj = fj (x, t, u, ux ), 1≤j≤n
and E = {(x, 0)} near the origin. The fj are real analytic in (x, t, ζ0 , ζ), and
holomorphic in (ζ0 , ζ) in a neighborhood of the origin in Rm × Rn × C × Cm .
Proof. Let p ∈ E. Let U be the domain of local coordinates x1 , . . . , xN that van-
ish at p. We may assume that we have n real analytic functions F1 (x, ζ0 , ζ), . . . ,
Fn (x, ζ0 , ζ) that are holomorphic in (ζ0 , ζ), such that (1.2) and (1.6) hold and
Fj (x, u(x), ux (x)) = 0, 1 ≤ j ≤ n. (3.2)
Condition (1.2) allows us to apply the implicit function theorem to (3.2) to
get new functions (after relabelling coordinates)
Fj (x, ζ) = ζj − fj (x, ζ0 , ζn+1 , . . . , ζN ), 1≤j≤n
where the fj are real analytic and holomorphic in (ζ0 , ζ).
Write tj = xj for 1 ≤ j ≤ n, τj = ζj for 1 ≤ j ≤ n, xi instead of xn+i and ζi
instead of ζn+i for n < i ≤ m = N − n. With this notation, u satisfies:
utj = fj (x, t, u, ux ), 1 ≤ j ≤ n.
The linearized bundle V is generated by
u

∂
m
∂fj ∂
Luj = − (x, t, u, ux ) , 1 ≤ j ≤ n.
∂tj ∂ζl ∂xl
l=1
Near the origin, we may assume that E has the form
E = {(x , g(x , t ), t , h(x , t ))},
where g and h are real analytic, x = (x , x ), t = (t , t ), x ∈ Rk , x ∈ Rm−k ,
t ∈ Rr , t ∈ Rn−r and k + r = m. Change coordinates to y = (y  , y  ) and
 On Involutive Systems of First-order Nonlinear Pdes 41

s = (s , s ) where y  = x , y  = x − g(x , t ), s = t and s = t − h(x , t ). In

these new coordinates, E = {(y  , 0, s , 0)}, and if η = (η  , η  ) and σ = (σ  , σ  )
denote the duals of y = (y  , y  ) and s = (s , s ), u satisfies equations of the form
usj = pj (y, s, u, uy , us ), 1≤j≤n (3.3)
where the pj (y, s, η0 , η, σ ) are real analytic and holomorphic in (η, σ). Let
Fj (y, s, η0 , η, σ) = σj − pj (y, s, η0 , η, σ  ), 1 ≤ j ≤ n.
Since the linearized bundle V of (3.3) is transversal to E = {(y  , 0, s , 0)},
u

the Jacobian
∂(F1 , . . . , Fn )
∂(η  , σ )
is invertible. By the implicit function theorem, the equations
Fj (y, s, η0 , η, σ) = 0 1 ≤ j ≤ n
lead to solutions
ηj = fj (y, s, η0 , η  , σ  ), k+1≤j ≤m
and
σl = fl (y, s, η0 , η  , σ  ), r + 1 ≤ l ≤ n
where the fi are real analytic, holomorphic in (η0 , η, σ). We can now relabel the
coordinates and their duals and assume that we have coordinates (x, t), x ∈ Rm ,
t ∈ Rn , with dual coordinates (ζ, τ ) such that near the origin, E = {(x, 0)}, and
u satisfies
utj = fj (x, t, u, ux ), 1 ≤ j ≤ n
with fj real analytic, holomorphic in (ζ0 , ζ). 
Proof of Theorem 2.1. Using Lemma 3.5, we may assume we have coordinates
(x, t) near 0 ∈ E, x ∈ Rm , t ∈ Rn , E = {(x, 0)} near 0 and on the wedge W,
utj = fj (x, t, u, ux ), 1≤j≤n (3.4)
Let
∂ ∂fj
m
∂
Luj = − (x, t, u, ux ) .
∂tj ∂ζk ∂xk
k=1
Note that the Luj are C on W. Consider the principal parts of the holomorphic
1

Hamiltonians
m m

∂ ∂fj ∂ ∂fj ∂fj ∂
Hj◦ = − (x, t, ζ0 , ζ) + + τl
∂tj ∂ζk ∂xk ∂tl ∂ζ0 ∂τl
k=1 l=1

  m 
∂fj
m
∂fj ∂ ∂fj ∂
+ + ζi + − ζi + fj .
i=1
∂xi ∂ζ0 ∂ζ i
i=1
∂ζi ∂ζ 0

Since the system (3.4) is involutive, there exist real analytic functions
Z1 (x, t, ζ0 , ζ), . . . , Zm (x, t, ζ0 , ζ), W0 (x, t, ζ0 , ζ), . . . , Wm (x, t, ζ0 , ζ),
42 S. Berhanu

holomorphic in (ζ0 , ζ) and satisfying

Hj0 Zk = 0, Zk (x, 0, ζ0 , ζ) = xk , Hj0 Wl = 0, Wl (x, 0, ζ0 , ζ) = ζl (3.5)
for 1 ≤ k ≤ m, 0 ≤ l ≤ m, near (x, t) = 0 in R m+n
and (ζ0 , ζ) = 0 in C m+1
.
Let
Zku (x, t) = Zk (x, t, u(x, t), ux (x, t)), and Wlu (x, t) = Wl (x, t, u(x, t), ux (x, t)).
By Lemma 3.1 and (3.5),
Luj (Zku ) = 0 and Luj (Wlu ) = 0
in the wedge W. In particular, since Zku (x, 0) = xk , the bundle V u is locally
integrable in the wedge W near 0. We have
W0u (x, 0) = u(x, 0) = u0 (x) and Wju (x, 0) = u0xj (x) (3.6)
for x ∈ E.
Let σ ∈ T0∗ (E) such that σ ∈ / (ΓT0 (W))0 . Then there exists L0 ∈ V0u with
L0 ∈ Γ0 (W), L0 ∈ T0 E such that σ, L0
< 0. Write
m 
m
∂ ∂
n
∂
L0 = aj +i bi + cl
j=1
∂xj i=1
∂xi ∂tl
l=1

where the aj , bi and cl ∈ R. Since L0 ∈

/ T0 E, cl = 0 for some l. Let
Y = {(x, c1 s, . . . , cn s) : x ∈ E, 0 < s < δ}
for some small δ > 0. Then Y is a wedge in an m + 1-dimensional space with edge
E and Y ⊆ W.
Y inherits a locally integrable structure from V u generated by a vector field
L that is C 1 up to E and L(0) = L0 . The restriction of W0u (x, y) to Y is a solution
of L. We can then proceed as in the proof of Theorem III.1 in [EG] to conclude
that σ ∈/ W Fa (W0u (x, 0)), that is, σ ∈
/ W Fa (u0 ). 

We will next derive an edge of the wedge result as a consequence of Corollary

2.2 using ideas from the proof of Theorem X.3.1 in [T1].
Corollary 3.6. Let p, Σ, W + , W − , E, u+ , u− , u0 be as in Corollary 2.2. Assume
+ −
that Vp = Vpu = Vpu is elliptic, that is, Tp◦ (M) = ∅. Then there exists a real
analytic function u in a neighborhood of p that is a solution and agrees with u+
on W + and with u− on W − .
Proof. We may assume that we are in the coordinates (x, t) where E = {(x, 0)}
and p is 0,
− − − −
tj = fj (x, t, u , ux ) in W , and utj = fj (x, t, u , ux ) in W ,
u+ 1 ≤ j ≤ n.
+ + +

We also have
u+ (x, 0) = u0 (x) = u− (x, 0).
 On Involutive Systems of First-order Nonlinear Pdes 43

By hypothesis and Corollary 2.2, u0 (x) is a real analytic function. Consider the
map
F (z, w, ζ0 , ζ) = (Z(z, w, ζ0 , ζ), w, W (z, w, ζ0 , ζ))
which is holomorphic near (0, 0, u0 (0), u0x (x)) and
F (0, 0, u0 (0), u0x (0)) = (0, 0, u0 (0), u0x (x)).
Here Z = (Z1 , . . . , Zm ) and W = (W0 , . . . , Wm ) are as in (3.5). Let
G(z  , w , ζ0 , ζ  ) = (P (ζ  , w , ζ0 , ζ  ), w , Q(ζ  , w , ζ0 , ζ  ))
denote the inverse of F . Then Q is holomorphic and
(ζ0 , ζ) = Q(Z(z, w, ζ0 , ζ), w, W (z, w, ζ0 , ζ)).
In particular,
+ + + +
u+ (x, t) = Q0 (Z u (x, t), t, W u (x, t)), u+
xj (x, t) = Qj (Z
u
(x, t), t, W u (x, t))
(3.7)
for 1 ≤ j ≤ m, and similarly,
− − − −
u− (x, t) = Q0 (Z u (x, t), t, W u (x, t)), u−
xj (x, t) = Qj (Z
u
(x, t), t, W u (x, t))
(3.8)
for 1 ≤ j ≤ m. Since u0 is real analytic, the function u0 (Z u+ (x, t)) is a solu-
+ + +
tion of V u in W + . Recall that W0u (x, t) is also a solution of V u in W + and
+ +
u0 (Z u (x, 0)) = u0 (x) = W0u (x, 0). By the wedge version of the Baouendi-Treves
approximation theorem (see [HM], [BCH]), it follows that on W + ,
+ +
W0u (x, t) = u0 (Z u (x, t)),
and likewise on W − ,
− −
W0u (x, t) = u0 (Z u (x, t)).
We also have
+ +
Wju (x, t) = u0xj (Z u (x, t)) on W + , and
− −
Wju (x, t) = u0xj (Z u (x, t)) on W − , for 1 ≤ j ≤ m.
Going back to (3.7), we get:
+ + +
u+ (x, t) = Q0 (Z u (x, t), t, u0 (Z u (x, t)), u0x (Z u (x, t))) (3.9)
+ + +
u+
xi (x, t) = Qi (Z
u
(x, t), t, u0 (Z u (x, t)), u0x (Z u (x, t))),
for 1 ≤ i ≤ m on W + . Next, since
∂Z
(0, 0, u0 (0), ux (0)) = 0 ∀ j = 0, . . . , m,
∂ζj
we can apply the implicit function theorem to the system
ζ0 = Q0 (Z(x, t, ζ0 , ζ), t, u0 (Z(x, t, ζ0 , ζ)), u0x (Z(x, t, ζ0 , ζ)))
ζi = Qi (Z(x, t, ζ0 , ζ), t, u0 (Z(x, t, ζ0 , ζ)), u0x (Z(x, t, ζ0 , ζ))), 1 ≤ i ≤ m
44 S. Berhanu

to find real analytic function ζ0 = v0 (x, t), ζ = v(x, t) such that

v0 (x, t) = Q0 (Z(x, t, v0 , v), u0 (Z(x, t, v0 , v)), u0x (Z(x, t, v0 , v)))
and
vi (x, t) = Qi (Z(x, t, v0 , v), u0 (Z(x, t, v0 , v)), u0x (Z(x, t, v0 , v)))
for 1 ≤ i ≤ m. By uniqueness and (3.9), we conclude that v0 (x, t) = u+ (x, t)
in W + . Likewise, v0 (x, t) = u− (x, t) in W − . Set u(x, t) = v0 (x, t). Since utj =
fj (x, t, u, ux ) on W + for 1 ≤ j ≤ n, by analyticity, it follows that
utj = fj (x, t, u, ux ) for all j
in a full neighborhood of the origin in RN (N = m + n). 

4. Proofs of Theorem 2.4 and Theorem 2.7

To prove Theorem 2.4, we will use the coordinates and notations of the proof of
Theorem 2.1. In particular, E = {(x, 0)} near our central point p = 0. The function
u is C 2 on W and solves
utj = fj (x, t, u, ux ) on W, 1 ≤ j ≤ n.
We are given a C 1 section L of V0u on W such that L0 = L(0) ∈ ΓV ◦
0 (W), σ ∈ T0 M,
and i σ, [L, L]
< 0.
1

We can write
m 

m
∂ ∂ n
∂
L0 = aj +i bk + cl
∂xj ∂xk ∂tl
j=1 k=1 l=1

where the aj , bk , and cl ∈ R. We also know that cl = 0 for some l. As before, let
Y = {(x, c1 s, . . . , cn s) : x ∈ E, 0 < s < δ}
for some small δ > 0 so that Y ⊆ W. Assume without loss of generality that
c1 = 0. After a linear change of coordinates that preserves E we may assume that
Y = {(x, t, 0, . . . , 0) : x ∈ E, 0 < t < δ}.
Y inherits a locally integrable structure generated by a C 1 vector field which we
still denote by L such that L0 ∈ ΓV 0 (W) and i σ, [L, L]
< 0. The C solution u
1 2
2
restricts to a C function u(x, t) on Y . Recall next that Zj (x, 0, ζ0 , ζ) = xj for
1 ≤ j ≤ m.
It follows that
Zj (x, t) = Zj (x, t, u(x, t), ux (x, t))
have the form
Zj (x, t) = xj + tΨj (x, t)
 On Involutive Systems of First-order Nonlinear Pdes 45

where Ψ1 (x, t), . . . , Ψm (x, t) are C 1 functions on Y . Since L0 ∈

/ T0 E, we may
assume that L has the form
∂
m
∂
L=i + cj (x, t) (4.1)
∂t ∂xj
j=1
1
where the cj are C on Y .
Observe that at a point (x, 0) near the origin, the characteristic set of L is
given by
Char L|(x,0) = {(x, 0; ξ, τ ) : Ψ(x, 0) · ξ = 0, τ = Ψ(x, 0) · ξ, (ξ, τ ) = (0, 0)}.
(4.2)
The latter follows from the equations,
c(x, t) = −iZx−1 · Zt , Zx = I + tΨx , and Zt = Ψ + tΨt .
The proof of Theorem 2.4 now follows from the following lemma from [B].
Lemma 4.1. Suppose L as above has C 1 coefficients and the Ψj ∈ C 1 (Y ). Let h ∈
C 1 (Y ) be a solution of Lh = 0. If σ = (0, 0; ξ 0 , τ 0 ) ∈ Char L and 2i
1
σ([L, L]) < 0,
then (0, ξ ) ∈
0
/ W Fa h(x, 0).
Proof. By adding a variable we may assume that L is a CR vector field near the
origin. Indeed, we can add the variable s and consider the vector field
∂
L = L +
∂s
which is locally integrable with first integrals
Z1 (x, t), . . . , Zm (x, t), and Zm+1 (x, t, s) = s + it.
Note that L h = 0 for t > 0 and L is a CR vector field. We can thus assume that
our original L is CR. This means that for some j, Ψj (0) = 0. Without loss of
generality assume that
Ψ1 (0) = 0. (4.3)
Observe next that the linear change of coordinates
xl = xl + tΨl (0), t = t
allow us to assume, after dropping the primes, that
Ψj (0) = 0, for all j = 1, . . . , m. (4.4)
We can use (4.3) and (4.4) to replace Z2 , . . . , Zm by a linear combination of
Z1 , . . . , Zm and apply a linear change of coordinates to get
Zj = xj + tΨj , 1 ≤ j ≤ m, and Ψ1 (0) = i, Ψj (0) = 0, for 2 ≤ j ≤ m. (4.5)
The equation LZl = 0 implies that

 m
∂Ψl ∂Ψl
i Ψl + t + cl + cj t =0 (4.6)
∂t j=1
∂xj
46 S. Berhanu

and so from (4.5) and (4.6),

c1 (0) = 1 and cj (0) = 0 for j ≥ 2. (4.7)
The condition that (0, 0; ξ , τ ) ∈ Char L therefore means that τ = 0 =
0 0 0
ξ10 and
ξj0 = 0 for some j ≥ 2. In particular, ξ 0 = 0 and
ξ 0 · Ψ(0) = 0. (4.8)
We may assume that
ξ 0 = (0, 1, 0, . . . , 0). (4.9)
We have

m
∂
[L, L] = Al (x, t)
∂xl
l=1
where

∂cl ∂cl
m
∂cl ∂cl
m
Al (x, t) = i + + cj − cj . (4.10)
∂t ∂t ∂xj ∂xj
j=1 j=1
We will express Al (0, 0) using the Ψj . From (4.6) we have
iΨl(x, 0) + cl (x, 0) = 0. (4.11)
Subtract (4.11) from (4.6), divide by t, and let t → 0 to arrive at (recalling that
Ψ and L are C 1 ):
∂Ψl ∂cl m
∂Ψl
2i (x, 0) + (x, 0) + cj (x, 0) (x, 0) = 0. (4.12)
∂t ∂t j=1
∂xj

From (4.11) and (4.12), we get:

∂cl ∂Ψl m
∂Ψl
(x, 0) = −2i (x, 0) + i Ψj (x, 0) (x, 0). (4.13)
∂t ∂t j=1
∂xj

Thus from (4.5), (4.10), (4.11) and (4.13), we have


∂cl ∂cl ∂cl ∂cl
Al (0, 0) = i (0, 0) + (0, 0) + (0, 0) − (0, 0)
∂t ∂t ∂x1 ∂x1


∂Ψl ∂Ψl
= −2 (0, 0) − (0, 0)
∂t ∂t
∂Ψl
= 4i (0).
∂t
1
Therefore, the assumption that 2i σ([L, L]) < 0 implies that
∂Ψ2 ∂Ψ
(0) = (0) · ξ 0 < 0. (4.14)
∂t ∂t
Next, we show that coordinates (x, t) and first integrals Zl = xl + tΨl can be
chosen so that (4.5), (4.9) and (4.14) still hold and in addition,
∂Ψl
(0) = 0 for all l, j.
∂xj
 On Involutive Systems of First-order Nonlinear Pdes 47

Define

m
Zl (x, t) = Zl + alk Z1 Zk , l = 1, . . . , m,
k=1
where ⎧
⎪ ∂Ψl
⎨ − 12 (0), k = 1
al,k = ∂xk
⎪
⎩ −
∂Ψl
(0), 2 ≤ k ≤ m.
∂xk
Note that

m
Zl (x, t) = xl + l (x, t),
alk x1 xk + tΨ
k=1
where

l (x, t) = Ψl +
Ψ al,k (x1 Ψk + xk Ψ1 + tΨ1 Ψk ).
k

By the choice of the alk and the fact that Ψ1 (0) = i, we have
"l
∂ Ψ
(0) = 0 for all l, j.
∂xj
Introduce new coordinates

x̃l = xl + alk x1 xk , t̃ = t, 1 ≤ l ≤ m.
k=1

These change of coordinates are smooth and hence L is still C 1 in these coordinates.
After dropping the tildes both in the new coordinates and the first integrals, we
have:
∂Ψl
Zj = xj + tΨj with (0) = 0 for all l, j (4.15)
∂xj
and (4.5), (4.9) and (4.14) still hold. Moreover, the new coordinates preserve the
set {t = 0} and so L still has the form

∂ m
∂
L=i + cj (x, t) .
∂t j=1 ∂xj

Let η(x) ∈ C0∞ (Br (0)), where Br (0) is a ball of small radius r centered at 0 ∈ Rm
and η(x) ≡ 1 when |x| ≤ r/2. We will be using the FBI transform

2
Fκ (t, z, ζ) = eiζ.(z−Z(x,t))−κ ζ
[z−Z(x,t)] η(x)h(x, t) dZ
Rm
m
where for z ∈ Cm , we write [z]2 = j=1 zj2 , ζ
= (ζ · ζ)1/2 is the main branch of
the square root, dZ = dZ1 ∧ · · · ∧ dZm = det Zx (x, t)dx1 ∧ · · · ∧ dxm , and κ > 0 is
a parameter which will be chosen later.
48 S. Berhanu

To prove that (0, ξ 0 ) ∈/ W Fa (h(x, 0)), we need to show that for some κ > 0
and constants C1 , C2 > 0,
# #
# #
#
|Fκ (0, z, ζ)| = # e iζ.(z−x)−κ ζ
[z−x]2
η(x)h(x, 0) dx##
(4.16)
−c2 |ζ|
≤ C1 e
for z near 0 in Cm and ζ in a conic neighborhood of ξ 0 in Cm . Let U = Br (0)×(0, δ)
for some δ small. Since h and the Zj are solutions, the form
2
ω = eiζ.(z−Z(x,t))−κ ζ
[z−Z(x,t)] h(x, t)dZ1 ∧ dZ2 ∧ · · · ∧ dZm
is a closed form. This is well known when the Zj are C 2 and when they are only
C 1 as in our case, one can prove that ω is closed by approximating the Zj by
smoother functions. By Stokes’ theorem, we therefore have
  
Fκ (0, z, ζ) = ηω = ηω − dη ∧ ω. (4.17)
{t=0} t=δ U

We will show that κ, δ and r > 0 can be chosen so that each of the two integrals
on the right side of (4.17) satisfies an estimate of the form (4.16). Set
(iζ · (z − Z(x, t)) − κ ζ
[z − Z(x, t)]2 )
Q(z, ζ, x, t) = .
|ζ|
Since Q is homogeneous of degree 0 in ζ, it is sufficient to show that there is
C > 0 so that Q(0, ξ 0 , x, t) ≤ −C for (x, t) ∈ (supp η × {δ}) ∪ (supp dη × [0, δ]).
For then, Q(z, ζ, x, t) ≤ −C/2 for the same (x, t), z near 0 in Cm , and ζ in a conic
neighborhood of ξ 0 in Cm . We recall that ξ 0 = (0, 1, . . . , 0), and so |ξ 0 | = 1. We
have:
Q(0, ξ 0 , x, t) = (−iξ 0 · (x + tΨ) − κ[x + tΨ]2 )
(4.18)
= tξ 0 · Ψ(x, t) − κ[|x|2 + t2 |Ψ|2 + 2t x, Ψ
− t2 |Ψ|2 ].
Since Ψ is C 1 , using (4.5), (4.14), and (4.15),
t(ξ 0 · Ψ(x, t)) = −C1 t2 + o(|x|t + t2 ) (4.19)
∂Ψ
where C1 = − (0) · ξ 0 > 0. Let C ≥ |Ψ|2 + 1 on U , and set α = C1
8C . Note
∂t
that (4.19) allows us to choose r and δ small enough so that on U ,
C1 2
t(ξ 0 · Ψ(x, t)) ≤ − t + α|x|2 . (4.20)
2
From (4.18) and (4.20), we get:
C1 2
Q(0, ξ 0 , x, t) ≤ −
t + α|x|2 − κ[|x|2 − 2t|x||Ψ| − t2 |Ψ|2 ].
2
Since Ψ(0) = 0, we may assume r and δ are small enough so that
2t|x||Ψ| ≤ t2 + |x|2 /2
 On Involutive Systems of First-order Nonlinear Pdes 49

and hence using C ≥ |Ψ|2 + 1,

C1 2
Q(0, ξ 0 , x, t) ≤ − t + α|x|2 − κ|x|2 − κ|x|2 /2 + κCt2 .
2
3C1 C1
Choose κ = 8C
. Recalling that α = 8C
, we get:
C1 2 C1
Q(0, ξ 0 , x, t) ≤ −
t − |x|2
8 16C
and so on supp η × {δ} ∪ (supp (dη) × [0, δ]), Q(0, ξ 0 , x, t) ≤ −C for some C > 0.
This proves the lemma. 

To prove Theorem 2.7, once again we will use the coordinates and notations
of the proof of Theorem 2.4. Thus E = {(x, 0)} near p = 0, u ∈ C 3 (W) and
utj = fj (x, t, u, ux ) on W, 1 ≤ j ≤ n.
We may assume that for some δ > 0,
Y = {(x, t, 0, . . . , 0) : x ∈ E, 0 < t < δ} ⊆ W,
and L is a C 2 vector field on Y of the form
m
∂ ∂
L= aj (x, t) +i
j=1
∂x j ∂t

such that σ, L
= 0, σ, [L, L̄]
= 0, and
√
3 | σ, [L, [L, L̄]]
| <  σ, [L, [L, L̄]]
.
Since u is C 3 on W, the functions ψj (x, t) are C 2 on Y . The rest of the proof
proceeds as in the proof of Lemma 3.2 in [B] starting from (3.27) in that paper.
Note in particular that unlike the reasoning used in [B], L does not have to satisfy
the solvability of the Cauchy problems (3.22) in [B]. This approach therefore is
simpler than the one employed in [B].

References
[A] C. Asano, on the C ∞ wave-front set of solutions of first-order nonlinear pde’s,
Proc. Amer. Math. Soc. 123 (1995), 3009–3019.
[BGT] M.S. Baouendi, C. Goulaouic, and F. Treves, Uniqueness in certain first-order
nonlinear complex Cauchy problems, Communications on Pure and Applied
Mathematics XXXVIII (1985), 109–123.
[B] S. Berhanu, On microlocal analyticity of solutions of first-order nonlinear PDE,
Ann. Inst. Fourier, Grenoble 59, 4 (2009) 1267–1290.
[BCH] S. Berhanu, P.D. Cordaro, and J. Hounie, An Introduction to Involutive Struc-
tures, Cambridge University Press (2008).
[Ca] C. Carathéodory, Calculus of Variations and Partial Differential Equations of
the First Order, Part I. Holden-Day, Inc., San Francisco (1965).
50 S. Berhanu

[CCCF] D. Chae, A. Cordoba, D. Cordoba, and M. Fontelos, Finite time singularities

in a 1D model of the quasi-geostrophic equation, Advances in Mathematics 194
(2005), 203–223.
[C] C.H. Chang, Hypo-analyticity with vanishing Levi form, Bull. Inst. Math. Acad.
Sinica 13(1985), 123–136.
[Che] J.Y. Chemin, Calcul paradifférentiel précisé et applications à des équations aux
dérivées partielles non semilinéaires, Duke Math.J, 56(1988), 431–469.
[EG] M.G. Eastwood and C.R. Graham, Edge of the wedge theory in hypo-analytic
manifolds, Commun. Partial Differ. Equations 28 (2003), 2003–2028.
[HT] N. Hanges and F. Treves, On the analyticity of solutions of first-order nonlinear
PDE, Trans. Amer. Math. Soc. 331 (1992), 627–638.
[H1] A.A. Himonas, On analytic microlocal hypoellipticity of linear partial differential
operators of principal type, Commun. Partial Differ. Equations 11 (1986), 1539–
1574.
[H2] A.A. Himonas, Semirigid partial differential operators and microlocal analytic
hypoellipticity, Duke Math. J. 59 (1989), 265–287.
[HM] J. Hounie and P. Malagutti, On the convergence of the Baouendi-Treves ap-
proximation formula, Comm. P.D.E., 23 (1998), 1305–1347.
[KO1] R. Kenyon and A. Okounkov, Limit shapes and the complex Burger’s equation,
http://arXiv.org/abs/math-ph/0507007.
[KO2] R. Kenyon and A. Okounkov, Dimers, the complex Burger’s equation and curves
inscribed in polygons,
http://www.math.ubc.ca/∼kenyon/talks/browncolloquium.pdf
[LMX] N. Lerner, Y. Morimoto, and C.-J. Xu, Instability of the Cauchy-Kovalevskaya
solution for a class of nonlinear systems, American Jour. of Mathematics, Vol-
ume 132, Number 1 (2010) 99–123.
[M] G. Metivier, Remarks on the well-posedness of the nonlinear Cauchy problem,
Contemp. Math.368 (2005), 337–356.
[T1] Treves, F., Hypo-Analytic Structures, Princeton University Press (1992).
[T2] Treves, F., Remarks on the integrability of first-order complex PDE, J. Funct.
Anal.106 (1992), 329–352.

S. Berhanu
Department of Mathematics
Temple University
Philadelphia, PA 19122, USA
Complex Analysis
Trends in Mathematics, 51–73

c 2010 Springer Basel AG

Gevrey Hypoellipticity for an

Interesting Variant of Kohn’s Operator
Antonio Bove, Marco Mughetti and David S. Tartakoff

Dedicated to Professor Linda Preiss Rothschild

on the occasion of her 60th birthday

Abstract. In this paper we consider the analogue of Kohn’s operator but with
a point singularity,
P = BB ∗ + B ∗ (t2 + x2k )B, B = Dx + ixq−1 Dt .
We show that this operator is hypoelliptic and Gevrey hypoelliptic in a cer-
q k
tain range, namely k < q, with Gevrey index q−k = 1 + q−k . Work in
progress by the present authors suggests that, outside the above range of the
parameters, i.e., when k ≥ q, the operator is not even hypoelliptic.

Mathematics Subject Classification (2000). 35H10, 35A20, 35B65.

Keywords. Sums of squares of complex vector fields; hypoellipticity; Gevrey
hypoellipticity; pseudodifferential operators.

1. Introduction
In J.J. Kohn’s recent paper [11] (see also [6]) the operator
∂ ∂
Em,k = Lm Lm + Lm |z|2k Lm , Lm = − iz|z|2(m−1)
∂z ∂t
was introduced and shown to be hypoelliptic, yet to lose 2 + k−1
m derivatives in L
2

Sobolev norms. Christ [7] showed that the addition of one more variable destroyed
hypoellipticity altogether.
In a recent volume, dedicated to J.J. Kohn, A. Bove and D.S. Tartakoff, [5],
showed that Kohn’s operator with an added Oleinik-type singularity, of the form
studied in [4],
Em,k + |z|2(p−1) Ds2
52 A. Bove, M. Mughetti and D.S. Tartakoff

is s-Gevrey hypoelliptic for any s ≥ p−k

2m
, (here 2m > p > k). A related result is
that for the ‘real’ version, with X = Dx + ixq−1 Dt , where Dx = i−1∂x ,
Rq,k + x2(p−1) Ds2 = XX ∗ + (xk X)∗ (xk X) + x2(p−1) Ds2
is sharply s-Gevrey hypoelliptic for any s ≥ q
p−k
, where q > p > k and q is an
even integer.
In this paper we consider the operator
P = BB ∗ + B ∗ (t2 + x2k )B, B = Dx + ixq−1 Dt , (1.1)
where k,  and q are positive integers, q even.
Observe that P is a sum of three squares of complex vector fields, but, with
a small change not altering the results, we could make P a sum of two squares
of complex vector fields in two variables, depending on the same parameters, e.g.,
BB ∗ + B ∗ (t2 + x2k )2 B.
Let us also note that the characteristic variety of P is {x = 0, ξ = 0}, i.e.,
a codimension two analytic symplectic submanifold of T ∗ R2 \ 0, as in the case
of Kohn’s operator. Moreover the Poisson-Treves stratification for P has a single
stratum thus coinciding with the characteristic manifold of P .
We want to analyze the hypoellipticity of P , both in C ∞ and in Gevrey
classes. As we shall see the Gevrey classes play an important role. Here are our
results:
Theorem 1.1. Let P be as in (1.1), q even.
(i) Suppose that
k
> . (1.2)
q
Then P is C ∞ hypoelliptic (in a neighborhood of the origin) with a loss of
2 q−1+k
q
derivatives.
(ii) Assume that the same condition as above is satisfied by the parameters , k
and q. Then P is s-Gevrey hypoelliptic for any s, with
q
s≥ . (1.3)
q − k
(iii) Assume now that
k
≤ . (1.4)
q
Then P is not C ∞ hypoelliptic.
The proof of the above theorem is lengthy, will be given in a forthcoming
paper [3]. In this paper we prove items (i) and (ii) of the theorem.
It is worth noting that the operator P satisfies the complex Hörmander condi-
tion, i.e., the brackets of the fields of length up to k + q generate a two-dimensional
complex Lie algebra C2 . Note that in the present case the vector fields involved
are B ∗ , xk B and t B, but only the first two enter in the brackets spanning C2 .
 Gevrey Hypoellipticity for Kohn’s Operator 53

A couple of remarks are in order. The above theorem seems to us to suggest

strongly that Treves conjecture cannot be extended to the case of sums of squares
of complex vector fields, since lacking C ∞ hypoellipticity we believe that P is not
analytic hypoelliptic for any choice of the parameters. We will address this point
further in the subsequent paper.
The second and trivial remark is that, even in two variables, there are exam-
ples of sums of squares of complex vector fields, satisfying the Hörmander condi-
tion, that are not hypoelliptic. In this case the characteristic variety is a symplectic
manifold. In our opinion this is due to the point singularity exhibited by the second
and third vector field, or by (t2 + x2k )B in the two-fields version.
Restricting ourselves to the case q even is no loss of generality, since the
operator (1.1) corresponding to an odd integer q is plainly hypoelliptic and actually
subelliptic, meaning by that term that there is a loss of less than two derivatives.
This fact is due to special circumstances, i.e., that the operator B ∗ has a trivial
kernel in that case. We stress the fact that the Kohn’s original operator, in the
complex variable z, automatically has an even q, while in the “real case” the parity
of q matters.
We also want to stress microlocal aspects of the theorem: the characteristic
manifold of P is symplectic in T ∗ R2 of codimension 2 and as such it may be
identified with T ∗ R \ 0 ∼ {(t, τ ) | τ = 0} (leaving aside the origin in the τ
variable). On the other hand, the operator P (x, t, Dx , τ ), thought of as a differential
operator in the x-variable depending on (t, τ ) as parameters, for τ > 0 has an
eigenvalue of the form τ 2/q (t2 + a(t, τ )), modulo a non zero function of t. Here
a(t, τ ) denotes a (non-classical) symbol of order −1 defined for τ > 0 and such that
2k
a(0,τ ) ∼ τ − q . Thus we may consider the pseudodifferential operator Λ(t, Dt ) =
Op τ 2/q (t2 + a(t, τ )) as defined in a microlocal neighborhood of our base point in
the characteristic manifold of P . One can show that the hypoellipticity properties
of P are shared by Λ, e.g., P is C ∞ hypoelliptic iff Λ is.
The last section of this paper includes a computation of the symbol of Λ as
well as the proof that P is hypoelliptic if Λ is hypoelliptic. This is done following
ideas of Boutet de Monvel, Helffer and Sjöstrand.

2. The operator P is C ∞ hypoelliptic

Theorem 2.1. Under the restriction that k < q, q even, the operator P is hypoel-
liptic.
Denoting by Wj , j = 1, 2, 3, 4 the operators
k−1
W1 = B ∗ , W2 = t B, W3 = xk B, and W4 = Dt
− q ,
(σ( Dt
) = (1 + |τ | ) ) then for v ∈
2 1/2
C0∞and of small support near (0, 0) we have
the estimate, following [11] and [6],
4
Wj v2  | P v, v
| + v2−∞
j=1
54 A. Bove, M. Mughetti and D.S. Tartakoff

where, unless otherwise noted, norms and inner products are in L2 (R2 ). Here the
last norm indicates a Sobolev norm of arbitrarily negative order. This estimate
was established in [11] without the norm of t B (and without the term B ∗ t2 B in
the operator) and our estimate follows at once in our setting.
A first observation is that we may work microlocally near the τ axis, since
away from that axis (conically) the operator is elliptic.
A second observation is that no localization in space is necessary, since away
from the origin (0, 0), we have estimates on both Bv2 and B ∗ v2 , and hence
the usual subellipticity (since q − 1 brackets of B and B ∗ generate the ‘missing’
∂
vector field ).
∂t
Our aim will be to show that for a solution u of P u = f ∈ C ∞ and arbi-
trary N ,

N
∂
u ∈ L2loc .
∂t
To do this, we pick a Sobolev space to which the solution belongs, i.e., in view
of the ellipticity of P away from the τ axis, we pick s0 such that Dt
−s0 u ∈ L2loc
and from now on all indices on norms will be in the variable t only.
Actually we will change our point of view somewhat and assume that the
left-hand side of the a priori estimate is finite locally for u with norms reduced
by s0 and show that this is true with the norms reduced by only s0 − δ for some
(fixed) δ > 0.
4
Taking s0 = 0 for simplicity, we will assume that
4 1 Wj u0 < ∞ and
show that in fact 1 Wj uδ < ∞ for some positive δ. Iterating this ‘bootstrap
operation will prove that the solution is indeed smooth.
The main new ingredient in proving hypoellipticity is the presence of the term
t B, which will result in new brackets. As in Kohn’s work and ours, the solution u
will initially be smoothed out in t so that the estimate may be applied freely, and
at the end the smoothing will be allowed to tend suitably to the identity and we
will be able to apply a Lebesgue bounded convergence theorem to show that the
4
1 Wj uδ are also finite, leading to hypoellipticity.
Without loss of generality, as observed above, we may assume that the
solution u to P u = f ∈ C0∞ has small support near the origin (to be more
thorough, we could take a localizing function of small support, ζ, and write
P ζu = ζP u + [P, ζ]u = ζf mod C0∞ so that P ζu ∈ C0∞ since we have already
seen that u will be smooth in the support of derivatives of ζ by the hypoellipticity
of P away from the origin.)
In order to smooth out the solution in the variable t, we introduce a standard
cut-off function χ(τ ) ∈ C0∞ (|τ | ≤ 2), χ(τ ) ≡ 1, |τ | ≤ 1, and set χM (τ ) = χ(τ /M ).
Thus χM (D) is infinitely smoothing (in t) and, in supp χM , τ ∼ M and |χM | ∼
(j)
−j
M . Further, as M → ∞, χM (D) → Id in such a way that it suffices to show
χM (D)wr ≤ C independent of M to conclude that w ∈ H r .
 Gevrey Hypoellipticity for Kohn’s Operator 55

Introducing of χM , however, destroys compact support, so we shall introduce

v = ψ(x, t) Dt
δ χM (D)u into the a priori estimate and show that the left-hand
remains bounded uniformly in M as M → ∞.
For clarity, we restate the estimate in the form in which we will use it, sup-
pressing the spatial localization now as discussed above:
k−1
B ∗ Dt
δ χM u2 + t B Dt
δ χM u2 + xk B Dt
δ χM u2 +  Dt
− q Dt
δ χM u2
 (P Dt
δ χM u, Dt
δ χM u) +  Dt
δ χM u2−∞ .
Clearly the most interesting bracket which will enter in bringing Dt
δ χM past
the operator P, and the only term which has not been handled in the two papers
cited above, is when tl is differentiated, as in
([B ∗ t2 B, Dt
δ χM ]u, Dt
δ χM u) ∼ (B ∗ [t2 , Dt
δ χM ]Bu, Dt
δ χM u)

∼ (t2−j ( Dt
δ χM )(j) Bu, B Dt
δ χM u)
in obvious notation. Here the derivatives on the symbol of Dt
δ χM are denoted
( Dt
δ χM )(j) .
So a typical term would lead, after using a weighted Schwarz inequality and
absorbing a term on the left-hand side of the estimate, to the need to estimate a
constant times the norm
t−j ( Dt
δ χM )(j) Bu2 .
Now we are familiar with handling such terms, although in the above-cited
works it was powers of x (or z in the complex case) instead of powers of t. The
method employed is to ‘raise and lower’ powers of t and of τ on one side of an inner
product and lower them on the other. That is, if we denote by A the operator
A = t Dt
ρ ,
we have
Ar w2 = |(Aρ w, Aρ w)| N w2 + AN w2
for any desired positive N ≥ r (repeated integrations by parts or by interpolation,
since the non-self-adjointness of A is of lower order), together with the observation
that a small constant may be placed in front of either term on the right, and the
notation N means that the constants involved may depend on N, but N will
always be bounded.
In our situation, looking first at the case j = 1,
t−1 B( Dt
δ χM ) u2
= |(A−1 Dt
−(−1)ρ B( Dt
δ χM ) u, A−1 Dt
−(−1)ρ B( Dt
δ χM ) u)|
≤ A Dt
−(−1)ρ B( Dt
δ χM ) u2 +  Dt
−(−1)ρ B( Dt
δ χM ) u2
= t Dt
ρ B( Dt
δ χM ) u2 +  Dt
−(−1)ρ B( Dt
δ χM ) u2
∼ t B χ̃M u2ρ+δ−1 + B χ̃M u2−(−1)ρ+δ−1
56 A. Bove, M. Mughetti and D.S. Tartakoff

modulo further brackets, where χ̃M is another function of τ such as Dt

χM , with
symbol uniformly bounded in τ independently of M and of compact support. χ̃M
will play the same role as χM in future iterations of the a priori estimate.
We are not yet done – the first term on the right will be handled inductively
provided ρ − 1 < 0, but the second contains just B without the essential powers
of t.
However, as in [11], we may integrate by parts, thereby converting B to B ∗
which is maximally controlled in the estimate, but modulo a term arising from the
bracket of B and B ∗ .
As in [6] or [11], or by direct computation, we have
q−2
Bw2r  B ∗ w2r + x 2 wr+1/2

and while this power of x may not be directly useful, we confronted the same issue
in [6] (in the complex form – the ‘real’ one is analogous). In that context, the
exponent q − 2/2 was denoted m − 1, but the term was well estimated in norm
− 2m
1
+ 12 − k−1q
, which in this context reads − q1 + 12 − k−1
q
= 12 − kq . We have
−( − 1)ρ − 1 + 12 , and under our hypothesis that  > k/q our norm is less than
1/2 − k/q for any choice of ρ ≤ 1 as desired.
Finally, the terms with j > 1 work out similarly.
This means that we do indeed have a weaker norm so that with a different
cut off in τ, which we have denoted χ̃M , there is a gain, and that as M → ∞ this
term will remain bounded.

3. Gevrey hypoellipticity
Again we write the example as

3
P = Wj∗ Wj ,
1

with
W1 = B ∗ , W2 = t B, W3 = xk B, B = Dx + ixq−1 Dt
and omit localization as discussed above, and set v = T p u, the a priori estimate
we have is
4
k−1
Wj v20  |(P v, v)|, W4 = Dt
− q .
1

The principal (bracketing) errors come from [Wj , T p]v, j = 1, 2, 3, and the
worst case occurs when j = 2 :

[W2 , T p]v = pt−1 BT p−1 v.

 Gevrey Hypoellipticity for Kohn’s Operator 57

Raising and lowering powers of t as above,

t−1 BT p−1 u  t BT p−1+δ u + BT p−1−(−1)δ u

 W2 T p−1+δ u + B ∗ T p−1−(−1)δ u + xq−1 T p−(−1)δ u

using the fact that B − B ∗ = ±ixq−1 T. Again we raise and lower powers of x to
obtain

xq−1 T p−(−1)δ u  T p−(−1)δ−(q−1)ρ u + {xk+q−1 T } T p−1−(−1)δ+kρ u

since the term in braces is a linear combination of xk B and B ∗ , both of which are
optimally estimated. The result is that

t−1 BT p−1 u  t BT p−1+δ u + BT p−1−(−1)δ u

 W2 T p−1+δ u + W1 T p−1−(−1)δ u + W1 T p−1−(−1)δ+kρ u
k−1
+ W4 T p−(−1)δ−(q−1)ρ+  u + W3 T p−1−(−1)δ+kρ u

where the third term on the right clearly dominates the second. In all, then,

4
t −1
BT p−1
u  Xj T p−σ u
j=1

where
σ = min sup si
i 0<ρ<1
0<δ<1

with

s1 = 1 − δ
l−1
s2 = ( − 1)δ + (q − 1)ρ −
q
s3 = 1 + ( − 1)δ − kρ.

The desired value of σ is achieved when all three are equal by a standard
minimax argument, and this occurs when
k 1
δ= , ρ=
q q
q − k q k
resulting in σ = , which yields G q−k = G1+ q−k hypoellipticity.
q
The restriction that k < q for hypoellipticity at all takes on greater meaning
given this result.
58 A. Bove, M. Mughetti and D.S. Tartakoff

4. Computing Λ
4.1. q-pseudodifferential calculus
The idea, attributed by J. Sjöstrand and Zworski [13] to Schur, is essentially a
linear algebra remark: assume that the n × n matrix A has zero in its spectrum
with multiplicity one. Then of course A is not invertible, but, denoting by e0 the
zero eigenvector of A, the matrix (in block form)
$ %
A e0
t
e0 0
is invertible as a (n + 1) × (n + 1) matrix in Cn+1 . Here t e0 denotes the row
vector e0 .
All we want to do is to apply this remark to the operator P whose part
BB ∗ has the same problem as the matrix A, i.e., a zero simple eigenvalue. This
occurs since q is even. Note that in the case of odd q, P may easily be seen to be
hypoelliptic.
It is convenient to use self-adjoint derivatives from now on, so that the vector
field B ∗ = Dx − ixq−1 Dt , where Dx = i−1 ∂x . It will also be convenient to write
B(x, ξ, τ) for the symbol of the vector field B, i.e., B(x, ξ, τ) = ξ + ixq−1 τ and
analogously for the other vector fields involved. The symbol of P can be written
as
P (x, t, ξ, τ ) = P0 (x, t, ξ, τ ) + P−q (t, x, ξ, τ ) + P−2k (x, t, ξ, τ ), (4.1.1)
where
P0 (x, t, ξ, τ ) = (1 + t2 )(ξ 2 + x2(q−1) τ 2 ) + (−1 + t2 )(q − 1)xq−2 τ ;
P−q (x, t, ξ, τ ) = −2t2−1xq−1 (ξ + ixq−1 τ );
P−2k (x, t, ξ, τ ) = x2k (ξ 2 + x2(q−1) τ 2 ) − i2kx2k−1 (ξ + ixq−1 τ ) + (q − 1)x2k+q−2 τ.
It is evident at a glance that the different pieces into which P has been decomposed
include terms of different order and vanishing speed. We thus need to say something
about the adopted criteria for the above decomposition.
Let μ be a positive number and consider the following canonical dilation in
the variables (x, t, ξ, τ ):
x → μ−1/q x, t → t, ξ → μ1/q ξ, τ → μτ.
It is then evident that P0 has then the following homogeneity property
P0 (μ−1/q x, t, μ1/q ξ, μτ ) = μ2/q P0 (x, t, ξ, τ ). (4.1.2)
Analogously
P−q (μ−1/q x, t, μ1/q ξ, μτ ) = μ2/q−1 Pq (x, t, ξ, τ ) (4.1.3)
and
P−2k (μ−1/q x, t, μ1/q ξ, μτ ) = μ2/q−(2k)/q P2k (x, t, ξ, τ ). (4.1.4)
Now the above homogeneity properties help us in identifying some symbol classes
suitable for P .
 Gevrey Hypoellipticity for Kohn’s Operator 59

Following the ideas of [1] and [2] we define the following class of symbols:
Definition 4.1.1. We define the class of symbols Sqm,k (Ω, Σ) where Ω is a conic
neighborhood of the point (0, e2 ) and Σ denotes the characteristic manifold {x =
0, ξ = 0}, as the set of all C ∞ functions such that, on any conic subset of Ω with
compact base,
 k− q−1
γ
−δ
# α β γ δ #
#∂t ∂τ ∂x ∂ξ a(x, t, ξ, τ )#  (1 + |τ |)m−β−δ |ξ| + |x|q−1 + 1q−1 .
|τ | |τ | q
(4.1.5)
We write Sqm,k for Sqm,k (R2 × R2 , Σ).
 
By a straightforward computation, see, e.g., [2], we have Sqm,k ⊂ Sqm ,k iff
m ≤ m and m − q−1  q−1 
q k ≤ m − q k . Sq
m,k
can be embedded in the Hörmander
m+ q−1
q k−
classes Sρ,δ , where k− = max{0, −k}, ρ = δ = 1/q ≤ 1/2. Thus we immedi-
q 2k
2,2+ q−1 2,2+ q−1
ately deduce that P0 ∈ Sq2,2 , P−q ∈ Sq1,2 ⊂ Sq and finally P−2k ∈ Sq .
We shall need also the following
Definition 4.1.2 ([2]). Let Ω and Σ be as above. We define the class Hqm (Ω, Σ) by
q
m−j,− q−1 j
Hqm (Ω, Σ) = ∩∞
j=1 Sq (Ω, Σ).
We write Hqm for Hqm (R2 × R , Σ).
2

Now it is easy to see that P0 , as a differential operator w.r.t. the variable

x, depending on the parameters t, τ ≥ 1 has a non negative discrete spectrum.
Moreover the dependence on τ of the eigenvalue is particularly simple, because of
(4.1.2). Call Λ0 (t, τ ) the lowest eigenvalue of P0 . Then
2
Λ0 (t, τ ) = τ q Λ̃0 (t).
Moreover Λ0 has multiplicity one and Λ̃0 (0) = 0, since BB ∗ has a null eigen-
value with multiplicity one. Denote by ϕ0 (x, t, τ ) the corresponding eigenfunction.
Because of (4.1.2), we have the following properties of ϕ0 :
a) For fixed (t, τ ), ϕ0 is exponentially decreasing w.r.t. x as x → ±∞. In fact,
q
because of (4.1.2), setting y = xτ 1/q , we have that ϕ0 (y, t, τ ) ∼ e−y /q .
b) It is convenient to normalize ϕ0 in such a way that ϕ0 (·, t, τ )L2 (Rx ) = 1.
This implies that a factor ∼ τ 1/2q appears. Thus we are led to the definition
of a Hermite operator (see [9] for more details).
Let Σ1 = πx Σ be the space projection of Σ. Then we write
Definition 4.1.3. We write Hqm for Hqm (R2x,t × Rτ , Σ1 ), i.e., the class of all smooth
q
m−j,− j
functions in ∩∞ j=1 Sq
q−1
(R2x,t × Rτ , Σ1 ). Here Sqm,k (R2x,t × Rτ , Σ1 ) denotes the
set of all smooth functions such that
 k− q−1
γ
# α β γ # 1
#∂t ∂τ ∂x a(x, t, τ )#  (1 + |τ |)m−β |x|q−1 + q−1 . (4.1.6)
|τ | q
60 A. Bove, M. Mughetti and D.S. Tartakoff

Define the action of a symbol a(x, t, τ ) in Hqm as the map a(x, t, Dt ) : C0∞ (Rt ) −→
C ∞ (R2x,t ) defined by

a(x, t, Dt )u(x, t) = (2π)−1 eitτ a(x, t, τ )û(τ )dτ.

Such an operator, modulo a regularizing operator (w.r.t. the t variable) is called

a Hermite operator and we denote by OP Hqm the corresponding class.
We need also the adjoint of the Hermite operators defined in Definition 4.1.3.
Definition 4.1.4. Let a ∈ Hqm . We define the map a∗ (x, t, Dt ) : C0∞ (R2x,t ) −→
C ∞ (Rt ) as
 
∗ −1
a (x, t, Dt )u(t) = (2π) eitτ a(x, t, τ )û(x, τ )dxdτ.

We denote by OP Hq∗ m the related set of operators.

Lemma 4.1.1. Let a ∈ Hqm , b ∈ Sqm,k ; then
(i) the formal adjoint a(x, t, Dt )∗ belongs to OP Hq∗ m and its symbol has the
asymptotic expansion

N −1
1 α α
σ(a(x, t, Dt )∗ ) − ∂ D a(x, t, τ ) ∈ Hqm−N . (4.1.7)
α=0
α! τ t
(ii) The formal adjoint (a∗ (x, t, Dt ))∗ belongs to OP Hqm and its symbol has the
asymptotic expansion

N −1
1 α α
σ(a∗ (x, t, Dt )∗ ) − ∂ D a(x, t, τ ) ∈ Hqm−N . (4.1.8)
α=0
α! τ t
(iii) The formal adjoint b(x, t, Dx , Dt )∗ belongs to OP Sqm,k and its symbol has the
asymptotic expansion

N −1 q
1 α m−N,k−N q−1
σ(a(x, t, Dx , Dt )∗ ) − α
∂(ξ,τ ) D(x,t) a(x, t, ξ, τ ) ∈ Sq . (4.1.9)
α=0
α!
The following is a lemma on compositions involving the two different types
of Hermite operators defined above. First we give a definition of “global” homo-
geneity:
Definition 4.1.5. We say that a symbol a(x, t, ξ, τ ) is globally homogeneous (abbre-
viated g.h.) of degree m, if, for λ ≥ 1, a(λ−1/q x, t, λ1/q ξ, λτ ) = λm a(x, t, ξ, τ ).
Analogously a symbol, independent of ξ, of the form a(x, t, τ ) is said to be globally
homogeneous of degree m if a(λ−1/q x, t, λτ ) = λm a(x, t, τ ).
j
m,k+
Let f−j (x, t, ξ, τ ) ∈ Sq q−1
, j ∈ N, then there exists f (x, t, ξ, τ ) ∈ Sqm,k
 N −1 N
m,k+ q−1
such that f ∼ j≥0 f−j , i.e., f − j=0 f−j ∈ Sq , thus f is defined modulo
a symbol in Sqm,∞
= ∩h≥0 Sq .
m,h
 Gevrey Hypoellipticity for Kohn’s Operator 61

Analogously, let f−j be globally homogeneous of degree m − k q−1

q
− j
q
and
such that for every α, β ≥ 0 satisfies the estimates
# #  k− q−1
# γ β # α
−β
#∂(t,τ ) ∂xα ∂ξ f−j (x, t, ξ, τ )#  |ξ| + |x|q−1 + 1 , (x, ξ) ∈ R2 , (4.1.10)

for (t, τ ) in a compact subset of R × R \ 0 and every multiindex γ. Then f−j ∈

j
m,k+ q−1
Sq .
m− j
Accordingly, let ϕ−j (x, t, τ ) ∈ Hq q , then there exists ϕ(x, t, τ ) ∈ Hqm such
 N −1 m− N
that ϕ ∼ j≥0 ϕ−j , i.e., ϕ − j=0 ϕ−j ∈ Hq q , so that ϕ is defined modulo a
symbol regularizing (w.r.t. the t variable.)
Similarly, let ϕ−j be globally homogeneous of degree m − qj and such that
for every α,  ≥ 0 satisfies the estimates
# #  −− q−1
# β # α
#∂(t,τ ) ∂xα ϕ−j (x, t, τ )#  |x|q−1 + 1 , x ∈ R, (4.1.11)

for (t, τ ) in a compact subset of R × R \ 0 and every multiindex β. Then ϕ−j ∈

m− qj
Hq .
As a matter of fact in the construction below we deal with asymptotic series
of homogeneous symbols.
Next we give a brief description of the composition of the various types of
operator introduced so far.
 
Lemma 4.1.2 ([9], Formula 2.4.9). Let a ∈ Sqm,k , b ∈ Sqm ,k , with asymptotic
globally homogeneous expansions
j
m,k+ q−1 q−1 j
a ∼ a−j , a−j ∈ Sq , g.h. of degree m − k−
q q
j≥0
m ,k + q−1
i
q − 1 i
b ∼ b−i , b−i ∈ Sq , g.h. of degree m − k − .
q q
i≥0

 
Then a ◦ b is an operator in OP Sqm+m ,k+k with

N −1 1
σ(a ◦ b) − σ (∂τα a−j (x, t, Dx , τ ) ◦x Dtα b−i (x, t, Dx , τ ))
s=0 qα+i+j=s
α!
 
∈ Sqm+m −N,k+k . (4.1.12)

Here ◦x denotes the composition w.r.t. the x-variable.

62 A. Bove, M. Mughetti and D.S. Tartakoff


Lemma 4.1.3 ([2], Section 5 and [9], Sections 2.2, 2.3). Let a ∈ Hqm , b ∈ Hqm and
m
λ ∈ S1,0 (Rt × Rτ ) with homogeneous asymptotic expansions
m− qj j
a ∼ a−j , a−j ∈ Hq , g.h. of degree m −
q
j≥0
m − qi i
b ∼ b−i , b−i ∈ Hq , g.h. of degree m −
q
i≥0
m − q 
λ ∼ λ− , λ− ∈ S1,0 , homogeneous of degree m −
q
≥0

Then
m+m − 1q
(i) a ◦ b∗ is an operator in OP Hq (R2 , Σ) with

N −1 1 α
σ(a ◦ b∗ )(x, t, ξ, τ ) − e−ixξ ∂ a−j (x, t, τ )Dtα ˆb̄−i (ξ, t, τ )
s=0 qα+i+j=s
α! τ
m+m − 1q − N
∈ Hq q
, (4.1.13)

where the Fourier transform in Dtαˆb̄−i (ξ, t, τ ) is taken w.r.t. the x-variable.
m+m − 1q
(ii) b∗ ◦ a is an operator in OP S1,0 (Rt ) with

N −1 
∗ 1
σ(b ◦ a)(t, τ ) − ∂τα b̄−i (x, t, τ )Dtα a−j (x, t, τ )dx
s=0 qα+j+i=s
α!
m+m − q1 − N
∈ S1,0 q
(Rt ). (4.1.14)

(iii) a ◦ λ is an operator in OP Hqm+m . Furthermore its asymptotic expansion is
given by

N −1 1 α m+m − N
σ(a ◦ λ) − ∂τ a−j (x, t, τ )Dtα λ− (t, τ ) ∈ Hq q
. (4.1.15)
s=0 qα+j+=s
α!

Lemma 4.1.4. Let a(x, t, Dx , Dt ) be an operator in the class OP Sqm,k (R2 , Σ) and

b(x, t, Dt ) ∈ OP Hqm with g.h. asymptotic expansions
j
m,k+ q−1 q−1 j
a ∼ a−j , a−j ∈ Sq k−
, g.h. of degree m −
q q
j≥0
m − i i
b ∼ b−i , b−i ∈ Hq q−1 , g.h. of degree m − .
q
i≥0
 Gevrey Hypoellipticity for Kohn’s Operator 63

m+m −k q−1
Then a ◦ b ∈ OP Hq q
and has a g.h. asymptotic expansion of the form

N −1 1 
σ(a ◦ b) − ∂ a−j (x, t, Dx , τ )(Dt b−i (·, t, τ ))
s=0 q+i+j=s
! τ
m+m −k q−1 N
q − q
∈ Hq . (4.1.16)

Lemma 4.1.5. Let a(x, t, Dx , Dt ) be an operator in the class OP Sqm,k (R2 , Σ), b∗ (x,

m
t, Dt ) ∈ OP Hq∗ m and λ(t, Dt ) ∈ OP S1,0 (Rt ) with homogeneous asymptotic ex-
pansions
m,k+ q−1 j
q−1 j
a ∼ a−j , a−j ∈ Sq , g.h. of degree m − k−
q q
j≥0
m − i i
b ∼ b−i , b−i ∈ Hq q−1 , g.h. of degree m −
q
i≥0
m − q 
λ ∼ λ− , λ− ∈ S1,0 , homogeneous of degree m −
q
≥0

Then
 q−1
(i) b∗ (x, t, Dt ) ◦ a(x, t, Dx , Dt ) ∈ OP Hq∗ m+m − q k with g.h. asymptotic expan-
sion

N −1 1 
σ(b∗ ◦ a) − D (a−j (x, t, Dx , τ ))∗ (∂τ b−i (·, t, τ ))
s=0 q+i+j=s
! t
 q−1 N
∈ Hq m+m −k q − q . (4.1.17)
 
(ii) λ(t, Dt ) ◦ b∗ (x, t, Dt ) ∈ OP Hq∗ m +m with asymptotic expansion

N −1
∗ 1 α m +m − N
σ(λ ◦ b ) − ∂τ λ− (t, τ )Dtα b−i (x, t, τ ) ∈ Hq q
. (4.1.18)
s=0 qα+i+=s
α!

The proofs of Lemmas 4.1.2–4.1.4 are obtained with a q-variation of the

calculus developed by Boutet de Monvel and Helffer, [2], [9]. The proof of Lemma
4.1.5 is performed taking the adjoint and involves a combinatoric argument; we
give here a sketchy proof.

Proof. We prove item (i). The proof of (ii) is similar and simpler.
∗
Since b∗ (x, t, Dt ) ◦ a(x, t, Dx , Dt ) = (a(x, t, Dx , Dt )∗ ◦ b∗ (x, t, Dt )∗ ) , using
Lemma 4.1.1 and 4.1.2, we first compute
σ(a(x, t, Dx , Dt )∗ ◦ b∗ (x, t, Dt )∗ )
1  
= ∂τα+p Dtα (a−j (x, t, Dx , τ ))∗ ∂τ Dt+p b−i (·, t, τ )
!α!p!
α,,p,i,j≥0
64 A. Bove, M. Mughetti and D.S. Tartakoff
⎛ ⎞
1 1  
= ∂ γ Dγ ⎝ (−Dt )β (a−j (x, t, Dx , τ ))∗ ∂τβ b−i (·, t, τ ) ⎠ ,
γ! τ t β!
γ≥0 β,i,j≥0

where (−Dt )β (a−j (x, t, Dx , τ ))∗ denotes the formal adjoint of the operator with
symbol Dtβ a−j (x, t, ξ, τ ) as an operator in the x-variable, depending on (t, τ ) as
parameters. Here we used Formula (A.2) in the Appendix. Hence
1
σ(b∗ (x, t, Dt ) ◦ a(x, t, Dx , Dt )) = ∂  D
! τ t
≥0
⎛ ⎛ ⎞⎞−
1 1 ∗  
×⎝ ∂γ D ⎝
γ
(−Dt )β (a−j (x, t, Dx , τ )) ∂τβ b−i (·, t, τ ) ⎠⎠
γ! τ t β!
γ≥0 β,i,j≥0

1 β ∗ 
= D (a−j (x, t, Dx , τ )) ∂τβ b−i (·, t, τ )
β! t
β,i,j≥0
1 β ∗ 
= Dt (a−j (x, t, Dx , τ )) ∂τβ b−i (·, t, τ ) ,
β!
s≥0 qβ+i+j=s

because of Formula (A.3) of the Appendix. 

4.2. The actual computation of the eigenvalue

We are now in a position to start computing the symbol of Λ.
Let us first examine the minimum eigenvalue and the corresponding eigen-
function of P0 (x, t, Dx , τ ) in (4.1.1), as an operator in the x-variable. It is well
known that P0 (x, t, Dx , τ ) has a discrete set of non negative, simple eigenvalues
depending in a real analytic way on the parameters (t, τ ).
P0 can be written in the form LL∗ + t2 L∗ L, where L = Dx + ixq−1 τ .
The kernel of L∗ is a one-dimensional vector space generated by ϕ0,0 (x, τ ) =
1 q
c0 τ 2q exp(− xq τ ), c0 being a normalization constant such that ϕ0,0 (·, τ )L2 (Rx )
= 1. We remark that in this case τ is positive. For negative values of τ the opera-
tor LL∗ is injective. Denoting by ϕ0 (x, t, τ ) the eigenfunction of P0 corresponding
to its lowest eigenvalue Λ0 (t, τ ), we obtain that ϕ0 (x, 0, τ ) = ϕ0,0 (x, τ ) and that
Λ0 (0, τ ) = 0. As a consequence the operator
P = BB ∗ + B ∗ (t2 + x2k )B, B = Dx + ixq−1 Dt , (4.2.1)
is not maximally hypoelliptic, i.e., hypoelliptic with a loss of 2 − 2q derivatives.
Next we give a more precise description of the t-dependence of both the
eigenvalue Λ0 and its corresponding eigenfunction ϕ0 of P0 (x, t, Dx , τ ).
It is well known that there exists an ε > 0, small enough, such that the
operator
&
1
Π0 = (μI − P0 (x, t, Dx , τ ))−1 dμ
2πi |μ|=ε
 Gevrey Hypoellipticity for Kohn’s Operator 65

is the orthogonal projection onto the eigenspace generated by ϕ0 . Note that Π0

depends on the parameters (t, τ ). The operator LL∗ is thought of as an unbounded
operator in L2 (Rx ) with domain Bq2 (Rx ) = {u ∈ L2 (Rx ) | xα Dxβ u ∈ L2 , 0 ≤
α
β + q−1 ≤ 2}. We have
  
(μI − P0 )−1 = I + t2 −A(I + t2 A)−1 (μI − LL∗ )−1 ,

where A = (LL∗ − μI)−1 L∗ L. Plugging this into the formula defining Π0 , we get
& &
1 1 2
Π0 = (μI − LL∗ )−1 dμ − t A(I + t2 A)−1 (μI − LL∗ )−1 dμ.
2πi |μ|=ε 2πi |μ|=ε

Hence
&
1
ϕ0 = Π0 ϕ0,0 = ϕ0,0 − t 2
A(I + t2 A)−1 (μI − LL∗ )−1 ϕ0,0 dμ
2πi |μ|=ε

= ϕ0,0 (x, τ ) + t2 ϕ̃0 (x, t, τ ). (4.2.2)

Since Π0 is an orthogonal projection then ϕ0 (·, t, τ )L2 (Rx ) = 1.

As a consequence we obtain that

Λ0 (t, τ ) = P0 ϕ0 , ϕ0
= t2 Lϕ0,0 2 + O(t4 ). (4.2.3)

We point out that Lϕ0,0 = 0. Observe that, in view of (4.1.2),

Λ0 (t, μτ ) = min P0 (x, t, Dx , μτ )u(x), u(x)

u∈Bq2
uL2 =1

u(μ−1/q x) u(μ−1/q x)
= min2 P0 (μ−1/q x, t, μ1/q Dx , μτ ) , −1/(2q)

u∈Bq μ−1/(2q) μ
uL2 =1
2
= μq min P0 (x, t, Dx , τ )v(x), v(x)

v∈Bq2
vL2 =1
2
= μ q Λ0 (t, τ ). (4.2.4)

This shows that Λ0 is homogeneous of degree 2/q w.r.t. the variable τ .

Since ϕ0 is the unique normalized solution of the equation (P0 (x, t, Dx , τ ) −
Λ0 (t, τ ))u(·, t, τ ) = 0, from (4.1.2) and (4.2.4) it follows that ϕ0 is globally homo-
geneous of degree 1/(2q). Moreover ϕ0 is rapidly decreasing w.r.t. the x-variable
smoothly dependent on (t, τ ) in a compact subset of R2 \ 0. Using estimates of the
1/(2q)
form (4.1.11) we can conclude that ϕ0 ∈ Hq .

Let us start now the construction of a right parametrix of the operator

$ %
P (x, t, Dx , Dt ) ϕ0 (x, t, Dt )
ϕ∗0 (x, t, Dt ) 0
66 A. Bove, M. Mughetti and D.S. Tartakoff

as a map from C0∞ (R2(x,t) ) × C0∞ (Rt ) into C ∞ (R2(x,t) ) × C ∞ (Rt ). In particular we
are looking for an operator such that
$ % $ %
P (x, t, Dx , Dt ) ϕ0 (x, t, Dt ) F (x, t, Dx , Dt ) ψ(x, t, Dt )
◦
ϕ∗0 (x, t, Dt ) 0 ψ ∗ (x, t, Dt ) −Λ(t, Dt )
$ %
IdC0∞ (R2 ) 0
≡ . (4.2.5)
0 IdC0∞ (R)

Here ψ and ψ ∗ denote operators in OP Hq and OP Hq∗ 1/2q , F ∈ OP Sq−2,−2 and

1/2q

2/q
Λ ∈ OP S1,0 . Here ≡ means equality modulo a regularizing operator.
From (4.2.5) we obtain four relations:
P (x, t, Dx , Dt ) ◦ F (x, t, Dx , Dt) + ϕ0 (x, t, Dt ) ◦ ψ∗ (x, t, Dt ) ≡ Id, (4.2.6)
P (x, t, Dx , Dt ) ◦ ψ(x, t, Dt ) − ϕ0 (x, t, Dt ) ◦ Λ(t, Dt ) ≡ 0, (4.2.7)
ϕ∗0 (x, t, Dt ) ◦ F (x, t, Dx , Dt ) ≡ 0, (4.2.8)

ϕ∗0 (x, t, Dt ) ◦ ψ(x, t, Dt ) ≡ Id. (4.2.9)

We are going to find the symbols F , ψ and Λ as asymptotic series of globally
homogeneous symbols:

F ∼ F−j , ψ∼ ψ−j , Λ∼ Λ−j , (4.2.10)
j≥0 j≥0 j≥0

From Lemma 4.1.2 we obtain that

1
σ(P ◦ F ) ∼ σ(∂τα P−j (x, t, Dx , τ ) ◦x Dtα F−i (x, t, Dx , τ )),
qα+i+j=s
α!
s≥0

where we denoted by P−j the globally homogeneous parts of degree 2q − qj of the

symbol of P , so that P = P0 + P−q + P−2k . Furthermore from Lemma 4.1.3(i) we
may write that
1
σ(ϕ0 ◦ ψ ∗ ) ∼ e−ixξ ∂ α ϕ0 (x, t, τ )Dtα ψ̄ˆ−i (ξ, t, τ ).
α! τ
s≥0 qα+i=s

Analogously Lemmas 4.1.4, (4.1.3)(iii) give

1
σ(P ◦ ψ) ∼ ∂  P−j (x, t, Dx , τ )(Dt ψ−i (·, t, τ )),
! τ
s≥0 q+i+j=s
1 α
σ(ϕ0 ◦ Λ) ∼ ∂ ϕ0 (x, t, τ )Dtα Λ− (t, τ ).
α! τ
s≥0 qα+=s

Finally Lemmas 4.1.5(i) and 4.1.3(ii) yield

1  ∗
σ(ϕ∗0 ◦ F ) ∼ Dt F−j (x, t, Dx , τ ) (∂τ ϕ0 (·, t, τ )),
!
s≥0 q+j=s
 Gevrey Hypoellipticity for Kohn’s Operator 67

and 
1
σ(ϕ∗0 ◦ ψ) ∼ ∂τα ϕ¯0 (x, t, τ )Dtα ψ−j (x, t, τ )dx.
α!
s≥0 qα+j=s
Let us consider the terms globally homogeneous of degree 0. We obtain the relations
P0 (x, t, Dx , τ ) ◦x F0 (x, t, Dx , τ ) + ϕ0 (x, t, τ ) ⊗ ψ0 (·, t, τ ) = Id (4.2.11)
P0 (x, t, Dx , τ )(ψ0 (·, t, τ )) − Λ0 (t, τ )ϕ0 (x, t, τ ) = 0 (4.2.12)
(F0 (x, t, Dx , τ ))∗ (ϕ0 (·, t, τ )) = 0 (4.2.13)

ϕ̄0 (x, t, τ )ψ0 (x, t, τ )dx = 1. (4.2.14)

Here we denoted by ϕ0 ⊗ ψ0 the operator u = u(x) → ϕ0 ψ̄0 udx; ϕ0 ⊗ ψ0 must
be a globally homogeneous symbol of degree zero.
Conditions (4.2.12) and (4.2.14) imply that ψ0 = ϕ0 . Moreover (4.2.12) yields
that
Λ0 (t, τ ) = P0 (x, t, Dx , τ )ϕ0 (x, t, τ ), ϕ0 (x, t, τ )
L2 (Rx ) ,
coherently with the notation chosen above. Conditions (4.2.11) and (4.2.13) are
rewritten as
P0 (x, t, Dx , τ ) ◦x F0 (x, t, Dx , τ ) = Id − Π0
F0 (x, t, Dx , τ )(ϕ0 (·, t, τ )) ∈ [ϕ0 ]⊥ ,
whence
'
(P0 (x, t, Dx , τ )|[ϕ ⊥ ∩B 2
)−1 on [ϕ0 ]⊥
F0 (x, t, Dx , τ ) = 0] q (4.2.15)
0 on [ϕ0 ].
Since P0 is q-globally elliptic w.r.t. (x, ξ) smoothly depending on the parameters
(t, τ ), one can show that F0 (x, t, Dx , τ ) is actually a pseudodifferential operator
whose symbol verifies (4.1.10) with m = k = −2, j = 0, and is globally homoge-
neous of degree −2/q.
From now on we assume that q < 2k and that 2k is not a multiple of q; the
complementary cases are analogous.
Because of the fact that P−j = 0 for j = 1, . . . , q − 1, relations (4.2.11)–
(4.2.14) are satisfied at degree −j/q, j = 1, . . . , q − 1, by choosing F−j = 0,
ψ−j = 0, Λ−j = 0. Then we must examine homogeneity degree −1 in Equations
(4.2.6)–(4.2.9). We get
P−q ◦x F0 + P0 ◦x F−q + ∂τ P0 ◦x Dt F0
+ϕ0 ⊗ ψ−q + ∂τ ϕ0 ⊗ Dt ϕ0 = 0 (4.2.16)
P0 (ψ−q ) + P−q (ϕ0 ) + ∂τ P0 (Dt ϕ0 )
−Λ−q ϕ0 − Dt Λ0 ∂τ ϕ0 = 0 (4.2.17)
∗
(F−q ) (ϕ0 ) − (Dt F0∗ )(∂τ ϕ0 ) = 0 (4.2.18)
ψ−q , ϕ0
L2 (Rx ) + Dt ϕ0 , ∂τ ϕ0
L2 (Rx ) = 0. (4.2.19)
68 A. Bove, M. Mughetti and D.S. Tartakoff

⊥
First we solve w.r.t. ψ−q = ψ−q , ϕ0
L2 (Rx ) ϕ0 + ψ−q ∈ [ϕ0 ] ⊕ [ϕ0 ]⊥ . From (4.2.19)
we get immediately that
ψ−q , ϕ0
L2 (Rx ) = − Dt ϕ0 , ∂τ ϕ0
L2 (Rx ) . (4.2.20)
(4.2.17) implies that
⊥
P0 ( ψ−q , ϕ0
ϕ0 ) + P0 (ψ−q ) = −P−q (ϕ0 ) − ∂τ P0 (Dt ϕ0 ) + Λ−q ϕ0 + Dt Λ0 ∂τ ϕ0 .
Thus, using (4.2.20) we obtain that

[ϕ0 ]⊥  P0 (ψ−q
⊥
)
= −P−q (ϕ0 ) − ∂τ P0 (Dt ϕ0 ) + Λ−q ϕ0 + Dt Λ0 ∂τ ϕ0 + Dt ϕ0 , ∂τ ϕ0
Λ0 ϕ0 ,
whence
Λ−q = P−q (ϕ0 ) + ∂τ P0 (Dt ϕ0 ) − DtΛ0 ∂τ ϕ0 , ϕ0
L2 (Rx ) − Dt ϕ0 , ∂τ ϕ0
Λ0 . (4.2.21)

ψ−q = − Dt ϕ0 , ∂τ ϕ0
L2 (Rx ) ϕ0
+ F0 (−P−q (ϕ0 ) − ∂τ P0 (Dt ϕ0 ) + Dt Λ0 ∂τ ϕ0 ) , (4.2.22)
since, by (4.2.15), F0 ϕ0 = 0. From (4.2.18) we deduce that, for every u ∈ L2 (Rx ),
Π0 F−q u = u, (Dt F0∗ )(∂τ ϕ0 )
L2 (Rx ) ϕ0 = [ϕ0 ⊗ (Dt F0∗ )(∂τ ϕ0 )] u
Let −ω−q = P−q ◦x F0 + ∂τ P0 ◦x Dt F0 + ϕ0 ⊗ ψ−q + ∂τ ϕ0 ⊗ Dt ϕ0 . Then from
(4.2.15), applying F0 to both sides of (4.2.16), we obtain that
(Id − Π0 )F−q = −F0 ω−q .
Therefore we deduce that
F−q = ϕ0 ⊗ (Dt F0∗ )(∂τ ϕ0 ) − F0 ω−q . (4.2.23)
1
2q −1
Inspecting (4.2.22), (4.2.23) we see that ψ−q ∈ Hq , globally homogeneous of
q
−2,−2+ q−1
degree 1/2q − 1, F−q ∈ Sq globally homogeneous of degree −2/q − 1.
,
2/q−1
From (4.2.21) we have that Λ−q ∈ S1,0 homogeneous of degree 2/q − 1.
Moreover P−q is O(t2−1 ), Dt ϕ0 is estimated by t2−1 , for t → 0, because of (4.2.2),
Dt Λ0 is also O(t2−1 ) and Λ0 = O(t2 ) because of (4.2.3). We thus obtain that
Λ−q (t, τ ) = O(t2−1 ). (4.2.24)
This ends the analysis of the terms of degree −1 in (4.2.5).
The procedure can be iterated arguing in a similar way. We would like to
point out that the first homogeneity degree coming up and being not a negative
integer is −2k/q (we are availing ourselves of the fact that 2k is not a multiple of
q. If it is a multiple of q, the above argument applies literally, but we need also
the supplementary remark that we are going to make in the sequel.)
At homogeneity degree −2k/q we do not see the derivatives w.r.t. t or τ of the
symbols found at the previous levels, since they would only account for a negative
integer homogeneity degrees.
 Gevrey Hypoellipticity for Kohn’s Operator 69

In particular condition (4.2.7) for homogeneity degree −2k/q reads as

P0 ψ−2k + P−2k ϕ0 − ϕ0 Λ−2k = 0.

Taking the scalar product of the above equation with the eigenfunction ϕ0 and
recalling that ϕ0 (·, t, τ )L2 (Rx ) = 1, we obtain that

Λ−2k (t, τ ) = P−2k ϕ0 , ϕ0

L2 (Rx ) + P0 ψ−2k , ϕ0
L2 (Rx ) . (4.2.25)

Now, because of the structure of P−2k , P−2k ϕ0 , ϕ0

L2 (Rx ) > 0, while the second
term on the right, which is equal to ψ−2k , ϕ0
Λ̄0 , vanishes for t = 0. Thus if t is
small enough we deduce that

Λ−2k (t, τ ) > 0. (4.2.26)

From this point on the procedure continues exactly as above.

We have thus proved the

Theorem 4.2.1. The operator Λ defined in (4.2.5) is a pseudodifferential operator

2/q
with symbol Λ(t, τ ) ∈ S1,0 (Rt × Rτ ). Moreover, if j0 is a positive integer such that
j0 q < 2k < (j0 + 1)q, the symbol of Λ has an asymptotic expansion of the form

j0  
Λ(t, τ ) ∼ Λ−jq (t, τ ) + Λ−2k−sq (t, τ ) + Λ−(j0 +1)q−sq (t, τ ) . (4.2.27)
j=0 s≥0

Here Λ−p has homogeneity 2/q − p/q and

a) Λ−jq (t, τ ) = O(t2−j ) for j = 0, . . . , j0 .
b) Λ−2k satisfies (4.2.26).

4.3. Hypoellipticity of P
In this section we give a different proof of the C ∞ hypoellipticity of P . This is
accomplished by showing that the hypoellipticity of P follows from the hypoellip-
ticity of Λ and proving that Λ is hypoelliptic if condition (1.2) is satisfied. As a
matter of fact the hypoellipticity of P is equivalent to the hypoellipticity of Λ, so
that the structure of Λ in Theorem 4.2.1, may be used to prove assertion (iii) in
Theorem 1.1 (see [3].)
We state without proof the following

Lemma 4.3.1.
(a) Let a ∈ Sqm,k , properly supported, with k ≤ 0. Then Op a is continuous from
s s−m+k q−1
q
Hloc (R2 ) to Hloc (R2 ).
m+ 1
(b) Let ϕ ∈ Hq 2q , s
properly supported. Then Op ϕ is continuous from Hloc (R) to
s−m 2 ∗ s 2 s−m
Hloc (R ). Moreover ϕ (x, t, Dt ) is continuous from Hloc (R ) to Hloc (R).
70 A. Bove, M. Mughetti and D.S. Tartakoff

Mirroring the argument above, we can find symbols F ∈ Sq−2,−2 , ψ ∈ Hq

1/2q

2/q
and Λ ∈ S1,0 as in (4.2.10), such that
$ % $ %
F (x, t, Dx , Dt ) ψ(x, t, Dt ) P (x, t, Dx , Dt) ϕ0 (x, t, Dt )
◦
ψ ∗ (x, t, Dt ) −Λ(t, Dt ) ϕ∗0 (x, t, Dt ) 0
$ %
IdC0∞ (R2 ) 0
≡ . (4.3.1)
0 IdC0∞ (R)
From (4.3.1) we get the couple of relations
F (x, t, Dx , Dt ) ◦ P (x, t, Dx , Dt ) = Id − ψ(x, t, Dt ) ◦ ϕ∗0 (x, t, Dt ) (4.3.2)
ψ ∗ (x, t, Dt ) ◦ P (x, t, Dx , Dt ) = Λ(t, Dt ) ◦ ϕ∗0 (x, t, Dt ). (4.3.3)
Proposition 4.3.1. If Λ is hypoelliptic with a loss of δ derivatives, then P is also
hypoelliptic with a loss of derivatives equal to
q−1
2 + max{0, δ}.
q
Proof. Assume that P u ∈ Hloc s
(R2 ). From Lemma 4.3.1 we have that F P u ∈
Hloc (R ). By (4.3.2) we have that u−ψϕ∗0 u ∈ Hloc (R2 ). Again, using Lemma
s+2/q 2 s+2/q
∗ ∗
4.3.1, ψ P u ∈ Hloc (R), so that, by (4.3.3), Λϕ0 u ∈ Hloc
s s
(R). The hypoellipticity of
s+ 2 −δ
Λ yields then that ϕ∗0 u ∈ Hloc q (R). From Lemma 4.3.1 we obtain that ψϕ∗0 u ∈
s+ 2q −δ s+ 2 −max{0,δ}
Hloc (R). Thus u = (Id − ψϕ∗0 )u + ψϕ∗0 u ∈ Hloc q . This proves the
proposition. 
Next we prove the hypoellipticity of Λ under the assumption that  > k/q.
First we want to show that there exists a smooth non negative function
M (t, τ ), such that
|Λ(β) (t, τ )| ≤ Cα,β M (t, τ )(1 + |τ |)−ρα+δβ ,
(α)
M (t, τ ) ≤ C|Λ(t, τ )|, (4.3.4)
where α, β are non negative integers, C, Cα,β suitable positive constants and
the inequality holds for t in a compact neighborhood of the origin and |τ | large.
Moreover ρ and δ are such that 0 ≤ δ < ρ ≤ 1.
We actually need to check the above estimates for Λ only when τ is positive
and large.
k
Let us choose ρ = 1, δ = q < 1 and
2
 2k

M (t, τ ) = τ q t2 + τ − q ,
for τ ≥ c ≥ 1. It is then evident, from Theorem 4.2.1, that the first of the conditions
(4.3.4) is satisfied. The second condition in (4.3.4) is also straightforward for Λ0 +
Λ−2k , because of (4.2.26) and (4.2.3). To verify the second condition in (4.3.4) for
Λ−jq , q ∈ {1, . . . , j0 }, we have to use property a- in the statement of Theorem
4.2.1. Finally the verification is straightforward for the lower order parts of the
symbol in Formula (4.2.27). Using Theorem 22.1.3 of [10], we see that there exists a
 Gevrey Hypoellipticity for Kohn’s Operator 71

parametrix for Λ. Moreover from the proof of the above-quoted theorem we get that
the symbol of any parametrix satisfies the same estimates that Λ−1 satisfies, i.e.,
 2 2k
−1 k
|Dtβ Dτα Λ(t, τ )| ≤ Cα,β τ q t2 + τ − q (1 + τ )−α+ q β
2k
− 2q −α+ q
k
≤ Cα,β (1 + τ ) q β
,
for t in a compact set and τ ≥ C. Thus the parametrix obtained from Theorem
2k
− q2
22.1.3 of [10] has a symbol in S1,q k .
q
We may now state the
Theorem 4.3.1. Λ is hypoelliptic with a loss of 2k
q derivatives, i.e., Λu ∈ Hloc
s

s+ 2 − 2k
implies that u ∈ Hloc q q .
Theorem 4.3.1 together with Proposition 4.3.1 prove assertion (i) of Theo-
rem 1.1.

A. Appendix
We prove here a well-known formula for the adjoint of a product of two pseudodif-
0
ferential operators using just symbolic calculus. Let a, b symbols in S1,0 (Rt ). We
want to show that
(a#b)∗ = b∗ #a∗ , (A.1)
where # denotes the usual symbolic composition law (a higher-dimensional exten-
sion involves just a more cumbersome notation.)
We may write
(−1)α  
(a#b)∗ = ∂  D ∂ α āDtα b̄
α!! τ t τ
,α≥0
(−1)α


= ∂ α+r Dt−s ā ∂τ−r Dtα+s b̄.
α!! r s τ
,α≥0 r,s≤

Let us change the summation indices according to the following prescription; j =

α + r, β + j =  − s, i = α + s, so that  − r = i + β, we may rewrite the last
equality in the above formula as

(a#b)∗



(−1)i−s β + j + s β + j + s i+β i j β+j
= ∂τ Dt b̄ ∂τ Dt ā.
(i − s)!(β + j + s)! j − i + s s
i,j,β≥0 s≤i

Let us examine the s-summation; we claim that

i

(−1)i−s 1 β+j =s 1
= .
s=0
(i − s)! (β + i)!(j − i + s)! s β!i!j!
72 A. Bove, M. Mughetti and D.S. Tartakoff

This is actually equivalent to

i



i−s i β+j+s β+j
(−1) = .
s=0
s β+i j
Setting i − s = ν ∈ {0, 1, . . . , i}, the above relation is written as
i



i β+i+j−ν β+j
(−1)ν = ,
ν=0
ν β+i j
and this is precisely identity (12.15) in W. Feller [8], vol. 1.
Thus we may conclude that
1
(a#b)∗ = ∂ i+β Dti b̄∂τj Dtj+β ā
β!i!j! τ
i,j,β
⎛ ⎞ ⎛ ⎞
1 1 1
= ∂β ⎝ ∂ i Di b̄⎠ Dtβ ⎝ ∂ j Dj ā⎠
β! τ i! τ t j! τ t
β≥0 i≥0 j≥0
∗ ∗
= b #a .
This proves (A.1).
As a by-product of the above argument we get the following identity
1 (−1)α  
∂τi+β Dti b̄∂τj Dtj+β ā = ∂  D ∂ α āDtα b̄ , (A.2)
β!i!j! α!! τ t τ
i,j,β ,α≥0

which is the purpose of the present Appendix.

We would like to point out that the relation (a∗ )∗ = a rests on the identity
⎛ ⎞
1 1
∂  D ⎝ ∂ α Dα ā⎠
! τ t α! τ t
≥0 α≥0
 
1 s! 1
= (−1)α
∂τs Dts a = (1 − 1)s ∂τs Dts a = a. (A.3)
s! !α! s!
s≥0 +α=s s≥0

References
[1] L. Boutet de Monvel, F. Treves, On a class of pseudodifferential operators with double
characteristics, Inv. Math. 24(1974), 1–34.
[2] L. Boutet de Monvel, Hypoelliptic operators with double characteristics and related
pseudodifferential operators, Comm. Pure Appl. Math. 27(1974), 585–639.
[3] A. Bove, M. Mughetti and D.S. Tartakoff, paper in preparation.
[4] A. Bove and D.S. Tartakoff, Optimal non-isotropic Gevrey exponents for sums of
squares of vector fields, Comm. Partial Differential Equations 22 (1997), no. 7–8,
1263–1282.
 Gevrey Hypoellipticity for Kohn’s Operator 73

[5] A. Bove and D.S. Tartakoff, Gevrey Hypoellipticity for Non-subelliptic Operators,
preprint, 2008.
[6] A. Bove, M. Derridj, J.J. Kohn and D.S. Tartakoff, Sums of Squares of Complex
Vector Fields and (Analytic-) Hypoellipticity, Math. Res. Lett. 13(2006), no. 5-6,
683–701.
[7] M. Christ, A remark on sums of squares of complex vector fields, arXiv:math.CV/
0503506.
[8] W. Feller, An Introduction to Probability Theory and Its Applications, John Wiley
and Sons, New York, London, Sidney, 1967.
[9] B. Helffer, Sur l’hypoellipticité des opérateurs à caractéristiques multiples (perte de
3/2 dérivées), Mémoires de la S.M.F., 51-52(1977), 13–61.
[10] L. Hörmander, The Analysis of Partial Differential Operators, III, Springer Verlag,
1985.
[11] J.J. Kohn, Hypoellipticity and loss of derivatives, Ann. of Math. (2) 162 (2005) 943–
982.
[12] C. Parenti, A. Parmeggiani, On the hypoellipticity with a big loss of derivatives,
Kyushu J. Math. 59(2005), 155–230.
[13] J. Sjöstrand and M. Zworski, Elementary linear algebra for advanced spectral prob-
lems, Festival Yves Colin de Verdière. Ann. Inst. Fourier (Grenoble) 57(2007), 2095–
2141.

Antonio Bove and Marco Mughetti

Dipartimento di Matematica
Università di Bologna,
Piazza di Porta San Donato 5
Bologna, Italy
e-mail: [email protected]
[email protected]
David S. Tartakoff
Department of Mathematics
University of Illinois at Chicago
m/c 249, 851 S. Morgan St.,
Chicago, IL 60607, USA
e-mail: [email protected]
Complex Analysis
Trends in Mathematics, 75–94

c 2010 Springer Basel AG

Subelliptic Estimates
David W. Catlin and John P. D’Angelo
Dedicated to Linda Rothschild

Abstract. This paper gathers old and new information about subelliptic es-
timates for the ∂-Neumann problem on smoothly bounded pseudoconvex do-
mains. It discusses the failure of effectiveness of Kohn’s algorithm, gives an
algorithm for triangular systems, and includes some new information on sharp
subelliptic estimates.
Mathematics Subject Classification (2000). 32T25, 32T27, 32F10, 32W05,
35N15, 35H20.
Keywords. Subelliptic estimates, ∂-Neumann problem, pseudoconvexity, pluri-
subharmonic functions, orders of contact, finite type, triangular systems.

1. Introduction
The purpose of this paper is to clarify some issues concerning subelliptic estimates
for the ∂-Neumann problem on (0, 1) forms. Details of several of the results and
examples here do not appear in the literature, but versions of them have been
known to the authors and a few others for a long time, and some have been
mentioned without proof such as in [DK]. Recent interest in this subject helps
justify including them. Furthermore, the situation in two complex dimensions has
long been completely understood; one of the main results there is due to Rothschild
and Stein ([RS]) and hence fits nicely into this volume.
First we briefly recall the definition of subelliptic estimate and one conse-
quence of such an estimate. See [BS], [C1], [C2], [C3], [DK], [K4], [K5], [KN] for
considerable additional discussion. We then discuss the situation in two complex
dimensions, where things are completely understood. We go on to describe two
methods for proving such estimates, Kohn’s method of subelliptic multipliers and
Catlin’s method of construction of bounded plurisubharmonic functions with large
Hessians.
We provide in Proposition 4.4 an example exhibiting the failure of effec-
tiveness for Kohn’s algorithm for finding subelliptic multipliers, and we give a
76 D.W. Catlin and J.P. D’Angelo

simplified situation (Theorem 5.1) in which one can understand this algorithm
perfectly. This section is taken from [D5]. We go on to discuss some unpublished
examples of the first author. These examples provide surprising but explicit infor-
mation about how the largest possible value of the parameter  that arises in a
subelliptic estimate is related to the geometry of the boundary. See Example 7.1
and Theorem 7.2.
Both authors acknowledge discussions with Joe Kohn over the years, and the
second author acknowledges support from NSF Grant DMS-07-53978.

2. Definition of subelliptic estimates

Let Ω be a pseudoconvex domain in Cn with smooth boundary, and assume that
p ∈ bΩ. Let T 1,0 bΩ be the bundle whose sections are (1, 0) vectors tangent to bΩ.
We may suppose that there is a neighborhood of p on which bΩ is given by the
n of a smooth function r with dr(p) = 0. In coordinates, a vector field
vanishing
L = j=1 aj ∂z∂ j is a local section of T 1,0bΩ if, on bΩ

n
aj (z)rzj (z) = 0. (1)
j=1

Then bΩ is pseudoconvex at p if, whenever (1) holds we have

n
rzj zk (p)aj (p)ak (p) ≥ 0. (2)
j,k=1

It is standard to express (2) more invariantly. The bundle T 1,0(bΩ) is a sub-

bundle of T (bΩ) ⊗ C. The intersection of T 1,0 (bΩ) with its complex conjugate
bundle is the zero bundle, and their direct sum has fibers of codimension one in
T (bΩ) ⊗ C. Let η be a non-vanishing purely imaginary 1-form that annihilates this
direct sum. Then (1) and (2) together become
λ(L, L) = η, [L, L]
≥ 0 (3)
1,0
on bΩ for all local sections of T (bΩ). Formula (3) defines a Hermitian form λ
on T 1,0 (bΩ) called the Levi form. The Levi form is defined only up to a multiple,
but this ambiguity makes no difference in what we will do. The domain Ω or
its boundary bΩ is called pseudoconvex if the Levi form is definite everywhere
on bΩ; in this case, we multiply by a constant to ensure that it is nonnegative
definite. The boundary is strongly pseudoconvex at p if the Levi form is positive
definite there. Each smoothly bounded domain has an open subset of strongly
pseudoconvex boundary points; the point farthest from the origin must be strongly
pseudoconvex, and strong pseudoconvexity is an open condition.
Subelliptic estimates arise from considering the ∂-complex on the closed do-
main Ω. As usual in complex geometry we have notions of smooth differential
forms of type (p, q). We will be concerned only with the case of (0, 1) forms here;
similar examples and results apply for forms of type (p, q).
 Subelliptic Estimates 77


A smooth differential (0, 1) form nj=1 φj dz j , defined near p, lies in the do-
∗ n
main of ∂ if the vector field j=1 φj ∂z∂ j lies in Tz1,0 bΩ for z near p. The boundary
∗  ∂r
condition for being in the domain of ∂ therefore becomes φj ∂zj = 0 on the set
where r = 0. Let ψ denote the L norm and let ψ denote the Sobolev  norm
2

of ψ, where ψ can be either a function or a differential form. The Sobolev norm

involves fractional derivatives of order  of the components of ψ.
Definition 2.1. A subelliptic estimate holds on (0, 1) forms at p if there is a neigh-
borhood U of p and positive constants C and  such that (4) holds for all forms
∗
φ, compactly supported in U and in the domain of ∂ .
 ∗

φ2 ≤ C ∂φ2 + ∂ φ2 + φ2 . (4)

In this paper we relate the largest possible value of the parameter  for which
(4) holds to the geometry of bΩ.
Perhaps the main interest in subelliptic estimates is the fundamental local
regularity theorem of Kohn and Nirenberg [KN]. In the statement of the theorem,
the canonical solution to the inhomogeneous Cauchy-Riemann equation is the
unique solution orthogonal to the holomorphic functions.
Theorem 2.1. Let Ω be a smoothly bounded pseudoconvex domain, and assume that
there is a subelliptic estimate at a boundary point p. Then there is a neighborhood
U of p in Ω with the following property. Let α be a (0, 1) form with L2 coefficients
and ∂α = 0. Let u be the canonical solution to ∂u = α. Then u is smooth on any
open subset of U on which α is smooth.
It has been known for nearly fifty years ([K1], [K2], [FK]) that there is a
subelliptic estimate with  = 12 at each strongly pseudoconvex boundary point.
One is also interested in global regularity. See [BS] for a survey of results on global
regularity of the canonical solution. In particular, on each smoothly bounded pseu-
doconvex domain, there is a smooth solution to ∂u = α when α is smooth and
∂α = 0, but the canonical solution itself need not be smooth.

3. Subelliptic estimates in two dimensions

Let Ω be a pseudoconvex domain in C2 with smooth boundary M , and suppose
p ∈ M . The statement of Theorem 3.1 below, resulting by combining the work of
several authors, completely explains the situation.
Assume that r is a defining function for M near p. We may choose coordinates
such that p is the origin and
r(z) = 2Re(z2 ) + f (z1 , Im(z2 )), (5)
where df (0) = 0. We let T(M, p) denote the maximum order of contact of one-
dimensional complex analytic curves with M at p, and we let Treg (M, p) denote
the maximum order of contact of one-dimensional regular complex analytic curves
78 D.W. Catlin and J.P. D’Angelo

with M at p. We let t(M, p) denote the type of M at p, defined as follows. Let L

be a type (1, 0) vector field on M , with L(p) = 0. Then type(L, p) is the smallest
integer k such that there is an iterated bracket Lk = [. . . [L1 , L2 ], . . . , Lk ] for which
each Lj is either L or L and such that
Lk , η
(p) = 0.
This number measures the degeneracy of the Levi form at p. It is independent of
the choice of L, as Tp1,0 M is one-dimensional. We put t(M, p) = type(L, p).
In two dimensions there is an equivalent method for computing t(M, p). Con-
sider the Levi form λ(L, L) as a function defined near p. We ask how many deriva-
tives one must take in either the L or L direction to obtain something non-zero at
p. Then c(L, p) is defined to be two more than this minimum number of deriva-
tives; we add two because the Levi form already involves two derivatives. In two
dimensions it is easy to see that type(L, p) = c(L, p). This conclusion is false in
higher dimensions when the Levi form has eigenvalues of opposite signs at p. It is
likely to be true on pseudoconvex domains; see [D1] for more information.
In C2 there are many other ways to compute the type of a point. The easiest
one involves looking at the defining function directly. With f as in (5), both of
these concepts and also both versions of orders of contact mentioned above equal
the order of vanishing of the function f (z1 , 0) at the origin. Things are much more
subtle and interesting in higher dimensions regarding these various measurements.
See [D1]. Both the geometry and the estimates are easier in C2 than in higher
dimensions; the following theorem explains fully the two-dimensional case.
Theorem 3.1. Let Ω be a smoothly bounded pseudoconvex domain in C2 , and sup-
pose p ∈ bΩ. The following are equivalent:
1
1) There is a subelliptic estimate at p with  = 2m , but for no larger value of .
2) For L a (1, 0) vector field on bΩ with L(p) = 0, we have type(L, p) = 2m.
3) For L as in 2), we have c(L, p) = 2m.
4) There is an even integer 2m such that T(bΩ, p) = 2m.
5) There is an even integer 2m such that Treg (M, p) = 2m.

Kohn [K3] established the first subelliptic estimate for domains in C2 , assum-
ing that type(L, p) was finite. Greiner [Gr] established the converse. To establish
the sharp result that  could be chosen to be the reciprocal of type(L, p), Kohn in-
voked results of Rothschild-Stein [RS] based on the notion of nilpotent Lie groups.
These difficult results establish the equivalence of 1) and 2) above. Also see for
example [CNS] among the many references for estimates in other function spaces
for solving the Cauchy-Riemann equations in two dimensions.
The geometry in two dimensions is easy to understand; it is quite easy to
establish that condition 2) is equivalent to the other conditions from Theorem 3.1,
and hence we listed all five conditions. In higher dimensions, however, the geometry
is completely different. Nonetheless, based on Theorem 3.1, one naturally seeks a
geometric condition for subellipticity in higher dimensions.
 Subelliptic Estimates 79

4. Subelliptic multipliers
We next consider the approach of Kohn from [K4] for proving subelliptic estimates.
Let E denote the ring of germs of smooth functions at p. Recall that u denotes
the L2 -norm of u; we use this notation whether u is a function, a 1-form, or a
2-form. We write u for the Sobolev  norm.
Definition 4.1. Assume f ∈ E. We say that f is a subelliptic multiplier at p if there
are positive constants C and  and a neighborhood U such that
 ∗

f φ2 ≤ C ∂φ2 + ∂ φ2 + φ2 (6)
∗
for all forms φ supported in U and in the domain of ∂ .
∗
We will henceforth write Q(φ, φ) for ∂φ2 + ∂ φ2 + φ2 . By Definitions
2.1 and 4.1, a subelliptic estimate holds at p if and only if the constant function 1
is a subelliptic multiplier at p. We recall that when bΩ is strongly pseudoconvex
at p we can take  = 12 in (4).
The collection of subelliptic multipliers is a non-trivial ideal in E closed under
taking radicals. Furthermore, the defining function r and the determinant of the
Levi form det(λ) are subelliptic multipliers. We state these results of Kohn [K4]:
Proposition 4.1. The collection I of subelliptic multipliers is a radical ideal in E;
in particular, if f N ∈ I for some N , then f ∈ I. Also, r and det(λ) are in I.
rφ21 ≤ CQ(φ, φ) (7)
det(λ)φ 1 ≤ CQ(φ, φ).
2
(8)
2

Kohn’s algorithm starts with these two subelliptic multipliers and constructs
additional ones. We approach the process via the concept of allowable rows. An
n-tuple (f1 , . . . , fn ) of germs of functions is an allowable row if there are positive
constants C and  such that, for all φ as in the definition of subelliptic estimate,
( (2
( (
( fj φj ( ≤ CQ(φ, φ). (9)
j 
The most important example of allowable row is, for each j, the jth row of the
Levi form, namely the n-tuple (rz1 z j , . . . , rzn zj ).
The following fundamental result of Kohn enables us to pass between allow-
able rows and subelliptic multipliers:
Proposition 4.2. Let f be a subelliptic multiplier such that
f φ22 ≤ Q(φ, φ). (10)
∂f ∂f
Then the n-tuple of functions ( ∂z 1
, . . . , ∂z 1
)
is an allowable row, and we have:
( (
( ∂f ( 2
( (
( j ∂zj φj ( ≤ CQ(φ, φ). (11)

Conversely, consider any n × n matrix (fij ) of allowable rows. Then det(fij ) is a
subelliptic multiplier.
80 D.W. Catlin and J.P. D’Angelo

Proof. See [K4] or [D1]. 

For domains with real analytic boundary, Kohn’s process always terminates
in finitely many steps, depending on only the dimension. Following the process
produces two lists of finite length; one of modules of allowable rows, the other of
subelliptic multipliers. The value of the  obtained from this process depends on
both the length of this list and the number of radicals taken in each step. We
will show that there is no positive lower bound on the value of  in a subelliptic
estimate obtained from Kohn’s process in general. In order to do so we recall some
geometric information and notation from [D1] and [D2].
For a real hypersurface M in Cn , we recall that T(M, p) denotes the max-
imum order of contact of one-dimensional complex analytic varieties with M at
p. We compute this number as follows. Let ν(z) denote the order of vanishing
operator. Let z be a parametrized holomorphic curve with z(0) = p. We compute
∗
the ratio T(M, p, z) = ν(z r)
ν(z) and call it the order of contact of the curve z with M
at p. Then T(M, p) is the supremum over z of T(M, p, z). Later we will generalize
this concept.
Next we consider the ring of germs of holomorphic functions O at 0 in Cn .
Some of the ideas also apply to the formal power series ring; at times we write R
or Rn when the statement applies in either setting. See [Cho] for a treatment of
Kohn’s algorithm in the formal power series setting.
The maximal ideal in O is denoted by m. If I is a proper ideal in O, then
the Nullstellensatz guarantees that its variety V(I) is an isolated point if and only
if the radical of I equals m. In this case the intersection number D(I) plays an
important role in our discussions. We put
D(I) = dimC O/I.
For such an ideal I we also consider its order of contact T(I), defined analogously to
the order of contact with a hypersurface. This number provides a slightly different
measurement of the singularity than does D(I). See [D1] and [D5] for precise
information.
The following proposition is a special case of results from [D2] and [K4]. It
gives a simple situation where one can relate the geometry to the estimates. Note
that the geometric conditions 3) through 6) state in various ways that there is no
complex analytic curve in bΩ through 0.
Proposition 4.3. Let Ω be a pseudoconvex domain in Cn for which 0 ∈ bΩ, and
there are holomorphic functions hj such that the defining equation near 0 can be
written as
N
r(z) = Re(zn ) + |hj (z)|2 . (12)
j=1

The following are equivalent:

1) There is a subelliptic estimate on (0, 1) forms.
 Subelliptic Estimates 81

2) There is no complex analytic (one-dimensional) curve passing through 0 and

lying in bΩ.
3) T(bΩ, 0) is finite.
4) V(zn , h1 , . . . , hN ) = {0}.
5) The radical of the ideal (zn , h1 , . . . , hN ) is m.
6) D(zn , h1 , . . . , hN ) is finite.
Our next example is of the form (12), but it illustrates a new quantitative
result. Let Ω be a pseudoconvex domain in C3 whose defining equation near the
origin is given by
r(z) = Re(z3 ) + |z1M |2 + |z2N + z2 z1K |2 . (13)
We assume that K > M ≥ 2 and N ≥ 3. We note that T(bΩ, 0) = 2max(M, N )
and that D(z1M , z2N + z2 z1K , z3 ) = M N . In the next result we show that Kohn’s
algorithm for finding subelliptic multipliers gives no lower bound for  in terms of
the dimension and the type.
Proposition 4.4 (Failure of effectiveness). Let Ω be a pseudoconvex domain whose
boundary contains 0, and which is defined near 0 by (13). Then the root taken in the
radical required in the second step of Kohn’s algorithm for subelliptic multipliers is
at least K, and hence it is independent of the type at 0. In particular, the procedure
in [K4] gives no positive lower bound for  in terms of the type.
Proof. Let Ω be a domain in Cn+1 defined near the origin by (13). By the discussion
in [K4], [D1] or [D5], Kohn’s algorithm reduces to an algorithm in the ring O
in two dimensions. We therefore write the variables as (z, w) and consider the
ideal (h) defined by (z M , wN + wz K ) in two variables. The exponents are positive
integers; we assume K > M ≥ 2 and N ≥ 3. Note that D(h) = M N and T(h) =
max(M, N ). We write g(z, w) = wN + wz K and we use subscripts on g to denote
partial derivatives.
The algorithm begins with the collection M0 of allowable rows spanned by
(14) and the ideal I0 given in (15):

M−1 
z 0
(14)
gz gw
There is only one determinant to take, and therefore
I0 = rad(z M−1 gw ) = (zgw ). (15)
By definition M1 is the union of M0 and d(zgw ) = (zgwz + gw )dz + zgww dw.
Using the row notation as before we see that the spanning rows of M1 are given
by (16): ⎛ M−1 ⎞
z 0
⎝ gz gw ⎠ . (16)
zgwz + gw zgww
It follows that I1 is the radical of the ideal J1 generated by the three possible
determinants.
82 D.W. Catlin and J.P. D’Angelo

The ideal generated by zgw and the two new determinants is

J1 = (zgw , z M gww , zgz gww − zgw gzw − gw
2
). (17)
It is easy to see that
I1 = rad(J1 ) = m. (18)
Thus M2 includes dz and dw and hence I2 = (1).
The crucial point concerning effectiveness involves the radical taken in passing
from J1 to I1 . We prove that we cannot bound this root in terms of M and N .
To verify this statement we claim that z K−1 is not an element of J1 . This claim
shows that the number of roots taken must be at least K. Since K can be chosen
independently of M and N and also arbitrarily large, there is no bound on the
number of roots taken in terms of the dimension 2 and the intersection number
D(h) = M N or the order of contact T(I) = max(M, N ).
It remains to prove the claim. If z K−1 ∈ J1 , then we could write
z K−1 = a(z, w)zgw + b(z, w)z M gww + c(z, w)(zgz gww − zgzw − gw
2
) (19)
K
for some a, b, c. We note that gww (z, 0) = 0, that gw (z, 0) = z , and gzw (z, 0) =
Kz K−1 . Using this information we set w = 0 in (19) and obtain
z K−1 = a(z, 0)z K + b(z, 0)0 + c(z, 0)(−zKz K−1 + 0). (20)
It follows from (20) that z K−1 is divisible by z K ; this contradiction proves that
z K−1 is not in J1 , and hence that passing to I1 requires at least K roots. (It is easy
to show, but the information is not needed here, that taking K roots suffices.) 

This proposition shows that one cannot take radicals in a controlled fash-
ion unless one revises the algorithm. One might naturally ask whether we can
completely avoid taking radicals. The following example shows otherwise.
Example 4.1. Put n = 2, and let h denote the three functions (z 2 , zw, w2 ). Then the
three Jacobians obtained are (z 2 , 2w2 , 4zw). If we tried to use the ideal generated
by them, instead of its radical, then the algorithm would get stuck. We elaborate;
the functions z 2 , zw, w2 are not known to be subelliptic multipliers at the start.
After we compute I0 , however, they are known to be subelliptic multipliers and
hence we are then allowed to take the radical. This strange phenomenon (we cannot
use these functions at the start, but we can use them after one step) illustrates
one of the subtleties in Kohn’s algorithm.

5. Triangular systems
Two computational difficulties in Kohn’s algorithm are finding determinants and
determining radicals of ideals. We describe a nontrivial class of examples for which
finding the determinants is easy. At each stage we require only determinants of tri-
angular matrices. Furthermore we avoid the computation of uncontrolled radicals;
for this class of examples we never take a root of order larger than the underlying
 Subelliptic Estimates 83

dimension. In order to do so, we deviate from Kohn’s algorithm by treating the

modules of (1, 0) forms differently.
We call this class of examples triangular systems. The author introduced
a version of these examples in [D4], using the term regular coordinate domains,
but the calculations there give a far from optimal value of the parameter  in a
subelliptic estimate. The version in this section thus improves the work from [D4].
Catlin and Cho [CC] and independently Kranh and Zampieri [KZ] have recently
established subelliptic estimates in some specific triangular systems. The crucial
point in this section is that triangular systems enable one to choose allowable rows
in Kohn’s algorithm, one at a time and with control on all radicals. In Theorem
5.1 we establish a decisive result on effectiveness for triangular systems.
Definition 5.1 (Triangular Systems). Let H be a collection of nonzero elements
of m ⊂ Rn . We say that H is a triangular system of full rank if, possibly after a
linear change of coordinates, there are elements, h1 , . . . , hn ∈ H such that
1) For each i with 1 ≤ i ≤ n, we have ∂h i
∂zj = 0 whenever j > i. In other words,
hi depends on only the variables z1 , . . . , zi .
2) For each i with 1 ≤ i ≤ n, hi (0, zi ) = 0. Here (0, zi ) is the i-tuple (0, . . . , 0, zi ).
It follows from 1) that the derivative matrix dh = ( ∂h ∂zj ) for 1 ≤ i, j ≤ n
i

is lower triangular. (All the entries above the main diagonal vanish identically.)
∂zi (0, zi ) = 0. By combining these facts we see that J =
It follows from 2) that ∂h i

det(dh) is not identically zero. Our procedure makes no use of the other elements
of H.
Of course any ideal defining a zero-dimensional variety contains a triangular
system of full rank. We are assuming here additionally that the differentials of
these functions define the initial module of allowable rows.
Remark 5.1. Triangular systems of rank less than n are useful for understand-
ing the generalization of the algorithm where we consider q by q minors. We do
not consider these systems here, and henceforth we drop the phrase of full rank,
assuming that our triangular systems have full rank.
Let H be a triangular system. After renumbering, we may assume that h1 is
a function of z1 alone, h2 is a function of (z1 , z2 ), and so on. Note that h1 (z1 ) =
z1m1 u1 (z1 ) for a unit u1 , that h2 (z1 , z2 ) = z2 u2 (z2 ) + z1 g2 (z1 , z2 ) for a unit u2 ,
and so on. After changing coordinates again we may assume that these units are
constant. For example z1m1 u1 (z1 ) = ζ1m1 , where ζ1 is a new coordinate. We may
therefore assume that a triangular system includes functions h1 , . . . , hn as follows:
h1 (z) = z1m1 (21.1)
h2 (z) = z2m2
+ z1 g21 (z1 , z2 ) (21.2)
h3 (z) = z3m3 + z1 g31 (z1 , z2 , z3 ) + z2 g32 (z1 , z2 , z3 ) (21.3)

n−1
hn (z) = znmn + zj gnj (z1 , . . . , zn ). (21.n)
j=1
84 D.W. Catlin and J.P. D’Angelo

In (21) the holomorphic germs gkl are arbitrary. Our approach works uniformly in
them (Corollary 5.1), but the  from Kohn’s algorithm depends upon them.
Each hj depends upon only the first j variables and has a pure monomial in
zj . A useful special case is where each hj is a Weierstrass polynomial of degree mj
in zj whose coefficients depend upon only the first j − 1 variables.
Example 5.1. Write the variables (z, w) in two dimensions. The pair of functions
h(z, w) = (h1 (z, w), h2 (z, w)) = (z m , wn + zg(z, w)), (22)
where g is any element of R2 , form a triangular system.
Lemma 5.1. Let h1 , . . . , hn define a triangular system in Rn and let (h) denote the
ideal generated by them. Then
)
n
D(h) = mj . (23)
j=1

Proof. There are many possible proofs. One is to compute the vector space di-
mension of Rn /(h) by listing a basis of this algebra. The collection {z α } for
0 ≤ αi ≤ mi − 1 is easily seen to be a basis. 

We next provide an algorithm that works uniformly over all triangular sys-
tems. The result is a finite list of pairs of subelliptic multipliers; the length of the
list is the multiplicity from (23). The first pair of multipliers is (A1 , B1 ) where
both A1 and B1 equal the Jacobian. The last pair is (1, 1). The number of pairs
in the list is exactly the multiplicity (or length) of the ideal (h). The key point is
that each Aj is obtained from Bj by taking a controlled root of some of its factors.
In other words, each Bj divides a power of Aj , and the power never exceeds the
dimension.
We remark that the proof appears at first glance to be inefficient, as delicate
machinations within it amount to lowering an exponent by one. This inefficiency
arises because the proof works uniformly over all choices of the gij in (21). Perhaps
the proof could be rewritten as an induction on the multiplicity.
Theorem 5.1. There is an effective algorithm for establishing subelliptic estimates
for (domains defined by) triangular
* systems. That is, let h1 , . . . , hn define a trian-
gular system with L = D(h) = mj . The following hold:
1) There is a finite list of pairs of subelliptic multipliers (B1 , A1 ), . . . , (BL , AL )
such that B1 = A1 = det( ∂h i
∂zj ), also BL = AL , and BL is a unit.
2) Each Bj divides a power of Aj . The power depends on only the dimension
n and not on the functions hj . In fact, we never require any power larger
than n.
3) The length L of the list equals the multiplicity D(h) given in (23).
Proof. The proof is a complicated multiple induction. For clarity we write out the
cases n = 1 and n = 2 in full.
 Subelliptic Estimates 85

When n = 1 we never need to take radicals. When n = 1 we may assume

h1 (z1 ) = z1m1 . We set B1 = A1 = ( ∂z∂ 1 )h1 , and we set Bj = Aj = ( ∂z∂ 1 )j h1 . Then
B1 is a subelliptic multiplier, and each Bj+1 is the derivative of Bj and hence also
a subelliptic multiplier; it is the determinant of the one-by-one matrix given by
dBj . Since h1 vanishes to order m1 at the origin, the function Bm1 is a non-zero
constant. Thus 1) holds. Here L = m1 and hence 3) holds. Since Bj = Aj we also
verify that the power used never exceeds the dimension, and hence 3) holds. Thus
the theorem holds when n = 1.
We next write out the proof when n = 2. The initial allowable rows are dh1
and dh2 , giving a lower triangular two-by-two matrix, because ∂h 1
∂z2
= 0. We set
∂hi
B1 = A1 = det( ) = Dh1 Dh2 ,
∂zj
where we use the following convenient notation:
∂hk
Dhk = . (24)
∂zk
For 1 ≤ j ≤ m2 we set
Bj = (Dh1 )2 Dj h2 (25.1)
Aj = Dh1 Dj h2 . (25.2)
Each Bj+1 is a subelliptic multiplier, obtained by taking the determinant of the
allowable matrix whose first row is dh1 and second row is dAj . Recall that Dm2 h2
is a unit. When j = m2 in (25.2) we therefore find that Am2 is a unit times Dh1 .
The collection of multipliers is an ideal, and hence Dh1 is a subelliptic multiplier.
We may use d(Dh1 ) as a new allowable first row. Therefore
Bm2 +1 = D2 (h1 )Dh2 .
Using d(h1 ) as the first row and d(Bm2 +1 ) as the second row, we obtain
Bm2 +2 = (D2 h1 )2 D2 h2
Am2 +2 = D2 h1 D2 h2 .
Notice again that we took only a square root of the first factor; more precisely,
A2k is divisible by Bk , where k = m2 + 2. Thus each Ak is a multiplier as well.
Proceeding in this fashion we obtain
Am2 +j = D2 (h1 )D j h2 ,
and therefore A2m2 is a unit times D2 (h1 ). Thus d(D2 h1 ) is an allowable row. We
increase the index by m2 in order to differentiate h1 once! Applying this procedure
a total of m1 times we see that Bm1 m2 is a unit.
We started with A1 = B1 ; otherwise each Bj divides A2j . Since each Bj is
a determinant of a matrix of allowable rows, each Bj is a subelliptic multiplier.
Therefore each Aj is a subelliptic multiplier, and AL = BL is a unit when L =
m1 m2 . We have verified 1), 2), and 3).
We pause to repeat why we needed to take radicals of order two in the
above. After establishing that Aj = Dh1 D j h2 is a multiplier, we use dAj as an
86 D.W. Catlin and J.P. D’Angelo

allowable row. The next determinant becomes v = (Dh1 )2 Dj+1 h2 . If we use v as

a multiplier, then we obtain both Dh1 and D2 h1 as factors. Instead we replace v
with Dh1 Dj+1 h2 in order to avoid having both Dh1 and D 2 h1 appear.
We now describe this aspect of the process when n = 3 before sketching the
induction. For n = 3, we will obtain
A1 = B1 = (Dh1 )(Dh2 )(Dh3 ).
After m3 steps we will find that Am3 is a unit times Dh1 Dh2 . To compute the
next determinant we use dh1 as the first row, d(Dh1 Dh2 ) as the second row, and
dA1 as the third row. Each of these includes Dh1 as a factor, and hence (Dh1 )3 is
a factor of the determinant. Hence we need to take a radical of order three.
For general n, each matrix of allowable rows used in this process is lower
triangular, and hence each determinant taken is a product of precisely n expres-
sions. As above, the largest number of repeated factors is precisely equal to the
dimension.
Now we make the induction hypothesis: we assume that n ≥ 2, and that
h1 , . . . , hn defines a triangular system. We assume that 1) and 2) hold for all
triangular systems in n − 1 variables. We set
∂hi
B1 = A1 = det( ) = Dh1 Dh2 · · · Dhn . (26)
∂zj
We replace the last allowable row by dAn and take determinants, obtaining
B2 = Dh1 Dh2 · · · Dhn−1 Dh1 Dh2 · · · Dhn−1 D 2 hn (27)
as a subelliptic multiplier. Taking a root of order two, we obtain
A2 = Dh1 Dh2 · · · Dhn−1 D2 hn (28)
as a subelliptic multiplier. Repeating this process mn times we obtain
Amn = Dh1 Dh2 · · · Dhn−1 (29)
as a subelliptic multiplier. We use its differential dAmn as the n − 1-st allowable
row, and use dhn as the nth allowable row. Taking determinants shows that
Amn +1 = Dh1 Dh2 · · · Dhn−2 Dh1 Dh2 · · · D2 hn−1 Dhn (30)
is a subelliptic multiplier.
What we have done? We are in the same situation as before, but we have
differentiated the function hn−1 one more time, and hence we have taken one
step in decreasing the multiplicity of the singularity. We go through the same
process mn−1 mn times and we determine that Amn mn−1 is a subelliptic multiplier
which depends upon only the first n − 2 variables. We then use its differential
as the n − 2-nd allowable row. We obtain, after mn mn−1 mn−2 steps, a nonzero
subelliptic
* multiplier independent of the last three variables. By another induction,
after mj steps, we obtain a unit. Each determinant is the product of n diagonal
elements. At any stage of the process we can have a fixed derivative of h1 appearing
as a factor to at most the first power in each of the diagonal elements. Similarly
 Subelliptic Estimates 87

a derivative of Dh2 can occur as a factor only in the last n − 1 diagonal elements.
It follows that we never need to take more than nth root in passing from the Bk
(which is a determinant) to the Ak . After L steps in all we obtain the unit
D m1 h1 Dm2 h2 . . . Dmn hn = AL = BL
as a subelliptic multiplier. Thus 1), 2), and 3) hold. 

Corollary 5.1. Let Ω be a domain defined near 0 by

Re(zn+1 ) + |hj (z)|2 ,
where hj are as in (21). There is  > 0 such that the subelliptic estimate (4) holds
at 0 for all choices of the arbitrary function gjk in (21).
The algorithm used in the proof of Theorem 5.1 differs from Kohn’s algorithm.
At each stage we choose a single function A with two properties. Some power of A
is divisible by the determinant of a matrix of allowable rows, and the differential
dA provides a new allowable row. The algorithm takes exactly D(h) steps. Thus
we do not consider the modules Mk ; instead we add one row at a time to the list
of allowable (1, 0) forms. By being so explicit we avoid the uncontrolled radicals
required in Proposition 4.2.
Remark 5.2. The difference in this approach from [K4] can be expressed as follows.
We replace the use of uncontrolled radicals by allowing only nth roots of specific
multipliers. On the other hand, we must pay by taking derivatives more often. The
special case when n = 1 clarifies the difference.
The multiplicity D(h) is the dimension over C of the quotient algebra R/(h).
This algebra plays an important role in commutative algebra, and it is worth notic-
ing that the process in Theorem 5.1 seems to be moving through basis elements
for this algebra as it finds the Aj . We note however that the multipliers Bj might
be in the ideal and hence 0 in the algebra. We give a simple example.
Example 5.2. Let h(z, w) = (z 2 , w2 ). The multiplicity is 4. We follow the proof of
Theorem 5.1. We have (A1 , B1 ) = (zw, zw). We have (A2 , B2 ) = (z, z 2 ). We have
(A3 , B3 ) = (w, w2 ), and finally (A4 , B4 ) = (1, 1). Notice that the Aj give the basis
for the quotient algebra, whereas two of the Bj lie in the ideal (h).
To close this section we show that we cannot obtain 1 as a subelliptic multi-
plier when the initial set does not define an m-primary ideal. This result indicates
why the presence of complex analytic curves in the boundary precludes subelliptic
estimates on (0, 1) forms. In Theorem 6.2 we state a more precise result from [C1].
Proposition 5.1. Let hj ∈ m for each j, and suppose (h1 , . . . , hK ) is not m-
primary. Then the stabilized ideal from the algorithm is not the full ring Rn .
Proof. Since the (analytic or formal) variety defined by the hj is positive dimen-
sional, we can find a (convergent or formal) nonconstant n-tuple of power series in
88 D.W. Catlin and J.P. D’Angelo

one variable t, written z(t), such that hj (z(t)) = 0 in R1 for all j. Differentiating
yields
∂hj
(z(t))zk (t) = 0. (31)
∂zk
∂h
Hence the matrix ∂zkj has a nontrivial kernel, and so each of its n by n minor
determinants J vanishes after substitution of z(t). Since J(z(t)) = 0,
∂J
(z(t))zk (t) = 0. (32)
∂zk
Hence including the 1-form dJ does not change the collection of vectors annihilated
by a matrix of allowable rows. Continuing we see that z  (t) lies in the kernel of
all new matrices we form from allowable rows, and hence g(z(t)) vanishes for all
functions g in the stabilized ideal. Since z(t) is not constant, we conclude that
the variety of the stabilized ideal is positive dimensional, and hence the stabilized
ideal is not Rn . 

6. Necessary and sufficient conditions for subellipticity

In the previous sections we have seen a sufficient condition for subellipticity. A
subelliptic estimate holds if and only if the function 1 is a subelliptic multiplier;
there is an algorithmic procedure to construct subelliptic multipliers beginning
with the defining function and the determinant of the Levi form. Each step of the
process decreases the value of  known to work in (4). If, however, the process
terminates in finitely many steps, then (4) holds for some positive . Using an
important geometric result from [DF], Kohn [K4] established that the process
must terminate when the boundary is real-analytic, and that 1 is a subelliptic
multiplier if and only if there is no complex variety of positive dimension passing
through p and lying in the boundary.
In this section we recall from [C1], [C2], [C3] a different approach to these es-
timates. The sufficient condition for an estimate involves the existence of plurisub-
harmonic functions with certain properties. Such functions can be used as weight
functions in proving L2 estimates. See also [He]. A related approach to the esti-
mates appears in [S]; existence of good plurisubharmonic functions implies subel-
lipticity. This intuitive method even works on domains with Lipschitz boundaries.
We wish to relate the estimate (4) to the geometry of the boundary. Let r be
a smooth local defining function of a pseudoconvex domain Ω, and assume 0 ∈ bΩ.
We consider families {Mt} of holomorphic curves through p and how these curves
contact bΩ there. For t > 0 we consider nonsingular holomorphic curves gt as
follows:
1) gt : {|ζ| < t} → Cn and gt(0) = 0.
2) There is a positive constant c2 (independent of t) such that, on {|ζ| < 1}, we
have |gt (ζ)| ≤ c2 .
3) There is a positive constant c1 such that c1 ≤ |gt (0)|.
 Subelliptic Estimates 89

We say that the order of contact of the family {Mt } (of holomorphic curves
parametrized by gt ) with bΩ is η0 if η0 is the supremum of the set of real numbers
η for which
supζ |r(gt (ζ))| ≤ Ctη . (33)
The holomorphic curves gt considered in this definition are all nonsingular. There-
fore this approach differs somewhat from the approach in [D1] and [D2], where
allowing germs of curves with singularities at 0 is crucial. Our next example pro-
vides some insight.

Example 6.1. 1) Define r as follows:

r(z) = Re(z3 ) + |z12 − z2 z3 |2 + |z2 |4 . (34)

By [D1] we have T(bΩ, 0) = 4. Each curve ζ → g(ζ) whose third component
vanishes has contact 4 at the origin. On the other hand, consider a nearby boundary
point of the form (0, 0, ia) for a real. Then the curve
ζ2
ζ → (ζ, , ia) = ga (ζ) (35)
ia
has order of contact 8 at (0, 0, ia). By [D2] this jump is the maximum possible;
see (39) below for the sharp inequality in general.
2) Following [C1] we jazz up this example by considering

r(z) = Re(z3 ) + |z12 − z2 z3l |2 + |z2 |4 + |z1 z3m |2 (36)

for positive integers l, m with 2 ≤ l ≤ m. Again we have T(bΩ, 0) = 4. We will
construct a family of regular holomorphic curves gt with order of contact 4(2m+l)
m+2l .
For |ζ| < t, and α to be chosen, put
ζ2
gt (ζ) = (ζ, , itα ). (37)
(itα )l
Then, pulling back r to gt we obtain
|ζ|8
r(gt (ζ)) = + |ζ|2 |t|2αm . (38)
|t|4αl
Setting the two terms in (38) equal, we obtain |ζ|6 = |t|4αl+2αm . Put α = 3
m+2l
and then we get |ζ| = |t|. It follows that
supζ |r(gt (ζ))| = 2|t|η ,

where η = 4(2m+l)
m+2l
. Hence the order of contact of this family is at least η; in fact
it is precisely this value. Furthermore, by Theorems 6.1 and 6.2 below, there is a
subelliptic estimate at 0 for  = η1 and this value is the largest possible. Depending
on l and m, the possible values of the upper bound on  live in the interval [ 18 , 14 ].
90 D.W. Catlin and J.P. D’Angelo

We next recall the equivalence of subelliptic estimates on (0, 1) forms with

finite type.

Theorem 6.1. See [C2] and [C3]. Suppose that bΩ is smooth and pseudoconvex and
T(bΩ, p0 ) is finite. Then the subelliptic estimate (4) holds for some  > 0.

Theorem 6.2. See [C1]. Suppose that the subelliptic estimate (4) holds for some
positive . If {Mt } is a family of complex-analytic curves of diameter t, then the
order of contact of {Mt } with bΩ is at most 1 .

In two dimensions, type of a point is an upper semi-continuous function: if

the type at p is t, then the type is at most t nearby. In higher dimensions the
type of a nearby point can be larger (as well as the same or smaller). Sharp local
bounds for the type indicate why relating the supremum of possible values of  in
a subelliptic estimate to the type is difficult in dimension at least 3.

Theorem 6.3. See [D2]. Let bΩ in Cn be smooth and pseudoconvex near p0 , and
assume T(bΩ, p0 ) is finite. Then there is a neighborhood of p0 on which
T(bΩ, p0 )n−1
T(bΩ, p) ≤ . (39)
2n−2
The bound (39) is sharp. When n = 2 we see that the type at a nearby
point can be no larger than the type at p0 . When n ≥ 3, however, the type can
be larger nearby. This failure of upper semi-continuity of the type shows that the
best epsilon in a subelliptic estimate cannot simply be the reciprocal of the type,
as holds in two dimensions. See [D1], [D2], [D3] for more information. Example 7.1
below generalizes Example 6.1. It is an unpublished result due to the first author.
All these examples are based upon a simple example found by the second
author in [D3] to illustrate the failure of upper semi-continuity of order of contact.
See [D1] and [D2] for extensions to higher dimensions and a proof of (39).

7. Sharp subelliptic estimates

Example 7.1. Consider the local defining function r given by
r(z) = 2Re(z3 ) + |z1m1 − f (z3 )z2 |2 + |z2m2 |2 + |z2 g(z3 )|2 , (40)
where m1 and m2 are integers at least 2 and f and g are functions to be chosen.
Let bΩ be the zero set of r. Assume f (0) = g(0) = 0. It follows by [D1] that
T(bΩ, 0) = 2max(m1 , m2 ) and mult(bΩ, 0) = 2m1 m2 . We will show that we can
obtain, for the reciprocal of the largest possible value of  in a subelliptic estimate,
any value in between these two numbers.
By Theorem 6.1 there is a subelliptic estimate. According to Theorem 6.2, to
find an upper bound for  we must find a family {Mt } of one-dimensional complex
curves with certain properties. We follow Example 6.1 and define this family {Mt}
 Subelliptic Estimates 91

as follows: Mt is the image of the holomorphic curve

ζ m1
γt (ζ) = (ζ, , it) (41)
f (it)
on the set where |ζ| ≤ t.
Pulling back r to this family of curves yields
ζ m1 2m2 ζ m1 2
r(γt (ζ)) = | | +| | |g(it)|2 . (42)
f (it) f (it)
Reasoning as in Example 6.1, we choose f and g to make the two terms in (42)
equal. The condition for equality is
|ζ|2m1 m2 −2m1 = |f |2m2 −2 |g|2 . (43)
The crucial difference now is that the functions f and g, which depend on only one
variable, can be chosen as we wish. In particular, choose a parameter λ ∈ (0, 1],
and assume that f and g are chosen such that log(|f |) = λlog(|g|). Then (43) gives
(2m1 (m2 − 1))log(|ζ|) = (2(m2 − 1)λ + 2)log(|g|). (44)
We obtain from (44) and (45)


|r(γt (ζ))| (2 − 2λ)2m1 (m2 − 1)
log( ) = 2m1 + log(|ζ|). (45)
2 2(m2 − 1)λ + 2)
In order to find a value η for which there is a constant C such that |r| ≤ C|ζ|η ,
log(|r|)
we take logs and see that we find the ratio log(|ζ|) . Using (45) we obtain the order
of contact T of this family of curves to be
2(1 − λ)m1 (m2 − 1)
T = 2m1 + . (46)
(m2 − 1)λ + 1
If in (46) we put λ = 1 then we get T = 2m1 . If in (46) we let λ tend to 0, we
obtain T = 2m1 + 2m1 (m2 − 1) = 2m1 m2 .

In the previous example we may, for example, choose f (z) = z p and g(z) = z q .
If we put λ = pq , then our calculations apply, and (46) is rational. On the other
hand, we can achieve the condition log(|f |) = λlog(|g|) by allowing f and g to
be functions vanishing to infinite order at 0 but which are holomorphic in the
half-plane Re(z3 ) < 0. For example we may define f by f (ζ) = exp( √−p −ζ
) and g
the same except that p is replaced by q. By doing so we can allow λ in (46) to be
real. It is easy to include the limiting value λ = 0, by setting g = 0.
In order to finish we have to discuss sufficiency. The first author uses the
method of weighted L2 estimates. We let H(Φ) denote the complex Hessian of a
smooth real-valued function Φ. We say that H(Φ) ≥ C if the minimum eigenvalue
of the Hessian is at least C at each point. One of the crucial steps in the proof of
Theorem 6.1 is the following result from [C2], based upon ideas from [C4].
92 D.W. Catlin and J.P. D’Angelo

Theorem 7.1. Let Ω be a smoothly bounded domain, defined near a boundary point
p by {r = 0}. Suppose that there is neighborhood U of p such that the following
holds: For each δ > 0, we can find a smooth function Φδ satisfying
1) |Φδ | ≤ 1 on U . Thus Φδ is uniformly bounded.
2) Φδ is plurisubharmonic on U . Thus H(Φδ ) ≥ 0 on U .
3) H(Φδ ) ≥ cδ −2 on U ∩ {−δ < r ≤ 0}. Thus the Hessian of Φδ blows up in a
precise manner as we approach the boundary.
Then there is a subelliptic estimate of order  at p.
Using this result it is possible to say more about Example 7.1. One can choose
f and g there such that there is a subelliptic estimate of order  at the origin, where
 is the reciprocal of the number T in (46). In particular, for every 0 in the range
[ 2m11 m2 , 2m
1
1
] there is a domain in C3 such that the largest possible value of  in a
subelliptic estimate is 0 . By changing the function g appropriately, one can create
the situation of part 2) of the next result.
Theorem 7.2. Let 0 be in the interval (0, 14 ].
1) There is a smooth pseudoconvex domain in C3 , with defining function
(40), such that the subelliptic estimate (4) holds with  equal to 0 , but for no
larger value of . In addition, if 0 (in the same range) is rational, then we can
choose the domain to be defined by (40), where f (z) = z p and g(z) = z q , and hence
the defining equation is a polynomial.
2) There is also a smooth pseudoconvex domain in C3 , with defining equation
(40), such that the estimate (4) holds for all  with 0 ≤  < 0 , but for which the
estimate fails at 0 .
Theorem 7.2 can be extended to higher dimensions. It is much harder to un-
derstand subelliptic estimates on (0, 1) forms in three or more dimensions than it
is in two dimensions. The theory for (0, 1) forms in two dimensions is analogous to
the theory for (0, n − 1) forms in n dimensions. In these cases there is no need to
consider the contact with singular varieties, and hence issues involving subelliptic
estimates are controlled by commutators. We conclude by observing that connec-
tions between the analysis and the commutative algebra involved do not reveal
themselves in two dimensions, or more generally, when we consider estimates on
(0, n − 1) forms. Hence Theorem 3.1 tells only a small part of the full story.

References
[BS] Boas, Harold P. and Straube, Emil J, Global regularity of the ∂-Neumann problem:
a survey of the L2-Sobolev theory, pages 79–111 in Several Complex Variables,
(M. Schneider and Y.T. Siu, eds.), Math. Sci. Res. Inst. Publ. 37, Cambridge Univ.
Press, Cambridge, 1999.
[C1] Catlin, D., Necessary conditions for subellipticity of the ∂-Neumann problem,
Annals of Math. 117(1983), 147–171.
 Subelliptic Estimates 93

[C2] , Boundary invariants of pseudoconvex domains, Annals of Math. 120(1984),

529–586.
[C3] , Subelliptic estimates for the ∂-Neumann problem on pseudoconvex domains,
Annals of Math. 126(1987), 131–191.
[C4] , Global regularity for the ∂-Neumann problem, pages 39–49 in Complex
Analysis of Several Variables (Yum-Tong Siu ed.), Proc. Symp. Pure Math. 41,
Amer. Math. Soc., Providence, RI, 1984.
[CC] Catlin, D. and Cho, Jae-Seong, Estimates for the ∂-Neumann problem on regular
coordinate domains, preprint.
[CNS] Chang, D.C., Nagel, A., and Stein, E.M., Estimates for the ∂-Neumann problem
in pseudoconvex domains of finite type in C2 . Acta Math. 169(1992), 153–228.
[CS] Chen, So-Chin and Shaw, Mei-chi, Partial differential equations in complex anal-
ysis, Amer. Math. Soc./International Press, 2001.
[Cho] Cho, Jae-Seong, An algebraic version of subelliptic multipliers, Michigan Math.
J., Vol. 54 (2006), 411–426.
[D1] D’Angelo, J.P., Several Complex Variables and the Geometry of Real Hypersur-
faces, CRC Press, Boca Raton, 1992.
[D2] , Real hypersurfaces, orders of contact, and applications, Annals of Math.
115(1982), 615–637.
[D3] , Subelliptic estimates and failure of semi-continuity for orders of contact,
Duke Math. J., Vol. 47(1980), 955–957.
[D4] , Finite type conditions and subelliptic estimates, pp. 63–78 in Modern Meth-
ods in Complex Analysis, Annals of Math. Studies 137, Princeton Univ. Press,
Princeton, 1995.
[D5] Real and complex geometry meet the Cauchy-Riemann equations, Park City
Math. Institute Notes, 2008 (to appear).
[DK] D’Angelo, J. and Kohn, J.J., Subelliptic estimates and finite type, pages 199–232
in Several Complex Variables (M. Schneider and Y.T. Siu, eds.), Math. Sci. Res.
Inst. Publ. 37, Cambridge Univ. Press, 1999.
[DF1] Diederich K., and Fornaess, J.E., Pseudoconvex domains with real analytic bound-
ary, Annals of Math. (2) 107(1978), 371–384.
[FK] Folland, G.B., and Kohn, J.J., The Neumann problem for the Cauchy-Riemann
complex, Annals of Math. Studies, 75, Princeton University Press, 1972.
[Gr] Greiner, P., Subellipticity estimates of the ∂-Neumann problem, J. Differential
Geometry 9 (1974), 239–260.
[He] Herbig, Anne-Katrin, A sufficient condition for subellipticity of the ∂-Neumann
operator. J. Funct. Anal. 242 (2007), no. 2, 337–362.
[K1] Kohn, J.J. Harmonic integrals on strongly pseudo-convex manifolds, I, Annals of
Mathematics, 78, 1963, 112–148.
[K2] Harmonic integrals on strongly pseudo-convex manifolds, II, Annals of Math-
ematics, 79, 1964, 450–472.
[K3] Boundary behavior of ∂¯ on weakly pseudo-convex manifolds of dimension
two, Journal of Differential Geometry, 6, 1972, 523–542.
94 D.W. Catlin and J.P. D’Angelo

[K4] ¯
Subellipticity of the ∂-Neumann problem on pseudo-convex domains: suffi-
cient conditions, Acta Mathematica, 142, March 1979.
[K5] ¯
A Survey of the ∂-Neumann Problem, Proc. of Symposia in Pure Math.,
Amer. Math. Soc. 41, 1984, 137–145.
[KN] Kohn, J.J. and Nirenberg, L. Non-coercive boundary value problems, Comm. Pure
Appl. Math., 18, 1965, 443–492.
[KZ] Khanh, Tran Vu and Zampieri, G., Precise subelliptic estimates for a class of
special domains, preprint, arXiv:0812.2560v2
[RS] Rothschild, Linda Preiss and Stein, E.M., Hypoelliptic differential operators and
nilpotent groups, Acta Math. 137 (1976), 247–320.
[S] Straube, Emil, Plurisubharmonic functions and subellipticity of the ∂-Neumann
problem on non-smooth domains, Math. Res. Lett. 4 (1997), 459–467.

David W. Catlin
Dept. of Mathematics
Purdue Univ.
W. Lafayette, IN 47906, USA
e-mail: [email protected]
John P. D’Angelo
Dept. of Mathematics
Univ. of Illinois
1409 W. Green St.
Urbana, IL 61801, USA
e-mail: [email protected]
Complex Analysis
Trends in Mathematics, 95–107

c 2010 Springer Basel AG

Invariant CR Mappings
John P. D’Angelo

Dedicated to Linda Rothschild

Abstract. This paper concerns CR mappings between hyperquadrics. We em-

phasize the case when the domain is a sphere and the maps are invariant
under a finite subgroup of the unitary group. We survey known results, pro-
vide some new examples, and use them to prove the following result. For each
d there is a CR polynomial mapping of degree 2d + 1 between hyperquadrics
that preserves the number of negative eigenvalues. Thus rigidity fails for CR
mappings between hyperquadrics.

Mathematics Subject Classification (2000). 32V99, 32H02, 32H35, 14P05,

11E25, 11E39.
Keywords. CR mappings, unitary groups, hyperquadrics, group-invariant
mappings, Hermitian forms, Hermitian symmetric polynomials.

1. Introduction
The subject of CR Geometry interacts with nearly all of mathematics. See [BER]
for an extensive discussion of many aspects of CR manifolds and mappings between
them. One aspect of the subject not covered in [BER] concerns CR mappings
invariant under groups. The purpose of this paper is to discuss interactions with
number theory and combinatorics that arise from the seemingly simple setting
of group-invariant CR mappings from the unit sphere to a hyperquadric. Some
elementary representation theory also arises.
The unit sphere S 2n−1 in complex Euclidean space Cn is the basic example of
a CR manifold of hypersurface type. More generally we consider the hyperquadric
Q(a, b) defined to be the subset of Ca+b defined by

a
a+b
|zj |2 − |zj |2 = 1. (1)
j=1 j=a+1
96 J.P. D’Angelo

Of course S 2n−1 is invariant under the unitary group U (n). Let Γ be a finite
subgroup of U (n). Assume that f : Cn → CN is a rational mapping invariant
under Γ and that f (S 2n−1 ) ⊂ S 2N −1 . For most Γ such an f must be a constant.
In other words, for most Γ, there is no non-constant Γ-invariant rational mapping
from sphere to sphere for any target dimension. In fact, for such a non-constant
invariant map to exist, Γ must be cyclic and represented in a rather restricted
fashion. See [Li1], [DL], and especially Corollary 7 on page 187 of [D1], for precise
statements and the considerable details required.
The restriction to rational mappings is natural; for n ≥ 2 Forstneric [F1]
proved that a proper mapping between balls, with sufficiently many continuous
derivatives at the boundary, must be a rational mapping. On the other hand, if
one makes no regularity assumption at all on the map, then (see [Li2]) one can
create group-invariant proper mappings between balls for any fixed-point free finite
unitary group. The restrictions on the group arise from CR Geometry and the
smoothness of the CR mappings considered. In this paper we naturally restrict
our considerations to the class of rational mappings. See [F2] for considerable
discussion about proper holomorphic mappings and CR Geometry.
In order to find group-invariant CR mappings from a sphere, we relax the
assumption that the target manifold be a sphere, and instead allow it to be a hy-
perquadric. We can then always find polynomial examples, as we note in Corollary
1.1. In this paper we give many examples of invariant mappings from spheres to
hyperquadrics. Our techniques allow us to give some explicit surprising examples.
In Theorem 6.1 for example, we show that rigidity fails for mappings between hy-
perquadrics; we find non-linear polynomial mappings between hyperquadrics with
the same number of negative eigenvalues in the defining equations of the domain
and target hyperquadrics. As in the well-known case of maps between spheres, we
must allow sufficiently many positive eigenvalues in the target for such maps to
exist.
To get started we recall that a polynomial R : Cn × Cn → C is called
Hermitian symmetric if R(z, w) = R(w, z) for all z and w. If R is Hermitian
symmetric, then R(z, z) is evidently real valued. By polarization, the converse
also holds. We note also that a polynomial in z = (z1 , . . . , zn ) and z is Hermitian
symmetric if and only if its matrix of coefficients is Hermitian symmetric in the
sense of linear algebra.
The following result from [D1] shows how to construct group-invariant map-
pings from spheres to hyperquadrics. Throughout the paper we will give explicit
formulas in many cases.
Theorem 1.1. Let Γ be a finite subgroup of U (n) of order p. Then there is a unique
Hermitian symmetric Γ-invariant polynomial ΦΓ (z, w) such that the following hold:
1) ΦΓ (0, 0) = 0.
2) The degree of ΦΓ in z is p.
3) ΦΓ (z, z) = 1 when z is on the unit sphere.
4) ΦΓ (γz, w) = ΦΓ (z, w) for all γ ∈ Γ.
 Invariant CR Mappings 97

Corollary 1.1. There are holomorphic vector-valued Γ-invariant polynomial map-

pings F and G such that we can write
ΦΓ (z, z) = ||F (z)||2 − ||G(z)||2 . (2)
The polynomial mapping z → (F (z), G(z)) restricts to a Γ-invariant mapping
from S 2n−1 to the hyperquadric Q(N+, N− ), where these integers are the numbers
of positive and negative eigenvalues of the matrix of coefficients of ΦΓ .
The results in this paper revolve around how the mapping (F, G) from Corol-
lary 1.1 depends on Γ. We clarify one point at the start; even if we restrict our
considerations to cyclic groups, then this mapping changes (surprisingly much) as
the representation of the group changes. The interesting things from the points
of view of CR Geometry, Number Theory, and Combinatorics all depend in non-
trivial ways on the particular representation. Therefore the results should be con-
sidered as statements about the particular subgroup Γ ⊂ U (n), rather than as
statements about the abstract group G for which π(G) = Γ.
The proof of Theorem 1 leads to the following formula for ΦΓ .
)
ΦΓ (z, w) = 1 − (1 − γz, w
). (3)
γ∈Γ

The first three properties from Theorem 1 are evident from (3), and the fourth
property is not hard to check. One also needs to verify uniqueness.
The starting point for this paper will therefore be formula (3). We will first
consider three different representations of cyclic groups and we note the consid-
erable differences in the corresponding invariant polynomials. We also consider
metacyclic groups. We also discuss some interesting asymptotic considerations, as
the order of the group tends to infinity. Additional asymptotic results are expected
in appear in the doctoral thesis [G] of D. Grundmeier.
An interesting result in this paper is the application in Section 6. In Theorem
6.1 we construct, for each odd 2p+1 with p ≥ 1, a polynomial mapping gp of degree
2p such that
gp : Q(2, 2p + 1) → Q(N (p), 2p + 1). (4)
These mappings illustrate a failure of rigidity; in many contexts restrictions on
the eigenvalues of the domain and defining hyperquadrics force maps to be linear.
See [BH]. Our new examples show that rigidity does not hold when we keep the
number of negative eigenvalues the same, as long as we allow a sufficient increase
in the number of positive eigenvalues. On the other hand, by a result in [BH], the
additional restriction that the mapping preserves sides of the hyperquadric does
then guarantee rigidity. It is quite striking that the construction of the polynomials
in Theorem 6.1 relies on the group-theoretic methods in the rest of the paper.
The author acknowledges support from NSF grant DMS-07-53978. He thanks
both Dusty Grundmeier and Jiri Lebl for many discussions on these matters. He
also acknowledges the referee who spotted an error in the original presentation of
Example 3.3.
98 J.P. D’Angelo

2. Properties of the invariant polynomials

Let Γ be a finite subgroup of the unitary group U (n), and let ΦΓ be the unique
polynomial defined by (3). Our primary interest concerns how this polynomial
depends on the particular representation of the group.
We remark at the outset that we will be considering reducible representations.
A simple example clarifies why. If G is cyclic of order p, then G has the irreducible
unitary (one-dimensional) representation Γ as the group of pth roots of unity. An
elementary calculation shows that the invariant polynomial ΦΓ becomes simply
ΦΓ (z, w) = (zw)p . (5)
On the other hand, there are many ways to represent G as a subgroup of U (n)
for n ≥ 2. We will consider these below; for now we mention one beautiful special
case.
Let p and q be positive integers with 1 ≤ q ≤ p − 1 and let ω be a primitive
pth root of unity. Let Γ(p, q) be the cyclic group generated by the diagonal 2-by-2
matrix A with eigenvalues ω and ω q :


ω 0
A= . (6)
0 ωq

Because A is diagonal, the invariant polynomial ΦΓ(p,q) (z, z) depends on only |z1 |2
and |z2 |2 . If we write x = |z1 |2 and y = |z2 |2 , then we obtain a corresponding
polynomial fp,q in x and y. This polynomial has integer coefficients; a combina-
torial interpretation of these coefficients appears in [LWW]. The crucial idea in
[LWW] is the interpretation of ΦΓ as a circulant determinant; hence permutations
arise and careful study of their cycle structure leads to the combinatorial result.
Asymptotic information about these integers as p tends to infinity appears in both
[LWW] and [D4]; the technique in [D4] gives an analogue of the Szegö limit theo-
rem. In the special case where q = 2, these polynomials provide examples of sharp
degree estimates for proper monomial mappings between balls. The polynomials
fp,2 have many additional beautiful properties. We pause to write down the for-
mula and state an appealing corollary. These polynomials will arise in the proof
of Theorem 6.1.

 + p  + p
p+1 p x+ x2 + 4y x − x2 + 4y
fp,2 (x, y) = (−1) y + + . (7)
2 2

Corollary 2.1 (D4). Let Sp be the sum of the coefficients of fp,2 . Then the limit as
1 √
p tends to infinity of Spp equals the golden ratio 1+ 5
2 .

Proof. The sum of the coefficients is fp,2 (1, 1), so put x = y = 1 in (7). The largest
(in absolute value) of the three terms is the middle term. Taking pth roots and
letting p tend to infinity gives the result. 
 Invariant CR Mappings 99

See [DKR] for degree estimates and [DLe] for number-theoretic information
concerning uniqueness results for degree estimates. The following elegant primality
test was proved in [D2].
Theorem 2.1. For each q, the congruence fp,q (x, y) ∼
= xp + y p mod (p) holds if and
only if p is prime.
We make a few comments. When q = 1, the polynomial fp,1 is simply (x+y)p
and the result is well known. For other values of q the polynomials are more com-
plicated. When q = 2 or when q = p − 1 there are explicit formulas for the integer
coefficients. For small q recurrences exist but the order of the recurrences grows
exponentially with q. See [D2], [D3], [D4] and [G]. There is no known general for-
mula for the integer coefficients. Nonetheless the basic theory enables us to reduce
the congruence question to the special case. Note also that the quotient space
L(p, q) = S 3 /Γ is a Lens space. It might be interesting to relate the polynomials
fp,q to the differential topology of these spaces.
We return to the general situation and repeat the crucial point; the invariant
polynomials depend on the representation in non-trivial and interesting ways, even
in the cyclic case. In order to express them we recall ideas that go back to E.
Noether. See [S] for considerable discussion. Given a subgroup Γ of the general
linear group, Noether proved that the algebra of polynomials invariant under Γ is
generated by polynomials of degree at most the order |Γ| of Γ. Given a polynomial
p we can create an invariant polynomial by averaging p over the group:
1
p ◦ γ. (8)
|Γ|
γ∈Γ

We find a basis for the algebra of invariant polynomials as follows. We average

each monomial z α of total degree at most |Γ| as in (8) to obtain an invariant
polynomial; often the result will be the zero polynomial. The nonzero polynomials
that result generate the algebra of polynomials invariant under Γ. In particular, the
number of polynomials required is bounded above by the dimension of the space
of homogeneous polynomials of degree |Γ| in n variables. Finally we can express
the F and G from (2) in terms of sums and products of these basis elements. The
invariant polynomials here are closely related to the Chern orbit polynomials from
[S]. The possibility of polarization makes our approach a bit different. It seems a
worthwhile project to deepen this connection. Some results in this direction will
appear in [G].

3. Cyclic groups
Let Γ be cyclic of order p. Then the elements of Γ are I, A, A2 , . . . , Ap−1 for some
unitary matrix A. Formula (3) becomes
)
p−1
ΦΓ (z, w) = 1 − (1 − Aj z, w
). (9)
j=0
100 J.P. D’Angelo

We begin by considering three different representations of a cyclic group of order

six; we give precise formulas for the corresponding invariant polynomials.
Let ω be a primitive sixth-root of unity, and let η be a primitive third-root of
unity. We consider three different unitary matrices; each generates a cyclic group
of order six.
Example 3.1. Let Γ be the cyclic group of order 6 generated by A, where


ω 0
A= . (10.1)
0 ω
The invariant polynomial satisfies the following formula:
ΦΓ (z, z) = (|z1 |2 + |z2 |2 )6 . (10.2)
It follows that Φ is the squared norm of the following holomorphic polynomial:
√ √ √ √ √
f (z) = (z16 , 6z15 z2 , 15z14 z22 , 20z13 z23 , 15z12 z24 , 6z1 z25 , z26 ). (10.3)
The polynomial f restricts to the sphere to define an invariant CR mapping
from S 3 to S 13 ⊂ C7 .
Example 3.2. Let Γ be the cyclic group of order 6 generated by A, where


ω 0
A= . (11.1)
0 ω
The invariant polynomial satisfies the following formula:
ΦΓ (z, z) = |z1 |12 + |z2 |12 + 6|z1 |2 |z2 |2 + 2|z1 |6 |z2 |6 − 9|z1 |4 |z2 |4 . (11.2)
Note that Φ is not a squared norm. Nonetheless we define f as follows:
√ √
f (z) = (z16 , z26 , 6z1 z2 , 2z13 z23 , 3z12 z22 ). (11.3)
Then
Φ = |f1 |2 + |f2 |2 + |f3 |2 + |f4 |2 − |f5 |2 ,
and the polynomial f restricts to the sphere to define an invariant CR mapping
from S 3 to Q(4, 1) ⊂ C5 .
Example 3.3. Let Γ be the cyclic group of order 6 generated by A, where


0 1
A= . (12.1)
η 0
The polynomial Φ can be expressed as follows:
Φ(z, z) = (x + y)3 + (ηs + ηt)3 − (x + y)3 (ηs + ηt)3 (12.2)
where
x = |z1 |2
y = |z2 |2
s = z2 z 1
t = z1 z 2 .
 Invariant CR Mappings 101

After diagonalization this information determines a (holomorphic) polyno-

mial mapping (F, G) such that
ΦΓ = ||F ||2 − ||G||2 .
It is somewhat complicated to determine the components of F and G. It is nat-
ural to use Noether’s approach. For this particular representation, considerable
computation then yields the following invariant polynomials:
z13 + z23 = p
z12 z2 + ηz1 z22 = q
z16 + z26 = f
1
z13 z23 = (p2 − f )
2
z1 z25 + η2 z15 z2 = k
1 4 2
(z z + η 2 z12 z24 ) = h
2 1 2
In order to write ΦΓ nicely, we let
g = c(z1 z25 + 3ηz13 z23 + η 2 z15 z2 ).
Then one can write ΦΓ , for some C > 0 as follows:
1
φΓ = |p|2 + |q|2 + (|f − z13 z23 |2 − |f + z13 z23 |2 ) + C(|g − h|2 − |g + h|2 ). (12.3)
2
We conclude that the invariant polynomial determines an invariant CR mapping
(F, G) from the unit sphere S 3 to the hyperquadric Q(4, 2) ⊂ C6 . We have


1 √
F = p, q, √ (f − z13 z23 ), C(g − h) (12.4)
2


1 √
G = √ (f + z1 z2 ), C(g + h) .
3 3
(12.5)
2
Consider these three examples together. In each case we have a cyclic group
of order six, represented as a subgroup of U (2). In each case we found an invariant
CR mapping. The image hyperquadrics were Q(7, 0), Q(4, 1), and Q(4, 2). The
corresponding invariant mappings had little in common. In the first case, the map
was homogeneous; in the second case the map was not homogeneous, although
it was a monomial mapping. In the third case we obtained a rather complicated
non-monomial map. It should be evident from these examples that the mappings
depend in non-trivial ways on the representation.

4. Asymptotic information
In this section we consider three families of cyclic groups, Γ(p, 1), Γ(p, 2), and
Γ(p, p − 1). For these groups it is possible to compute the invariant polynomials
ΦΓ exactly. In each case, because the group is generated by a diagonal matrix, the
102 J.P. D’Angelo

invariant polynomial depends on only x = |z1 |2 and y = |z2 |2 . We will therefore

often write the polynomials as functions of x and y.
For p = 1 we have
ΦΓ (z, z) = (|z1 |2 + |z2 |2 )p = (x + y)p . (13)
It follows that there is an invariant CR mapping to a sphere, namely the hyper-
quadric Q(p + 1, 0). We pause to prove (13) by establishing the corresponding
general result in any domain dimension.
Theorem 4.1. Let Γ be the cyclic group generated by ωI, where I is the identity
operator on Cn and ω is a primitive pth root of unity. Then ΦΓ (z, z) = ||z||2p =
1
||z ⊗p ||2 . Thus ΦΓ(p,1)
p
= ||z||2 and hence it is independent of p.
Proof. A basis for the invariant polynomials is given by the homogeneous mono-
mials of degree p. By Theorem 1.1 ΦΓ is of degree p in z and hence of degree 2p
overall. It must then be homogeneous of total degree 2p and it must take the value
1 on the unit sphere; it therefore equals ||z||2p . 
We return to the case where n = 2 where ||z||2 = |z1 |2 + |z2 |2 = x + y. In
1
p
the more complicated situation arising from Γ(p, q), the expression ΦΓ(p,q) is not
constant, but its behavior as p tends to infinity is completely analyzed in [D4].
As an illustration we perform this calculation when q = 2. By expanding (3)
the following formula holds (see [D4] for details and precise formulas for the nj ):

fp,2 (x, y) = ΦΓ(p,2) (z, z) = xp + (−1)p+1 y p + nj xp−2j y j . (14.1)
j

Here the nj are positive integers and the summation index j satisfies 2j ≤ p. The
target hyperquadric now depends on whether p is even or odd. When p = 2r − 1 is
odd, the target hyperquadric is the sphere, namely the hyperquadric Q(r + 1, 0).
When p = 2r is even, the+ target hyperquadric is Q(r + 1, 1). In any case, using (7)
under the condition x + x2 + 4y > 2y, we obtain
+
1
p
x + x2 + 4y 1
(fp,2 (x, y)) = (1 + hp (x, y)) p , (14.2)
2
where hp (x, y) tends to zero as p tends to infinity. Note that we recover Corollary
2.1 by setting x = y = 1. We summarize this example in the following result.
Similar results hold for the fp,q for q ≥ 3. See [D4].
+
Proposition 4.1. For x + x2 + 4y > 2y, the √ limit, as p tends to infinity, of the
x+ x2 +4y
left-hand side of (14.2) exists and equals 2 .
It is also possible to compute ΦΓ(p,p−1) exactly. After some computation we
obtain the following:

ΦΓ (z, z) = |z1 |2p + |z2 |2p + nj (|z1 |2 |z2 |2 )j = xp + y p + nj (xy)j , (15)
j
 Invariant CR Mappings 103

where the nj are integers. They are 0 when 2j > p, and otherwise non-zero. In
this range nj > 0 when j is odd, and nj < 0 when j is even. Explicit formulas for
the nj exist; in fact they are closely related to the coefficients for fp,2 . See [D3].
To see what is going on, we must consider the four possibilities for p modulo (4).
We illustrate by listing the polynomials of degrees 4, 5, 6, 7. As above we put
|z1 |2 = x and |z2 |2 = y. We obtain:
f4,3 (x, y) = x4 + y 4 + 4xy − 2x2 y 2 (16.4)
f5,4 (x, y) = x + y + 5xy − 5x y
5 5 2 2
(16.5)
f6,5 (x, y) = x6 + y 6 + 6xy − 9x2 y 2 + 2x3 y 3 (16.6)
f7,6 (x, y) = x + y + 7xy − 14x y + 7x y .
7 7 2 2 3 3
(16.7)
For Γ(p, p − 1) one can show the following. When p = 4k or p = 4k + 1, we
have k + 2 positive coefficients and k negative coefficients. When p = 4k + 2 or
p = 4k + 3, we have k + 3 positive coefficients and k negative coefficients. For q > 2
in general one obtains some negative coefficients when expanding fp,q , and hence
the target must be a (non-spherical) hyperquadric. The paper [LWW] provides a
method for determining the sign of the coefficients.
Given a finite subgroup Γ of U (n), the invariant polynomial ΦΓ is Hermitian
symmetric, and hence its underlying matrix of coefficients is Hermitian. We let
N+ (Γ) denote the number of positive eigenvalues of this matrix, and we let N− (Γ)
denote the number of negative eigenvalues. When Γ is cyclic of order p we some-
times write N+ (p) instead of N+ (Γ), but the reader should be warned that the
N+ (p)
numbers N+ and N− depend upon Γ and not just p. The ratio Rp = N+ (p)+N − (p)
is of some interest, but it can be hard to compute. We therefore consider its as-
ymptotic behavior.
For the class of groups considered above Theorem 4.1 holds. It is a special
case of a result to appear in the doctoral thesis [G] of Grundmeier, who has found
the limit of Rp for many classes of groups (not necessarily cyclic) whose order
depends on p. Many different limiting values can occur. Here we state only the
following simple version which applies to the three classes under consideration.
Proposition 4.2. For the three classes of cyclic groups whose invariant polynomials
are given by (13), (14), and (15), the limit of Rp as p tends to infinity exists. In
the first two cases the limit is 1. When ΦΓ satisfies (15), the limit is 12 .
Remark 4.1. For the class of groups Γ(p, q) the limit Lq of Rp exists and depends
on q. If one then lets q tend to infinity, the resulting limit equals 34 . Thus the
asymptotic result differs from the limit obtained by setting q = p − 1 at the start.
The subtlety of the situation is evident.
104 J.P. D’Angelo

5. Metacyclic groups
Let Cp denote a cyclic group of order p. A group G is called metacyclic if there is
an exact sequence of the form
1 → Cp → G → Cq → 1.
Such groups are also described in terms of two generators A and B such that
Ap = I, B q = I, and AB = B m A for some m. In this section we will consider
metacyclic subgroups of U (2) defined as follows. Let ω be a primitive pth root of
unity, and let A be the following element of U (2):


ω 0
. (17)
0 ω
For these metacyclic groups we obtain in (23) a formula for the invariant
polynomials in terms of known invariant polynomials for cyclic groups. We write
C(p, p − 1) for the cyclic subgroup of U (2) generated by A. Its invariant polyno-
mial is
)p
ΦC(p,p−1) = 1 − (1 − Ak z, z
). (18)
k=0
Now return to the metacyclic group Γ. Each group element of Γ will be of
the form B j Ak for appropriate exponents j, k. Since B is unitary, B ∗ = B −1 . We
may therefore write
B j Ak z, w
= Ak z, B −j w
. (19)
We use (19) in the product defining ΦΓ to obtain the following formula:
))
p−1 q−1
))
p−1 q−1
ΦΓ (z, z) = 1 − (1 − B j Ak z, z
) = 1 − (1 − Ak z, B −j z
). (20)
k=0 j=0 k=0 j=0

Notice that the term

)
p−1
(1 − Ak z, B −j z
) (21)
k=0
can be expressed in terms of the invariant polynomial for the cyclic group C(p, p −
1). We have
)
p−1
(1 − Ak z, B −j z
) = 1 − ΦC(p,p−1) (z, B −j z), (22)
k=0
and hence we obtain
)
q−1

ΦΓ (z, z) = 1 − 1 − ΦC(p,p−1) (z, B −j z) . (23)
j=0

The invariance of ΦΓ follows from the definition, but this property is not immedi-
ately evident from this polarized formula. The other properties from Theorem 1.1
are evident in this version of the formula. We have ΦΓ (0, 0) = 0. Also, ΦΓ (z, z) = 1
 Invariant CR Mappings 105

on the unit sphere, because of the term when j = 0. The degree in z is pq because
we have a product of q terms each of degree p.
The simplest examples of metacyclic groups are the dihedral groups. The
dihedral group Dp is the group of symmetries of a regular polygon of p sides. The
group Dp has order 2p; it is generated by two elements A and B, which satisfy the
relations Ap = I, B 2 = I, and AB = BAp−1 . Thus A corresponds to a rotation
and B corresponds to a reflection. We may represent Dp as a subgroup of U (2) by
putting

ω 0
A= (24.1)
0 ω −1


0 1
B= . (24.2)
1 0
Formula (23) for the invariant polynomial simplifies because the product in
(23) has only two terms. We obtain the following result, proved earlier in [D2].
Theorem 5.1. The invariant polynomial for the above representation of Dp satisfies
the following formula:
Φ(z, z) = fp,p−1 (|z1 |2 , |z2 |2 )+
+ fp,p−1 (z2 z 1 , z1 z 2 ) − fp,p−1 (|z1 |2 , |z2 |2 )fp,p−1 (z2 z 1 , z1 z 2 ). (25)

6. An application; failure of rigidity

In this section we use the group invariant approach to construct the first examples
of polynomial mappings of degree 2p from Q(2, 2p + 1) to Q(N (p), 2p + 1). The key
point of these examples is that the number of negative eigenvalues is preserved. The
mappings illustrate the failure of rigidity in the case where we keep the number of
negative eigenvalues the same but we are allowed to increase the number of positive
eigenvalues sufficiently. The mappings arise from part of a general theory being
developed [DLe2] by the author and J. Lebl. As mentioned in the introduction,
the additional assumption that the mapping preserves sides of the hyperquadric
does force linearity in this context. [BH]
Theorem 6.1. Let 2p + 1 be an odd number with p ≥ 1. There is an integer N (p)
and a holomorphic polynomial mapping gp of degree 2p such that
gp : Q(2, 2p + 1) → Q(N (p), 2p + 1).
and gp maps to no hyperquadric with smaller numbers of positive or negative eigen-
values.
Proof. We begin with the group Γ(2p, 2). We expand the formula given in (7) with
p replaced by 2p. The result is a polynomial f2p,2 in the two variables x, y with the
following properties. First, the coefficients are positive except for the coefficient
of y 2p which is −1. Second, we have f2p,2(x, y) = 1 on x + y = 1. Third, because
of the group invariance, only even powers of x arise. We therefore can replace x
106 J.P. D’Angelo

by −x and obtain a polynomial f (x, y) such that f (x, y) = 1 on −x + y = 1 and

again, all coefficients are positive except for the coefficient of y 2p . Next replace y
by Y1 + Y2 . We obtain a polynomial in x, Y1 , Y2 which has precisely 2p + 1 terms
with negative coefficients. These terms arise from expanding −(Y1 + Y2 )2p . All
other terms have positive coefficients. This polynomial takes the value 1 on the
set −x + Y1 + Y2 = 1. Now replace x by X1 + · · · + X2p+1 .
We now have a polynomial W (X, Y ) that is 1 on the set given by

2p+1

2
− Xj + Yj = 1.
1 1

It has precisely 2p + 1 terms with negative coefficients. There are many terms
with positive coefficients; suppose that the number is N (p). In order to get back
to the holomorphic setting, we put Xj = |zj |2 for 1 ≤ j ≤ 2p + 1 and we put
Y1 = |z2p+2 |2 and Y2 = |z2p+3 |2 . We note that this idea (an example of the
moment map) has been often used in this paper, as well as in the author’s work
on proper mappings between balls; see for example [D1] and [DKR]. Let gp (z) be
the mapping, determined up to a diagonal unitary matrix, with

N (p)

2p+1
|gj (z)|2 − |gj (z)|2 = W (X, Y ). (26)
j=1 j=1

Each component of gp is determined by (26) up to a complex number of modulus

1. The degree of gp is the same as the degree of w. We obtain all the claimed
properties. 
√
Example 6.1. We write out everything explicitly when p = 1. Let c = 2. The
proof of Theorem 6.1 yields the polynomial mapping g : Q(2, 3) → Q(8, 3) of
degree 2 defined by

g(z) = (z12 , z22 , z32 , cz1 z2 , cz1 z3 , cz2 z3 , cz4 , cz5 ; z42 , cz4 z5 , z52 ). (27)

Notice that we used a semi-colon after the first eight terms to highlight that g
maps to Q(8, 3). Summing the squared moduli of the first eight terms yields

(|z1 |2 + |z2 |2 + |z3 |2 )2 + 2(|z4 |2 + |z5 |2 ). (28)

Summing the squared moduli of the last three terms yields

(|z4 |2 + |z5 |2 )2 . (29)

The set Q(2, 3) is given by

|z4 |2 + |z5 |2 − 1 = |z1 |2 + |z2 |2 + |z3 |2 .

On this set we obtain 1 when we subtract (29) from (28).

 Invariant CR Mappings 107

References
[BER] Baouendi, M. Salah, Ebenfelt, Peter, and Rothschild, Linda Preiss, Real subman-
ifolds in complex space and their mappings, Princeton Mathematical Series 47,
Princeton University Press, Princeton, NJ, 1999.
[BH] Baouendi, M.S. and Huang, X., Super-rigidity for holomorphic mappings between
hyperquadrics with positive signature, J. Differential Geom. 69 (2005), no. 2,
379–398.
[D1] D’Angelo, J.P., Several Complex Variables and the Geometry of Real Hypersur-
faces, CRC Press, Boca Raton, 1992.
[D2] D’Angelo, J.P., Number-theoretic properties of certain CR mappings, J. of Geo-
metric Analysis, Vol. 14, Number 2 (2004), 215–229.
[D3] D’Angelo, J.P., Invariant holomorphic maps, J. of Geometric Analysis, Vol. 6,
1996, 163–179.
[D4] D’Angelo, J.P., Asymptotics for invariant polynomials, Complex Variables and
Elliptic Equations, Vol. 52, No. 4 (2007), 261–272.
[DKR] D’Angelo, J.P., Kos, Šimon, and Riehl E., A sharp bound for the degree of proper
monomial mappings between balls, J. of Geometric Analysis, Vol. 13, Number 4
(2003), 581–593.
[DL] D’Angelo, J.P. and Lichtblau, D.A., Spherical space forms, CR maps, and proper
maps between balls, J. of Geometric Analysis, Vol. 2, No. 5(1992), 391–416.
[DLe] D’Angelo, J.P. and Lebl, J., Complexity results for CR mappings between
spheres, International J. Math, Vol. 20, No. 2 (2009), 1–18.
[DLe2] D’Angelo, J.P. and Lebl, J. (work in progress).
[F1] Forstneric, F., Extending proper holomorphic mappings of positive codimension,
Invent. Math. 95 (1989), no. 1, 31–61.
[F2] Forstneric, F., Proper holomorphic mappings: a survey. Several complex vari-
ables (Stockholm, 1987/1988), 297–363, Math. Notes, 38, Princeton Univ. Press,
Princeton, NJ, 1993.
[G] Grundmeier, D., Doctoral thesis, University of Illinois, expected 2010.
[Li1] Lichtblau D.A., Invariant proper holomorphic maps between balls. Indiana Univ.
Math. J. 41 (1992), no. 1, 213–231.
[Li2] Lichtblau, D.A., Invariant proper holomorphic maps between balls, PhD thesis,
University of Illinois, 1991.
[LWW] Loehr, N.A., Warrington, G.S., and Wilf, H.S., The combinatorics of a three-line
circulant determinant, Israel J. Math. 143 (2004), 141–156.
[S] Smith, Larry, Polynomial Invariants of Finite Groups, Research Notes in Math-
ematics, 6. A.K. Peters, Ltd., Wellesley, MA, 1995.

John P. D’Angelo
Dept. of Mathematics
Univ. of Illinois
1409 W. Green St.
Urbana, IL 61801, USA
e-mail: [email protected]
Complex Analysis
Trends in Mathematics, 109–123

c 2010 Springer Basel AG

On the Subellipticity of Some

Hypoelliptic Quasihomogeneous Systems
of Complex Vector Fields
M. Derridj and B. Helffer
In honor of Linda P. Rothschild

Abstract. For about twenty five years it was a kind of folk theorem that
complex vector-fields defined on Ω × Rt (with Ω open set in Rn ) by
∂ ∂ϕ ∂
Lj = +i (t) , j = 1, . . . , n , t ∈ Ω, x ∈ R ,
∂tj ∂tj ∂x
with ϕ analytic, were subelliptic as soon as they were hypoelliptic. This was
indeed the case when n = 1 [Tr1] but in the case n > 1, an inaccurate
reading of the proof (based on a non standard subelliptic estimate) given by
Maire [Mai1] (see also Trèves [Tr2]) of the hypoellipticity of such systems,
under the condition that ϕ does not admit any local maximum or minimum,
was supporting the belief for this folk theorem. This question reappears in
the book of [HeNi] in connection with the semi-classical analysis of Witten
Laplacians. Quite recently, J.L. Journé and J.M. Trépreau [JoTre] show by
explicit examples that there are very simple systems (with polynomial ϕ’s)
which were hypoelliptic but not subelliptic in the standard L2 -sense. But
these operators are not quasihomogeneous.
In [De] and [DeHe] the homogeneous and the quasihomogeneous cases
were analyzed in dimension 2. Large classes of systems for which subellipticiity
can be proved were exhibited. We will show in this paper how a new idea for
the construction of escaping rays permits to show that in the analytic case
all the quasihomogeneous hypoelliptic systems in the class above considered
by Maire are effectively subelliptic in the 2-dimensional case. The analysis
presented here is a continuation of two previous works by the first author for
the homogeneous case [De] and the two authors for the quasihomogeneous
case [DeHe].

Mathematics Subject Classification (2000). Primary 35B65, Secondary 35N10.

Keywords. Subellipticity, complex vector fields, hypoellipticity.
110 M. Derridj and B. Helffer

1. Introduction and main result

1.1. Preliminaries on subellipticity and hypoellipticity
Let Ω an open set in Rn with 0 ∈ Ω and ϕ ∈ C 1 (Ω, R), with ϕ(0) = 0. We consider
the regularity properties of the following family of complex vector fields on Ω × R
∂ ∂ϕ ∂
(Lϕ ) Lj = +i (t) , j = 1, . . . , n , t ∈ Ω, x ∈ R , (1.1)
∂tj ∂tj ∂x

We will concentrate our analysis near a point (0, 0).

Many authors (among them [Ko2], [JoTre], [Mai1], [BDKT] and references
therein) have considered this type of systems. For a given Ω, they were in particular
interested in the existence, for some pair (s, N ) such that s+N > 0, of the following
family of inequalities.
For any pair of bounded open sets ω, I such that ω ⊂ Ω and I ⊂ R, there exists
a constant Cs,N (ω, I) such that
⎛ ⎞
n
||u||2s ≤ CN (ω, I) ⎝ ||Lj u||20 + ||u||2−N ⎠ , ∀u ∈ C0∞ (ω × I) , (1.2)
j=1

where || · ||r denotes the Sobolev norm in H r (Ω × R).

If s > 0, we say that we have a subelliptic estimate and when ϕ is C ∞ this
estimate is known to imply the hypoellipticity of the system. In [JoTre], there are
also results where s can be arbitrarily negative.
The system (1.1) being elliptic in the t variable, it is enough to analyze the
subellipticity microlocally near τ = 0, i.e., near (0, (0, ξ)) in (ω × I) × (Rn+1 \ {0})
with {ξ > 0} or {ξ < 0}.
This leads to the analysis of the existence of two constants Cs+ and Cs− such that
the two following semi-global inequalities hold:
 
ξ |,
2s
u(t, ξ)| dtdξ ≤ Cs
2 + - ξ)|2 dtdξ , ∀u ∈ C ∞ (ω × R) , (1.3)
|Lu(t, 0
ω×R+ ω×R+

,(t, ξ) is the partial Fourier transform of u with respect to the x variable,

where u
and
 
|ξ| |,
2s
u(t, ξ)| dtdξ ≤ Cs
2 − - ξ)|2 dtdξ , ∀u ∈ C0∞ (ω × R) . (1.4)
|Lu(t,
ω×R− ω×R−

When (1.3) is satisfied, we will speak of microlocal subellipticity in

{ξ > 0} and similarly when (1.4) is satisfied, we will speak of microlocal subellip-
ticity in {ξ < 0}. Of course, when s > 0, it is standard that these two inequalities
imply (1.2).
We now observe that (1.3) for ϕ is equivalent to (1.4) for −ϕ, so it is enough to
concentrate our analysis on the first case.
 Subellipticity of Hypoelliptic Quasihomogeneous Systems 111

1.2. The main results

As in [De] and [DeHe], we consider1 ϕ in C ∞ . We assume m and  be given2 in
R+ such that
m ≥ 2 ≥ 2 . (1.5)
We consider in R2 (t, s) as the variables (instead of t) and the functions ϕ ∈ C 1 (R2 )
will be (1, )-quasihomogeneous of degree m in the following sense
ϕ(λt, λ s) = λm ϕ(t, s) , ∀(t, s, λ) ∈ R2 × R+ . (1.6)
According to (1.6), the real function ϕ is determined by its restriction ϕ to
the distorted circle S
 := ϕ|S .
ϕ (1.7)
where S is defined by
S = {(t, s) ; t2 + s2 = 1} , (1.8)
Our main results will be obtained under the following assumptions
Assumption 1.1 (Hnew1).
(i) ϕ−1 (]0, +∞[) = ∅ .
 are not local maxima of ϕ.
(ii) The zeroes of ϕ 
Assumption 1.2 (Hnew2). ϕ  = ϕ/S has only a finite numbers of zeroes θj , each
one being of finite order.
Under this assumption, we denote by p ≥ 1, the smallest integer such that,
 there exists an integer k ≤ p, such that
if θ0 is any zero of ϕ,
• if θ0 = (0, ±1), then,
(k) (θ0 ) = 0 ,
ϕ (1.9)
• if θ0 = (0, ±1),
1
|ϕ(t, s)| ≥ |t|k , (1.10)
C
for t close to 0.
Theorem 1.3. Let ϕ ∈ C ∞ be a (1, )-quasihomogeneous function of order m (with
(m, ) satisfying (1.5)) and satisfying Assumptions 1.1 and 1.2. Then the associated
system Lϕ is microlocally -subelliptic in {ξ > 0} with
1
= sup(m, p) .

Theorem 1.4. Let ϕ be a real analytic (1, )-quasihomogeneous function of order
m with m ≥ 2 ≥ 2.
Under Assumption 1.1 the associated system Lϕ is microlocally -subelliptic in
{ξ > 0} for some  > 0.
1 Like in these papers, the C 1 case could also be considered but the statements will be more
complicate to formulate and we are mainly interested for this paper in the links between hypoel-
lipticity and subellipticity.
2  will be rational in the analytic case.
112 M. Derridj and B. Helffer

Theorem 1.5. Let ϕ be a real analytic (1, )-quasihomogeneous function of order

m with m ≥ 2 ≥ 2.
Then the system Lϕ is hypoelliptic if and only if it is -subelliptic for some  > 0.
1.3. Comparison with previous results
We recall two classical theorems.
Theorem 1.6. Let ϕ be C ∞ in a neighborhood of 0. Then if the system Lϕ is
microlocally hypoelliptic in {ξ > 0}, then there exists a neighborhood V of 0 ∈ Rn
such that ϕ has no local maximum in V
This is Theorem III.1.1 in [Tr2] and we refer to [JoTre] for an elementary proof.
Theorem 1.7. If ϕ is analytic without local maximum in a neighborhood of 0, then
the system Lϕ is (microlocally)-hypoelliptic in {ξ > 0}.
We recall that H. Maire has shown in [Mai1] that the corresponding C ∞
statement of Theorem 1.7 is false when n ≥ 2.
If ϕ does not admit any local maximum in a neighborhood of 0, and if ϕ is
quasi-homogeneous of order m for some m > 0, then this implies that ϕ  satisfies
Assumption 1.1. So if we prove when ϕ is analytic, that this condition implies that
the system is microlocally subelliptic in {ξ > 0}, we obtain immediately, using the
necessary condition of Trèves ([Tr2], Theorem III.1.1), that, for quasihomogneous
ϕ’s, the system Lϕ is microlocally hypoelliptic {ξ > 0} if and only if the system is
microlocally subelliptic {ξ > 0}.
We also obtain that the system Lϕ is hypoelliptic if and only if the system is
subelliptic. According to the counterexamples of Journé-Trépreau [JoTre], this
cannot be improved.
The results in [De] (homogeneous case) and [DeHe] (quasihomogeneous case)
were obtained under Assumptions (1.1) and (1.2), but with the additional condi-
tion:
Assumption 1.8 (Hadd).
(i) If Sj+ is a connected component of ϕ(−1) (]0, +∞[), then one can write Sj+ as
a finite union of arcs satisfying Property 1.9 below.
(ii) If Sj− is one connected component of ϕ (−1) (] − ∞, 0[), then ϕ̃ has a unique
−
minimum in Sj .
Here in the first item of Assumption 1.8, we mean by saying that a closed
arc [θ, θ ] has Property 1.9 the following:
Property 1.9. There exists on this arc a point θ, such that:
   
 is non decreasing on the arc θ, θ, and non increasing on the arc θ,
(a) ϕ , θ .

(So the restriction of ϕ to [θ, θ ] has a maximum at θ). ,

(b)
θ, | θ
 ≥ 0 and θ, | θ
 ≥ 0 , (1.11)
 Subellipticity of Hypoelliptic Quasihomogeneous Systems 113

where for θ = (α, β) and θ, = (, , in S ⊂ R2 ,

α, β)
θ, | θ
 := α
,α|, , .
α|−1 |α|−1 + ββ (1.12)

Note here that we could have ϕ  constant on Sθ,θ  and θ, = θ or θ . Moreover

item (b) says that the length of the two arcs is sufficiently small, more precisely
that the distorted “angles” (see Section 3) associated to [θ, θ ] are acute.

Remarks 1.10.
(i) Assumption 1.8 was not so restrictive for the positive components of
(−1) (]0, +∞[) (at least in the analytic case) but this was the condition (ii)
ϕ
on the uniqueness of the minimum which was introducing the most restrictive
technical condition.
(ii) The proof of Theorem 1.3 consists in showing that Assumptions 1.1 and 1.2
imply Assumption (H+ (α)), which was introduced in [De] and exploited in
[DeHe] and which will be recalled in Section 2.
(iii) If ϕ is analytic and  = 21 (with 1 and 2 mutually prime integers), all the
 can be reinterpreted as criteria involving the restriction
criteria involving ϕ
, of ϕ on
ϕ
S1 ,2 = {(t, s) ; t22 + s21 = 1} .
(iv) Due to Maire’s characterization of hypoellipticity [Mai1] and Journé-Trépreau
counterexamples our results are optimal.

Organization of the paper

As in [De] and [DeHe], the proof of our main theorem will be based on a rather
“abstract” criterion established in [De], which will be recalled in Section 2. We re-
call the terminology adapted to the quasihomogeneity of the problem in Section 3.
Section 4 will be devoted to the construction of escaping rays inside small sectors,
which is the main novelty. Because a big part of the proof is based on the results
of [DeHe], we will only emphasize in Section 5 on the new points of the proof
permitting to eliminate all the finally non necessary assumptions of this previous
paper.

2. Derridj’s subellipticity criterion

We now recall the criterion established in [De]. This involves, for a given α > 0,
the following geometric escape condition on ϕ. We do not have in this section the
restriction n = 2.
Assumption 2.1. (H+ (α))
There exist open sets ω ⊂ Ω and ω  ⊂ ω, with full Lebesgue measure in ω, and a
map γ:
 × [0, 1]  (t, τ ) → γ(t, τ ) ∈ Ω ,
ω
such that
114 M. Derridj and B. Helffer

(i) γ(t, 0) = t ; γ(t, 1) ∈ ω , ∀t ∈ ω

.
(ii) γ is of class C 1 outside a negligible set E and there exist C1 > 0, C2 > 0 and
C3 > 0 such that
(a) |∂τ γ(t, τ )| ≤ C2 , ∀(t, τ ) ∈ ω
 × [0, 1] \ E .
1
(b) | det(Dt γ)(t, τ )| ≥ , (2.1)
C1
where Dt γ denotes the Jacobian matrix of γ considered as a map from
 into R2 .
ω
1 α
(c) ϕ(γ(t, τ )) − ϕ(t) ≥ τ , ∀(t, τ ) ∈ ω
 × [0, 1] .
C3
Using this assumption, it is proved in [De] the following theorem.
Theorem 2.2.
– If ϕ satisfies (H+ (α)), then the associated system (1.1)ϕ is microlocally
α -subelliptic in {ξ > 0}.
1

– If −ϕ satisfies (H+ (α)), then the associated system (1.1)ϕ is microlocally

α -subelliptic in {ξ < 0}.
1

– If ϕ and −ϕ satisfy (H+ (α)), then the associated system (1.1)ϕ is

1
α
-subelliptic.

3. Quasihomogeneous structure
3.1. Distorted geometry
Condition (i) in Assumption 2.1 expresses the property that the curve is escaping
from ω. For the description of escaping curves, it appears useful to extend the
usual terminology used in the Euclidean space R2 in a way which is adapted to
the given quasihomogeneous structure. This is realized by introducing the dressing
map:  
(t, s) → d (t, s) = t |t|−1 , s . (3.1)
which is at least of class C 1 as  ≥ 1, and whose main role is to transport the
distorted geometry onto the Euclidean geometry.
The first example was the unit distorted circle (in short unit disto-circle or unit
“circle”) S introduced in (1.8) whose image by d becomes the standard unit circle
in R2 centered at (0, 0).
Similarly, we will speak of disto-sectors, disto-arcs, disto-rays, disto-disks. In par-
ticular, for (a, b) ∈ S, we define the disto-ray R(a,b) by
R(a,b) := {(λa, λ b) ; 0 ≤ λ ≤ 1} . (3.2)
 
The disto-scalar product of two vectors in R (t, s) et (t , s ) is then given by
2

(t, s) | (t , s )
 = tt |tt |−1 + ss . (3.3)
(for  = 1, we recover the standard scalar product).
 Subellipticity of Hypoelliptic Quasihomogeneous Systems 115

For (t, s) ∈ R2 , we introduce also the quasihomogeneous positive function 

defined on R2 by:
(t, s)2 = t2 + s2 . (3.4)
With this notation, we observe that, if (t, s) ∈ R \ {(0, 0)}, then
2


 t s
(t , s) := , ∈S , (3.5)
(t, s) (t, s)
and
(t, s) ∈ R(t,s) .
The open disto-disk D(R) is then defined by
D(R) = {(x, y) | (x, y) < R} . (3.6)
We can also consider a parametrization of the disto-circle by a parameter on the
corresponding circle ϑ ∈ R/2πZ (through the dressing map). We note that we
have a natural (anticlockwise) orientation of the disto-circle. In other cases it will
be better to parametrize by s (if t = 0) or by t (if s = 0). So a point in S will be
defined either by θ or by (a, b) ∈ R2 or by ϑ.
Once an orientation is defined on S, two points θ1 and θ2 (or (a1 , b1 ) and
(a2 , b2 )) on S will determine a unique unit “sector” S(θ1 , θ2 ) ⊂ D(1).
3.2. Distorted dynamics
The parametrized curves γ permitting us to satisfy Assumption 2.1 will actually
be “lines” (possibly broken) which will finally escape from a neighborhood of the
origin. Our aim in this subsection is to define these “lines” (actually distorted
parametrized lines).
In parametric coordinates, with
t(τ ) = t + ν τ , (3.7)
the curve γ starting from (t, s) and disto-parallel to (c, d) is defined by writing
that the vector (t(τ )|t(τ )|−1 − t|t|−1 , s(τ ) − s) is parallel to (c|c|−1 , d).
In the applications, we will only consider ν = ±c and c = 0.
So  
t(τ )|t(τ )|−1 − t|t|−1 d = c|c|−1 (s(τ ) − s) ,
and we find
d  
s(τ ) = s + −1
t(τ )|t(τ )|−1 − t|t|−1 , (3.8)
c|c|
We consider the map σ → f (σ) which is defined by
f (σ) = σ|σ|−1 . (3.9)
Note that
f (σ) = |σ|−1 ≥ 0 . (3.10)
With this new function, (3.8) can be written as
df (t(τ )) − s(τ )f (c) = df (t) − sf (c) . (3.11)
116 M. Derridj and B. Helffer

This leads us to use the notion of distorted determinant of two vectors in R2 .

For two vectors v := (v1 , v2 ) and w := (w1 , w2 ), we introduce:
Δ (v; w) = Δ (v1 , v2 , w1 , w2 ) := f (v1 )w2 − v2 f (w1 ) . (3.12)
With this notation, (3.11) can be written
Δ ((c, d); (t(τ ), s(τ ))) = Δ (c, d, t(τ ), s(τ )) = Δ (c, d, t, s) , (3.13)
When  = 1, we recover the usual determinant of two vectors in R . 2

We now look at the variation of ψ which is defined (for a given initial point (t, s))
by
τ → ψ(τ ) = ρ(τ )2 = t(τ )2 + s(τ )2 . (3.14)
We now need the following lemma proved in [DeHe].
Lemma 3.1. Under Condition
c ν > 0 , (c, d) | (s, t)
 > 0 , (3.15)
we have, for any τ ≥ 0, for any (t, s) ∈ R \ (0, 0) 2

ν τ 2
ρ(τ )2 − ρ(0)2 ≥ ( ) . (3.16)
2c
If instead c ν < 0, we obtain:
ν τ 2
ρ(τ )2 − ρ(0)2 ≤ −( ) . (3.17)
2c
We continue by analyzing the variation of s(τ ) and t(τ ) and more precisely
the variation on the disto-circle of:
 t(τ ) s(τ )
t(τ ) = , s(τ ) = . (3.18)
ρ(τ ) ρ(τ )
After some computations, we get, with
ν = ±c ,

 s(τ )
t (τ ) = ±|c|1− Δ (c, d, t, s) , (3.19)
ρ(τ )2+1
which can also be written in the form
s(τ )

t (τ ) = ±|c|1− Δ (c, d, 
t(τ ), s(τ )) . (3.20)
ρ(τ )
Similarly, we get for s ,
t(τ )2−1
s (τ ) = ∓|c|1− Δ (c, d, t, s) , (3.21)
ρ(τ )3
and

t2−1 (τ )
s (τ ) = ∓|c|1− Δ (c, d, 
t(τ ), s(τ )) . (3.22)
ρ(τ )
 Subellipticity of Hypoelliptic Quasihomogeneous Systems 117

 > 0)
4. Analysis of the quasielliptic case (ϕ
We first start the proof of the main theorem with the particular case when
≥μ>0.
ϕ (4.1)
This case is already interesting for presenting the main ingredients of the general
proof. We can remark indeed that what we are doing below in S can be done later
in a specific (disto)-arc of S. In addition, it will be applied ((with reverse time) in
the region where ϕ  is negative.

4.1. Construction of γ
We construct γ for points (s, t) belonging to a unit sector S(θ1 , θ2 ) associated to
some arc (θ1 , θ2 ). For a given pair (c, d) in S, in a sufficiently small neighborhood
of (θ1 , θ2 ), we now define γ (see (3.7)-(3.8), with ν = c = 0) by
d
γ(t, s, τ ) := (t(τ ), s(τ )) := (t + cτ, s + (f (t(τ )) − f (t)) . (4.2)
f (c)
Remark 4.1. Note that for any (t0 , s0 , τ ) the Jacobian of the map (t, s) → γ(t, s, τ )
at (t0 , s0 ) is 1.
4.2. Analysis of ρ(τ )m − ρm
Using (3.16) and
 m  m  m
ρ(τ )m − ρm = ρ(τ )2 2 − ρ2 2 ≥ ρ(τ )2 − ρ2 2 ,
where we note that m ≥ 2, we deduce
ρ(τ )m − ρm ≥ 2−m τ m , ∀τ ≥ 0 . (4.3)
A second trivial estimate, will be useful:
ρ(τ )m − ρm ≥ [ρ(τ )2 − ρ2 ]ρm−2 . (4.4)

4.3. The lower bound in the quasi-homogeneous case

Lemma 4.2. There exist for any μ > 0, δ(μ) > 0 and (μ) such that, for any
(−1) (μ, +∞[) of
truncated sector S(θ1 , θ2 ) attached to an arc (θ1 , θ2 ) contained in ϕ
openness less than δ(μ) and any pair (c, d) inside an (μ)-neighborhood of (θ1 , θ2 ),
with c = 0, then, for all (t, s) ∈ S(θ1 , θ2 ), all τ ≥ 0 such that (t(τ ), s(τ )) ∈
S(θ1 , θ2 ), there exists C > 0 such that
1 m
ϕ(t(τ ), s(τ )) − ϕ(t, s) ≥ τ . (4.5)
C
We start with the case
m = 2 ,
and will use the following decomposition for estimating from below the variation
ϕ(t(τ ), s(τ )) − ϕ(t, s).
ϕ(t(τ ), s(τ ))−ϕ(t, s) = (ρ(τ )2 −ρ2 )ϕ(θ
 τ )+ρ2 (ϕ(θ
 τ )−ϕ(θ))
 = (I) + (II) . (4.6)
118 M. Derridj and B. Helffer

So it is enough to show .
|II| ≤ I2 ,
(4.7)
I ≥ C1 τ 2 ,
for getting the right estimate for I + II:
1
I + II ≥ τ 2 . (4.8)
C
Take a small arc S(θ1 , θ2 ) on which
≥μ>0.
ϕ
We consider, starting from (s, t) in S(θ1 , θ2 ), a positive “half-ray” γ(t, s, τ ) with
direction (c, d) in an - neighborhood of (θ1 , θ2 ) (inside or outside of (θ1 , θ2 ) in S).
Remark 4.3. We emphasize that the choice of (c, d) is relatively free (we do not
have used actually that (c, d) is at a maximum point of ϕ  like in [DeHe]). It
is enough that (c, d) is in a sufficiently small neighborhood of (θ1 , θ2 ) (inside or
outside) and both cases will be used at the end.
Note that with
ψ(τ ) = ρ(τ )2 − ρ2 ,
we have
t(τ ) −1
ψ  (τ ) = 2|| (c, d), (t(τ ), s(τ ))
 . (4.9)
c
We will only estimate I and II for (t(τ ), s(τ )) inside S((θ1 , θ2 ) and τ ≤ 1.
This could be an effective restriction when (c, d) does not belong to (θ1 , θ2 ).
We observe that when the arc is of sufficiently small angle (measured by δ), then,
on this part of the curve, we have
(c, d) , (t(τ ), s(τ ))
 ≥ (1 − δ)ρ . (4.10)
We now distinguish two cases. The first one corresponds to the case when S(θ1 , θ2 )
is far from (0, ±1) and the second case corresponds to the case when S(θ1 , θ2 ) is
close to (0, ±1).
First case. We will assume that
1 1
|
t(τ )| ≥ > 0 and |c| ≥ . (4.11)
4 4
We then deduce from (4.9), (4.10) and (4.11) the following lower bound
1
ψ  (τ ) ≥ (1 − δ)ρ2−1 . (4.12)
C
Now we have
 
 τ ) − ϕ(θ)|
|ϕ(θ  ≤ C|θτ − θ| ≤ C̃ |t̃(τ ) − t̃| + |s̃(τ ) − s̃|
We will show, that if the angle of the sector is less than δ then
 
ρ2 |s̃ (τ )| + |t̃ (τ )| ≤ (δ)ψ  (τ ) , (4.13)
where (δ) tends to 0 as δ → 0.
This last inequality is immediate from (4.12), (3.22) and (3.20).
 Subellipticity of Hypoelliptic Quasihomogeneous Systems 119

This leads by integration to

II ≤ (δ)I . (4.14)
Second case. We now suppose that
1 1
|
t(τ )| ≤ and |c| ≤ . (4.15)
2 2
In this case, we have only
1
ψ  (τ ) ≥ |c|1− (1 − δ)|t̃(τ )|−1 ρ2−1 . (4.16)
C
On the other hand, we can parametrize S by t̃ and we get
1
 t̃, s̃) = ϕ(t̃, (1 − t̃2 ) 2 ) .
χ(t̃) := ϕ(
So let us consider
κ(τ ) = χ(t̃(τ )) .
We get

κ (τ ) = t̃ (τ )χ (t̃(τ )) =

 1 1 1

= t̃ (τ ) (∂t ϕ)(t̃(τ ), (1 − t̃(τ )2 ) 2 ) − t̃(τ )2−1 (1 − t̃2 )− 2 (∂s ϕ)(t̃, (1 − t̃2 ) 2 ) .

Using the property that ϕ satisfies (A.2) and that (4.15) and (3.20) are satisfied,
we obtain the existence of C > 0 and C , such that
 
|κ (τ )| ≤ C|t̃ (τ )| |t̃(τ )|−1 + | , 1 |c|1− |t̃(τ )|−1 .
t|2−1 ≤ C (4.17)
ρ
, tends to zero as the openness of the sector tends to
Moreover (3.20) shows that C
zero.
We can achieve the proof of this case by comparing the upper bound of
ρ2 κ (τ ) and the lower bound of ψ (τ ).

Remark 4.4. Till now, we have only treated the case m = 2. For the general case,
it is enough to use the inequality (4.4).

Remark 4.5. For the lower bound in the second line of (4.7), we can use (4.3),
which is proved under the weak condition (3.15), which will be satisfied in our case
because we consider sufficiently small sectors.

4.4. The case of arcs in ϕ ≥ 0 but with a zero at one end

So we consider a closed arc (θ0 , θ1 ) who has a zero of ϕ at one end, say θ0 . Then,
Assumption 1.2 implies that if the opening of this arc is sufficiently small ϕ  is
strictly increasing. We are actually in a situation that we have considered in [DeHe].
We have from Section 5 in [DeHe]
120 M. Derridj and B. Helffer

Lemma 4.6. There exists δ0 such that if the openness of (θ0 , θ1 ) is smaller than
δ0 , then
ϕ(t(τ ), s(τ )) − ϕ(t, s)) ≥ c0 τ sup(m,p) (4.18)
for τ ≥ 0 and any (t, s) ∈ S(θ1 , θ2 ).
Remark 4.7. Actually this analysis will be mainly applied in reverse time in the
negative zone for ϕ. This is indeed here that the assumptions in [DeHe] were
unnecessarily too strong. So we need the estimates (when taking an exit direction
which is exterior to the sector) for the part of the trajectory which remains inside
the sector.

5. Completion of the proof

Let us be more precise on how we can put the arguments together. The zeroes xj
 which are isolated by our assumptions determine disjoint arcs
(j = 1, . . . , k) of ϕ
(xj , xj+1 ) on which ϕ  has a constant sign. We call “negative” arc an arc on which
 is strictly negative in its interior.
ϕ
Our assumptions imply that two negative arcs can not touch and that there exists
at least a positive arc.
Associated to each arc, we have a natural sector and a natural truncated sector
(the sector intersected with a ball of radius less than 1 which will be shown as
uniformly bounded from below by a strictly positive constant). For each of these
sectors, we have to determine how we can escape starting from the truncated
sector.
The case of the “positive arcs” has been essentially treated in our previous
paper. The new arguments can give a partially alternative solution and actually
improvements: outside a neighborhood of the zeros, we do not need any assump-
tions in the C ∞ case.
The case of the “negative arcs” is more interesting because this was in this
case that we meet unnecessary conditions.
Let (θ0 , θ1 ) such an arc and S(θ0 , θ1 ) the associated sector. So we have
 0 ) = 0 = ϕ(θ
θ0 < θ1 , ϕ(θ  1) .
In addition, we have

ϕ(θ) < 0 , ∀θ ∈]θ0 , θ1 [ .
 changes of sign at θ0 and θ1 .
According to our assumptions, ϕ
We now construct a finite sequence ωj such that
θ0 < ω1 < ω2 < · · · < ωL < θ1 ,
and such that
• One can escape from the truncated sectors S(θ0 , ω1 ) and S(ωL , θ1 ) by the
method presented in our previous paper [DeHe] (this is possible by choosing
ω1 sufficiently close to θ0 and ωL sufficiently close to θ1 ).
 Subellipticity of Hypoelliptic Quasihomogeneous Systems 121

• For the other arcs (ωj , ωj+1 ) we construct escaping rays permitting to touch
the neighboring sector S(ωj+1 , ωj+2 ) (with the convention that ωL+1 = θ1 ).
So iterating at most L times we will arrive to a positive sector.
So we are reduced, modulo what we have done in the previous paper, to
control the situation in a strictly negative subsector S(ωj , ωj+1 ) and this problem
is solved under the condition that |ωj − ωj+1 | is small enough.
One choose indeed a point ωj−1 < ωj < ωj and we will escape of the sector
S(ωj , ωj+1 ) by considering the curve


dj
γ(t, s, τ ) = t − cj τ, s − (f (t(τ )) − f (t)) ,
f (cj )
with ωj = (cj , dj ).
Reversing the time, we can apply the results of Section 4 to −ϕ
 in the sectors
 is strictly negative.
where ϕ
The only new point is that we have to work with sufficiently small sectors
(because of the condition appearing in Lemma 4.2). This imposes possibly a larger
but finite number of broken lines. The control of the Jacobians in Section 6 of
[DeHe] is exactly the same.

1

!3
2

!2
+
+ -
3 - +

!1

!0
4

Figure 1. The escaping flow (after application of the dressing map)

Appendix A. A technical proposition

Proposition A.1. Let ϕ ∈ C ∞ be a (1, )-quasihomogeneous function of degree m,
with  ≥ 1 and suppose that
ϕ(0, 1) = 0 . (A.1)
122 M. Derridj and B. Helffer

Then, there exists a C ∞ function g such that

ϕ(t, s) = ϕ(0, s) + t g(t, s) , for s > 0 . (A.2)
Remark A.2. In the analytic case, we now assume that
 = 2 /1 , (A.3)
with 1 and 2 mutually prime integers. In this case, the quasihomogeneity As-
sumption (1.6) on ϕ implies that ϕ is actually a polynomial and we can write ϕ
in the form
ϕ(t, s) = aj,k tj sk , (A.4)
1 j+2 k=1 m
where (j, k) are integers and the aj,k are real. It results of (A.4), that under the
hypothesis A.1 one has 2 k = 1 m so m 
is an integer and a0, m = 0 . Moreover,
the j’s appearing in the sum giving (A.4) satisfy j is an integer. This shows in
particular (A.2).
Proof. The proof in the C ∞ case is quite close to the analytic case. Using the
quasihomogeneity, we have, for s > 0,
m 1
ϕ(t, s) = s  ϕ(s−  t, 1) .
Differentiating k-times with respect to t and taking t = 0, we get
m−k
(∂tk ϕ)(0, s) = s  (∂tk ϕ)(0, 1) . (A.5)
∞
The right-hand side is C under the condition that (∂tk ϕ)(0, 1)
= 0 for m−k

∈ N.
For k = 0, we obtain under our assumption that m  should be an integer.
Then we obtain, using this last property, that (∂tk ϕ)(0, 1) = 0 for k = 1, . . . ,  − 1,
hence
(∂tk ϕ)(0, s) = 0 , for k = 1, . . . ,  − 1 .
This gives immediately the proposition.

References
[BaTr] M.S. Baouendi and F. Trèves. A microlocal version of Bochner’s tube theorem.
Indiana Univ. Math. J. 31 (6), pp. 885–895 (1982).
[BDKT] A. Bove, M. Derridj, J. Kohn and D. Tartakoff. Sum of squares of complex
vector fields and (analytic)-hypoellipticity. Math. Res. Lett. 13 (5), pp. 683–702
(2006).
[CoHo] P.D. Cordaro and J.G. Hounie. Local solvability for a class of differential com-
plexes. Acta Math. 197, pp. 191–212 (2001).
[De] M. Derridj. Subelliptic estimates for some systems of complex vector fields. In
“Hyperbolic problems and regularity questions”. Series Trends in Mathematics.
Eds.: M. Padula and L. Zanghirati. Birkhäuser (2006), pp. 101–108.
[DeHe] M. Derridj and B. Helffer. Subelliptic estimates for some systems of complex
vector fields: quasihomogeneous case. Trans. Amer. Math. Soc. 361 (2009), no. 5,
pp. 2607–2630
 Subellipticity of Hypoelliptic Quasihomogeneous Systems 123

[HeNi] B. Helffer and F. Nier. Hypoelliptic estimates and spectral theory for Fokker-
Planck operators and Witten Laplacians. Lecture Notes in Math. 1862, Springer
Verlag, Berlin 2005.
[HeNo] B. Helffer and J. Nourrigat. Hypoellipticité maximale pour des opérateurs
polynômes de champs de vecteur. Progress in Mathematics, Birkhäuser, Vol. 58
(1985).
[Ho1] L. Hörmander. Hypoelliptic second-order differential equations. Acta Mathe-
matica 119 (1967), pp. 147–171.
[Ho2] L. Hörmander. Subelliptic operators. Seminar on singularities of solutions of
partial differential equations. Ann. Math. Studies 91 (1978), pp. 127–208.
[JoTre] J.L. Journé and J.M. Trépreau. Hypoellipticité sans sous-ellipticité: le cas des
systèmes de n champs de vecteurs complexes en (n + 1)- variables. Séminaire
EDP in Ecole Polytechnique, April 2006.
[Ko1] J. Kohn. Lectures on degenerate elliptic problems. Pseudodifferential operators
with applications, C.I.M.E., Bressanone 1977, pp. 89–151 (1978).
[Ko2] J.J. Kohn. Hypoellipticity and loss of derivatives, with an appendix by M. Der-
ridj and D. Tartakoff. Ann. of Math. 162 (2), pp. 943–986 (2005).
[Mai1] H.M. Maire. Hypoelliptic overdetermined systems of partial differential equa-
tions. Comm. Partial Differential Equations 5 (4), pp. 331–380 (1980).
[Mai2] H.M. Maire. Résolubilité et hypoellipticité de systèmes surdéterminés. Séminaire
Goulaouic-Schwartz 1979–1980, Exp. V, Ecole Polytechnique (1980).
[Mai3] H.M. Maire. Necessary and sufficient condition for maximal hypoellipticity of
∂¯b . Unpublished (1979).
[Mai4] H.M. Maire. Régularité optimale des solutions de systèmes différentiels et du
Laplacien associé: application au b . Math. Ann. 258, pp. 55–63 (1981).
[Ni] F. Nier. Hypoellipticity for Fokker-Planck operators and Witten Laplacians.
Course in China. Preprint September 2006.
[No1] J. Nourrigat. Subelliptic estimates for systems of pseudo-differential operators.
Course in Recife (1982). University of Recife.
[No2] J. Nourrigat. Systèmes sous-elliptiques. Séminaire Equations aux Dérivées Par-
tielles, 1986–1987, exposé V, Ecole Polytechnique (1987).
[No3] J. Nourrigat. Subelliptic systems II. Inv. Math. 104 (2) (1991), pp. 377–400.
[Tr1] F. Trèves. A new method of proof of the subelliptic estimates. Comm. Pure
Appl. Math. 24 (1971), pp. 71–115.
[Tr2] F. Trèves. Study of a model in the theory of complexes of pseudo-differential
operators. Ann. of Math. (2) 104, pp. 269–324 (1976). See also erratum: Ann.
of Math. (2) 113, p. 423 (1981).

M. Derridj
5 rue de la Juvinière
F-78 350 Les loges en Josas, France
B. Helffer
Laboratoire de Mathématiques, Univ Paris-Sud and CNRS,
F-91 405 Orsay Cedex, France
Complex Analysis
Trends in Mathematics, 125–144

c 2010 Springer Basel AG

Invariance of the Parametric Oka Property

Franc Forstnerič
Dedicated to Linda P. Rothschild

Abstract. Assume that E and B are complex manifolds and that π : E → B is

a holomorphic Serre fibration such that E admits a finite dominating family
of holomorphic fiber-sprays over a small neighborhood of any point in B.
We show that the parametric Oka property (POP) of B implies POP of E;
conversely, POP of E implies POP of B for contractible parameter spaces.
This follows from a parametric Oka principle for holomorphic liftings which
we establish in the paper.
Mathematics Subject Classification (2000). 32E10, 32E30, 32H02.
Keywords. Oka principle, Stein spaces, subelliptic submersions.

1. Oka properties
The main result of this paper is that a subelliptic holomorphic submersion π : E →
B between (reduced, paracompact) complex spaces satisfies the parametric Oka
property. Subellipticity means that E admits a finite dominating family of holomor-
phic fiber-sprays over a neighborhood of any point in B (Def. 2.3). The conclusion
means that for any Stein source space X, any compact Hausdorff space P (the pa-
rameter space), and any continuous map f : X × P → B which is X-holomorphic
(i.e., such that fp = f (· , p) : X → B is holomorphic for every p ∈ P ), a continuous
lifting F : X × P → E of f (satisfying π ◦ F = f ) can be homotopically deformed
through liftings of f to an X-holomorphic lifting. (See Theorem 4.2 for a precise
statement.)
xx; E
F x x
x
xx π
xx f 
X ×P /B

Supported by the research program P1-0291, Republic of Slovenia.

126 F. Forstnerič

The following result is an easy consequence. Suppose that E and B are com-
plex manifolds and that π : E → B is a subelliptic submersion which is also a Serre
fibration (such map is called a subelliptic Serre fibration), or is a holomorphic fiber
bundle whose fiber satisfies the parametric Oka property. Then the parametric
Oka property passes up from the base B to the total space E; it also passes down
from E to B if the parameter space P is contractible, or if π is a weak homotopy
equivalence (Theorem 1.2).
We begin by recalling the relevant notions. Among the most interesting phe-
nomena in complex geometry are, on the one hand, holomorphic rigidity, com-
monly expressed by Kobayashi-Eisenman hyperbolicity; and, on the other hand,
holomorphic flexibility, a term introduced in [7]. While Kobayashi hyperbolicity
of a complex manifold Y implies in particular that there exist no nonconstant
holomorphic maps C → Y , flexibility of Y means that it admits many nontrivial
holomorphic maps X → Y from any Stein manifold X; in particular, from any
Euclidean space Cn .
The most natural flexibility properties are the Oka properties which originate
in the seminal works of Oka [27] and Grauert [14, 15]. The essence of the classical
Oka-Grauert principle is that a complex Lie group, or a complex homogeneous
manifold, Y , enjoys the following:
Basic Oka Property (BOP) of Y : Every continuous map f : X → Y from a Stein
space X is homotopic to a holomorphic map. If in addition f is holomorphic
on (a neighborhood of) a compact O(X)-convex subset K of X, and if f |X  is
holomorphic on a closed complex subvariety X  of X, then there is a homotopy
f t : X → Y (t ∈ [0, 1]) from f 0 = f to a holomorphic map f 1 such that for every
t ∈ [0, 1], f t is holomorphic and uniformly close to f 0 on K, and f t |X  = f |X  .
All complex spaces in this paper are assumed to be reduced and paracompact.
A map is said to be holomorphic on a compact subset K of a complex space X
if it is holomorphic in an open neighborhood of K in X; two such maps are
identified if they agree in a (smaller) neighborhood of K; for a family of maps, the
neighborhood should be independent of the parameter.
When Y = C, BOP combines the Oka-Weil approximation theorem and the
Cartan extension theorem. BOP of Y means that, up to a homotopy obstruction,
the same approximation-extension result holds for holomorphic maps X → Y from
any Stein space X to Y .
Denote by C(X, Y ) (resp. by O(X, Y )) the space of all continuous (resp.
holomorphic) maps X → Y , endowed with the topology of uniform convergence
on compacts. We have a natural inclusion
O(X, Y ) → C(X, Y ). (1.1)
BOP of Y implies that every connected component of C(X, Y ) contains a compo-
nent of O(X, Y ). By [8, Theorem 5.3], BOP also implies the following
One-parametric Oka Property: A path f : [0, 1] → C(X, Y ) such that f (0) and
f (1) belong to O(X, Y ) can be deformed, with fixed ends at t = 0, 1, to a path in
 Invariance of the Parametric Oka Property 127

O(X, Y ). Hence (1.1) induces a bijection of the path connected components of the
two spaces.
Y enjoys the Weak Parametric Oka Property if for each finite polyhedron P
and subpolyhedron P0 ⊂ P , a map f : P → C(X, Y ) such that f (P0 ) ⊂ O(X, Y )
can be deformed to a map f: P → O(X, Y ) by a homotopy that is fixed on P0 :
P0 / O(X, Y )
f www
w;
w
incl
ww incl
 www f 
P / C(X, Y )

This implies that (1.1) is a weak homotopy equivalence [11, Corollary 1.5].
Definition 1.1. (Parametric Oka Property (POP)) Assume that P is a compact
Hausdorff space and that P0 is a closed subset of P . A complex manifold Y enjoys
POP for the pair (P, P0 ) if the following holds. Assume that X is a Stein space,
K is a compact O(X)-convex subset of X, X  is a closed complex subvariety of
X, and f : X × P → Y is a continuous map such that
(a) the map fp = f (· , p) : X → Y is holomorphic for every p ∈ P0 , and
(b) fp is holomorphic on K ∪ X  for every p ∈ P .
Then there is a homotopy f t : X × P → Y (t ∈ [0, 1]) such that f t satisfies
properties (a) and (b) above for all t ∈ [0, 1], and also
(i) fp1 is holomorphic on X for all p ∈ P ,
(ii) f t is uniformly close to f on K × P for all t ∈ [0, 1], and
(iii) f t = f on (X × P0 ) ∪ (X  × P ) for all t ∈ [0, 1].
The manifold Y satisfies POP if the above holds for each pair P0 ⊂ P of compact
Hausdorff spaces. Analogously we define POP for sections of a holomorphic map
Z → X. 
Restricting POP to pairs P0 ⊂ P consisting of finite polyhedra we get Gro-
mov’s Ell∞ property [16, Def. 3.1.A.]. By Grauert, all complex homogeneous mani-
folds enjoy POP for finite polyhedral inclusions P0 ⊂ P [14, 15]. A weaker sufficient
condition, called ellipticity (the existence of a dominating spray on Y , Def. 2.1 be-
low), was found by Gromov [16]. A presumably even weaker condition, subellipticity
(Def. 2.2), was introduced in [4].
If Y enjoys BOP or POP, then the corresponding Oka property also holds for
sections of any holomorphic fiber bundle Z → X with fiber Y over a Stein space
X [10]. See also Sect. 2 below and the papers [5, 21, 22, 23].
It is important to know which operations preserve Oka properties. The follow-
ing result was stated in [8] (remarks following Theorem 5.1), and more explicitly
in [9, Corollary 6.2]. (See also [16, Corollary 3.3.C’].)
Theorem 1.2. Assume that E and B are complex manifolds. If π : E → B is a sub-
elliptic Serre fibration (Def. 2.3 below), or a holomorphic fiber bundle with POP
fiber, then the following hold:
128 F. Forstnerič

(i) If B enjoys the parametric Oka property (POP), then so does E.

(ii) If E enjoys POP for contractible parameter spaces P (and arbitrary closed
subspaces P0 of P ), then so does B.
(iii) If in addition π : E → B is a weak homotopy equivalence then
POP of E =⇒ POP of B.
All stated implications hold for a specific pair P0 ⊂ P of parameter spaces.
The proof Theorem 1.2, proposed in [9], requires the parametric Oka property
for sections of certain continuous families of subelliptic submersions. When Finnur
Lárusson asked for explanation and at the same time told me of his applications
of this result [24] (personal communication, December 2008), I decided to write
a more complete exposition. We prove Theorem 1.2 in Sec. 5 as a consequence
of Theorem 4.2. This result should be compared with Lárusson’s [24, Theorem 3]
where the map π : E → B is assumed to be an intermediate fibration in the model
category that he constructed.
Corollary 1.3. Let Y = Ym → Ym−1 → · · · → Y0 , where each Yj is a complex
manifold and every map πj : Yj → Yj−1 (j = 1, 2, . . . , m) is a subelliptic Serre
fibration, or a holomorphic fiber bundle with POP fiber. Then the following hold:
(i) If one of the manifolds Yj enjoys BOP, or POP with a contractible parameter
space, then all of them do.
(ii) If in addition every πj is acyclic (a weak homotopy equivalence) and if Y is a
Stein manifold, then every manifold Yj in the tower satisfies the implication
BOP =⇒ POP.
Proof. Part (i) is an immediate consequences of Theorem 1.2. For (ii), observe
that BOP of Yj implies BOP of Y by Theorem 1.2 (i), applied with P a singleton.
Since Y is Stein, BOP implies that Y is elliptic (see Def. 2.2 below); for the simple
proof see [13, Proposition 1.2] or [16, 3.2.A.]. By Theorem 2.4 below it follows that
Y also enjoys POP. By part (iii) of Theorem 1.2, POP descends from Y = Ym to
every Yj . 

Remark 1.4. A main remaining open problem is whether the implication

BOP =⇒ POP (1.2)
holds for all complex manifolds. By using results of this paper and of his earlier
works, F. Lárusson proved this implication for a large class of manifolds, includ-
ing all quasi-projective manifolds [24, Theorem 4]. The main observation is that
there exists an affine holomorphic fiber bundle π : E → Pn with fiber Cn whose
total space E is Stein; since the map π is acyclic and the fiber satisfies POP, the
implication (1.2) follows from Corollary 1.3 (ii) for any closed complex subvariety
Y ⊂ Pn (since the total space E|Y = π−1 (Y ) is Stein). The same applies to com-
plements of hypersurfaces in such Y ; the higher codimension case reduces to the
hypersurface case by blowing up. 
 Invariance of the Parametric Oka Property 129

2. Subelliptic submersions and Serre fibrations

A holomorphic map h : Z → X of complex spaces is a holomorphic submersion if
for every point z0 ∈ Z there exist an open neighborhood V ⊂ Z of z0 , an open
neighborhood U ⊂ X of x0 = h(z0 ), an open set W in a Euclidean space Cp , and a
biholomorphic map φ : V → U × W such that pr1 ◦ φ = h, where pr1 : U × W → U
is the projection on the first factor.
Z ⊃V
φ
/ U ×W

h pr1
 
X ⊃U
id /U

Each fiber Zx = h−1(x) (x ∈ X) of a holomorphic submersion is a closed

complex submanifold of Z. A simple example is the restriction of a holomorphic
fiber bundle projection E → X to an open subset Z of E.
We recall from [16, 4] the notion of a holomorphic spray and domination.
Definition 2.1. Assume that X and Z are complex spaces and h : Z → X is a
holomorphic submersion. For x ∈ X let Zx = h−1 (x).
(i) A fiber-spray on Z is a triple (E, π, s) consisting of a holomorphic vector
bundle π : E → Z together with a holomorphic map s : E → Z such that for
each z ∈ Z we have
s(0z ) = z, s(Ez ) ⊂ Zh(z) .
(ii) A spray (E, π, s) is dominating at a point z ∈ Z if its differential
(ds) 0z : T0z E → Tz Z
at the origin 0z ∈ Ez = π−1 (z) maps the subspace Ez of T0z E surjectively
onto the vertical tangent space VTz Z = ker dhz . The spray is dominating (on
Z) if it is dominating at every point z ∈ Z.
(iii) A family of h-sprays (Ej , πj , sj ) (j = 1, . . . , m) on Z is dominating at the
point z ∈ Z if
(ds1 )0z (E1,z ) + (ds2 )0z (E2,z ) · · · + (dsm )0z (Em,z ) = VTz Z.
If this holds for every z ∈ Z then the family is dominating on Z.
(iv) A spray on a complex manifold Y is a fiber-spray associated to the constant
map Y → point.
The simplest example of a spray on a complex manifold Y is the flow Y ×C →
Y of a C-complete holomorphic vector field on Y . A composition of finitely many
such flows, with independent time variables, is a dominating spray at every point
where the given collection of vector fields span the tangent space of Y . Another
example of a dominating spray is furnished by the exponential map on a complex
Lie group G, translated over G by the group multiplication.
The following notion of an elliptic submersion is due to Gromov [16, Sect.
1.1.B]; subelliptic submersions were introduced in [4]. For examples see [4, 8, 16].
130 F. Forstnerič

Definition 2.2. A holomorphic submersion h : Z → X is said to be elliptic (resp.

subelliptic) if each point x0 ∈ X has an open neighborhood U ⊂ X such that
the restricted submersion h : Z|U → U admits a dominating fiber-spray (resp. a
finite dominating family of fiber-sprays). A complex manifold Y is elliptic (resp.
subelliptic) if the trivial submersion Y → point is such.
The following notions appear in Theorem 1.2.
Definition 2.3. (a) A continuous map π : E → B is Serre fibration if it satisfies the
homotopy lifting property for polyhedra (see [32, p. 8]).
(b) A holomorphic map π : E → B is an elliptic Serre fibration (resp. a
subelliptic Serre fibration) if it is a surjective elliptic (resp. subelliptic) submersion
and also a Serre fibration.
The following result was proved in [10] (see Theorems 1.4 and 8.3) by follow-
ing the scheme proposed in [13, Sect. 7]. Earlier results include Gromov’s Main
Theorem [16, Theorem 4.5] (for elliptic submersions onto Stein manifolds, without
interpolation), [13, Theorem 1.4] (for elliptic submersions onto Stein manifolds), [4,
Theorem 1.1] (for subelliptic submersion), and [8, Theorem 1.2] (for fiber bundles
with POP fibers over Stein manifolds).
Theorem 2.4. Let h : Z → X be a holomorphic submersion of a complex space
Z onto a Stein space X.
/∞Assume that X is exhausted by Stein Runge domains
D1  D2  · · · ⊂ X = j=1 Dj such that every Dj admits a stratification
Dj = X0 ⊃ X1 ⊃ · · · ⊃ Xmj = ∅ (2.1)
with smooth strata Sk = Xk \Xk+1 such that the restriction of Z → X to every
connected component of each Sk is a subelliptic submersion, or a holomorphic fiber
bundle with POP fiber. Then sections X → Z satisfy POP.
Remark 2.5. In previous papers [11, 12, 13, 4, 8, 9] POP was only considered for
pairs of parameter spaces P0 ⊂ P such that
(*) P is a nonempty compact Hausdorff space, and P0 is a closed subset of
P that is a strong deformation neighborhood retract (SDNR) in P .
Here we dispense with the SDNR condition by using the Tietze extension
theorem for maps into Hilbert spaces (see the proof of Proposition 4.4).
Theorem 2.4 also hold when P is a locally compact and countably compact
Hausdorff space, and P0 is a closed subspace of P . The proof requires only a minor
change of the induction scheme (applying the diagonal process).
On the other hand, all stated results remain valid if we restrict to pairs
P0 ⊂ P consisting of finite polyhedra; this suffices for most applications. 
Theorem 2.4 implies the following result concerning holomorphic liftings.
Theorem 2.6. Let π : E → B be a holomorphic submersion of a complex space E
onto a complex space B. Assume that B admits a stratification B = B0 ⊃ B1 ⊃
· · · ⊃ Bm = ∅ by closed complex subvarieties such that the restriction of π to every
connected component of each difference Bj \Bj+1 is a subelliptic submersion, or a
holomorphic fiber bundle with POP fiber.
 Invariance of the Parametric Oka Property 131

Given a Stein space X and a holomorphic map f : X → B, every continu-

ous lifting F : X → E of f (π ◦ F = f ) is homotopic through liftings of f to a
holomorphic lifting.

>E
F ~~~
~~ π
~~ f 
X /B

Proof. Assume first that X is finite dimensional. Then there is a stratification

X = X0 ⊃ X1 ⊃ · · · ⊃ Xl = ∅ by closed complex subvarieties, with smooth
differences Sj = Xj \Xj+1 , such that each connected component S of every Sj is
mapped by f to a stratum Bk \Bk+1 for some k = k(j). The pull-back submersion
f ∗ E = {(x, e) ∈ X × E : f (x) = π(e)} −→ X
then satisfies the assumptions of Theorem 2.4 with respect to this stratification of
X. Note that liftings X → E of f : X → B correspond to sections X → f ∗ E, and
hence the result follows from Theorem 2.4. The general case follows by induction
over an exhaustion of X by an increasing sequence of relatively compact Stein
Runge domains in X. 
A fascinating application of Theorem 2.6 has recently been found by Ivarsson
and Kutzschebauch [19, 20] who solved the following Gromov’s Vaserstein problem:
Theorem 2.7. (Ivarsson and Kutzschebauch [19, 20]) Let X be a finite-dimensional
reduced Stein space and let f : X → SLm (C) be a null-homotopic holomorphic
mapping. Then there exist a natural number N and holomorphic mappings G1 ,
. . . , GN : X → Cm(m−1)/2 (thought of as lower resp. upper triangular matrices)
such that




1 0 1 G2 (x) 1 GN (x)
f (x) = ...
G1 (x) 1 0 1 0 1
is a product of upper and lower diagonal unipotent matrices. (For odd N the last
matrix has GN (x) in the lower left corner.)
In this application one takes B = SLm (C), E = (Cm(m−1)/2 )N , and π : E →
B is the map




1 0 1 G2 1 GN
π(G1 , G2 , . . . , GN ) = ... .
G1 1 0 1 0 1
Every null-homotopic holomorphic map f : X → B = SLm (C) admits a continuous
lifting F : X → E for a suitably chosen N ∈ N (Vaserstein [31]), and the goal
is to deform F to a holomorphic lifting G = (G1 , . . . , GN ) : X → E. This is
done inductively by applying Theorem 2.6 to auxiliary submersions obtained by
composing π with certain row projections. Stratified elliptic submersions naturally
appear in their proof.
132 F. Forstnerič

3. Convex approximation property

In this section we recall from [8] a characterization of Oka properties in terms of
an Oka-Weil approximation property for entire maps Cn → Y .
Let z = (z1 , . . . , zn), zj = xj + i yj , be complex coordinates on Cn . Given
numbers aj , bj > 0 (j = 1, . . . , n) we set
Q = {z ∈ Cn : |xj | ≤ aj , |yj | ≤ bj , j = 1, . . . , n}. (3.1)
Definition 3.1. A special convex set in Cn is a compact convex set of the form
K = {z ∈ Q : yn ≤ φ(z1 , . . . , zn−1 , xn )}, (3.2)
where Q is a cube (3.1) and φ is a continuous concave function with values in
(−bn , bn ). Such (K, Q) is called a special convex pair in Cn .
Definition 3.2. A complex manifold Y enjoys the Convex Approximation Property
(CAP) if every holomorphic map f : K → Y on a special convex set K ⊂ Q ⊂ Cn
(3.2) can be approximated, uniformly on K, by holomorphic maps Q → Y .
Y enjoys the Parametric Convex Approximation Property (PCAP) if for every
special convex pair (K, Q) and for every pair of parameter spaces P0 ⊂ P as in
Def. 1.1, a map f : Q × P → Y such that fp = f (· , p) : Q → Y is holomorphic
for every p ∈ P0 , and is holomorphic on K for every p ∈ P , can be approximated
uniformly on K × P by maps f: Q × P → Y such that fp is holomorphic on Q for
all p ∈ P , and fp = fp for all p ∈ P0 .
The following characterization of the Oka property was found in [8, 9] (for
Stein source manifolds), thereby answering a question of Gromov [16, p. 881,
3.4.(D)]. For the extension to Stein source spaces see [10].
Theorem 3.3. For every complex manifold we have
BOP ⇐⇒ CAP, POP ⇐⇒ PCAP.
Remark 3.4. The implication PCAP =⇒ POP also holds for a specific pair of
(compact, Hausdorff) parameter spaces as is seen from the proof in [8]. More
precisely, if a complex manifold Y enjoys PCAP for a certain pair P0 ⊂ P , then it
also satisfies POP for that same pair. 

4. A parametric Oka principle for liftings

In this section we prove the main result of this paper, Theorem 4.2, which gen-
eralizes Theorem 2.6 to families of holomorphic maps. We begin by recalling the
relevant terminology from [13].
Definition 4.1. Let h : Z → X be a holomorphic map of complex spaces, and let
P0 ⊂ P be topological spaces.
(a) A P -section of h : Z → X is a continuous map f : X × P → Z such that
fp = f (· , p) : X → Z is a section of h for each p ∈ P . Such f is holomorphic
 Invariance of the Parametric Oka Property 133

if fp is holomorphic on X for each fixed p ∈ P . If K is a compact set in

X and if X  is a closed complex subvariety of X, then f is holomorphic on
K ∪ X  if there is an open set U ⊂ X containing K such that the restrictions
fp |U and fp |X  are holomorphic for every p ∈ P .
(b) A homotopy of P -sections is a continuous map H : X × P × [0, 1] → Z such
that Ht = H(· , · , t) : X × P → Z is a P -section for each t ∈ [0, 1].
(c) A (P, P0 )-section of h is a P -section f : X × P → Z such that fp = f (· , p) :
X → Z is holomorphic on X for each p ∈ P0 . A (P, P0 )-section is holomorphic
on a subset U ⊂ X if fp |U is holomorphic for every p ∈ P .
(d) A P -map X → Y to a complex space Y is a map X × P → Y . Similarly one
defines (P, P0 )-maps and their homotopies.
Theorem 4.2. Assume that E and B are complex spaces and π : E → B is a
subelliptic submersion (Def. 2.3), or a holomorphic fiber bundle with POP fiber
(Def. 1.1). Let P be a compact Hausdorff space and P0 a closed subspace of P .
Given a Stein space X, a compact O(X)-convex subset K of X, a closed complex
subvariety X  of X, a holomorphic P -map f : X × P → B, and a (P, P0 )-map
F : X × P → E that is a π-lifting of f (π ◦ F = f ) and is holomorphic on (a
neighborhood of) K and on the subvariety X  , there exists a homotopy of liftings
F t : X × P → E of f (t ∈ [0, 1]) that is fixed on (X × P0 ) ∪ (X  × P ), that
approximates F = F 0 uniformly on K × P , and such that Fp1 is holomorphic on
X for all p ∈ P .
If in addition F is holomorphic in a neighborhood of K∪X  then the homotopy
F can be chosen such that it agrees with F 0 to a given finite order along X  .
t

;E
Ft xx
xx
x
xx π
xx 
X ×P f /B

Definition 4.3. A map π : E → B satisfying the conclusion of Theorem 4.2 is said

to enjoy the parametric Oka property (c.f. Lárusson [22, 23, 24]). 
 = X × B,
Proof. The first step is a reduction to the graph case. Set Z = X × E, Z

 : Z → Z denote the map
and let π
(x, e) = (x, π(e)),
π x ∈ X, e ∈ E.
Then π  is a subelliptic submersion, resp. a holomorphic fiber bundle with POP
fiber. Let   = X × B → X denote the projection onto the first factor, and let
h: Z
h= h◦ π : Z → X. To a P -map f : X ×P → B we associate the P -section f(x, p) =
(x, f (x, p)) of 
h: Z → X. Further, to a lifting F : X × P → E of f we associate the

P -section F (x, p) = (x, F (x, p)) of h : Z → X. Then π  ◦ F = f. This allows us to
drop the tilde’s on π, f and F and consider from now on the following situation:
(i) Z and Z  are complex spaces,
(ii) π : Z → Z  is a subelliptic submersion, or a holomorphic fiber bundle with
POP fiber,
134 F. Forstnerič

(iii)   → X is a holomorphic map onto a Stein space X,

h: Z
(iv) f : X × P → Z  is a holomorphic P -section of 
h,
(v) F : X × P → Z is a holomorphic (P, P0 )-section of h = 
h ◦ π : Z → X such
that π ◦ F = f , and F is holomorphic on K ∪ X  .
We need to find a homotopy F t : X × P → Z (t ∈ [0, 1]) consisting of (P, P0 )-
sections of h : Z → X such that π ◦ F t = f for all t ∈ [0, 1], and
(α) F 0 = F ,
(β) F 1 is a holomorphic P -section, and
(γ) for every t ∈ [0, 1], F t is holomorphic on K, it is uniformly close to F 0 on
K × P , and it agrees with F 0 on (X × P0 ) ∪ (X  × P ).

<Z
xxx
xx
Ft
xx
π
xx 
X ×P f /Z


Set fp = f (· , p) : X → Z  for p ∈ P . The image fp (X) is a closed Stein

subspace of Z that is biholomorphic to X (since  h ◦ f is the identity on X).
When P = {p} is a singleton, there is only one section f = fp , and the
desired conclusion follows by applying Theorem 2.4 to the restricted submersion
π : Z|f (X) → f (X).
In general we consider the family of restricted submersions Z|fp (X) → fp (X)
(p ∈ P ). The proof of the parametric Oka principle [12, Theorem 1.4] requires
certain modifications that we now explain. It suffices to obtain a homotopy F t of
liftings of f over a relatively compact subset D of X with K ⊂ D; the proof is
then finished by induction over an exhaustion of X. The initial step is provided
by the following proposition. (No special assumption is needed on the submersion
π: Z → Z  for this result.)

Proposition 4.4. (Assumptions as above) Let D be an open relatively compact set in

X with K ⊂ D. There exists a homotopy of liftings of f over D from F = F 0 |D×P
to a lifting F  such that properties (α) and (γ) hold for F  , while (β) is replaced by
(β  ) Fp is holomorphic on D for all p in a neighborhood P0 ⊂ P of P0 .

The existence of such local holomorphic extension F  is used at several subse-

quent steps. We postpone the proof of the proposition to the end of this section and
continue with the proof of Theorem 4.2. Replacing F by F  and X by D, we assume
from now on that Fp is holomorphic on X for all p ∈ P0 (a neighborhood of P0 ).
Assume for the sake of discussion that X is a Stein manifold, that X  = ∅,
and that π : Z → Z is a subelliptic submersion. (The proof in the fiber bundle case
is simpler and will be indicated along the way. The case when X has singularities
or X  = ∅ uses the induction scheme from [10], but the details presented here
remain unchanged.) It suffices to explain the following:
 Invariance of the Parametric Oka Property 135

Main step: Let K ⊂ L be compact strongly pseudoconvex domains in X that are

O(X)-convex. Assume that F 0 = {Fp0 }p∈P is a π-lifting of f = {fp }p∈P such that
Fp0 is holomorphic on K for all p ∈ P , and Fp0 is holomorphic on X when p ∈ P0 .
Find a homotopy of liftings F t = {Fpt }p∈P (t ∈ [0, 1]) that are holomorphic on K,
uniformly close to F 0 on K × P , the homotopy is fixed for all p in a neighborhood
of P0 , and Fp1 is holomorphic on L for all p ∈ P .
Granted the Main Step, a solution satisfying the conclusion of Theorem 4.2
is then obtained by induction over a suitable exhaustion of X.
/
Proof of the Main Step. We cover the compact set fp (L\K) ⊂ Z by open
p∈P
 such that every restricted submersion π : Z|Uj → Uj admits a
sets U1 , . . . , UN ⊂ Z
finite dominating family of π-sprays. In the fiber bundle case we choose the sets Uj
such that Z|Uj is isomorphic to the trivial bundle Uj × Y → Uj with POP fiber Y .
Choose a Cartan /n string A = (A0 , A1 , . . . , An ) in X [12, Def. 4.2] such that
K = A0 and L = j=0 Aj . The construction is explained in [12, Corollary 4.5]:
It suffices to choose /each of thecompact sets Ak to be a strongly pseudoconvex
k−1
domain such that j=0 Aj , Ak is a Cartan pair for all k = 1, . . . , n. In addition,
we choose the sets A1 , . . . , An small enough such that fp (Aj ) is contained in one
of the sets Ul for every p ∈ P and j = 1, . . . , n.
We cover P by compact subsets P1 , . . . , Pm such that for every j = 1, . . . , m
and k = 1, . . . , n, there is a neighborhood Pj ⊂ P of Pj such that the set
/
p∈Pj fp (Ak ) is contained in one of the sets Ul .
As in [12] we denote by K(A) the nerve complex of A = (A0 , A1 , . . . , An ), i.e.,
a combinatorial simplicial complex consisting of all multiindices J = (j0 , j1 , . . . , jk ),
with 0 ≤ j0 < j1 < · · · < jk ≤ n, such that
AJ = Aj0 ∩ Aj1 ∩ · · · ∩ Ajk = ∅.
Its geometric realization, K(A), is a finite polyhedron in which every multiindex
J = (j0 , j1 , . . . , jk ) ∈ K(A) of length k + 1 determines a closed k-dimensional
face |J| ⊂ K(A), homeomorphic to the standard k-simplex in Rk , and every k-
dimensional face of K(A) is of this form. The face |J| is called the body (or carrier)
of J, and J is the vertex scheme of |J|. Given I, J ∈ K(A) we have |I|∩|J| = |I ∪J|.
The vertices of K(A) correspond to the individual sets Aj in A, i.e., to singletons
(j) ∈ K(A). (See [18] or [30] for simplicial complexes and polyhedra.)
Given a compact set A in X, we denote by ΓO (A, Z) the space of all sections
of h : Z → X that are holomorphic over some unspecified open neighborhood A in
Z, in the sense of germs at A.
Recall that a holomorphic K(A, Z)-complex [12, Def. 3.2] is a continuous
family of holomorphic sections
F∗ = {FJ : |J| → ΓO (AJ , Z), J ∈ K(A)}
satisfying the following compatibility conditions:
I, J ∈ K(A), I ⊂ J =⇒ FJ (t) = FI (t)|AJ (∀t ∈ |I|).
136 F. Forstnerič

Note that
• F(k) a holomorphic section over (a neighborhood of) Ak ,
• F(k0 ,k1 ) is a homotopy of holomorphic sections over Ak0 ∩ Ak1 connecting
F(k0 ) and F(k1 ) ,
• F(k0 ,k1 ,k2 ) is a triangle of homotopies with vertices F(k0 ) , F(k1 ) , F(k2 ) and sides
F(k0 ,k1 ) , F(k0 ,k2 ) , F(k1 ,k2 ) , etc.
Similarly one defines a continuous K(A, Z)-complex.
A K(A, Z; P )-complex is defined in an obvious way by adding the parameter
p ∈ P . It can be viewed as a K(A, Z)-complex of P -sections of Z → X, or as
a family of K(A, Z)-complexes depending continuously on the parameter p ∈ P .
Similarly, a K(A, Z; P, P0 )-complex /nis a K(A, Z; P )-complex consisting of holomor-
phic sections (over the set L = j=0 Aj ) for the parameter values p ∈ P0 . The
terminology of Def. 4.1 naturally applies to complexes of sections.
By choosing the sets A1 , . . . , An sufficiently small and by shrinking the neigh-
borhood P0 (furnished by Proposition 4.4) around P0 if necessary we can deform
F = F 0 to a holomorphic K(A, Z; P, P0 )-complex F∗,∗ = {F∗,p }p∈P such that
• every section in F∗,p projects by π : Z → Z to the section fp (such F∗,∗ is
called a lifting of the holomorphic P -section f = {fp}p∈P ),
• F(0),p is the restriction to A0 = K of the initial section Fp0 , and
• for p ∈ P0 , every section in F∗,p is the restriction of Fp0 to the appropriate
subdomain (i.e., the deformation from F 0 to F∗,∗ is fixed over P0 ).
A completely elementary construction of such initial holomorphic complex
F∗,∗ can be found in [12, Proposition 4.7].

Remark 4.5. We observe that, although the map h =  h ◦ π : Z → X is not neces-

 
sarily a submersion (since the projection h : Z → X may have singular fibers), the
construction in [12] still applies since we only work with the fiber component of Fp
(over fp ) with respect to the submersion π : Z → Z.  All lifting problems locally
reduce to working with functions. 
The rest of the construction amounts to finitely many homotopic modifica-
tions of the complex F∗,∗ . At every step we collapse one of the cells in the complex
and obtain a family (parametrized by P ) of holomorphic sections over the union
of the sets that determine the cell. In finitely many steps we obtain a family of
constant complexes F 1 = {Fp1 }p∈P , that is, Fp1 is a holomorphic section of Z → X
over L. This procedure is explained in [12, Sect. 5] (see in particular Proposition
5.1.). The additional lifting condition is easily satisfied at every step of the con-
struction. In the end, the homotopy of complexes from F 0 to F 1 is replaced by a
homotopy of constant complexes, i.e., a homotopy of liftings F t of f that consist
of sections over L (see the conclusion of proof of Theorem 1.5 in [12, p. 657]).
Let us describe more carefully the main step – collapsing a segment in a holo-
morphic complex. (All substeps in collapsing a cell reduce to collapsing a segment,
each time with an additional parameter set.)
 Invariance of the Parametric Oka Property 137

We have a special pair (A, B) of compact sets contained in L ⊂ X, called a

Cartan pair [13, Def. 4], with B contained in one of the sets A1 , . . . , An in our Car-
tan string A. (Indeed, B is the intersection of some of these sets.) Further, we have
an additional compact parameter set P (which appears in the proof) and families
of holomorphic sections of h : Z → X, a(p,p̃) over A and b(p,p̃) over B, depending
continuously on (p, p̃) ∈ P × P and projecting by π : Z → Z  to the section fp . For

p ∈ P0 we have a(p,p̃) = b(p,p̃) over A ∩ B. These two families are connected over
A ∩ B by a homotopy of holomorphic sections bt(p,p̃) (t ∈ [0, 1]) such that

b0(p,p̃) = a(p,p̃) , b1(p,p̃) = b(p,p̃) , π ◦ bt(p,p̃) = fp

hold for each p ∈ P and t ∈ [0, 1], and the homotopy is fixed for p ∈ P0 . These
two families are joined into a family of holomorphic sections a(p,p̃) over A ∪ B,
projecting by π to fp . The deformation consists of two substeps:
1. by applying the Oka-Weil theorem [11, Theorem 4.2] over the pair A ∩ B ⊂ B
we approximate the family a(p,p̃) sufficiently closely, uniformly on a neigh-
borhood of A ∩ B, by a family b(p,p̃) of holomorphic sections over B;
2. assuming that the approximation in (1) is sufficiently close, we glue the fam-
ilies a(p,p̃) and b(p,p̃) into a family of holomorphic sections 
a(p,p̃) over A ∪ B
such that π ◦  a(p,p̃) = fp .
For Substep (2) we can use local holomorphic sprays as in [8, Proposition
3.1], or we apply [11, Theorem 5.5]. The projection condition π ◦ a(p,p̃) = fp is a
trivial addition.
Substep (1) is somewhat more problematic as it requires a dominating family
of π-sprays on Z|U over an open set U ⊂ Z  to which the sections bt
(p,p̃) project.
(In the fiber bundle case we need triviality of the restricted bundle Z|U → U and
POP of the fiber.) Recall that B is contained in one of the sets Ak , and therefore
 
fp (B) ⊂ fp (Ak ) ⊂ Ul(j,k) .
p∈Pj p∈Pj

Since π ◦ bt(p,p̃) = fp and Z admits a dominating family of π-sprays over each set
Ul , Substep (1) applies separately to each of the m families

{bt(p,p̃) : p ∈ Pj , p̃ ∈ P , t ∈ [0, 1]}, j = 1, . . . , m.

To conclude the proof of the Main Step we use the stepwise extension method,
similar to the one in [12, pp. 138–139]. In each step we make the lifting holomor-
phic for the parameter values in one of the sets Pj , keeping the homotopy fixed
over the union of the previous sets.
We begin with P1 . The above shows that the Main Step can be accomplished
in finitely many applications of Substeps (1) and (2), using the pair of parame-
ter spaces P0 ∩ P1 ⊂ P1 (instead of P0 ⊂ P ). We obtain a homotopy of liftings
138 F. Forstnerič

{Fpt : p ∈ P1 , t ∈ [0, 1]} of fp such that Fp1 is holomorphic on L for all p in a
neighborhood of P1 , and Fpt = Fp0 for all t ∈ [0, 1] and all p in a relative neighbor-
hood of P0 ∩ P1 in P1 . We extend this homotopy to all values p ∈ P by replacing
tχ(p)
Fpt by Fp , where χ : P → [0, 1] is a continuous function that equals one near
P1 and has support contained in P1 . Thus Fp1 is holomorphic on L for all p in a
neighborhood V1 of P0 ∪ P1 , and Fp1 = Fp0 for all p in a neighborhood of P0 .
We now repeat the same procedure with F 1 as the ‘initial’ lifting of f , us-
ing the pair of parameter spaces (P0 ∪ P1 ) ∩ P2 ⊂ P2 . We obtain a homotopy of
liftings {Fpt }t∈[1,2] of fp for p ∈ P2 such that the homotopy is fixed for all p in a
neighborhood of (P0 ∪ P1 ) ∩ P2 in P2 , and Fp2 is holomorphic on L for all p in a
neighborhood of P0 ∪ P1 ∪ P2 in P .
In m steps of this kind we get a homotopy {F t }t∈[0,m] of liftings of f such that
Fp is holomorphic on L for all p ∈ P , and the homotopy is fixed in a neighborhood
m

of P0 in P . It remains to rescale the parameter interval [0, m] back to [0, 1].

This concludes the proof in the special case when X is a Stein manifold and
X  = ∅. In the general case we follow the induction scheme in the proof of the para-
metric Oka principle for stratified fiber bundles with POP fibers in [10]; Cartan
strings are now used inside the smooth strata.
When π : Z → Z is a fiber bundle, we apply the one-step approximation and
gluing procedure as in [8], without having to deal with holomorphic complexes.
The Oka-Weil approximation theorem in Substep (1) is replaced by POP of the
fiber. 

Proof of Proposition 4.4. We begin by considering the special case when π : Z =

Z ×C → Z is a trivial line bundle. We have Fp = (fp , gp ) where gp is a holomorphic
function on X for p ∈ P0 , and is holomorphic on K ∪ X  for all p ∈ P . We replace
X by a relatively compact subset containing D̄ and consider it as a closed complex
subvariety of a Euclidean space CN . Choose bounded pseudoconvex domains Ω 
Ω in CN such that D̄ ⊂ Ω ∩ X.
By [13, Lemma 3.1] there exist bounded linear extension operators
S : H ∞ (X ∩ Ω ) −→ H 2 (Ω) = L2 (Ω) ∩ O(Ω),
S  : H ∞ (X  ∩ Ω ) −→ H 2 (Ω),
such that S(g)|X∩ Ω = g|X∩ Ω , and likewise for S  . (In [13] we obtained an exten-
sion operator into H ∞ (Ω), but the Bergman space appeared as an intermediate
step. Unlike the Ohsawa-Takegoshi extension theorem [26], this is a soft result de-
pending on the Cartan extension theorem and some functional analysis; the price
is shrinking of the domain.) Set
hp = S(gp |X∩ Ω ) − S  (gp |X  ∩ Ω ) ∈ H 2 (Ω), p ∈ P0 .
Then hp vanishes on X  , and hence it belongs to the closed subspace HX 2
 (Ω)

consisting of all functions in H (Ω) that vanish on X ∩ Ω. Since these are Hilbert
2

spaces, the generalized Tietze extension theorem (a special case of Michael’s convex
 Invariance of the Parametric Oka Property 139

selection theorem; see [28, Part C, Theorem 1.2, p. 232] or [3, 25]) furnishes a
continuous extension of the map P0 → HX 2 
 (Ω), p → hp , to a map P  p → hp ∈
2
HX  (Ω). Set
Gp = 
hp + S  (gp |X  ∩ Ω ) ∈ H 2 (Ω), p ∈ P.
Then

Gp |X  ∩ Ω = gp |X  ∩ Ω (∀p ∈ P ), Gp |X∩ Ω = gp |X∩ Ω (∀p ∈ P0 ).

This solves the problem, except that Gp should approximate gp uniformly on K.

Choose holomorphic functions φ1 , . . . , φm on CN that generate the ideal sheaf of
the subvariety X  at every point in Ω . A standard application of Cartan’s Theorem
B shows that in a neighborhood of K we have

m
gp = Gp + φj ξj,p
j=1

for some holomorphic functions ξj,p in a neighborhood of K, depending con-

tinuously on p ∈ P and vanishing identically on X for p ∈ P0 . (See, e.g., [12,
Lemma 8.1].)
Since the set K is O(X)-convex, and hence polynomially convex in CN , an
extension of the Oka-Weil approximation theorem (using a bounded linear solution
operator for the ∂-equation, given for instance by Hörmander’s L2 -methods [17]
or by integral kernels) furnishes functions ξj,p ∈ O(Ω), depending continuously on
p ∈ P , such that ξj,p approximates ξj,p as close as desired uniformly on K, and it
vanishes on X ∩ Ω when p ∈ P0 . Setting

m

gp = G p + φj ξj,p , p∈P
j=1

gives the solution. This proof also applies to vector-valued maps by applying it
componentwise.
The general case reduces to the special case by using that for every p0 ∈ P0 ,
the Stein subspace Fp0 (X) (resp. fp0 (X)) admits an open Stein neighborhood in Z
 according to a theorem of Siu [2, 29]. Embedding these neighborhoods
(resp. in Z)
in Euclidean spaces and using holomorphic retractions onto fibers of π (see [10,
Proposition 3.2]), the special case furnishes neighborhoods Up0 ⊂ Up 0 of p0 in P
and a P -section F  : D̄ × P → Z, homotopic to F through liftings of f , such that
(i) π ◦ Fp = fp for all p ∈ P ,
(ii) Fp is holomorphic on D̄ when p ∈ Up0 ,
(iii) Fp = Fp for p ∈ P0 ∪ (P \Up 0 ),
(iv) Fp |X  ∩D = Fp |X  ∩D for all p ∈ P , and
(v) F  approximates F on K × P .
140 F. Forstnerič

(The special case is first used for parameter values p in a neighborhood Up 0 of p0 ;
the resulting family of holomorphic maps D̄ × Up 0 → CN is then patched with F
by using a cut-off function χ(p) with support in Up 0 that equals one on a neigh-
borhood Up0 of p0 , and applying holomorphic retractions onto the fibers of π.) In
finitely many steps of this kind we complete the proof. 

Remark 4.6. One might wish to extend Theorem 4.2 to the case when π : E → B is
a stratified subelliptic submersion, or a stratified fiber bundle with POP fibers. The
problem is that the induced stratifications on the pull-back submersions fp∗ E → X
may change discontinuously with respect to the parameter p. Perhaps one could
get a positive result by assuming that the stratification of E → B is suitably
compatible with the variable map fp : X → B. 

Recall (Def. 4.3) that a holomorphic map π : E → B satisfies POP if the

conclusion of Theorem 4.2 holds. We show that this is a local property.

Theorem 4.7. (Localization principle for POP) A holomorphic submersion π : E →

B of a complex space E onto a complex space B satisfies POP if and only if
every point x ∈ B admits an open neighborhood Ux ⊂ B such that the restricted
submersion π : E|Ux → Ux satisfies POP.

Proof. If π : E → B satisfies POP then clearly so does its restriction to any open
subset U of B.
Conversely, assume that B admits an open covering U = {Uα } by open
sets such that every restriction E|Uα → Uα enjoys POP. When proving POP for
π : E → B, a typical step amounts to choosing small compact sets A1 , . . . , An
in the source (Stein) space X such that, for a given compact set A0 ⊂ X, A =
(A0 , A1 , . . . , An ) is a Cartan string. We can choose the sets A1 , . . . , An sufficiently
small such that each map fp : X → B in the given family sends each Aj into one
of the sets Uα ∈ U.
To the string A we associate a K(A, Z; P, P0 )-complex F∗,∗ which is then
/n
inductively deformed into a holomorphic P -map F : j=0 Aj × P → E such that
π ◦ F = f . The main step in the inductive procedure amounts to patching a pair of
liftings over a Cartan pair (A , B  ) in X, where the set B  is contained in one of the
sets A1 , . . . , An in the Cartan string A. This is subdivided into substeps (1) and
(2) (see the proof of Theorem 4.2). Only the first of these substeps, which requires
a Runge-type approximation property, is a nontrivial condition on the submersion
E → B. It is immediate from the definitions that this approximation property
holds if there is an open set U ⊂ B containing the image fp (B  ) (for a certain set
of parameter values p ∈ P ) such that the restricted submersion E|U → U satisfies
POP. In our case this is so since we have insured that fp (B  ) ⊂ fp (Aj ) ⊂ Uα for
some j ∈ {1, . . . , n} and Uα ∈ U. 
 Invariance of the Parametric Oka Property 141

5. Ascent and descent of the parametric Oka property

In this section we prove Theorem 1.2 stated in Section 1.
Proof of (i): Assume that B enjoys POP (which is equivalent to PCAP). Let (K, Q)
be a special convex pair in Cn (Def. 3.1), and let F : Q × P → E be a (P, P0 )-map
that is holomorphic on K (Def. 4.1).
Then f = π ◦F : Q×P → B is a (P, P0 )-map that is holomorphic on K. Since
B enjoys POP, there is a holomorphic P -map g : Q × P → B that agrees with f
on Q × P0 and is uniformly close to f on a neighborhood of K × P in Cn × P .
If the latter approximation is close enough, there exists a holomorphic P -map
G : K × P → E such that π ◦ G = g, G approximates F on K × P , and G = F
on K × P0 . To find such lifting of g, we consider graphs of these maps (as in the
proof of Theorem 4.2) and apply a holomorphic retraction onto the fibers of π [10,
Proposition 3.2].
Since G = F on K × P0 , we can extend G to (K × P ) ∪ (Q × P0 ) by setting
G = F on Q × P0 .
Since π : E → B is a Serre fibration and K is a strong deformation retract of
Q (these sets are convex), G extends to a continuous (P, P0 )-map G : Q × P → E
such that π ◦ G = g. The extended map remains holomorphic on K.
By Theorem 4.2 there is a homotopy of liftings Gt : Q×P → E of g (t ∈ [0, 1])
which is fixed on Q × P0 and is holomorphic and uniformly close to G0 = G on
K × P . The holomorphic P -map G1 : Q × P → E then satisfies the condition in
Def. 3.2 relative to F . This proves that E enjoys PCAP and hence POP.
Proof of (ii): Assume that E enjoys POP. Let (K, Q) be a special convex pair, and
let f : Q × P → B be a (P, P0 )-map that is holomorphic on K. Assuming that P
is contractible, the Serre fibration property of π : E → B insures the existence of
a continuous P -map F : Q × P → E such that π ◦ F = f . (The subset P0 of P
does not play any role here.) Theorem 4.2 furnishes a homotopy F t : Q × P → E
(t ∈ [0, 1]) such that
(a) F 0 = F ,
(b) π ◦ F t = f for each t ∈ [0, 1], and
(c) F 1 is a (P, P0 )-map that is holomorphic on K.
This is accomplished in two steps: We initially apply Theorem 4.2 with Q × P0 to
obtain a homotopy F t : Q × P0 → E (t ∈ [0, 12 ]), satisfying properties (a) and (b)
1/2
above, such that Fp is holomorphic on Q for all p ∈ P0 . For trivial reasons this
homotopy extends continuously to all values p ∈ P . In the second step we apply
Theorem 4.2 over K × P , with F 1/2 as the initial lifting of f and keeping the
homotopy fixed for p ∈ P0 (where it is already holomorphic), to get a homotopy
F t (t ∈ [ 12 , 1]) such that π ◦ F t = f and Fp1 is holomorphic on K for all p ∈ P .
Since E enjoys POP, F 1 can be approximated uniformly on K × P by holo-
morphic P -maps
F : Q × P → E
such that F = F 1 on Q × P0 .
142 F. Forstnerič

Then
f = π ◦ F : Q × P → B
is a holomorphic P -map that agrees with f on Q × P0 and is close to f on K × P .
This shows that B enjoys PCAP for any contractible (compact, Hausdorff)
parameter space P and for any closed subspace P0 of P . Since the implication
PCAP=⇒POP in Theorem 3.3 holds for each specific pair (P0 , P ) of parameter
spaces, we infer that B also enjoys POP for such parameter pairs. This completes
the proof of (ii).
Proof of (iii): Contractibility of P was used in the proof of (ii) to lift the map
f : Q × P → B to a map F : Q × P → E. Such a lift exists for every topological
space if π : E → B is a weak homotopy equivalence. This is because a Serre
fibration between smooth manifolds is also a Hurewicz fibration (by Cauty [1]),
and a weak homotopy equivalence between them is a homotopy equivalence by the
Whitehead Lemma. 

Acknowledgement
I express my sincere thanks to Finnur Lárusson for his questions which led to this
paper, and for very helpful discussions and remarks. I also thank Petar Pavešić
and Dušan Repovš for advice on Tietze extension theorem used in the proof of
Proposition 4.4.

Added in proofs
Since the completion of this paper, the author gave a positive answer to the ques-
tion posed in Remark 1.4 for parameter spaces P0 ⊂ P that are compact sets in a
Euclidean space Rm (C. R. Acad. Sci. Paris, Ser. I 347, 1017–1020 (2009); C. R.
Acad. Sci. Paris, Ser. I (2009)).

References
[1] Cauty, R.: Sur les ouverts des CW-complexes et les fibrés de Serre, Colloq. Math.
63, 1–7 (1992)
[2] Demailly, J.-P., Cohomology of q-convex spaces in top degrees. Math. Z. 204, 283–295
(1990)
[3] Dowker, C.H.: Mapping theorems for non-compact spaces. Amer. J. Math. 69, 200–
242 (1947)
[4] Forstnerič, F.: The Oka principle for sections of subelliptic submersions. Math. Z.
241, 527–551 (2002)
[5] Forstnerič, F.: The homotopy principle in complex analysis: A survey. Explorations
in Complex and Riemannian Geometry: A Volume dedicated to Robert E. Greene,
pp. 73–99, Contemporary Mathematics, 332, American Mathematical Society, Prov-
idence, 2003.
[6] Forstnerič, F.: The Oka principle for multivalued sections of ramified mappings.
Forum Math. 15, 309–328 (2003)
 Invariance of the Parametric Oka Property 143

[7] Forstnerič, F.: Holomorphic flexibility properties of complex manifolds. Amer. J.

Math. 128, 239–270 (2006)
[8] Forstnerič, F.: Runge approximation on convex sets implies Oka’s property. Ann.
Math. (2) 163, 689–707 (2006)
[9] Forstnerič, F.: Extending holomorphic mappings from subvarieties in Stein mani-
folds. Ann. Inst. Fourier 55, 733–751 (2005)
[10] Forstnerič, F.: The Oka principle for sections of stratified fiber bundles. Pure Appl.
Math. Quarterly, 6, 843–874 (2010)
[11] Forstnerič, F., Prezelj, J.: Oka’s principle for holomorphic fiber bundles with sprays.
Math. Ann. 317, 117–154 (2000)
[12] Forstnerič, F., Prezelj, J.: Oka’s principle for holomorphic submersions with sprays.
Math. Ann. 322, 633–666 (2002)
[13] Forstnerič, F., Prezelj, J.: Extending holomorphic sections from complex subvarieties.
Math. Z. 236, 43–68 (2001)
[14] Grauert, H.: Holomorphe Funktionen mit Werten in komplexen Lieschen Gruppen.
Math. Ann. 133, 450–472 (1957)
[15] Grauert, H.: Analytische Faserungen über holomorph-vollständigen Räumen. Math.
Ann. 135, 263–273 (1958)
[16] Gromov, M.: Oka’s principle for holomorphic sections of elliptic bundles. J. Amer.
Math. Soc. 2, 851–897 (1989)
[17] Hörmander, L.: L2 estimates and existence theorems for the ∂¯ operator. Acta Math.
113 89–152 (1965)
[18] Hurewicz, W., Wallman, H.: Dimension Theory. Princeton Mathematical Series 4,
Princeton University Press, Princeton (1941)
[19] Ivarsson, B., Kutzschebauch, F.: A solution of Gromov’s Vaserstein problem. C. R.
Acad. Sci. Paris, Ser. I, 336 (2008)
[20] Ivarsson, B., Kutzschebauch, F.: Holomorphic factorization of maps into Sln (C).
Preprint (2008)
[21] Lárusson, F.: Excision for simplicial sheaves on the Stein site and Gromov’s Oka
principle. Internat. J. Math. 14, 191–209 (2003)
[22] Lárusson, F.: Model structures and the Oka principle. J. Pure Appl. Algebra 192,
203–223 (2004)
[23] Lárusson, F.: Mapping cylinders and the Oka principle. Indiana Univ. Math. J. 54,
1145–1159 (2005)
[24] Lárusson, F.: Applications of a parametric Oka principle for liftings. Preprint (2009)
[25] Lee, S.W. On the theory of selections. Honam Math. J. 19, 125–130 (1997)
[26] Ohsawa, T., Takegoshi, K.: On the extension of L2 holomorphic functions. Math. Z.
195, 197–204 (1987)
[27] Oka, K.: Sur les fonctions des plusieurs variables. III: Deuxième problème de Cousin.
J. Sc. Hiroshima Univ. 9, 7–19 (1939)
[28] Repovš, D., Semenov, P.V.: Continuous Selections of Multivalued Mappings. Math-
ematics and its Applications 455, Kluwer Academic Publishers, Dordrecht (1998)
[29] Siu, J.-T., Every Stein subvariety admits a Stein neighborhood. Invent. Math. 38,
89–100 (1976)
144 F. Forstnerič

[30] Spanier, E.H.: Algebraic topology. Corrected reprint. Springer-Verlag, New York-
Berlin (1981)
[31] Vaserstein, L.: Reduction of a matrix depending on parameters to a diagonal form
by addition operations. Proc. Amer. Math. Soc. 103, 741–746 (1988)
[32] Whitehead, G.W.: Elements of Homotopy Theory. Graduate Texts in Math. 61,
Springer-Verlag, Berlin (1978)

Franc Forstnerič
Faculty of Mathematics and Physics
University of Ljubljana
and Institute of Mathematics, Physics and Mechanics
Jadranska 19
1000 Ljubljana, Slovenia
e-mail: [email protected]
Complex Analysis
Trends in Mathematics, 145–158

c 2010 Springer Basel AG

¯
Positivity of the ∂-Neumann Laplacian
Siqi Fu
Dedicated to Professor Linda Rothschild

¯
Abstract. We study the ∂-Neumann Laplacian from spectral theoretic per-
spectives. In particular, we show how pseudoconvexity of a bounded domain
¯
is characterized by positivity of the ∂-Neumann Laplacian.
Mathematics Subject Classification (2000). 32W05.
¯
Keywords. Pseudoconvex, ∂-Neumann Laplacian, Dolbeault cohomology, L2 -
cohomology.

1. Introduction
Whether or not a given system has positive ground state energy is a widely studied
problem with significant repercussions in physics, particularly in quantum mechan-
ics. It follows from the classical Hardy inequality that the bottom of the spectrum
of the Dirichlet Laplacian on a domain in Rn that satisfies the outer cone con-
dition is positive if and only if its inradius is finite (see [D95]). Whereas spectral
behavior of the Dirichlet Laplacian is insensitive to boundary geometry, the story
¯
for the ∂-Neumann Laplacian is different. Since the work of Kohn [Ko63, Ko64]
and Hörmander [H65], it has been known that existence and regularity of the ∂- ¯
Neumann Laplacian closely depend on the underlying geometry (see the surveys
[BSt99, Ch99, DK99, FS01] and the monographs [CS99, St09]).
Let Ω be a domain in Cn . It follows from the classical Theorem B of Cartan
that if Ω is pseudoconvex, then the Dolbeault cohomology groups H 0,q (Ω) vanish
for all q ≥ 1. (More generally, for any coherent analytic sheaf F over a Stein man-
ifold, the sheaf cohomology groups H q (X, F) vanish for all q ≥ 1.) The converse is
also true ([Se53], p. 65). Cartan’s Theorem B and its converse were generalized by
Laufer [L66] and Siu [Siu67] to a Riemann domain over a Stein manifold. When Ω
is bounded, it follows from Hörmander’s L2 -existence theorem for the ∂-operator

The author was supported in part by NSF grant DMS-0805852.

146 S. Fu

that if Ω is in addition pseudoconvex, then the L2 -cohomology groups H  0,q (Ω)

vanish for q ≥ 1. The converse of Hörmander’s theorem also holds, under the as-
sumption that the interior of the closure of Ω is the domain itself. Sheaf theoretic
arguments for the Dolbeault cohomology groups can be modified to give a proof of
this fact (cf. [Se53, L66, Siu67, Br83, O88]; see also [Fu05] and Section 3 below).
In this expository paper, we study positivity of the ∂-Neumann Laplacian,
in connection with the above-mentioned classical results, through the lens of spec-
tral theory. Our emphasis is on the interplay between spectral behavior of the
∂-Neumann Laplacian and the geometry of the domains. This is evidently moti-
vated by Marc Kac’s famous question “Can one hear the shape of a drum?” [Ka66].
Here we are interested in determining the geometry of a domain in Cn from the
spectrum of the ∂-Neumann Laplacian. (See [Fu05, Fu08] for related results.) We
make an effort to present a more accessible and self-contained treatment, using
extensively spectral theoretic language but bypassing sheaf cohomology theory.

2. Preliminaries
¯
In this section, we review the spectral theoretic setup for the ∂-Neumann Lapla-
cian. The emphasis here is slightly different from the one in the extant literature
(cf. [FK72, CS99]). The ∂-Neumann Laplacian is defined through its associated
quadratic form. As such, the self-adjoint property and the domain of its square
root come out directly from the definition.
Let Q be a non-negative, densely defined, and closed sesquilinear form on
a complex Hilbert space H with domain D (Q). Then Q uniquely determines a
non-negative and self-adjoint operator S such that D (S 1/2 ) = D (Q) and
Q(u, v) = S 1/2 u, S 1/2 v

for all u, v ∈ D (Q). (See Theorem 4.4.2 in [D95], to which we refer the reader for
the necessary spectral theoretic background used in this paper.) For any subspace
L ⊂ D (Q), let λ(L) = sup{Q(u,u) | u ∈ L,u = 1}. For any positive integer j, let
λj (Q) = inf{λ(L) | L ⊂ D (Q), dim(L) = j}. (2.1)
The resolvent set ρ(S) of S consists of all λ ∈ C such that the operator S −
λI : D (S) → H is both one-to-one and onto (and hence has a bounded inverse
by the closed graph theorem). The spectrum σ(S), the complement of ρ(S) in C,
is a non-empty closed subset of [0, ∞). Its bottom inf σ(S) is given by λ1 (Q).
The essential spectrum σe (S) is a closed subset of σ(S) that consists of isolated
eigenvalues of infinite multiplicity and accumulation points of the spectrum. It is
empty if and only if λj (Q) → ∞ as j → ∞. In this case, λj (Q) is the j th eigen-
value of S, arranged in increasing order and repeated according to multiplicity.
The bottom of the essential spectrum inf σe (T ) is the limit of λj (Q) as j → ∞.
(When σe (S) = ∅, we set inf σe (S) = ∞.)
Let Tk : Hk → Hk+1 , k = 1, 2, be densely defined and closed operators on
Hilbert spaces. Assume that R (T1 ) ⊂ N (T2 ), where R and N denote the range
 ¯
Positivity of the ∂-Neumann Laplacian 147

and kernel of the operators. Let Tk∗ be the Hilbert space adjoint of Tk , defined in
the sense of Von Neumann by
D (Tk∗ ) = {u ∈ Hk+1 | ∃C > 0, | u, Tk v
| ≤ Cv, ∀v ∈ D (Tk )}
and
Tk∗ u, v
= u, Tk v
, for all u ∈ D (Tk∗ ) and v ∈ D (Tk ).
Then Tk∗ is also densely defined and closed. Let
Q(u, v) = T1∗ u, T1∗ v
+ T2 u, T2 v

with its domain given by D (Q) = D (T1∗ ) ∩ D (T2 ). The following proposition
elucidates the above approach to the ∂-Neumann Laplacian.
Proposition 2.1. Q(u, v) is a densely defined, closed, non-negative sesquilinear
form. The associated self-adjoint operator  is given by
D () = {f ∈ H2 | f ∈ D (Q), T2 f ∈ D (T2∗ ), T1∗ f ∈ D (T1 )},  = T1 T1∗ + T2∗ T2 .
(2.2)
Proof. The closedness of Q follows easily from that of T1 and T2 . The non-
negativity is evident. We now prove that D (Q) is dense in H2 . Since N (T2 )⊥ =
R (T2∗ ) ⊂ N (T1∗ ) and
 
D (T2 ) = N (T2 ) ⊕ D (T2 ) ∩ N (T2 )⊥ ,
we have
   
D (Q) = D (T1∗ ) ∩ D (T2 ) = N (T2 ) ∩ D (T1∗ ) ⊕ D (T2 ) ∩ N (T2 )⊥ .
Since D (T1∗ ) and D (T2 ) are dense in H2 , D (Q) is dense in N (T2 )⊕N (T2 )⊥ = H2 .
It follows from the above definition of  that f ∈ D () if and only if f ∈
D (Q) and there exists a g ∈ H2 such that
Q(u, f ) = u, g
, for all u ∈ D (Q) (2.3)
(cf. Lemma 4.4.1 in [D95]). Thus
D () ⊃ {f ∈ H2 | f ∈ D (Q), T2 f ∈ D (T2∗ ), T1∗ f ∈ D (T1 )}.
We now prove the opposite containment. Suppose f ∈ D (). For any u ∈ D (T2 ),
we write u = u1 + u2 ∈ (N (T1∗ ) ∩ D (T2 )) ⊕ N (T1∗ )⊥ . Note that N (T1∗ )⊥ ⊂
R (T2∗ )⊥ = N (T2 ). It follows from (2.3) that
| T2 u, T2 f
| = | T2 u1 , T2 f
= |Q(u1 , f )| = | u1 , g
| ≤ u · g.
Hence T2 f ∈ D (T2∗ ). The proof of T1∗ f ∈ D (T1 ) is similar. For any w ∈ D (T1∗ ), we
write w = w1 + w2 ∈ (N (T2 ) ∩ D (T1∗ )) ⊕ N (T2 )⊥ . Note that N (T2 )⊥ = R (T2∗ ) ⊂
N (T1∗ ). Therefore, by (2.3),
| T1∗ w, T1∗ f
| = | T1∗ w1 , T1∗ f
= |Q(w1 , f )| = | w1 , g
| ≤ w · g.
148 S. Fu

Hence T1∗ f ∈ D (T1∗∗ ) = D (T1 ). It follows from the definition of  that for any
f ∈ D () and u ∈ D (Q),
f, u
= 1/2 f, 1/2 u
= Q(f, u)
= T1∗ f, T1∗ u
+ T2 f, T2 u
= (T1 T1∗ + T2∗ T2 )f, u
.
Hence  = T1 T1∗ + T2∗ T2 . 
The following proposition is well known (compare [H65], Theorem 1.1.2 and
Theorem 1.1.4; [C83], Proposition 3; and [Sh92], Proposition 2.3). We provide a
proof here for completeness.
Proposition 2.2. inf σ() > 0 if and only if R (T1 ) = N (T2 ) and R (T2 ) is closed.
Proof. Assume inf σ() > 0. Then 0 is in the resolvent set of  and hence  has a
bounded inverse G : H2 → D (). For any u ∈ H2 , write u = T1 T1∗ Gu + T2∗T2 Gu. If
u ∈ N (T2 ), then 0 = (T2 u, T2 Gu) = (T2 T2∗ T2 Gu, T2 Gu) = (T2∗ T2 Gu, T2∗ T2 Gu).
Hence T2∗ T2 Gu = 0 and u = T1 T1∗ Gu. Therefore, R (T1 ) = N (T2 ). Similarly,
R (T2∗ ) = N (T1∗ ). Therefore T2∗ and hence T2 have closed range. To prove the
opposite implication, we write u = u1 + u2 ∈ N (T2 ) ⊕ N (T2 )⊥ , for any u ∈ D (Q).
Note that u1 , u2 ∈ D (Q). It follows from N (T2 ) = R (T1 ) and the closed range
property of T2 that there exists a positive constant c such that cu1 2 ≤ T1∗u1 2
and cu2 2 ≤ T2 u2 2 . Thus
cu2 = c(u1 2 + u2 2 ) ≤ T1∗ u1 2 + T2 u2 2 = Q(u, u).
Hence inf σ() ≥ c > 0 (cf. Theorem 4.3.1 in [D95]). 
Let N (Q) = N (T1∗ ) ∩ N (T2 ). Note that when it is non-trivial, N (Q) is the
eigenspace of the zero eigenvalue of . When R (T1 ) is closed, N (T2 ) = R (T1 ) ⊕
N (Q). For a subspace L ⊆ H2 , denote by PL⊥ the orthogonal projection onto
L⊥ and T2 |L⊥ the restriction of T2 to L⊥ . The next proposition clarifies and
strengthens the second part of Lemma 2.1 in [Fu05].
Proposition 2.3. The following statements are equivalent:
1. inf σe () > 0.
2. R (T1 ) and R (T2 ) are closed and N (Q) is finite dimensional.
3. There exists a finite-dimensional subspace L ⊂ D (T1∗ ) ∩ N (T2 ) such that
N (T2 ) ∩ L⊥ = PL⊥ (R (T1 )) and R (T2 |L⊥ ) is closed.
Proof. We first prove (1) implies (2). Suppose a = inf σe () > 0. If inf σ() > 0,
then N (Q) is trivial and (2) follows from Proposition 2.2. Suppose inf σ() = 0.
Then σ() ∩ [0, a) consists only of isolated points, all of which are eigenvalues of
finite multiplicity of  (cf. Theorem 4.5.2 in [D95]). Hence N (Q), the eigenspace
of the eigenvalue 0, is finite dimensional. Choose a sufficiently small c > 0 so that
σ()∩[0, c) = {0}. By the spectral theorem for self-adjoint operators (cf. Theorem
2.5.1 in [D95]), there exists a finite regular Borel measure μ on σ() × N and a
unitary transformation U : H2 → L2 (σ()×N, dμ) such that U U −1 = Mx , where
 ¯
Positivity of the ∂-Neumann Laplacian 149

Mx ϕ(x, n) = xϕ(x, n) is the multiplication operator by x on L2 (σ() × N, dμ). Let

PN (Q) be the orthogonal projection onto N (Q). For any f ∈ D (Q) ∩ N (Q)⊥ ,
U PN (Q) f = χ[0,c) U f = 0,
where χ[0,c) is the characteristic function of [0, c). Hence U f is supported on [c, ∞).
Therefore, 
Q(f, f ) = x|U f |2 dμ ≥ cU f 2 = cf 2 .
σ()×N
It then follows from Theorem 1.1.2 in [H65] that both T1 and T2 have closed range.
To prove (2) implies (1), we use Theorem 1.1.2 in [H65] in the opposite
direction: There exists a positive constant c such that
cf 2 ≤ Q(f, f ), for all f ∈ D (Q) ∩ N (Q)⊥ . (2.4)
Proving by contradiction, we assume inf σe () = 0. Let ε be any positive number
less than c. Since L[0,ε) = R (χ[0,ε) ()) is infinite dimensional (cf. Lemma 4.1.4 in
[D95]), there exists a non-zero g ∈ L[0,ε) such that g ⊥ N (Q). However,

Q(g, g) = xχ[0,ε) |U g|2 dμ ≤ εU g2 = εg2,
σ()×N

contradicting (2.4).
We do some preparations before proving the equivalence of (3) with (1) and
∗ 
(2). Let L be# any finite-dimensional # subspace of D (T1 )∩N (T2 ). Let H2 = H2 %L.
Let T2 = T2 #H  and let T1∗  = T1∗ #H  . Then T2 : H2 → H3 and T1∗  : H2 → H1 are
2 2
densely defined, closed operators. Let T1 : H1 → H2 be the adjoint of T1∗  . It
follows from the definitions that D (T1 ) ⊂ D (T1 ). The finite dimensionality of L
implies the opposite containment. Thus, D (T1 ) = D (T1 ). For any f ∈ D (T1 ) and
g ∈ D (T1∗  ) = D (T1∗ ) ∩ L⊥ ,
T1 f, g
= f, T1∗  g
= f, T1∗ g
= T1 f, g
.
Hence T1 = PL⊥ ◦ T1 and R (T1 ) = PL⊥ (R (T1 )) ⊂ N (T2 ). Let
∗ ∗
Q (f, g) = T1 f, T1 g
+ T2 f, T2 g

be the associated sesquilinear form on H2 with D (Q ) = D (Q) ∩ L⊥ .

We are now in position to prove that (2) implies (3). In this case, we can take
L = N (Q). By Theorem 1.1.2 in [H65], there exists a positive constant c such that
Q(f, f ) = Q (f, f ) ≥ cf 2 , for all f ∈ D (Q ).
We then obtain (3) by applying Proposition 2.2 to T1 , T2 , and Q (f, g).
Finally, we prove (3) implies (1). Applying Proposition 2.2 in the opposite
direction, we know that there exists a positive constant c such that
Q(f, f ) ≥ cf 2 , for all f ∈ D (Q) ∩ L⊥ .
The rest of the proof follows the same lines of the above proof of the implication
(2) ⇒ (1), with N (Q) there replaced by L. 
150 S. Fu

We now recall the definition of the ∂-Neumann Laplacian on a complex man-

∞
ifold. Let X be a complex hermitian manifold of dimension n. Let C(0,q) (X) =
∞ 0,q ∗ ∞
C (X, Λ T X) be the space of smooth (0, q)-forms on X. Let ∂ q : C(0,q) (X) →
∞
C(0,q+1) (X) be the composition of the exterior differential operator and the pro-
∞
jection onto C(0,q+1) (X).
∞
Let Ω be a domain in X. For u, v ∈ C(0,q) (X), let u, v
be the point-wise
inner product of u and v, and let

u, v

Ω = u, v
dV
Ω

be the inner product of u and v over Ω. Let L2(0,q) (Ω) be the completion of the
∞
space of compactly supported forms in C(0,q) (Ω) with respect to the above inner
product. The operator ∂ q has a closed extension on L2(0,q) (Ω). We also denote the
closure by ∂ q . Thus ∂ q : L2(0,q) (Ω) → L2(0,q+1) (Ω) is densely defined and closed. Let
∗
∂ q be its adjoint. For 1 ≤ q ≤ n − 1, let
∗ ∗
Qq (u, v) = ∂ q u, ∂ q v

Ω + ∂ q−1 u, ∂ q−1 v

Ω
∗
be the sesquilinear form on L2(0,q) (Ω) with domain D (Qq ) = D (∂ q )∩D (∂ q−1 ). The
self-adjoint operator q associated with Qq is called the ∂-Neumann Laplacian on
L2(0,q) (Ω). It is an elliptic operator with non-coercive boundary conditions [KN65].
The Dolbeault and L2 -cohomology groups on Ω are defined respectively by
∞
{f ∈ C(0,q) (Ω) | ∂ q f = 0} {f ∈ L2(0,q) (Ω) | ∂ q f = 0}
H 0,q (Ω) =  0,q (Ω) =
and H .
∞
{∂ q−1 g | g ∈ C(0,q−1) (Ω)} {∂ q−1 g | g ∈ L2(0,q−1) (Ω)}
These cohomology groups are in general not isomorphic. For example, when a
complex variety is deleted from Ω, the L2 -cohomology group remains the same but
the Dolbeault cohomology group could change from trivial to infinite-dimensional.
As noted in the paragraph preceding Proposition 2.3, when R (∂ q−1 ) is closed in
 0,q (Ω) ∼
L2(0,q) (Ω), H = N (q ). We refer the reader to [De] for an extensive treatise
on the subject and to [H65] and [O82] for results relating these cohomology groups.

3. Positivity of the spectrum and essential spectrum

Laufer proved in [L75] that for any open subset of a Stein manifold, if a Dolbeault
cohomology group is finite dimensional, then it is trivial. In this section, we es-
tablish the following L2 -analogue of this result on a bounded domain in a Stein
manifold:
Theorem 3.1. Let Ω ⊂⊂ X be a domain in a Stein manifold X with C 1 boundary.
Let q , 1 ≤ q ≤ n − 1, be the ∂-Neumann Laplacian on L2(0,q) (Ω). Assume that
N (q ) ⊂ W 1 (Ω). Then inf σ(q ) > 0 if and only if inf σe (q ) > 0.
 ¯
Positivity of the ∂-Neumann Laplacian 151

The proof of Theorem 3.1 follows the same line of arguments as Laufer’s. We
provide the details below.
Let H ∞ (Ω) be the space of bounded holomorphic functions on Ω. For any
f ∈ H ∞ (Ω), let Mf be the multiplication operator by f :
Mf : L2(0,q) (Ω) → L2(0,q) (Ω), Mf (u) = f u.

Then Mf induces an endomorphism on H  0,q (Ω). Let I be set of all holomorphic

∞
functions f ∈ H (Ω) such that Mf = 0 on H  0,q (Ω). Evidently, I is an ideal of
H ∞ (Ω). Assume inf σe (q ) > 0. To show that H  0,q (Ω) is trivial, it suffices to show
that 1 ∈ I.
Lemma 3.2. Let ξ be a holomorphic vector field on X and let f ∈ I. Then ξ(f ) ∈ I.
∞ ∞
Proof. Let D = ξ∂ : C(0,q) (Ω) → C(0,q) (Ω), where  denotes the contraction
operator. It is easy to check that D commute with the ∂ operator. Therefore,
D induces an endomorphism on H  0,q (Ω). (Recall that under the assumption,
 ∼
H (Ω) = N (q ) ⊂ W (Ω).) For any u ∈ N (q ),
0,q 1

D(f u) − f D(u) = ξ∂(f u) − f ξ∂u = ξ(f )u.

Notice that Ω is locally starlike near the boundary. Using partition of unity and
the Friedrichs Lemma, we obtain [D(f u)] = 0. Therefore, [ξ(f )u] = [D(f u)] −
[f D(u)] = [0]. 
We now return to the proof of the theorem. Let F = (f1 , . . . , fn+1 ) : X →
C2n+1 be a proper embedding of X into C2n+1 (cf. Theorem 5.3.9 in [H91]). Since
Ω is relatively compact in X, fj ∈ H ∞ (Ω). For any fj , let Pj (λ) be the character-
istic polynomial of Mfj : H  0,q (Ω) → H  0,q (Ω). By the Cayley-Hamilton theorem,
Pj (Mfj ) = 0 (cf. Theorem 2.4.2 in [HJ85]). Thus Pj (fj ) ∈ I.
The number of points in the set {(λ1 , λ2 , . . . , λ2n+1 ) ∈ C2n+1 | Pj (λj ) =
0, 1 ≤ j ≤ 2n + 1} is finite. Since F : X → C2n+1 is one-to-one, the number of
common zeroes of Pj (fj (z)), 1 ≤ j ≤ 2n+1, on X is also finite. Denote these zeroes
by z k , 1 ≤ k ≤ N . For each z k , let gk be a function in I whose vanishing order
at z k is minimal. (Since Pj (fj ) ∈ I, gk ≡ 0.) We claim that gk (z k ) = 0. Suppose
otherwise gk (z k ) = 0. Since there exists a holomorphic vector field ξ on X with
any prescribed holomorphic tangent vector at any given point (cf. Corollary 5.6.3
in [H91]), one can find an appropriate choice of ξ so that ξ(gj ) vanishes to lower
order at z k . According to Lemma 3.2, ξ(gj ) ∈ I. We thus arrive at a contradiction.
Now we know that there are holomorphic functions, Pj (fj ), 1 ≤ j ≤ 2n + 1,
and gk , 1 ≤ k ≤ N , that have no common zeroes on X. It then follows that there
exist holomorphic functions hj on X such that

Pj (fj )hj + gk hk = 1.
(See, for example, Corollary 16 on p. 244 in [GR65], Theorem 7.2.9 in [H91],
and Theorem 7.2.5 in [Kr01]. Compare also Theorem 2 in [Sk72].) Since Pj (fj ) ∈
152 S. Fu

I, gk ∈ I, and hj ∈ H ∞ (Ω), we have 1 ∈ I. We thus conclude the proof of

Theorem 3.1.
Remark. (1) Unlike the above-mentioned result of Laufer on the Dolbeault co-
homology groups [L75], Theorem 3.1 is not expected to hold if the boundedness
condition on Ω is removed (compare [W84]). It would be interesting to know
whether Theorem 3.1 remains true if the assumption N (q ) ⊂ W 1 (Ω) is dropped
and whether it remains true for unbounded pseudoconvex domains.
(2) Notice that in the above proof, we use the fact that R (∂ q−1 ) is closed,
as a consequence of the assumption inf σe (q ) > 0 by Proposition 2.3. It is well
known that for any infinite-dimensional Hilbert space H, there exists a subspace
R of H such that H/R is finite dimensional but R is not closed. However, the
construction of such a subspace usually involves Zorn’s lemma (equivalently, the
axiom of choice). It would be of interest to know whether there exists a domain
Ω in a Stein manifold such that H 0,q (Ω) is finite dimensional but R (∂ q−1 ) is not
closed.
(3) We refer the reader to [Sh09] for related results on the relationship be-
tween triviality and finite dimensionality of the L2 -cohomology groups using the
∂-Cauchy problem. We also refer the reader to [B02] for a related result on em-
bedded CR manifolds.

4. Hearing pseudoconvexity
The following theorem illustrates that one can easily determine pseudoconvexity
from the spectrum of the ∂-Neumann Laplacian.
Theorem 4.1. Let Ω be a bounded domain in Cn such that int (cl (Ω)) = Ω. Then
the following statements are equivalent:
1. Ω is pseudoconvex.
2. inf σ(q ) > 0, for all 1 ≤ q ≤ n − 1.
3. inf σe (q ) > 0, for all 1 ≤ q ≤ n − 1.
The implication (1) ⇒ (2) is a consequence of Hörmander’s fundamental L2 -
estimates of the ∂-operator [H65], in light of Proposition 2.2, and it holds with-
out the assumption int (cl (Ω)) = Ω. The implications (2) ⇒ (1) and (3) ⇒ (1)
are consequences of the sheaf cohomology theory dated back to Oka and Cartan
(cf. [Se53, L66, Siu67, Br83, O88]). A elementary proof of (2) implying (1), as
explained in [Fu05], is given below. The proof uses sheaf cohomology arguments
in [L66]. When adapting Laufer’s method to study the L2 -cohomology groups,
one encounters a difficulty: While the restriction to the complex hyperplane of
the smooth function resulting from the sheaf cohomology arguments for the Dol-
beault cohomology groups is well defined, the restriction of the corresponding L2
function is not. This difficulty was overcome in [Fu05] by appropriately modifying
the construction of auxiliary (0, q)-forms (see the remark after the proof for more
elaborations on this point).
 ¯
Positivity of the ∂-Neumann Laplacian 153

We now show that (2) implies (1). Proving by contradiction, we assume that
Ω is not pseudoconvex. Then there exists a domain Ω   Ω such that every holo-

morphic function on Ω extends to Ω. Since int (cl (Ω)) = Ω, Ω\  cl (Ω) is non-empty.
After a translation and a unitary transformation, we may assume that the origin
 \ cl (Ω) and there is a point z 0 in the intersection of zn -plane with Ω that
is in Ω
is in the same connected component of the intersection of the zn -plane with Ω. 
Let m be a positive integer (to be specified later). Let kq = n. For any
{k1 , . . . , kq−1 } ⊂ {1, 2, . . . , n − 1}, we define

(q − 1)!(z̄k1 · · · z̄kq )m−1

q
u(k1 , . . . , kq ) = q

(−1)j z̄kj dz̄k1 ∧ · · · ∧ dz̄ kj ∧ · · · ∧ dz̄kq ,
rm
j=1
(4.1)
where rm = |z1 |2m + · · · + |zn |2m . As usual, dz̄ kj indicates the deletion of dz̄kj
from the wedge product. Evidently, u(k1 , . . . , kq ) ∈ L2(0,q−1) (Ω) is a smooth form
on Cn \ {0}. Moreover, u(k1 , . . . , kq ) is skew-symmetric with respect to the indices
(k1 , . . . , kq−1 ). In particular, u(k1 , . . . , kq ) = 0 when two kj ’s are identical.
We now fix some notional conventions. Let K = (k1 , . . . , kq ) and J a collection
of indices from {k1 , . . . , kq }. Write dz̄K = dz̄k1 ∧· · ·∧dz̄kq , z̄K
m−1
= (z̄k1 · · · z̄kq )m−1 ,
and dz̄ " 
kj = dz̄k1 ∧ · · · ∧ dz̄kj ∧ . . . ∧ dz̄kq . Denote by (k1 , . . . , kq | J) the tu-
ple of remaining indices after deleting those in J from (k1 , . . . , kq ). For example,
(2, 5, 3, 1 | (4, 1, 6 | 4, 6)) = (2, 5, 3).
It follows from a straightforward calculation that
m−1 
q!mz̄K 
n
 
q

∂u(k1 , . . . , kq ) = − rm dz̄K + z̄m−1 zm dz̄ ∧ "
(−1)j z̄kj dz̄ kj
q+1
rm =1 j=1
m−1
q!mz̄K
n

q

= − zm z̄m−1 z̄ dz̄K + dz̄ ∧ "
(−1)j z̄kj dz̄ kj
q+1
rm =1 j=1
=k1 ,...,kq

n−1
=m zm u(, k1 , . . . , kq ). (4.2)
=1

It follows that u(1, . . . , n) is a ∂-closed (0, n − 1)-form.

By Proposition 2.2, we have R (∂ q−1 ) = N (∂ q ) for all 1 ≤ q ≤ n − 1.
We now solve the ∂-equations inductively, using u(1, . . . , n) as initial data. Let
v ∈ L2(0,n−2) (Ω) be a solution to ∂v = u(1, . . . , n). For any k1 ∈ {1, . . . , n − 1},
define
w(k1 ) = −mzkm1 v + (−1)1+k1 u(1, . . . , n | k1 ).

Then it follows from (4.2) that ∂w(k1 ) = 0. Let v(k1 ) ∈ L2(0,n−3) (Ω) be a solution
of ∂v(k1 ) = w(k1 ).
154 S. Fu

Suppose for any (q − 1)-tuple K  = (k1 , . . . , kq−1 ) of integers from {1, . . . , n −

1}, q ≥ 2, we have constructed v(K  ) ∈ L2(0,n−q−1) (Ω) such that it is skew-
symmetric with respect to the indices and satisfies

q−1


∂v(K ) = m (−1)j zkmj v(K  | kj ) + (−1)q+|K | u(1, . . . , n | K  ) (4.3)
j=1

where |K  | = k1 + · · · + kq−1 as usual. We now construct a (0, n − q − 2)-forms

v(K) satisfying (4.3) for any q-tuple K = (k1 , . . . , kq ) of integers from {1, . . . , n−1}
(with K  replaced by K). Let

q
w(K) = m (−1)j zkmj v(K | kj ) + (−1)q+|K| u(1, . . . , n | K).
j=1

Then it follows from (4.2) that

∂w(K)

q
=m (−1)j zkmj ∂v(K | kj ) + (−1)q+|K| ∂u(1, . . . , n | K)
j=1

q 
=m (−1)j zkmj m (−1)i zkmi v(K | kj , ki ) + m (−1)i−1 zkmi v(K | kj , ki )
j=1 1≤i<j j<i≤q

− (−1) q+|K|−kj
u(1, . . . , n | (K | kj )) + (−1)q+|K| ∂u(1, . . . , n | K)

q

= (−1)q+|K| − m (−1)j−kj zkmj u(1, . . . , n | (K | kj )) + ∂u(1, . . . , n | K)
j=1


q

= (−1)q+|K| − m zkmj u(kj , (1, . . . , n | K)) + ∂u(1, . . . , n | K) = 0.
j=1

Therefore, by the hypothesis, there exists a v(K) ∈ L2(0,n−q−2) (Ω) such that
∂v(K) = w(K). Since w(K) is skew-symmetric with respect to indices K, we
may also choose a likewise v(K). This then concludes the inductive step.
Now let

n−1
n(n−1)
F = w(1, . . . , n − 1) = m zjm v(1, . . . , j, . . . , n − 1) − (−1)n+ 2 u(n),
j=1

where u(n) = −z̄nm /rm , as given by (4.1). Then F (z) ∈ L2 (Ω) and ∂F (z) = 0.
By the hypothesis, F (z) has a holomorphic extension to Ω.  We now restrict F (z)

to the coordinate hyperplane z = (z1 , . . . , zn−1 ) = 0. Notice that so far we only
choose the v(K)’s and w(K)’s from L2 -spaces. The restriction to the coordinate
hyperplane z  = 0 is not well defined.
 ¯
Positivity of the ∂-Neumann Laplacian 155

To overcome this difficulty, we choose m > 2(n − 1). For sufficiently small
ε > 0 and δ > 0,
' 01/2
# n(n−1)  #
# F + (−1)n+ 2 u(n) (δz  , zn ))# dV (z)
2

{|z  |<ε}∩Ω
' 01/2

n−1
≤ mδ m εm |v(1, . . . , ĵ, . . . , n − 1)(δz  , zn )|2 dV (z)
j=1 {|z  |<ε}∩Ω

n−1
≤ mδ m−2(n−1) εm v(1, . . . , ĵ, . . . , n − 1)L2 (Ω) .
j=1

Letting δ → 0, we then obtain

n(n−1) n(n−1)
F (0, zn ) = −(−1)n+ 2 u(n)(0, zn) = (−1)n+ 2 zn−m .
for zn near zn0 . (Recall that z 0 ∈ Ω is in the same connected component of {z  =
0} ∩ Ω as the origin.) This contradicts the analyticity of F near the origin. We
therefore conclude the proof of Theorem 4.1.
Remark. (1) The above proof of the implication (2) ⇒ (1) uses only the fact that
the L2 -cohomology groups H  0,q (Ω) are trivial for all 1 ≤ q ≤ n−1. Under the (pos-
sibly) stronger assumption inf σ(q ) > 0, 1 ≤ q ≤ n − 1, the difficulty regarding
the restriction of the L2 function to the complex hyperplane in the proof becomes
superficial. In this case, the ∂-Neumannn Laplacian q has a bounded inverse. The
interior ellipticity of the ∂-complex implies that one can in fact choose the forms
v(K) and w(K) to be smooth inside Ω, using the canonical solution operator to
the ∂-equation. Therefore, in this case, the restriction to {z  = 0} ∩ Ω is well de-
fined. Hence one can choose m = 1. This was indeed the choice in [L66], where the
forms involved are smooth and the restriction posts no problem. It is interesting
to note that by having the freedom to choose m sufficiently large, one can leave
out the use of interior ellipticity. Also, the freedom to choose m becomes crucial
when one proves an analogue of Theorem 4.1 for the Kohn Laplacian because the
∂ b -complex is no longer elliptic. The construction of u(k1 , . . . , kq ) in (4.1) with the
exponent m was introduced in [Fu05] to handle this difficulty.
(2) One can similarly give a proof of the implication (3) ⇒ (1). Indeed,
the above proof can be easily modified to show that the finite dimensionality of
H 0,q (Ω), 1 ≤ q ≤ n − 1, implies the pseudoconvexity of Ω. In this case, the u(K)’s
are defined by
(α + q − 1)!z̄nmα (z̄k1 · · · z̄kq )m−1
u(k1 , . . . , kq ) = α+q
rm

q
× 
(−1)j z̄kj dz̄k1 ∧ · · · ∧ dz̄ kj ∧ · · · ∧ dz̄kq ,
j=1
156 S. Fu

where α is any non-negative integers. One now fixes a choice of m > 2(n − 1) and
let α runs from 0 to N for a sufficiently large N , depending on the dimensions of
the L2 -cohomology groups. We refer the reader to [Fu05] for details.
(3) As noted in Sections 2 and 3, unlike the Dolbeault cohomology case, one
cannot remove the assumption int (cl (Ω)) = Ω or the boundedness condition on
Ω from Theorem 4.1. For example, a bounded pseudoconvex domain in Cn with a
complex analytic variety removed still satisfies condition (2) in Theorem 3.1.
(4) As in [L66], Theorem 4.1 remains true for a Stein manifold. More gener-
ally, as a consequence of Andreotti-Grauert’s theory [AG62], the q-convexity of a
bounded domain Ω in a Stein manifold such that int (cl (Ω)) = Ω is characterized
by inf σ(k ) > 0 or inf σe (k ) > 0 for all q ≤ k ≤ n − 1.
(5) It follows from Theorem 3.1 in [H04] that for a domain Ω in a complex
hermitian manifold of dimension n, if inf σe (q ) > 0 for some q between 1 and
n−1, then wherever the boundary is C 3 -smooth, its Levi-form cannot have exactly
n − q − 1 positive and q negative eigenvalues. A complete characterization of a
domain in a complex hermitian manifold, in fact, even in Cn , that has inf σe (q ) >
0 or inf σ(q ) > 0 is unknown.

Acknowledgment
We thank Professor Yum-Tong Siu for drawing our attention to the work of Laufer
[L66], by which our work was inspired. We also thank Professors Mei-Chi Shaw
and Emil Straube, and the referee for stimulating discussions and constructive
suggestions.

References
[AG62] Aldo Andreotti and Hans Grauert, Théorème de finitude pour la cohomologie des
espaces complexes, Bull. Soc. Math. France 90 (1962), 193–259.
[BSt99] Harold P. Boas and Emil J. Straube, Global regularity of the ∂-Neumann problem:
a survey of the L2 -Sobolev theory, Several Complex Variables (M. Schneider and
Y.-T. Siu, eds.), MSRI Publications, vol. 37, 79–112, 1999.
[B02] Judith Brinkschulte, Laufer’s vanishing theorem for embedded CR manifolds,
Math. Z. 239(2002), 863–866.
[Br83] Thorsten Broecker, Zur L2 -Kohomologie beschränkter Gebiete, Bonner Mathe-
matische Schriften, vol. 145, Universität Bonn, 1983.
[C83] David Catlin, Necessary conditions for subellipticity of the ∂-Neumann problem,
Ann. Math. 117 (1983), 147–171.
[CS99] So-Chin Chen and Mei-Chi Shaw, Partial differential equations in several complex
variables, AMS/IP, 2000.
[Ch99] Michael Christ, Remarks on global irregularity in the ∂-Neumann problem, Sev-
eral Complex Variables (M. Schneider and Y.-T. Siu, eds.), MSRI Publications,
vol. 37, 161–198, 1999.
 ¯
Positivity of the ∂-Neumann Laplacian 157

[DK99] J. D’Angelo and J.J. Kohn, Subelliptic estimates and finite type, Several Complex
Variables (M. Schneider and Y.-T. Siu, eds.), MSRI Publications, vol. 37, 199–
232, 1999.
[D95] E.B. Davies, Spectral theory and differential operators, Cambridge Studies in
advanced mathematics, vol. 42, Cambridge University Press, 1995.
[De] J.-P. Demailly, Complex analytic and algebraic geometry, book available at
http://www-fourier.ujf-grenoble.fr/demailly/books.html.
[FK72] G.B. Folland and J.J. Kohn, The Neumann problem for the Cauchy-Riemann
complex, Annals of Mathematics Studies, no. 75, Princeton University Press,
1972.
[Fu05] Siqi Fu, Hearing pseudoconvexity with the Kohn Laplacian, Math. Ann. 331
(2005), 475–485.
[Fu08] , Hearing the type of a domain in C2 with the d-bar-Neumann Laplacian,
Adv. in Math. 219 (2008), 568–603.
[FS01] Siqi Fu and Emil J. Straube, Compactness in the ∂-Neumann problem, Complex
Analysis and Geometry, Proceedings of Ohio State University Conference, vol. 9,
141–160, Walter De Gruyter, 2001.
[GR65] Robert C. Gunning and Hugo Rossi, Analytic functions of several complex vari-
ables, Prentice-Hall, Englewood Cliffs, New Jersey, 1965.
[H65] Lars Hörmander, L2 estimates and existence theorems for the ∂ operator, Acta
Math. 113 (1965), 89–152.
[H91] , An introduction to complex analysis in several variables, third ed., El-
sevier Science Publishing, 1991.
[H04] ¯
, The null space of the ∂-Neumann operator, Ann. Inst. Fourier (Grenoble)
54 (2004), 1305–1369.
[HJ85] Roger A. Horn and Charles R. Johnson, Matrix analysis, Cambridge University
Press, 1985.
[Ka66] Marc Kac, Can one hear the shape of a drum? Amer. Math. Monthly 73 (1966),
1–23.
[Ko63] J.J. Kohn, Harmonic integrals on strongly pseudo-convex manifolds, I, Ann.
Math. 78 (1963), 112–148.
[Ko64] , Harmonic integrals on strongly pseudo-convex manifolds, II, Ann. Math.
79 (1964), 450–472.
[KN65] J.J. Kohn and L. Nirenberg, Non-coercive boundary value problems, Comm. Pure
Appl. Math. 18 (1965), 443–492.
[Kr01] Steven G. Krantz, Function theory of several complex variables, AMS Chelsea
Publishing, Providence, RI, 2001.
[L66] Henry B. Laufer, On sheaf cohomology and envelopes of holomorphy, Ann. Math.
84 (1966), 102–118.
[L75] , On the finite dimensionality of the Dolbeault cohomology groups, Proc.
Amer. Math. Soc. 52(1975), 293–296.
[O82] T. Ohsawa, Isomorphism theorems for cohomology groups of weakly 1-complete
manifolds, Publ. Res. Inst. Math. Sci. 18 (1982), no. 1, 191–232.
158 S. Fu

[O88] Takeo Ohsawa, Complete Kähler manifolds and function theory of several com-
plex variables, Sugaku Expositions 1 (1988), 75–93.
[Se53] J.-P. Serre, Quelques problèmes globaux relatifs aux variétés de Stein, Colloque
sur les Fonctions de Plusieurs Variables, 57–68, Brussels, 1953.
[Sh92] Mei-Chi Shaw, Local existence theorems with estimates for ∂ b on weakly pseudo-
convex CR manifolds, Math. Ann. 294 (1992), no. 4, 677–700.
[Sh09] , The closed range property for ∂¯ on domains with pseudoconcave bound-
ary, preprint, 2009, 307–320 in this volume.
[Siu67] Yum-Tong Siu, Non-countable dimensions of cohomology groups of analytic
sheaves and domains of holomorphy, Math. Z. 102 (1967), 17–29.
[Sk72] Henri Skoda, Application des techniques L2 à la théorie des idéaux d’une algèbre
de fonctions holomorphes avec poids, Ann. Sci. École Norm. Sup. (4) 5 (1972),
545–579.
[St09] Emil Straube, Lectures on the L2 -Sobolev theory of the ∂-Neumann problem, to
appear in the series ESI Lectures in Mathematics and Physics, European Math.
Soc.
[W84] Jan Wiegerinck, Domains with finite-dimensional Bergman space, Math. Z. 187
(1984), 559–562.

Siqi Fu
Department of Mathematical Sciences
Rutgers University-Camden
Camden, NJ 08102, USA
e-mail: [email protected]
Complex Analysis
Trends in Mathematics, 159–174

c 2010 Springer Basel AG

Compactness Estimates for the ∂-Neumann

Problem in Weighted L2 -spaces
Klaus Gansberger and Friedrich Haslinger
Dedicated to Linda Rothschild

Abstract. In this paper we discuss compactness estimates for the ∂-Neumann

problem in the setting of weighted L2 -spaces on Cn . For this purpose we use
a version of the Rellich-Lemma for weighted Sobolev spaces.
Mathematics Subject Classification (2000). Primary 32W05; Secondary 32A36,
35J10.
Keywords. ∂-Neumann problem, Sobolev spaces, compactness.

1. Introduction
Let Ω be a bounded pseudoconvex domain in Cn . We consider the ∂-complex
∂ ∂ ∂ ∂
L2 (Ω) −→ L2(0,1) (Ω) −→ · · · −→ L2(0,n) (Ω) −→ 0 ,
where L2(0,q) (Ω) denotes the space of (0, q)-forms on Ω with coefficients in L2 (Ω).
The ∂-operator on (0, q)-forms is given by
 
 n
 ∂aJ
∂ aJ dz J = dz j ∧ dz J ,
j=1
∂z j
J J

where means that the sum is only taken over strictly increasing multi-indices J.
The derivatives are taken in the sense of distributions, and the domain of ∂
consists of those (0, q)-forms for which the right-hand side belongs to L2(0,q+1) (Ω).
So ∂ is a densely defined closed operator, and therefore has an adjoint operator
∗
from L2(0,q+1) (Ω) into L2(0,q) (Ω) denoted by ∂ .

Partially supported by the FWF-grant P19147.

160 K. Gansberger and F. Haslinger

∗ ∗
The complex Laplacian  = ∂ ∂ + ∂ ∂ acts as an unbounded selfadjoint
operator on L2(0,q) (Ω), 1 ≤ q ≤ n, it is surjective and therefore has a continuous
inverse, the ∂-Neumann operator Nq . If v is a ∂-closed (0, q + 1)-form, then u =
∗
∂ Nq+1 v provides the canonical solution to ∂u = v, namely the one orthogonal to
the kernel of ∂ and so the one with minimal norm (see for instance [ChSh]).
A survey of the L2 -Sobolev theory of the ∂-Neumann problem is given in [BS].
The question of compactness of Nq is of interest for various reasons. For
example, compactness of Nq implies global regularity in the sense of preservation
of Sobolev spaces [KN]. Also, the Fredholm theory of Toeplitz operators is an
immediate consequence of compactness in the ∂-Neumann problem [HI], [CD].
There are additional ramifications for certain C ∗ -algebras naturally associated to
a domain in Cn [SSU]. Finally, compactness is a more robust property than global
regularity – for example, it localizes, whereas global regularity does not – and it
is generally believed to be more tractable than global regularity.
A thorough discussion of compactness in the ∂-Neumann problem can be
found in [FS1] and [FS2].
The study of the ∂-Neumann problem is essentially equivalent to the study
of the canonical solution operator to ∂:
The ∂-Neumann operator Nq is compact from L2(0,q) (Ω) to itself if and only
if the canonical solution operators
∗ ∗
∂ Nq : L2(0,q) (Ω) −→ L2(0,q−1) (Ω) and ∂ Nq+1 : L2(0,q+1) (Ω) −→ L2(0,q) (Ω)
are compact.
Not very much is known in the case of unbounded domains. In this paper we
continue the investigations of [HaHe] concerning existence and compactness of the
canonical solution operator to ∂ on weighted L2 -spaces over Cn , where we applied
ideas which were used in the spectral analysis of the Witten Laplacian in the real
case, see [HeNi].
Let ϕ : Cn −→ R+ be a plurisubharmonic C 2 -weight function and define the
space 
L2 (Cn , ϕ) = {f : Cn −→ C : |f |2 e−ϕ dλ < ∞},
Cn
where λ denotes the Lebesgue measure, the space L2(0,1) (Cn , ϕ) of (0, 1)-forms with
coefficients in L2 (Cn , ϕ) and the space L2(0,2) (Cn , ϕ) of (0, 2)-forms with coefficients
in L2 (Cn , ϕ). Let 
f, g
ϕ = f ge−ϕ dλ
Cn
denote the inner product and

f 2ϕ = |f |2 e−ϕ dλ
Cn

the norm in L2 (Cn , ϕ).

 Compactness Estimates for the ∂-Neumann Problem 161

We consider the weighted ∂-complex

∂ ∂
L2 (Cn , ϕ) −→ L2(0,1) (Cn , ϕ) −→ L2(0,2) (Cn , ϕ),
←− ←−
∂∗
ϕ ∂∗
ϕ

∗
where ∂ ϕ is the adjoint operator to ∂ with respect to the weighted inner product.
 ∗
For u = nj=1 uj dz j ∈ dom(∂ ϕ ) one has
n

∗ ∂ ∂ϕ
∂ ϕu = − − uj .
j=1
∂zj ∂zj

The complex Laplacian on (0, 1)-forms is defined as

∗ ∗
ϕ := ∂ ∂ ϕ + ∂ ϕ ∂,
where the symbol ϕ is to be understood as the maximal closure of the operator
initially defined on forms with coefficients in C0∞ , i.e., the space of smooth functions
with compact support.
ϕ is a selfadjoint and positive operator, which means that
ϕ f, f
ϕ ≥ 0 , for f ∈ dom(ϕ ).
The associated Dirichlet form is denoted by
∗ ∗
Qϕ (f, g) = ∂f, ∂g
ϕ + ∂ ϕ f, ∂ ϕ g
ϕ ,
∗
for f, g ∈ dom(∂)∩dom(∂ ϕ ). The weighted ∂-Neumann operator Nϕ is – if it exists
– the bounded inverse of ϕ .
There is an interesting connection between ∂ and the theory of Schrödinger
operators with magnetic fields, see for example [Ch], [B], [FS3] and [ChF] for recent
contributions exploiting this point of view.
Here we use a Rellich-Lemma for weighted Sobolev spaces to establish com-
pactness estimates for the ∂-Neumann operator Nϕ on L2(0,1) (Cn , ϕ) and we use
this to give a new proof of the main result of [HaHe] without spectral theory of
Schrödinger operators.

2. Weighted basic estimates

In the weighted space L2(0,1) (Cn , ϕ) we can give a simple characterization of
∗
dom(∂ ϕ ):
 ∗
Proposition 2.1. Let f = fj dz j ∈ L2(0,1) (Cn , ϕ). Then f ∈ dom(∂ ϕ ) if and only if
n

∂fj ∂ϕ
− fj ∈ L2 (Cn , ϕ).
j=1
∂zj ∂zj
162 K. Gansberger and F. Haslinger

n  ∂fj 
j=1 ∂zj − ∂zj fj
∂ϕ
Proof. Suppose first that ∈ L2 (Cn , ϕ), which equivalently

means that eϕ j=1 ∂z∂ j (fj e−ϕ ) ∈ L2 (Cn , ϕ). We have to show that there exists
n

a constant C such that | ∂g, f

ϕ | ≤ Cgϕ for all g ∈ dom(∂). To this end let
(χR )R∈N be a family of radially symmetric smooth cutoff functions, which are
identically one on BR , the ball with radius R, such that the support of χR is
contained in BR+1 , supp(χR ) ⊂ BR+1 , and such that furthermore all first-order
derivatives of all functions in this family are uniformly bounded by a constant M .
Then for all g ∈ C0∞ (Cn ):
n 1 2 
∂  
n
∂g
∂g, χR f
ϕ = , χR fj =− g χR f j e−ϕ dλ,
j=1
∂zj ϕ j=1
∂z j

by integration by parts, which in particular means

# #
#
n
∂   #
| ∂g, f
ϕ | = lim | ∂g, χR f
ϕ | = lim ## g χR f j e−ϕ dλ##.
R→∞ R→∞ C n
j=1
∂z j

Now we use the triangle inequality, afterwards Cauchy–Schwarz, to get

# #
# n
∂   #
lim ## g χR f j e−ϕ dλ##
R→∞ Cn
j=1
∂zj
# # # #
# ∂  −ϕ  ## # ∂χR −ϕ ##
n n
≤ lim ## χR g f e dλ# + lim ## fjg e dλ#
R→∞ Cn
j=1
∂z j j R→∞ Cn
j=1
∂z j
( (
( ϕ n
∂  −ϕ (
≤ lim χR gϕ (e ( fj e ( + M gϕ f ϕ
R→∞ ∂z j
(
j=1 ϕ
( (
( ϕ ∂  −ϕ (
n
= gϕ ( fj e ( + M gϕ f ϕ .
(e ∂zj (
j=1 ϕ

Hence by assumption,
( (
( ϕ n
∂  −ϕ (
| ∂g, f
ϕ | ≤ gϕ(e f e ( + M gϕf ϕ ≤ Cgϕ
( ∂zj
j (
j=1 ϕ

for all g ∈ C0∞ (Cn ), and by density of C0∞ (Cn ) this is true for all g ∈ dom(∂).
∗
Conversely, let f ∈ dom(∂ ϕ ) and take g ∈ C0∞ (Cn ). Then g ∈ dom(∂) and
n 1 2
∗ ∂g
g, ∂ ϕ f
ϕ = ∂g, f
ϕ = , fj
j=1
∂z j ϕ
1 n 2 1 2
∂  −ϕ 
n
∂  −ϕ 
= − g, fj e = − g, eϕ fj e .
j=1
∂zj L2 j=1
∂zj ϕ
 Compactness Estimates for the ∂-Neumann Problem 163

Since C0∞ (Cn ) is dense in L2 (Cn , ϕ), we conclude that

∗ n
∂  −ϕ 
∂ ϕ f = −eϕ fe ,
j=1
∂z j

n
which in particular implies that eϕ ∂
j=1 ∂zj (fj e−ϕ ) ∈ L2 (Cn , ϕ). 

The following lemma will be important for our considerations.

∗
Lemma 2.2. Forms with coefficients in C0∞ (Cn ) are dense in dom(∂) ∩ dom(∂ ϕ ) in
∗ 1
the graph norm f → (f 2ϕ + ∂f 2ϕ + ∂ ϕ f 2ϕ ) 2 .

Proof. First we show that compactly supported L2 -forms are dense in the graph
norm. So let {χR }R∈N be a family of smooth radially symmetric cutoffs identically
one on BR and supported in BR+1 , such that all first-order derivatives of the
functions in this family are uniformly bounded in R by a constant M .
∗ ∗
Let f ∈ dom(∂) ∩ dom(∂ ϕ ). Then, clearly, χR f ∈ dom(∂) ∩ dom(∂ ϕ ) and χR f → f
in L2(0,1) (Cn , ϕ) as R → ∞. As observed in Proposition 2.1, we have

∗ n
∂  −ϕ 
∂ ϕ f = −eϕ fj e ,
j=1
∂zj

hence
∗
n
∂  
∂ ϕ (χR f ) = −eϕ χR fj e−ϕ .
∂zj
j=1

We need to estimate the difference of these expressions

∗ ∗ ∗ ∗
n
∂χR
∂ ϕ f − ∂ ϕ (χR f ) = ∂ ϕ f − χR ∂ ϕ f + fj ,
j=1
∂zj

which is by the triangle inequality

n 
∗ ∗ ∗ ∗
∂ ϕ f − ∂ ϕ (χR f )ϕ ≤∂ ϕ f − χ R ∂ ϕ f ϕ +M |fj |2 e−ϕ dλ.
j=1 n
C \BR

Now both terms tend to 0 as R → ∞, and one can see similarly that also ∂(χR f ) →
∂f as R → ∞.
So we have density of compactly supported forms in the graph norm, and density
of forms with coefficients in C0∞ (Cn ) will follow by applying Friedrich’s Lemma,
see Appendix D in [ChSh], see also [Jo]. 

As in the case of bounded domains, the canonical solution operator to ∂,

∗
which we denote by Sϕ , is given by ∂ ϕ Nϕ . Existence and compactness of Nϕ and
Sϕ are closely related. At first, we notice that equivalent weight functions have the
same properties in this regard.
164 K. Gansberger and F. Haslinger

Lemma 2.3. Let ϕ1 and ϕ2 be two equivalent weights, i.e., C −1 .ϕ1 ≤ .ϕ2 ≤
C.ϕ1 for some C > 0. Suppose that Sϕ2 exists. Then Sϕ1 also exists and Sϕ1 is
compact if and only if Sϕ2 is compact.
An analog statement is true for the weighted ∂-Neumann operator.
Proof. Let ι be the identity ι : L2(0,1) (Cn , ϕ1 ) → L2(0,1) (Cn , ϕ2 ), ιf = f , let j be the
identity j : L2ϕ2 → L2ϕ1 and let furthermore P be the orthogonal projection onto
ker(∂) in L2ϕ1 . Since the weights are equivalent, ι and j are continuous, so if Sϕ2
is compact, j ◦ Sϕ2 ◦ ι gives a solution operator on L2(0,1) (Cn , ϕ1 ) that is compact.
Therefore the canonical solution operator Sϕ1 = P ◦ j −1 ◦ Sϕ2 ◦ ι is also compact.
Since the problem is symmetric in ϕ1 and ϕ2 , we are done.
The assertion for the Neumann operator follows by the identity
Nϕ = Sϕ Sϕ∗ + Sϕ∗ Sϕ . 

Note that whereas existence and compactness of the weighted ∂-Neumann

operator is invariant under equivalent weights by Lemma 2.3, regularity is not. For
examples on bounded pseudoconvex domains, see for instance [ChSh], Chapter 6.
Now we suppose that the lowest eigenvalue λϕ of the Levi-matrix

2 
∂ ϕ
Mϕ =
∂zj ∂z k jk
of ϕ satisfies
lim inf λϕ (z) > 0. (∗ )
|z|→∞

Then, by Lemma 2.3, we may assume without loss of generality that λϕ (z) >  for
some  > 0 and all z ∈ Cn , since changing the weight function on a compact set
does not influence our considerations. So we have the following basic estimate
Proposition 2.4. For a plurisubharmonic weight function ϕ satisfying (∗), there is
a C > 0 such that
∗
u2ϕ ≤ C(∂u2ϕ + ∂ ϕ u2ϕ )
∗
for each (0, 1)-form u ∈ dom(∂) ∩ dom(∂ ϕ ).
Proof. By Lemma 2.2 and the assumption on ϕ it suffices to show that
 n
∂2ϕ ∗
uj uk e−ϕ dλ ≤ ∂u2ϕ + ∂ ϕ u2ϕ ,
n
C j,k=1 ∂z j ∂z k
n ∞
k=1 uk dz k with coefficients uk ∈ C0 (C ), for k =
n
for each (0, 1)-form u =
1, . . . , n.
For this purpose we set δk = ∂z∂k − ∂z ∂ϕ
k
and get since

∂uj 
∂uk
∂u = − dz j ∧ dz k
∂z k ∂z j
j<k
 Compactness Estimates for the ∂-Neumann Problem 165

that
 # #2 
∗ # ∂uj ∂uk ## −ϕ n
∂u2ϕ + ∂ ϕ u2ϕ
= # − e dλ + δj uj δk uk e−ϕ dλ
# ∂z j #
Cn j<k ∂z k Cn j,k=1
n  # # n 

# ∂uj #2 −ϕ ∂uj ∂uk
= # # e dλ + δ u δ u − e−ϕ dλ
# ∂z k # j j k k
∂z ∂z
j,k=1 C
n
j,k=1 C
n k j
 # #2 n  $ %
n
# ∂uj # −ϕ ∂
= # # e dλ + δ , uj uk e−ϕ dλ,
# # j
Cn ∂z k
j,k=1 Cn ∂z k
j,k=1

where we used the fact that for f, g ∈ C0∞ (Cn ) we have

1 2
∂f
,g = − f, δk g
ϕ .
∂z k ϕ
Since % $
∂ ∂ 2ϕ
= δj , ,
∂z k ∂zj ∂zk
and ϕ satisfies (*) we are done (see also [H]). 
Now it follows by Proposition 2.4 that there exists a uniquely determined
∗
(0, 1)-form Nϕ u ∈ dom(∂) ∩ dom(∂ ϕ ) such that
∗ ∗
u, v
ϕ = Qϕ (Nϕ u, v) = ∂Nϕ u, ∂v
ϕ + ∂ ϕ Nϕ u, ∂ ϕ v
ϕ ,
and again by 2.4 that
∗
∂Nϕ u2ϕ + ∂ ϕ Nϕ u2ϕ ≤ C1 u2ϕ
as well as
∗
Nϕ u2ϕ ≤ C2 (∂Nϕ u2ϕ + ∂ ϕ Nϕ u2ϕ ) ≤ C3 u2ϕ,
where C1 , C2 , C3 > 0 are constants. Hence we get that Nϕ is a continuous linear
operator from L2(0,1) (Cn , ϕ) into itself (see also [H] or [ChSh]).

3. Weighted Sobolev spaces

We want to study compactness of the weighted ∂-Neumann operator Nϕ . For this
purpose we define weighted Sobolev spaces and prove, under suitable conditions,
a Rellich-Lemma for these weighted Sobolev spaces. We will also have to consider
their dual spaces, which already appeared in [BDH] and [KM].
Definition 3.1. For k ∈ N let
W k (Cn , ϕ) := {f ∈ L2 (Cn , ϕ) : Dα f ∈ L2 (Cn , ϕ) for |α| ≤ k},
∂ |α|
where Dα = ∂ α1 x1 ...∂ α2n yn for (z1 , . . . , zn ) = (x1 , y1 , . . . , xn , yn ) with norm

f 2k,ϕ = Dα f 2ϕ .
|α|≤k
166 K. Gansberger and F. Haslinger

We will also need weighted Sobolev spaces with negative exponent. But it
turns out that for our purposes it is more reasonable to consider the dual spaces
of the following spaces.
Definition 3.2. Let
∂ ∂ϕ ∂ ∂ϕ
Xj = − and Yj = − ,
∂xj ∂xj ∂yj ∂yj
for j = 1, . . . , n and define
W 1 (Cn , ϕ, ∇ϕ) = {f ∈ L2 (Cn , ϕ) : Xj f, Yj f ∈ L2 (Cn , ϕ), j = 1, . . . , n},
with norm

n
f 2ϕ,∇ϕ = f 2ϕ + (Xj f 2ϕ + Yj f 2ϕ ).
j=1

In the next step we will analyze the dual space of W 1 (Cn , ϕ, ∇ϕ).
By the mapping f → (f, Xj f, Yj f ), the space W 1 (Cn , ϕ, ∇ϕ) can be identi-
fied with a closed product of L2 (Cn , ϕ), hence each continuous linear functional L
on W 1 (Cn , ϕ, ∇ϕ) is represented (in a non-unique way) by

L(f ) = f (z)g0 (z)e−ϕ(z) dλ(z)
Cn
n 
+ (Xj f (z)gj (z) + Yj f (z)hj (z))e−ϕ(z) dλ(z),
j=1 Cn

for some gj , hj ∈ L (Cn , ϕ).

2

For f ∈ C0∞ (Cn ) it follows that

 n 

∂gj (z) ∂hj (z)
L(f ) = f (z)g0 (z)e−ϕ(z) dλ(z) − f (z) + e−ϕ(z) dλ(z).
Cn j=1 Cn ∂xj ∂yj

Since C0∞ (Cn ) is dense in W (C , ϕ, ∇ϕ) we have shown

1 n

Lemma 3.3. Each element u ∈ W −1 (Cn , ϕ, ∇ϕ) := (W 1 (Cn , ϕ, ∇ϕ)) can be rep-
resented in a non-unique way by
n

∂gj ∂hj
u = g0 + + ,
j=1
∂xj ∂yj

where gj , hj ∈ L2 (Cn , ϕ).

The dual norm u−1,ϕ,∇ϕ := sup{|u(f )| : f ϕ,∇ϕ ≤ 1} can be expressed
in the form

n
u2−1,ϕ,∇ϕ = inf{g0 2 + (gj 2 + hj 2 ),
j=1

where the infimum is taken over all families (gj , hj ) in L2 (Cn , ϕ) representing the
functional u (see for instance [T]).
 Compactness Estimates for the ∂-Neumann Problem 167

In particular each function in L2 (Cn , ϕ) can be identified with an element of

−1
W (Cn , ϕ, ∇ϕ).
Proposition 3.4. Suppose that the weight function satisfies
lim (θ|∇ϕ(z)|2 + 'ϕ(z)) = +∞,
|z|→∞

for some θ ∈ (0, 1), where

# # # # 

n
# ∂ϕ #2 # ∂ϕ #2
|∇ϕ(z)| =2 # # # #
# ∂xk # + # ∂yk # .
k=1

Then the embedding of W (C , ϕ, ∇ϕ) into L2 (Cn , ϕ) is compact.

1 n

Proof. We adapt methods from [BDH] or [Jo], Proposition 6.2., or [KM]. For the
vector fields Xj from 3.2 and their formal adjoints Xj∗ = − ∂x
∂
j
we have
∂ϕ ∂ 2ϕ
(Xj + Xj∗ )f = − f and [Xj , Xj∗ ]f = − 2 f,
∂xj ∂xj
for f ∈ C0∞ (Cn ), and
[Xj , Xj∗ ]f, f
ϕ = Xj∗ f 2ϕ − Xj f 2ϕ ,
(Xj + Xj∗ )f 2ϕ ≤ (1 + 1/)Xj f 2ϕ + (1 + )Xj∗ f 2ϕ
for each  > 0. Similar relations hold for the vector fields Yj . Now we set
Ψ(z) = |∇ϕ(z)|2 + (1 + )'ϕ(z).
It follows that

n
Ψf, f
ϕ ≤ (2 +  + 1/) (Xj f 2ϕ + Yj f 2ϕ ).
j=1

Since C0∞ (Cn )

is dense in W (C , ϕ, ∇ϕ) by definition, this inequality holds for
1 n

all f ∈ W (C , ϕ, ∇ϕ).
1 n

If (fk )k is a sequence in W 1 (Cn , ϕ, ∇ϕ) converging weakly to 0, then (fk )k

is a bounded sequence in W 1 (Cn , ϕ, ∇ϕ) and our the assumption implies that
Ψ(z) = |∇ϕ(z)|2 + (1 + )'ϕ(z)
is positive in a neighborhood of ∞. So we obtain

|fk (z)|2 e−ϕ(z) dλ(z)
Cn
 
Ψ(z)|fk (z)|2
≤ |fk (z)|2 e−ϕ(z) dλ(z) + e−ϕ(z) dλ(z)
inf{Ψ(z) : |z| ≥ R}
|z|<R |z|≥R

C fk 2ϕ,∇ϕ
≤ Cϕ,R fk 2L2 (BR ) + .
inf{Ψ(z) : |z| ≥ R}
168 K. Gansberger and F. Haslinger

Hence the assumption and the fact that the injection W 1 (BR ) → L2 (BR ) is
compact (see for instance [T]) show that a subsequence of (fk )k tends to 0 in
L2 (Cn , ϕ). 
Remark 3.5. It follows that the adjoint to the above embedding, the embedding of
L2 (Cn , ϕ) into (W 1 (Cn , ϕ, ∇ϕ)) = W −1 (Cn , ϕ, ∇ϕ) (in the sense of 3.3) is also
compact.
Remark 3.6. Note that one does not need plurisubharmonicity of the weight func-
tion in Proposition 3.4. If the weight is plurisubharmonic, one can of course drop
θ in the formulation of the assumption.

4. Compactness estimates
The following Proposition reformulates the compactness condition for the case
of a bounded pseudoconvex domain in Cn , see [BS], [Str]. The difference to the
compactness estimates for bounded pseudoconvex domains is that here we have to
assume an additional condition on the weight function implying a corresponding
Rellich-Lemma.
Proposition 4.1. Suppose that the weight function ϕ satisfies (∗) and
lim (θ|∇ϕ(z)|2 + 'ϕ(z)) = +∞,
|z|→∞

for some θ ∈ (0, 1), then the following statements are equivalent.
1. The ∂-Neumann operator N1,ϕ is a compact operator from L2(0,1) (Cn , ϕ) into
itself.
∗
2. The embedding of the space dom(∂) ∩ dom(∂ ϕ ), provided with the graph norm
∗
u → (u2ϕ + ∂u2ϕ + ∂ ϕ u2ϕ )1/2 , into L2(0,1) (Cn , ϕ) is compact.
3. For every positive  there exists a constant C such that
∗
u2ϕ ≤ (∂u2ϕ + ∂ ϕ u2ϕ ) + C u2−1,ϕ,∇ϕ,
∗
for all u ∈ dom(∂) ∩ dom(∂ ϕ ).
4. The operators
∗
∂ ϕ N1,ϕ : L2(0,1) (Cn , ϕ) ∩ ker(∂) −→ L2 (Cn , ϕ) and
∗
∂ ϕ N2,ϕ : L2(0,2) (Cn , ϕ) ∩ ker(∂) −→ L2(0,1) (Cn , ϕ)
are both compact.
Proof. First we show that (1) and (4) are equivalent: suppose that N1,ϕ is compact.
For f ∈ L2(0,1) (Cn , ϕ) it follows that
∗
∂ ϕ N1,ϕ f 2ϕ ≤ f, N1,ϕ f
ϕ ≤ f 2ϕ + C N1,ϕ f 2ϕ
∗
by Lemma 2 of [CD]. Hence ∂ ϕ N1,ϕ is compact. Applying the formula
∗ ∗ ∗ ∗
N1,ϕ − (∂ ϕ N1,ϕ )∗ (∂ ϕ N1,ϕ ) = (∂ ϕ N2,ϕ )(∂ ϕ N2,ϕ )∗ ,
 Compactness Estimates for the ∂-Neumann Problem 169

∗
(see for instance [ChSh]), we get that also ∂ ϕ N2,ϕ is compact. The converse follows
easily from the same formula.
Now we show (4) =⇒ (3) =⇒ (2) =⇒ (1). We follow the lines of [Str],
where the case of a bounded pseudoconvex domain is handled.
Assume (4): if (3) does not hold, then there exists 0 > 0 and a sequence
∗
(un )n in dom (∂) ∩dom (∂ ϕ ) with un ϕ = 1 and
∗
un 2ϕ ≥ 0 (∂un 2ϕ + ∂ ϕ un 2ϕ ) + nun 2−1,ϕ,∇ϕ
−1
for each n ≥ 1, which implies that un → 0 in W(0,1) (Cn , ϕ, ∇ϕ). Since un can be
written in the form
∗ ∗ ∗
un = (∂ ϕ N1,ϕ )∗ ∂ ϕ un + (∂ ϕ N2,ϕ ) ∂un ,
(4) implies there exists a subsequence of (un )n converging in L2(0,1) (Cn , ϕ) and the
limit must be 0, which contradicts un ϕ = 1.
To show that (3) implies (2) we consider a bounded sequence in dom (∂) ∩
∗
dom (∂ ϕ ). By 2.4 this sequence is also bounded in L2(0,1) (Cn , ϕ). Now 3.4 implies
−1
that it has a subsequence converging in W(0,1) (Cn , ϕ, ∇ϕ). Finally use (3) to show
that this subsequence is a Cauchy sequence in L2(0,1) (Cn , ϕ), therefore (2) holds.
Assume (2): by 2.4 and the basic facts about N1,ϕ , it follows that
∗
N1,ϕ : L2(0,1) (Cn , ϕ) −→ dom (∂) ∩ dom (∂ ϕ )
is continuous in the graph topology, hence
∗
N1,ϕ : L2(0,1) (Cn , ϕ) −→ dom (∂) ∩ dom (∂ ϕ ) → L2(0,1) (Cn , ϕ)
is compact. 

Remark 4.2. Suppose that the weight function ϕ is plurisubharmonic and that the
lowest eigenvalue λϕ of the Levi-matrix Mϕ satisfies
lim λϕ (z) = +∞ . (∗∗ )
|z|→∞

This condition implies that N1,ϕ is compact [HaHe].

It also implies that the condition of the Rellich-Lemma 3.4 is satisfied.
This follows from the fact that we have for the trace tr(Mϕ ) of the Levi-matrix
1
tr(Mϕ ) ='ϕ,
4
and since for any invertible (n × n)-matrix T
tr(Mϕ ) = tr(T Mϕ T −1),
it follows that tr(Mϕ ) equals the sum of all eigenvalues of Mϕ . Hence our assump-
tion on the lowest eigenvalue λϕ of the Levi-matrix implies that the assumption
of Proposition 3.4 is satisfied.
170 K. Gansberger and F. Haslinger

In order to use Proposition 4.1 to show compactness of Nϕ we still need

Proposition 4.3 (Gårding’s inequality). Let Ω be a smooth bounded domain. Then
for any u ∈ W 1 (Ω, ϕ, ∇ϕ) with compact support in Ω
 ∗

u21,ϕ,∇ϕ ≤ C(Ω, ϕ) ∂u2ϕ + ∂ ϕ u2ϕ + u2ϕ .
Proof. The operator −ϕ is strictly elliptic since its principal part equals the
∗ ∗
Laplacian. Now −ϕ = −(∂ ⊕ ∂ ϕ )∗ ◦ (∂ ⊕ ∂ ϕ ), so from general PDE theory follows
∗
that the system ∂ ⊕∂ ϕ is elliptic. This is, because a differential operator P of order
s is elliptic if and only if (−1)s P ∗ ◦ P is strictly elliptic. So because of ellipticity,
one has on each smooth bounded domain Ω the classical Gårding inequality
 ∗

u21 ≤ C(Ω) ∂u2 + ∂ ϕ u2 + u2

for any (0,1)-form u with coefficients in C0∞ . But our weight ϕ is smooth on Ω,
hence the weighted and unweighted L2 -norms on Ω are equivalent, and therefore
u21,ϕ,∇ϕ ≤ C1 (u21,ϕ + u2ϕ) ≤ C2 (u21 + u2 )
∗ ∗
≤ C3 (∂u2 + ∂ ϕ u2 + u2 ) ≤ C4 (∂u2ϕ + ∂ ϕ u2ϕ + u2ϕ). 
We are now able to give a different proof of the main result in [HaHe].
Theorem 4.4. Let ϕ be plurisubharmonic. If the lowest eigenvalue λϕ (z) of the
Levi-matrix Mϕ satisfies (∗∗ ), then Nϕ is compact.
Proof. By Proposition 3.4 and Remark 4.2, it suffices to show a compactness es-
timate and use Proposition 4.1. Given  > 0 we choose M ∈ N with 1/M ≤ /2
and R such that λ(z) > M whenever |z| > R. Let χ be a smooth cutoff function
identically one on BR . Hence we can estimate
 ∂2ϕ
M f 2ϕ ≤ fj f k e−ϕ dλ + M χf 2ϕ
∂zj ∂z k
j,k Cn \B
R

≤Qϕ (f, f ) + M χf, f

ϕ
≤Qϕ (f, f ) + M χf 1,ϕ,∇ϕf −1,ϕ,∇ϕ
≤Qϕ (f, f ) + M aχf 21,ϕ,∇ϕ + a−1 M f 2−1,ϕ,∇ϕ,
where a is to be chosen a bit later. Now we apply Gårding’s inequality 4.3 to the
second term, so there is a constant CR depending on R such that
 
M f 2ϕ ≤ Qϕ (f, f ) + M aCR Qϕ (f, f ) + f 2ϕ + a−1 M f 2−1,ϕ,∇ϕ.
By Proposition 2.4 and after increasing CR we have
M f 2ϕ ≤ Qϕ (f, f ) + M aCR Qϕ (f, f ) + a−1 M f 2−1,ϕ,∇ϕ.
Now choose a such that aCR ≤ /2, then
f 2ϕ ≤ Qϕ (f, f ) + a−1 f 2−1,ϕ,∇ϕ
and this estimate is equivalent to compactness by 4.1. 
 Compactness Estimates for the ∂-Neumann Problem 171

Remark 4.5. Assumption (∗∗ ) on the lowest eigenvalue of Mϕ is the analog of

property (P ) introduced by Catlin in [Ca] in case of bounded pseudoconvex domains.
Therefore the proof is similar.
Remark 4.6. We mention that for the weight ϕ(z) = |z|2 the ∂-Neumann operator
fails to be compact (see [HaHe]), but the condition
lim (θ|∇ϕ(z)|2 + 'ϕ(z)) = +∞
|z|→∞

of the Rellich-Lemma is satisfied.

m
Remark 4.7. Denote by Wloc (Cn ) the space of functions which locally belong to
the classical unweighted Sobolev space W m (Cn ). Suppose that ϕ v = g and g ∈
(0,1) (C ). Then v ∈ Wloc (0,1) (C ). In particular, if there exists a weighted
m n m+2 n
Wloc
∞
∂-Neumann operator Nϕ , it maps C(0,1) (Cn ) ∩ L2(0,1) (Cn , ϕ) into itself.
ϕ is strictly elliptic, and the statement in fact follows from interior regular-
ity of a general second-order elliptic operator. The reader can find more on elliptic
regularity for instance in [Ev], Chapter 6.3.
An analog statement is true for Sϕ . If there exists a continuous canonical solu-
∞
tion operator Sϕ , it maps C(0,1) (Cn ) ∩ L2(0,1) (Cn , ϕ) into itself. This follows from
ellipticity of ∂.
Although ϕ is strictly elliptic, the question whether Sϕ is globally or exactly
regular is harder to answer. This is, because our domain is not bounded and neither
are the coefficients of ϕ . Only in a very special case the question is easy – this
is, when A2ϕ (the weighted space of entire functions) is zero. In this case, there is
only one solution operator to ∂, namely the canonical one, and if f ∈ Wϕk (0,1) and
u = Sϕ f , it follows that ∂Dα u = Dα f , since ∂ commutes with ∂x ∂
j
. Now Sϕ is
continuous, so D uϕ ≤ CD f ϕ , meaning that u ∈ Wϕ . So in this case Sϕ is
α α k

a bounded operator from Wϕk (0,1) → Wϕk .

Remark 4.8. Let A2(0,1) (Cn , ϕ) denote the space of (0, 1)-forms with holomorphic
coefficients belonging to L2 (Cn .ϕ).
We point out that assuming (∗∗ ) implies directly – without use of Sobolev
spaces – that the embedding of the space
∗
A2(0,1) (Cn , ϕ) ∩ dom (∂ ϕ )
∗
provided with the graph norm u → (u2ϕ + ∂ ϕ u2ϕ )1/2 into A2(0,1) (Cn , ϕ) is com-
pact. Compare 4.1 (2).
∗
For this purpose let u ∈ A2(0,1) (Cn , ϕ) ∩ dom (∂ ϕ ). Then we obtain from the
proof of 2.4 that
 n
∗ ∂ 2ϕ
∂ ϕ uϕ =
2
uj uk e−ϕ dλ.
∂zj ∂z k
Cn j,k=1
172 K. Gansberger and F. Haslinger


Let us for u = nj=1 uj dz j identify u(z) with the vector (u1 (z), . . . , un(z)) ∈ Cn .
Then, if we denote by ., .
the standard inner product in Cn , we have

n
u(z), u(z)
= |uj (z)|2
j=1

and

n
∂ 2 ϕ(z)
Mϕ u(z), u(z)
= uj (z)uk (z).
∂zj ∂z k
j,k=1

Note that the lowest eigenvalue λϕ of the Levi-matrix Mϕ can be expressed as

Mϕ u(z), u(z)

λϕ (z) = inf .
u(z)=0 u(z), u(z)

So we get
  
u, u
e−ϕ dλ ≤ u, u
e−ϕ dλ + [ ninf λϕ (z)]−1 λϕ (z) u, u
e−ϕ dλ
Cn BR C \BR Cn \BR
 
≤ u, u
e−ϕ dλ + [ ninf λϕ (z)]−1 Mϕ u, u
e−ϕ dλ.
BR C \BR Cn

For a given  > 0 choose R so large that

[ ninf λϕ (z)]−1 < ,
C \BR

and use the fact that for Bergman spaces of holomorphic functions the embedding
of A2 (BR1 ) into A2 (BR2 ) is compact for R2 < R1 . So the desired conclusion follows.
Remark 4.9. Part of the results, in particular Theorem 4.4, are taken from [Ga].
We finally mention that the methods used in this paper can also be applied to treat
unbounded pseudoconvex domains with boundary, see [Ga].
Acknowledgement
The authors thank the referee for two corrections in the bibliography and a re-
mark increasing the readability of the paper. Moreover they are thankful to Emil
Straube for pointing out an inaccuracy in an earlier version of the paper and help-
ful questions and suggestions. Finally they thank Anne-Katrin Herbig for many
inspiring discussions on this topic, which actually led to the idea to write this
paper.

References
[B] B. Berndtsson, ∂ and Schrödinger operators, Math. Z. 221 (1996), 401–413.
[BS] H.P. Boas and E.J. Straube, Global regularity of the ∂-Neumann problem: a survey
of the L2 -Sobolev theory, Several Complex Variables (M. Schneider and Y.-T. Siu,
eds.) MSRI Publications, vol. 37, Cambridge University Press, 1999, pg. 79–111.
 Compactness Estimates for the ∂-Neumann Problem 173

[BDH] P. Bolley, M. Dauge and B. Helffer, Conditions suffisantes pour l’injection com-
pacte d’espace de Sobolev à poids, Séminaire équation aux dérivées partielles
(France), vol.1, Université de Nantes (1989), 1–14.
[Ca] D.W. Catlin, Global regularity of the ∂-Neumann problem, Proc. Symp. Pure
Math. 41 (1984), 39–49
[CD] D. Catlin and J. D’Angelo, Positivity conditions for bihomogeneous polynomials,
Math. Res. Lett. 4 (1997), 555–567.
[Ch] M. Christ, On the ∂ equation in weighted L2 norms in C1 , J. of Geometric Anal-
ysis 1 (1991), 193–230.
[ChF] M. Christ and S. Fu, Compactness in the ∂-Neumann problem, magnetic
Schrödinger operators, and the Aharonov-Bohm effect, Adv. in Math. 197 (2005),
1–40.
[ChSh] So-Chin Chen and Mei-Chi Shaw, Partial differential equations in several complex
variables, Studies in Advanced Mathematics, Vol. 19, Amer. Math. Soc. 2001.
[Ev] L.C. Evans, Partial Differential Equations, Graduate Studies in Mathematics, 19,
American Mathematical Society, 1998.
[FS1] S. Fu and E.J. Straube, Compactness of the ∂-Neumann problem on convex do-
mains, J. of Functional Analysis 159 (1998), 629–641.
[FS2] S. Fu and E.J. Straube, Compactness in the ∂-Neumann problem, Complex Anal-
ysis and Geometry (J. McNeal, ed.), Ohio State Math. Res. Inst. Publ. 9 (2001),
141–160.
[FS3] S. Fu and E.J. Straube, Semi-classical analysis of Schrödinger operators and
compactness in the ∂ Neumann problem, J. Math. Anal. Appl. 271 (2002), 267–
282.
[Ga] K. Gansberger, Compactness of the ∂-Neumann operator, Dissertation, Univer-
sity of Vienna, 2009.
[HK] P. Hajlasz and P. Koskela, Sobolev met Poincaré, Memoirs AMS 688 (2000).
[HaHe] F. Haslinger and B. Helffer, Compactness of the solution operator to ∂ in weighted
L2 -spaces, J. of Functional Analysis, 243 (2007), 679–697.
[Hei] J. Heinonen, Nonsmooth calculus, Bull. AMS 44 (2007), 163–232.
[HeNi] B. Helffer and F. Nier, Criteria to the Poincaré inequality associated with Dirichlet
forms in Rd , d ≥ 2, Int. Math. Res. Notices 22 (2003), 1199–1223.
[HI] G. Henkin and A. Iordan, Compactness of the ∂-Neumann operator for hyper-
convex domains with non-smooth B-regular boundary, Math. Ann. 307 (1997),
151–168.
[H] L. Hörmander, An introduction to complex analysis in several variables, North
Holland, Amsterdam etc., 1990.
[Jo] J. Johnsen, On the spectral properties of Witten Laplacians, their range projec-
tions and Brascamp-Lieb’s inequality, Integral Equations Operator Theory 36 (3),
2000, 288–324.
[KM] J.-M. Kneib and F. Mignot, Equation de Schmoluchowski généralisée, Ann. Math.
Pura Appl. (IV) 167 (1994), 257–298.
[KN] J. Kohn and L. Nirenberg, Non-coercive boundary value problems, Comm. Pure
and Appl. Math. 18 (1965), 443–492.
174 K. Gansberger and F. Haslinger

[SSU] N. Salinas, A. Sheu and H. Upmeier, Toeplitz operators on pseudoconvex domains

and foliation C ∗ -algebras, Ann. of Math. 130 (1989), 531–565.
[Str] E. Straube, The L2 -Sobolev theory of the ∂-Neumann problem, ESI Lectures in
Mathematics and Physics, EMS (to appear).
[T] F. Treves, Basic Linear Partial Differential, Dover Books on Mathematics, Dover
Publications, Inc., Mineola, NY, 2006.

Klaus Gansberger and Friedrich Haslinger

Institut für Mathematik
Universität Wien
Nordbergstrasse 15
A-1090 Wien, Austria
e-mail: [email protected]
[email protected]
Complex Analysis
Trends in Mathematics, 175–185

c 2010 Springer Basel AG

Remarks on the Homogeneous Complex

Monge-Ampère Equation
Pengfei Guan
Dedicated to Professor Linda Rothchild on the occasion of her 60th birthday

Abstract. We refine the arguments in [12] to show that the extended norm
of Bedford-Taylor is in fact exact the same as the original Chern-Levine-
Nirenberg intrinsic norm, thus provides a proof of the Chern-Levine-Nirenberg
conjecture. The result can be generalized to deal with homogeneous Monge-
Ampère equation on any complex manifold.
Mathematics Subject Classification (2000). 32W20, 35J65.
Keywords. Complex Monge-Ampère equation, Dirichlet problem,
intrinsic norms.

This short note concerns the homogeneous complex Monge-Ampère equation aris-
ing from the Chern-Levine-Nirenberg holomorphic invariant norms in [9]. In [9],
Chern-Levine-Nirenberg found close relationship of the intrinsic norms with the
variational properties and regularity of the homogeneous complex Monge-Ampère
equation. It is known that solutions to the homogeneous complex Monge-Ampère
equation fail to be C 2 in general, since the equation is degenerate and the best reg-
ularity is C 1,1 by examples of Bedford-Forneass [2]. In a subsequential study under-
taken by Bedford-Taylor [4], to overcome regularity problem for the homogeneous
complex Monge-Ampère equation, they developed the theory of weak solutions and
they extended the definition of intrinsic norm to a larger class of plurisubharmonic
functions. Furthermore, they related it to an extremal function determined by the
weak solution of the homogeneous complex Monge-Ampère equation. Among many
important properties of the extremal function, they obtained the Lipschitz regu-
larity for the solution of the homogeneous complex Monge-Ampère equation, and
proved an estimate for the intrinsic norm in terms of the extremal function and
the defining function of the domain. In [12], the optimal C 1,1 regularity was estab-
lished for the extremal function. As a consequence, the variational characterization

Research was supported in part by an NSERC Discovery Grant.

176 P. Guan

of the intrinsic norm of Bedford-Taylor is validated, along with the explicit formula
for the extended norm speculated in [4].
This paper consists of two remarks related to the results in [12]. First is
that the approximation of the extremal function constructed in [12] can be used
to show that the extended norm of Bedford-Taylor is in fact exact the same as
the Chern-Levine-Nirenberg intrinsic norm, thus it provides a proof of the origi-
nal Chern-Levine-Nirenberg conjecture. The second is that the results in [12] can
be generalized to any complex manifold, with the help of the existence of the
plurisubharmonic function obtained in [12]. That function was used in a crucial
way in [12] to get C 2 boundary estimate following an argument of Bo Guan [10]. We
will use this function to establish global C 1 estimate for the homogeneous complex
Monge-Ampère equation on general manifolds. We also refer Chen’s work [8] on
homogeneous complex Monge-Ampère equation arising from a different geometric
context.
Let’s recall the definitions of the Chern-Levine-Nirenberg norm [9] and the
extended norm defined by Bedford-Taylor [4]. Let M be a closed complex manifold
with smooth boundary ∂M = Γ1 ∪ Γ0 , set
F = {u ∈ C 2 (M ) | u plurisubharmonic and 0 < u < 1 on M },

Fk = {u ∈ F | (ddc u)k = 0, dim γ = 2k − 1, or du ∧ (ddc u)k = 0, dim γ = 2k}.

∀γ ∈ H∗ (M, R) be a homology class in M ,
N2k−1 {γ} = sup inf |T (dc u ∧ (ddc u)k−1 )|, if dim γ = 2k − 1; (1)
u∈F T ∈γ

N2k {γ} = sup inf |T (du ∧ dc u ∧ (ddc u)k−1 )|, if dim γ = 2k, (2)
u∈F T ∈γ

where T runs over all currents which represent γ.

It is pointed out in [9] that the intrinsic norm Nj may also be obtained as the
supremum over the corresponding subclass of C 2 solutions of homogeneous com-
plex Hessian equations in Fk . The most interesting case is k = 2n − 1, elements of

F2n−1 are plurisubharmonic functions satisfying the homogeneous complex Monge-
Ampère equation
(ddc u)n = 0. (3)
In this case, associated to N2n−1 , there is an extremal function satisfying the
Dirichlet boundary condition for the homogeneous complex Monge-Ampère equa-
tion:
⎧
⎨ (ddc u)n = 0 in M 0
u|Γ1 =1 (4)
⎩
u|Γ0 =0 ,
where dc = i(∂¯ − ∂), M 0 is the interior of M , and Γ1 and Γ0 are the corresponding
outer and inner boundaries of M respectively.
 Remarks on the Homogeneous Complex Monge-Ampère Equation 177

Due to the lack of C 2 regularity for solutions of equation (4), an extended

norm Ñ was introduced by Bedford-Taylor [4]. Set

F̃ = {u ∈ C(M )|u plurisubharmonic, 0 < u < 1 on M }.

Ñ {γ} = sup inf |T (dc u ∧ (ddc u)k−1 )|, if dim γ = 2k − 1, (5)
T ∈γ
u∈F̃

Ñ {γ} = sup inf |T (du ∧ dc u ∧ (ddc u)k−1 )|, if dim γ = 2k, (6)
T ∈γ
u∈F̃

where the infimum this time is taken over smooth, compactly supported currents
which represent γ.
Ñ enjoys similar properties of N , and N ≤ Ñ < ∞. They are invariants of
the complex structure, and decrease under holomorphic maps.
Chern-Levine-Nirenberg observed in [9] that equation (3) also arises as the
Euler equation for the functional

I(u) = du ∧ dc u ∧ (ddc u)n−1 . (7)
M

Let

B = {u ∈ F | u = 1 on Γ1 , u = 0 on Γ0 }. (8)

If v ∈ B, let γ denote the (2n − 1)-dimensional homology class of the level hyper-
surface v = constant. Then ∀ T ∈ γ, if v satisfies (ddc v)n = 0,
 
dv ∧ (ddc v)n−1 = dv ∧ dc v ∧ (ddc v)n−1 = I(v).
T M

Chern-Levine-Nirenberg Conjecture [9]: N {Γ} = inf u∈B I(u).

The relationship between the intrinsic norms and the extremal function u of
(4) was investigated by Bedford-Taylor [4]. They pointed out that: if the extremal
function u in (4) is C 2 , one has the following important representation formula,

n
∂u
Ñ ({Γ1 }) = dc r ∧ (ddc r)n−1 , (9)
Γ1 dr

where Γ1 is the outer boundary of M and r is a defining function of Γ1 . They also

observed that if the extremal function u of (4) can be approximated by functions

in Fn , then N = Ñ and the Chern-Levine-Nirenberg conjecture would be valid.

The problem is that functions in Fn are C 2 plurisubharmonic functions satisfying
the homogeneous complex Monge-Ampère equation. It is hard to construct such
approximation due to the lack of C 2 regularity for such equation. Though in some
special cases, for example on Reinhardt domains ([4]) or a perturbation of them
([1] and [16]), the extremal function is smooth.
178 P. Guan

We note that in order for equation (4) to have a plurisubharmonic solution,

it is necessary that there is a plurisubharmonic subsolution v. Now suppose M is
of the following form,
⎛ ⎞

N
M = Ω̄∗ \ ⎝ Ωj ⎠ , (10)
j=1
∗
where Ω , Ω1 , . . . , ΩN are bounded strongly smooth pseudoconvex domains in
/N
Cn . Ω̄j ⊂ Ω∗ , ∀ j = 1, . . . , N , Ω̄1 , . . . , Ω̄N are pairwise disjoint, and j=1 Ωj
/
is holomorphic convex in Ω∗ , and Γ1 = ∂Ω∗ and Γ0 = j=1 ∂Ωj . If Γ = {v =
N

constant} for some v ∈ B, Γ ∼ {v = 1} ∼ {v = 0} in H2n−1 (M ), the hypersurface

{v = 1} is pseudoconvex, and the hypersurface {v = 0} is pseudoconcave. If
M is embedded in Cn , v is strictly plurisubharmonic, and M must be of the
form (10). The reverse is proved in [12]: if M is of the form (10), there is v ∈
P SH(M 0 ) ∩ C ∞ (M )
(ddc V )n > 0 in M, (11)
such that Γ1 = {V = 1} and Γ0 = {V = 0}.
The following was proved in [12].
Theorem 1. If M is of the form (10), for the unique solution u of (4), there is a
sequence {uk } ⊂ B such that
uk C 2 (M) ≤ C, ∀ k, lim sup(ddc uk )n = 0.
k→∞

In particular, u ∈ C 1,1 (M ) and limk→∞ uk − uC 1,α (M) = 0, ∀ 0 < α < 1. And
we have

n
∂u
Ñ ({Γ1 }) = dc r ∧ (ddc r)n−1 (12)
Γ1 dr
where r is any defining function of Ω. Moreover,

Ñ ({Γ1 }) = inf dv ∧ dc v ∧ (ddc v)n−1 . (13)
v∈B M

In this paper, we establish

Theorem 2. If M is of the form (10), we have N ({Γ1 }) = Ñ ({Γ1 }), and the
Chern-Levine-Nirenberg conjecture is valid, that is

N ({Γ1 }) = inf dv ∧ dc v ∧ (ddc v)n−1 . (14)
v∈B M

We will work on general complex manifold M , which may not necessary to

be restricted as a domain in Cn . We assume that
/
M is a complex manifold, ∂M = Γ1 Γ0 with both Γ1 and
Γ0 are compact hypersurfaces of M , and there is V ∈ B such (15)
that (ddc V (z))n > 0, ∀z ∈ M .
 Remarks on the Homogeneous Complex Monge-Ampère Equation 179

We will prove the following generalization of Theorem 1.

Theorem 3. Suppose M is of the form (15), there is a unique solution u of (4),
there exist a constant C > 0 and a sequence {uk } ⊂ B such that
|Δuk (z)| ≤ C, ∀ k, lim sup (ddc uk (z))n = 0. (16)
k→∞ z∈M

In particular, 0 ≤ Δu(z) ≤ C and limk→∞ uk − uC 1,α (M) = 0, ∀ 0 < α < 1.
Furthermore,

n
∂u
N ({Γ1 }) = Ñ ({Γ1 }) = dc r ∧ (ddc r)n−1 (17)
Γ1 dr
where r is any defining function of Ω. Finally,

N ({Γ1 }) = inf dv ∧ dc v ∧ (ddc v)n−1 . (18)
v∈B M

Theorem 3 implies Theorem 2. The proof Theorem 3 relies on the regularity

study of equation (4). It is a degenerate elliptic fully nonlinear equation.
If M is a domain in Cn , Caffarelli-Kohn-Nirenberg-Spruck [6] establishes
1,1
C regularity for solutions in strongly pseudoconvex domains with homogeneous
boundary condition. For the Dirichlet problem (4), some pieces of the boundary
are concave. In [12], we made use of the subsolution method of [10] for the second
derivative estimates on the boundary (in the real case, this method was intro-
duced by Hoffman-Rosenberg-Spruck [15] and Guan-Spruck in [11]). This type of
estimates is of local feature, so the second derivative estimates on the boundary
can be treated in the same way. What we will work on is the interior estimates
for the degenerate complex Monge-Ampère equation on Kähler manifold. Such C 2
estimate has been established by Yau in [17]. The contribution of this paper is
an interior C 1 estimate for the degenerate complex Monge-Ampère equation in
general Kähler manifolds.
We remark here that the subsolution V in (15) can be guaranteed if we
impose certain holomorphic convexity condition on M as one can use the pasting
method developed in [12]. The subsolution V played important role in the proof
boundary estimates in [12]. In this paper, the subsolution V will be crucial to prove
the interior estimate. Since we are dealing the equation (4) on a general complex
manifold, there may not exist a global coordinate chart. Instead, we treat equation
(4) as a fully nonlinear equation on Kähler manifold (M, g), where g = (gij̄ ) = (Vij̄ )
is defined by the function V in (15). We will work on the following equation with
parameter 0 ≤ t < 1,
⎧
⎪
⎪ det(gij̄ + φij̄ ) = (1 − t) det(gij̄ )f,
⎨
(gij̄ + φij̄ ) > 0,
(19)
⎪
⎪ φ|Γ1 = 0,
⎩
φ|Γ0 = 0,
where f is a given positive function (f = 1 for (4), but we will consider general
positive function f ). Equation (19) is elliptic for 0 ≤ t < 1. We want to prove
180 P. Guan

that equation (19) has a unique smooth solution with a uniform bound on Δφ
(independent of t). We emphasize that V is important for the C 1 estimate solutions
to equation (19), and it also paves way for us to use Yau’s interior C 2 estimate
in [17]. We set u = V + φ, where φ is the solution of equation (19). Therefore, u
satisfies
⎧
⎨ det(uij̄ ) = (1 − t)f det(Vij̄ )
u|Γ1 = 1 (20)
⎩
u|Γ0 = 0.

Theorem 4. If M as in (15), there is a constant C depending only on M (inde-

pendent of t) such that for each 0 ≤ t < 1, there is a unique smooth solution u of
(19) with
|Δφ(z)| ≤ C, ∀z ∈ M. (21)

We first deduce Theorem 3 from Theorem 4, following the same lines of

arguments in [12].

Proof of Theorem 3. For each 0 ≤ t < 1, let φt be the solution of equation

(19). Set ut = V + φt . From Theorem 4, there is a sequence of strictly smooth
plurisubharmonic functions {ut} satisfying (20). By (21), there is a subsequence
{tk } that tends to 1, such that {utk } converges to a plurisubharmonic function u
in C 1,α (M ) for any 0 < α < 1. By the Convergence Theorem for complex Monge-
Ampère measures (see [3]), u satisfies equation (4). Again by (21), 0 ≤ Δu ≤ C.
For the sequence {uk }, we have

duk ∧ dc uk ∧ (ddc uk )n−1
M
 
= d uk ∧ (dd uk )
c c n−1
− uk (ddc uk )n
Γ1 M

n 
∂uk
= dc r ∧ (ddc r)n−1 − uk (ddc uk )n
Γ1 ∂r M

n 
∂uk
= dc r ∧ (ddc r)n−1 − (1 − tk ) uk (ddc V )n .
Γ1 ∂r M
 ∂u n  ∂u n
Since uk → u in C 1,α
(M ), ∂r
k
→ uniformly on Γ1 . Therefore,
∂r
 
n
∂u
dc u ∧ (ddc u)n−1 = dc r ∧ (ddc r)n−1 .
Γ1 Γ1 ∂r

The proof of Theorem 3.2 in [4] yields Ñ ({Γ1 }) = Γ1 dc u ∧ (ddc u)n−1 . Since
u = 1 on Γ1 , by the Stokes Theorem,

Ñ ({Γ1 }) = du ∧ dc u ∧ (ddc u)n−1 .
M
 Remarks on the Homogeneous Complex Monge-Ampère Equation 181

∀ v ∈ B if v ≡ u, one must have v < u in M int . By the Comparison Theorem,

 
du ∧ d u ∧ (dd u)
c c n−1
≤ dv ∧ dc v ∧ (ddc v)n−1 .
M M
That is  
du ∧ d u ∧ (dd u)
c c n−1
≤ inf dv ∧ dc v ∧ (ddc v)n−1 .
M v∈B M
On the other hand, by the Convergent Theorem for complex Monge-Ampère mea-
sures
 
lim inf duk ∧ dc uk ∧ (ddc uk )n−1 = du ∧ dc u ∧ (ddc u)n−1 .
k→∞ M M
That is,
 
Ñ (Γ1 ) = du ∧ d u ∧ (dd u)
c c n−1
= inf dv ∧ dc v ∧ (ddc v)n−1 . (22)
M v∈B

Finally, if T is homological to Γ1 , there is ω such that ∂ω = Γ1 − T . For any

v ∈ B,  
T (dc v ∧ (ddc v)n−1 ) = dc v ∧ (ddc v)n−1 − (ddc v)n .
Γ1 ω
Applying this to uk , we obtain
 
|T (d uk ∧ (dd uk )
c c n−1
)| ≥ d uk ∧ (dd uk )
c c n−1
− (1 − tk ) (ddc V )n .
Γ1 M
This implies
 
N (Γ1 ) ≥ d uk ∧ (dd uk )
c c n−1
− (1 − tk ) (ddc V )n .
Γ1 M
Taking k → ∞, 
N (Γ1 ) ≥ dc u ∧ (ddc u)n−1 = Ñ (Γ1 ).
Γ1
Since Ñ ({Γ1 }) ≥ N ({Γ1 }) by definition, we must have N (Γ1 ) = Ñ (Γ1 ). The
Chern-Levine-Nirenberg conjecture now follows from (22). 
The rest of this paper will be devoted to the proof of Theorem 4.
Proof of Theorem 4. We show that ∀ 0 ≤ t < 1, ∃! ut ∈ C ∞ , ut strongly plurisub-
harmonic, such that ut solves (20) and ∃ C > 0, ∀ 0 ≤ t < 1
0 ≤ Δu ≤ C. (23)
The uniqueness is a consequence of the comparison theorem for complex Monge-
Ampère equations. In the rest of the proof, we will drop the subindex t.
We first note that since u is plurisubharmonic in M 0 , and 0 ≤ u ≤ 1 on ∂M ,
the maximum principle gives 0 ≤ u(z) ≤ 1 ∀ z ∈ M . The estimate for Δu is also
easy. We have Δu = ΔV + Δφ = n + Δφ. Here we will make use of Yau’s estimate
[17]. Let Rij̄ij̄ be the holomorphic bisectional curvature of the Kähler metric g, let
C be a positive constant such that C + Rij̄ij̄ ≥ 2 for all i, j. Let ϕ = exp(−Cφ)Δu.
182 P. Guan

Lemma 1. [Yau] There is C1 depending only on supM −Δf , supM | inf i=j Rij̄ij̄ |,
supM f, n, if the maximum of ϕ is achieved at an interior point z0 , then
Δu(z0 ) ≤ C1 . (24)
By Yau’s interior C 2 estimate, we only need to get the estimates of the second
derivatives of u on the boundary of M . The boundary of M consists of pieces
of compact strongly pseudoconvex and pseudoconcave hypersurfaces. The second
derivative estimates on strongly pseudoconvex hypersurface have been established
in [6]. For general boundary under the existence of subsolution, the C 2 boundary
estimate was proved by Bo Guan [10]. In [12], following the arguments in [6, 10],
boundary C 2 estimates were established for M as in the form of (10), that is, M
is a domain in Cn . As these C 2 estimates are of local feature, they can be adapted
to general complex manifolds without any change. Therefore, we have a uniform
bound on Δu. Once Δu is bounded, the equation is uniformly elliptic and concave
(for each t < 1). The Evans-Krylov interior and the Krylov boundary estimates
can be applied here to get global C 2,α regularity (since they can be localized). In
fact, with sufficient smooth boundary data, the assumption of u ∈ C 1,γ for some
γ > 0 is sufficent to get global C 2,α regularity (e.g., see Theorem 7.3 in [7]).
What is left is the gradient estimate. We will prove C 1 estimate for solution
of equation (20) independent of Δu. We believe this type of estimate will be useful.
Lemma 2. Suppose φ satisfies equation
det(gij̄ + φij̄ ) = det(gij̄ )f, (25)
where gij̄ = Vij̄ for some smooth strictly plurisubharmonic function V and f is a
positive function. Let u = V + φ and W = |∇u|2 . There exist constants A and C2
1 1
depending only on supM f n , supM |∇f n |, supM |V |, inf M Rij̄ij̄ , if the maximum of
function H = eAV W is achieved at an interior point p, then
H(p) ≤ C2 . (26)
Let’s first assume Lemma 2 to finish the global C 1 estimate. We only need
to estimate ∇u on ∂M . Let h be the solution of
⎧
⎨ Δh = 0 in M 0
h|Γ1 = 1 (27)
⎩
h|Γ0 = 0.
Since 0 < det(uij̄ ) = (1 − t)f0 ≤ det(Vij̄ ), and
Δu > 0 = Δh,
and
u|∂M = V |∂M = h|∂M ,
by the Comparison Principle, V (z) ≤ u(z) ≤ h(z), ∀ z ∈ M . Therefore
|∇u(z)| ≤ max (|∇V (z)|, |∇h(z)|) ≤ c ∀ z ∈ ∂M, (28)
 Remarks on the Homogeneous Complex Monge-Ampère Equation 183

i.e., max∂M |∇u| ≤ c. In turn,

max |∇u| ≤ c. (29)
M
We now prove Lemma 2. Suppose the maximum of H is attained at some
interior point p. We pick a holomorphic orthonormal coordinate system at the
point such that (uij̄ ) = (gij̄ + φij̄ ) is diagonal at that point. We also have ∇gij̄ =
∇g αβ̄ = 0. We may also assume that W (p) ≥ 1.
All the calculations will be performed at p.
Wi Wī
+ AVi = 0, + AVī = 0. (30)
W W
We have

Wi = uαi uᾱ + uα uiᾱ , Wī = uαī uᾱ + uα uīᾱ ,
αβ̄
Wiī = g,iī uα uβ̄ + [|uiα |2 + uα uiīᾱ + uᾱ uiīα ] + u2iī ,

|Wi |2 = uᾱ uβ uiα uīβ̄ + |ui |2 u2iī + uī uiī uᾱ uiα + ui uiī uα uīᾱ .
By (30),

uᾱ uiα = −AW Vi − ui uiī , uα uīᾱ = −AW Vī − uī uiī ,
and by equation (25)
fα
(log det(uij̄ ))α = .
f
We have

|Wi |2 = | uᾱ uiα |2 − |ui |2 u2iī − AW uiī (Vi uī + Vī ui ).


Wiī |Wi |2
0≥ u iī
− + AViī
i
W W2


iī
g,iαīβ̄ uα uβ̄
= u + AViī
W
1
+ [uα (log det(uij̄ ))ᾱ + uᾱ (log det(uij̄ ))α ]
W $
  %
|uiα |2 | uα uīᾱ |2 Auiī (Vi uī + Vī ui )
+ u iī
− +
W W2 W

uiī 
|ui |2 uiī
+ +
W W2

g,iαīβ̄ uα uβ̄ 
1  
≥ uiī + AViī + 2 Re uα fᾱ (31)
W Wf
$
  %
|uiα | 2
| uα uīᾱ |2 Auiī
+ uiī − + 2 Re(V u
i ī ) .
W W2 W
184 P. Guan

αβ̄
g,i u u
ī α β̄
Since W is controlled by inf M Rij̄ij̄ , it follows from the Cauchy-Schwartz
inequality,
 
|uα fα | |Vi uī |
0≥ uiī (inf Rkl̄kl̄ + AViī ) − 2 − 2A (32)
M fW W
Now we may pick A sufficient large, such that
inf Rkl̄kl̄ + A ≥ 1.
M

This yields
 
A|∇V | + |∇ log f | 1
1
Af n |∇V | + |∇f n |
1

0≥ u −2
iī
1 ≥ nf − n 1−2 1 . (33)
W2 W2
Lemma 2 follows directly from (33). 

Added in proof
A general gradient estimate for complex Monge-Ampère equation det(gij̄ + φij̄ ) =
f det(gij̄ ) on Kähler manifolds has been proved by Blocki in [5], also by the author
in [13]. We would like to thank Phillipe Delanoe for pointing out to us that this
type of gradient estimates in fact were proved by Hanani in [14] in general setting
on Hermitian manifolds.

References
[1] E. Bedford, Stability of envelopes of holomorphy and the degenerate Monge-Ampère
equation, Math. Ann. 259 (1982), 1–28.
[2] E. Bedford and J.-E. Forneass, Counterexamples to regularity for the complex Monge-
Ampère equation, Invent. Math. 50 (1979), 129–134.
[3] E. Bedford and A. Taylor, A new capacity for pluriharmonic functions, Acta Math.
149 (1982), 1–40.
[4] E. Bedford and A. Taylor, Variational properties of the complex Monge-Ampère equa-
tion, II. Intrinsic norms, Amer. J. Math., 101 (1979), 1131–1166.
[5] Z. Blocki, A gradient estimate in the Calabi-Yau theorem, Math. Ann. 344 (2009),
317–327.
[6] L. Caffarelli, J. Kohn, L. Nirenberg, and J. Spruck, The Dirichlet problem for nonlin-
ear second-order elliptic equations, II: Complex Monge-Ampère and uniformly elliptic
equations, Commu. Pure Appl. Math., 38 (1985), 209–252.
[7] Y. Chen and L. Wu, Second-order elliptic equations and systems of elliptic equations,
Science Press, Beijing, 1991.
[8] X. Chen, The space of Kähler metrics, J. Differential Geom. 56 (2000), 189–234.
[9] S.S. Chern, H. Levine and L. Nirenberg, Intrinsic norms on a complex manifold,
Global Analysis, Papers in honor of K. Kodaira, Princeton Univ. Press, (1969), 119–
139.
 Remarks on the Homogeneous Complex Monge-Ampère Equation 185

[10] B. Guan, The Dirichlet problem for complex Monge-Ampère equations and the regu-
larity of the pluri-complex Green function, Comm. Anal. and Geom. 6 (1998), 687–
703.
[11] B. Guan and J. Spruck, Boundary value problem on S n for surfaces of constant
Gauss curvature, Ann. of Math., 138 (1993), 601–624.
[12] P. Guan, Extremal Function associated to Intrinsic Norms, Annals of Mathematics,
156 (2002), 197–211.
[13] P. Guan, A gradient estimate for complex Monge-Ampère equation, private circula-
tion.
[14] A. Hanani, Équations du type de Monge-Ampère sur les variétés hermitiennes com-
pactes. J. Funct. Anal. 137 (1996), 49–75.
[15] D. Hoffman, H. Rosenberg and J. Spruck, Boundary value problem for surfaces of
constant Gauss curvature. Comm. Pure and Appl. Math., 45, (1992), 1051–1062.
[16] R. Moriyon, Regularity of the Dirichlet problem for the degenerate complex Monge-
Ampère equation, Comm. Pure and Appl. Math., 35 (1982), 1–27.
[17] S.T. Yau, On the Ricci curvature of a complex Kähler manifold and the complex
Monge-Ampère equation, I, Comm. Pure and Appl. Math., 31 (1978), 339–411.

Pengfei Guan
Department of Mathematics and Statistics
McGill University
Montreal, Quebec, H3A 2K6, Canada
e-mail: [email protected]
Complex Analysis
Trends in Mathematics, 187–203

c 2010 Springer Basel AG

A Radó Theorem for Locally

Solvable Structures of Co-rank One
J. Hounie
Dedicated to Linda Rothschild

Abstract. We extend the classical theorem of Radó to locally solvable struc-

tures of co-rank one. One of the main tools in the proof is a refinement of the
Baouendi-Treves approximation theorem that may be of independent interest.

Mathematics Subject Classification (2000). Primary 35F05, 35B60; Secondary

35D99.
Keywords. Radó theorem, locally integrable structures, local solvability,
Baouendi-Treves approximation theorem.

1. Introduction
A classical theorem of Radó, in the form given by Cartan, states that a continuous
function defined on an open set of the complex plane which is holomorphic outside
the closed set where it vanishes is holomorphic everywhere. This theorem implies
easily that the same result also holds for functions of several complex variables.
Radó’s theorem may be regarded as a theorem about removing singularities of the
Cauchy-Riemann operator, but in that theory it is customary to impose additional
restrictions on the set outside which the equation holds and is wished to be re-
moved (for instance, the set to be removed may be required to have null capacity
or to have null or bounded Hausdorff measure of some dimension). The beauty of
the classical result of Radó lies in the fact that the set u−1 (0) is removed without
any assumption about its size or geometric properties. The theorem was extended
by replacing the set u−1 (0) by u−1 (E) where E is a compact subset of null ana-
lytic capacity ([St]) or is a null-set for the holomorphic Dirichlet class ([C2]). A
generalization for a more general class of functions was given by Rosay and Stout

Work supported in part by CNPq, FINEP and FAPESP.

188 J. Hounie

[RS] who extended Radó’s result to CR functions on strictly pseudoconvex hyper-

surfaces of Cn and other extension (in the spirit of removing singularities of CR
functions) was given in [A]. For homogeneous solutions of locally solvable vector
fields with smooth coefficients, a Radó type theorem was proved in [HT].
In this paper we extend Radó’s theorem to homogeneous solutions of locally
integrable structures of co-rank one that are locally solvable in degree one. Thus,
we deal with an overdetermined system of equations
⎧
⎪
⎪ L1 u = 0,
⎪
⎨L u = 0,
2
(1.1)
⎪
⎪ · · ·······
⎪
⎩
Ln u = 0,
where L1 , . . . , Ln , n ≥ 1, are pairwise commuting smooth complex vector fields
defined on an open subset of Rn+1 and assume that this system of vector fields has
local first integrals at every point and it is solvable in the sense that the equation
⎧
⎪
⎪ L1 u = f 1 ,
⎪
⎨L u = f ,
2 2
(1.2)
⎪
⎪ · · · · · · · · ·
⎪
⎩
Ln u = f n ,
can be locally solved for all smooth right-hand sides that satisfy the compatibility
conditions Lj fk = Lk fj , 1 ≤ j, k ≤ n (see Section 3 for precise statements). Under
these condition it is shown that if u is continuous and satisfies (1.1) outside u−1 (0)
then it satisfies (1.1) everywhere (Theorem 4.1). This solvability hypothesis can
be characterized in terms of the connectedness properties of the fibers of local
first integrals ([CT], [CH]) and this characterization is one of the main ingredients
in the proof of Theorem 4.1, which is given in Section 4. Another key tool is a
refinement of the Baouendi-Treves approximation theorem, which seems to have
interest per se and it is stated and proved in Section 2 for general locally integrable
structures. In Section 5 we apply Theorem 4.1 to obtain a result on uniqueness in
the Cauchy problem for continuous solutions with Cauchy data on rough initial
surfaces.

2. The approximation theorem

The approximation formula ([BT1], [BT2], [T1], [T2], [BCH]) is of local nature
and we will restrict our attention to a locally integrable structure L defined in an
open subset Ω of RN over which L⊥ is spanned by the differentials dZ1 , . . . , dZm
of m smooth functions Zj ∈ C ∞ (Ω), j = 1, . . . , m, at every point of Ω. Thus, if n
is the rank of L, we recall that N = n + m.
Given a continuous function u ∈ C(Ω) we say that u is a homogeneous
solution of L and write Lu = 0 if, for every local section L of L defined on an open
 A Radó Theorem for Locally Solvable Structures 189

subset U ⊂ Ω,
Lu = 0 on U in the sense of distributions.
Simple examples of homogeneous solutions of L are the constant functions and
also the functions Z1 , . . . , Zm , since LZj = dZj , L
= 0 because dZj ∈ L⊥ ,
j = 1, . . . , m. By the Leibniz rule, any product of smooth homogeneous solutions
is again a homogeneous solution, so a polynomial with constant coefficients in the
m functions Zj , i.e., a function of the form

P (Z) = cα Z α , α = (α1 , . . . , αm ) ∈ Zm , cα ∈ C, (2.1)
|α|≤d

is also a homogeneous solution. The classical approximation theorem for contin-

uous functions states that any continuous solution u of Lu = 0 can be uniformly
approximated by polynomial solutions such as (2.1). More precisely:
Theorem 2.1. Let L be a locally integrable structure on Ω and assume that dZ1 , . . . ,
dZm span L⊥ at every point of Ω. Then, for any p ∈ Ω, there exist two open sets
U and W , with p ∈ U ⊂ U ⊂ W ⊂ Ω, such that
every u ∈ C(W ) that satisfies Lu = 0 on W is the uniform limit of a sequence
of polynomial solutions Pj (Z1 , . . . , Zm ):
u = lim Pj ◦ Z uniformly in U .
j→∞

In this section we will prove a refinement of the approximation theorem (The-

orem 2.2) that we will later use in the proof of the paper’s main result. In order
to prove this variation, we start by reviewing the main steps in the proof of the
classical approximation theorem. The first one is to choose local coordinates
{x1 , . . . , xm , t1 , . . . , tn }
defined on a neighborhood of the point p and vanishing at p so that, for some
smooth, real-valued functions ϕ1 , . . . , ϕm defined on a neighborhood of the origin
and satisfying
ϕk (0, 0) = 0, dx ϕk (0, 0) = 0, k = 1, . . . , m,
the functions Zk , k = 1, . . . , m, may be written as
Zk (x, t) = xk + iϕk (x, t), k = 1, . . . , m, (2.2)
on a neighborhood of the origin. Then we choose R > 0 such that if
V = {q : |x(q)| < R, |t(q)| < R}
then, on a neighborhood of V we have
(
(
( ∂ϕj (x, t) ( 1
( (< , (x, t) ∈ V , (2.3)
( ∂xk ( 2
where the double bar indicates the norm of the matrix ϕx (x, t) = (∂ϕj (x, t)/∂xk )
as a linear operator in Rm . Modifying the functions ϕk ’s off a neighborhood of V
we may assume without loss of generality that the functions ϕk (x, t), k = 1, . . . , m,
190 J. Hounie

are defined throughout RN , have compact support and satisfy (2.3) everywhere,
that is (
(
( ∂ϕj (x, t) ( 1
( ( < , (x, t) ∈ RN . (2.3 )
( ∂xk ( 2
Modifying also L off a neighborhood of V we may assume as well that the differ-
entials dZj , j = 1, . . . , m, given by (2.2), span L⊥ over RN . Of course, the new
structure L and the old one coincide on V so any conclusion we draw about the
new L on V will hold as well for the original L. The vector fields
m
∂
Mk = μk (x, t) , k = 1, . . . , m,
∂x
=1
characterized by the relations
Mk Z = δk k,  = 1, . . . , m,
and the vectors fields
∂ ∂ϕkm
Lj = −i (x, t)Mk , j = 1, . . . , n,
∂tj ∂tj
k=1
are linearly independent and satisfy Lj Zk = 0, for j = 1, . . . , n, k = 1, . . . , m.
Hence, L1 , . . . , Ln span L at every point while the N = n + m vector fields
L 1 , . . . , Ln , M 1 . . . , M m ,
are pairwise commuting and span CTp (RN ), p ∈ RN . Since
dZ1 , . . . , dZm , dt1 , . . . , dtn span CT ∗ RN
the differential dw of a C 1 function w(x, t) may be expressed in this basis. In fact,
we have
n
m
dw = Lj w dtj + Mk w dZk
j=1 k=1
which may be checked by observing that Lj Zk = 0 and Mk tj = 0 for 1 ≤ j ≤ n
and 1 ≤ k ≤ m, while Lj tk = δjk for 1 ≤ j, k ≤ n and Mk Zj = δjk for 1 ≤ j, k ≤ m
(δjk = Kronecker delta). At this point, the open set W in the statement of Theorem
2.1 is chosen as any fixed neighborhood of V in Ω. That u ∈ C(W ) satisfies
Lu = 0 is equivalent to saying that it satisfies on W the overdetermined system
of equations ⎧
⎪
⎪ L1 u = 0,
⎪
⎨L u = 0,
2
(2.4)
⎪· · · · · · · · ·
⎪
⎪
⎩
Ln u = 0.
Given such u we define a family of functions {Eτ u} that depend on a real parameter
τ , 0 < τ < ∞, by means of the formula

 2
Eτ u(x, t) = (τ /π)m/2
e−τ [Z(x,t)−Z(x ,0)] u(x , 0)h(x ) det Zx (x , 0) dx
Rm
 A Radó Theorem for Locally Solvable Structures 191

which we now discuss. For ζ = (ζ1 , . . . , ζm ) ∈ Cm we will use the notation [ζ]2 =
ζ12 + · · · + ζm
2
, which explains the meaning of [Z(x, t) − Z(x , 0)]2 in the formula.
The function h(x) ∈ Cc∞ (Rm ) satisfies h(x) = 0 for |x| ≥ R and h(x) = 1 in a
neighborhood of |x| ≤ R/2 (R was defined right before (2.3)). Since u is assumed
to be defined in a neighborhood of V , the product u(x , 0)h(x ) is well defined on
Rm , compactly supported and continuous. Furthermore, since the exponential in
the integrand is an entire function of (Z1 , . . . , Zm ), the chain rule shows that it
satisfies the homogenous system of equations (2.4) and the same holds for Eτ u(x, t)
by differentiation under the integral sign. Then Theorem 2.1 is proved by showing
that Eτ u(x, t) → u(x, t) as τ → ∞ uniformly for |x| < R/4 and |t| < T < R if
T is conveniently small. In particular, the set U in the statement of Theorem 2.1
may be taken as
U = B1 × B2 ,
B1 = {x ∈ Rm : |x| < R/4},
B2 = {t ∈ R : n
|t| < T }.
2
Once this is proved, approximating the exponential e−τ [ζ] (for fixed large τ ) by
the partial sum of√degree k, Pk (ζ), of its Taylor series on a fixed polydisk that
contains the set { τ (Z(x, t) − Z(x , 0)) : |x|, |x | < R, |t| < R}, a sequence of
polynomials in Z(x, t) that approximate uniformly Eτ u(x, t) for |x| < R/4 and
|t| < T as k → ∞ is easily constructed.
Thus, the main task is to prove that Eτ u(x, t) → u(x, t) as τ → ∞ uniformly
on B1 × B2 and in order to do that one proves first that a convenient modification
of the operator Eτ , to wit,

 2
Gτ u(x, t) = (τ /π)m/2 e−τ [Z(x,t)−Z(x ,t)] u(x , t)h(x ) det Zx (x , t) dx ,
Rm
converges uniformly to u on B1 × B2 as τ → ∞. This is easy because (2.3 ) ensures
that the operator u → Gτ u is very close to the convolution of u with a Gaussian,
which is a well-known approximation of the identity. In particular, the uniform
convergence Gτ → u holds for any continuous u and it is irrelevant at this point
whether u satisfies the equation Lu = 0 or not. After Gτ u → u has been proved,
it remains to estimate the difference Rτ u = Gτ u − Eτ u and it is here that the fact
that Lu = 0 is crucial.
Let B1 and B2 be as described before in the outline of the proof of Theorem
2.1 and set B̃1 = {x ∈ Rm : |x| < R}. In the version we want to prove u need
not be a solution in a neighborhood of V but on a smaller open subset of B̃1 × B2 .
With this notation we have
Theorem 2.2. Let u be continuous on B̃ 1 × B 2 and assume that there is an open
and connected set ω, 0 ∈ ω ⊂ B2 , such that
Lu = 0, on B̃1 × ω.
Then Eτ u(x, t) → u(x, t) uniformly on compact subsets of B1 × ω as τ → ∞.
192 J. Hounie

Remark 2.3. The main point in Theorem 2.2 is that as soon as the equation holds
on B̃1 × ω, in order to obtain a set where the approximation holds we do not need
to replace ω by a smaller subset, although we must shrink B̃1 to B1 and the radius
T of B2 has been initially taken small as compared to the radius R of B̃1 . Under
some more restrictive circumstances, we may even avoid taking T is small, as the
proof of Theorem 4.1 below shows.
Proof. The formula that defines Eτ u(x, t) only takes into account the values of
u(x, 0). It will be enough to prove that Eτ u → u uniformly on compact subsets of
B1 × ω as j → ∞. The argument that shows in the classical setup that Gτ u → u
uniformly on B̃1 × B2 applies here word by word, because it only uses the fact that
u is continuous on the closure of B̃1 × B2 and it is carried out by freezing t ∈ B2
and showing that Gτ u is an approximation of the identity on Rm , uniformly in
t ∈ B2 . Hence, the proof is reduced to showing that Rτ u = Gτ u − Eτ u converges
uniformly to 0 on compact subsets of B1 × ω.
When u satisfies Lu = 0 throughout B̃1 × B2 , we have the formula
 n
Rτ u(x, t) = rj (x, t, t , τ ) dtj , (2.5)
[0,t] j=1

where
rj (x, t, t , τ ) = (2.6)

  2
(τ /π)m/2 e−τ [Z(x,t)−Z(x ,t )] u(x , t )Lj h(x , t ) det Zx (x , t ) dx
Rm

and [0, t] denotes the straight segment joining 0 to t. This may be shown by writing
for fixed ζ and τ

  2
g(t ) = G̃τ u(ζ, t ) = e−τ [ζ−Z(x ,t )] u(x , t )h(x ) det Zx (x , t ) dx
Rm

and applying the fundamental theorem of calculus


n
∂g  
g(t) − g(0) = (t ) dtj . (2.7)
[0,t] ∂tj
j=1

Then a computation that exploits that Lj u = 0, j = 1, . . . , n, shows that (see

[BCH, p. 64] for details)
∂g 
(t ) = r̃j (ζ, t , τ ) (2.8)
∂tj
where

  2
r̃j (ζ, t , τ ) = e−τ [ζ−Z(x ,t )] u(x , t )Lj h(x , t ) det Zx (x , t ) dx .
Rm

Hence, (2.7) for ζ = Z(x, t) shows that Rτ u = Gτ u − Eτ u is given by (2.5) and

(2.6).
 A Radó Theorem for Locally Solvable Structures 193

Let’s return to the case in which u is only known to satisfy Lj u = 0,

j = 1, . . . , n, on B̃1 × ω. We still have, for t ∈ ω,
 n
∂g  
g(t) − g(0) =  (t ) dtj . (2.7 )
γt j=1 ∂tj

where γt denotes a polygonal path contained in ω that joins the origin to t. On

the other hand, (2.8) remains valid in the new situation. This is true because its
proof depends on integration by parts with respect to x – which can be performed
as well on B̃1 × ω – and on local arguments. Thus, we get

n
Rτ u(x, t) = rj (x, t, t , τ ) dtj , (x, t) ∈ B̃1 × ω. (2.5 )
γt j=1

This gives the estimate

|Rτ u(x, t)| ≤ C|γt | max sup |rj (x, t, t , τ )|, (x, t) ∈ B1 × ω.
1≤j≤n t ∈ω

However, due to the fact that the factor Lj h(x , t ) vanishes for |x | ≥ R/2, we
have
# #
# −τ [Z(x,t)−Z(x ,t )]2 #
#e # ≤ e−cτ , (x, t) ∈ B1 × B2 , |x | ≥ R/2, t ∈ B2 ,

for some c > 0. This follows, taking account of (2.3 ), from

[Z(x, t) − Z(x , t )]2 ≥ |x − x |2 − |ϕ(x, t) − ϕ(x , t )|2

|x − x |2
≥ − |ϕ(x , t) − ϕ(x , t )|2
2
≥ c. (2.9)

Note that |x − x | ≥ R/4 for |x | ≥ R/2 and |x| ≤ R/4, while the term |ϕ(x , t) −
ϕ(x , t )| ≤ C|t − t | will be small if t and t are both small which may be obtained
by taking T small. Thus

|Rτ u(x, t)| ≤ C|γt |e−cτ , (x, t) ∈ B1 × ω. (2.10)

If K ⊂⊂ ω, there is a constant CK such that any t ∈ K can be reached from

the origin by a polygonal line of length bounded by CK so (2.10) shows that
|Rτ u(x, t)| → 0 uniformly on B1 × K. 

Corollary 2.4. Under the hypotheses of the theorem, there is a sequence of polyno-
mial solutions Pj (Z1 , . . . , Zm ) that converges uniformly to u on compact subsets
of B1 × ω as j → ∞. 
194 J. Hounie

3. Structures of co-rank one

A smooth locally integrable structure L of rank n ≥ 1 defined on an open subset
Ω ⊂ Rn+1 is said to be a structure of co-rank one. Thus L⊥ is locally spanned by
a single function Z that, in appropriate local coordinates (x, t1 , . . . , tn ) centered
around a given point, may be written as
Z(x, t) = x + iϕ(x, t), |x| < a, |t| < r,
where ϕ(x, t) is smooth, real valued and satisfies ϕ(0, 0) = ϕx (0, 0) = 0. Then, L
is locally spanned by the vector fields
∂ ϕtj ∂
Lj = −i , j = 1, . . . , n,
∂tj 1 + iϕx ∂x
.
on X = (−a, a) × {|t| < r}. It turns out that [Lj , Lk ] = 0, 1 ≤ j, k ≤ n. Given an
open set Y ⊂ X, consider the space of p-forms
3 4 5
.
C ∞ (Y, p ) = u = uJ (x, t)dtJ , uJ ∈ C ∞ (Y )
|J|=p

as well as the differential complex

3 3
L : C ∞ (Y, p
) −→ C ∞ (Y, p+1
)
defined by

n
L= Lj uJ (x, t) dtj ∧ dtJ .
|J|=p j=1

The fact that L2 = 0 ensues from the relations [Lj , Lk ] = 0, 1 ≤ j, k ≤ n.

Definition 3.1. The operator L is said to be solvable at ω0 ∈ Ω in 6 degree q,
1 ≤ q ≤ n, if for every open neighborhood Y ⊂ X of ω0 and f ∈ C ∞ (Y, q ) such
that Lf = 0,6there exists an open neighborhood Y  ⊂ Y of ω0 and a (q − 1)-form
u ∈ C ∞ (Y  , q−1 ) such that Lu = f in Y  . If this holds for every ω0 ∈ Ω we say
that the structure L is locally solvable in Ω in degree q.
If z0 = x0 + iy0 ∈ C and Y ⊂ X we will refer to the set
F (z0 , Y ) = {(x, t) ∈ Y : Z(x, t) = z0 } = Z −1 (z0 ) ∩ Y
as the fiber of the map Z : X −→ C over Y . The local solvability in degree q can be
characterized in terms of the homology of the fibers of Z for any degree 1 ≤ q ≤ n,
as it was conjectured by Treves in [T3]. The full solution of this conjecture took
several years (see [CH] and the references therein). In this paper we will only
consider locally solvable structures of co-rank one that are locally solvable in degree
q = 1. For q = 1, the geometric characterization of local solvability at the origin
means that ([CT],[CH],[MT]), given any open neighborhood X of the origin there
is another open neighborhood Y ⊂ X of the origin such that, for every regular
value z0 ∈ C of Z : X −→ C, either F (z0 , Y ) is empty or else the homomorphism
H̃0 (F(z0 , Y ), C) −→ H̃0 (F (z0 , X), C) (∗)0
 A Radó Theorem for Locally Solvable Structures 195

induced by the inclusion map F(z0 , Y ) ⊂ F(z0 , X) is identically zero. We are de-
noting by H̃0 (M, C) the 0th reduced singular homology space of M with complex
coefficients. In other words, (∗)0 means that there is at most one connected com-
ponent Cq of F (z0 , X) that intersects Y . Thus, if q ∈ Y , z0 = Z(q) is a regular
value and Cq is the connected component of F (z0 , X) that contains q, it follows
that
F (z0 , Y ) = Y ∩ F(z0 , X) = Y ∩ Cq .

4. A Radó theorem for structures of co-rank one

Consider a smooth locally integrable structure L of rank n ≥ 1 defined on an open
subset Ω ⊂ Rn+1 . A function u defined on Ω is a Radó function if
i) u ∈ C(Ω) and
ii) satisfies the differential equation Lu = 0 on Ω \ u−1 (0), in the weak sense.
We say that L has the Radó property if every Radó function is a homogeneous
solution on Ω, i.e., the singular set where u vanishes and where the equation is a
priori not satisfied can be removed and the equation Lu = 0 holds everywhere.

Theorem 4.1. Assume that L is locally solvable in degree 1 in Ω. Then L has the
Radó property.

The Radó property has a local nature: it is enough to show that a Radó
function u satisfies the equation Lu = 0 in a neighborhood of an arbitrary point
p ∈ Ω such that u(p) = 0. We may choose local coordinates x, t1 , . . . , tn such that
x(p) = tj (p) = 0, j = 1, . . . , n, in which a first integral Z(x, t) has the form
Z(x, t) = x + iϕ(x, t), |x| ≤ 1, |t| = |(t1 , . . . , tn )| ≤ 1,

where ϕ is real valued and ϕ(0, 0) = ϕx (0, 0) = 0. Let us write

I = (−1, 1), B = {t : |t| < 1}, X = I × B.
By Theorem 2.1, we may further assume without loss of generality that any contin-
uous solution of Lv = 0 defined on a neighborhood of |x| ≤ 1, |t| ≤ 1, is uniformly
approximated by Eτ v for |x| < a, |t| < T , where a and T are convenient positive
small values. Of course, we cannot apply this to our Radó function u since u is
not known to satisfy the equation everywhere.
In order to prove that Lu = 0 in some neighborhood of the origin we will
consider different cases.

Case 1
Assume that ∇t ϕ(0, 0) = 0, say ∂ϕ(0, 0)/∂t1 = 0 (this is the simple elliptic case).
Then, replacing the coordinate function t1 by ϕ and leaving t2 , . . . , tn unchanged,
196 J. Hounie

we obtain a local change of coordinates defined in a small ball centered at the

origin. In the new coordinates we have ϕ(x, t) ≡ t1 . Then the system (2.4) becomes
⎧

⎪
⎪ ∂ ∂
⎪
⎪ −i u = 0,
⎪
⎪ ∂t1 ∂x
⎪
⎪
⎪
⎨ ∂
u = 0,
∂t2
⎪
⎪
⎪
⎪ ············
⎪
⎪
⎪
⎪ ∂
⎪
⎩ u = 0.
∂tn
Take an arbitrary point p = (x0 , τ1 , τ  ), τ  = (τ2 , . . . , τn ) such that u(p) = 0
and a cube Q centered at p that does not intersect the zero set of u. Choosing Q
sufficiently small, we may approximate u uniformly on Q by polynomials in the first
integral Z = x+it1 . Then, for fixed t , the restricted function ut (t1 , x) = u(x, t1 , t )
is a holomorphic function of x + it1 on a slice of Q. Keeping τ  = t0 fixed and
varying (x0 , τ1 ), it turns out that ut0 (t1 , x) = u(x, t1 , t0 ) is a holomorphic function
of x + it1 outside its zero set so, by the classical Radó theorem it is a holomorphic
everywhere. Similarly, keeping (x0 , τ1 ) fixed and letting t vary, we see that the
function t → u(x0 , τ1 , t ) is locally constant on the set {t : u(x0 , τ1 , t ) = 0},
thus constant on its connected components. The continuity of u then shows that
t → u(x0 , τ1 , t ) is constant for fixed (x0 , τ1 ). Hence, u is independent of t =
(t2 , . . . , tn ) and the restricted function ut0 (t1 , x) = u(x, t1 , t0 ) is a holomorphic
function of x + it1 so u satisfies Lu = 0 in a neighborhood of the origin.
We already know that ϕx (0, 0) = 0 and in view of Case 1 we will assume
from now on that
∇x,t ϕ(0, 0) = 0. (4.1)
We recall that, for z0 = x0 + iy0 ∈ C and Y ⊂ X, the set
F(z0 , Y ) = {(x, t) ∈ Y : Z(x, t) = z0 } = Z −1 (z0 ) ∩ Y
is referred to as the fiber of the map Z : X −→ C over Y . To deal with the next
cases, the following lemma will be important; its proof is a consequence of the
classical approximation theorem.
Lemma 4.2. The Radó function u is constant on the connected components of the
fibers of Z over X.
Proof. Let q ∈ X. Assume first that u(q) = 0. Then u is a homogeneous solution of
L on a neighborhood of q and by a standard consequence of Theorem 2.1 applied
at the point q, u must be constant on the fibers of some first integral Z1 of L
defined on a sufficiently small neighborhood of q. The germs of the fibers at q are
invariant objects attached to L and do not depend on the particular first integral,
i.e., replacing Z1 by Z, there exists a neighborhood W of q such that u is constant
on F(z0 , W ), z0 = Z(q) = x0 + iy0 . Let {x0 } × Cq be the connected component
of F(z0 , X) \ u−1(0) that contains q and denote by {x0 } × Cq# the connected
 A Radó Theorem for Locally Solvable Structures 197

component of F (z0 , X) that contains q, so Cq ⊂ Cq# . We have seen that u is locally

constant on {x0 } × Cq , hence it assumes a constant value c = 0 on {x0 } × Cq . This
implies that u cannot vanish at any point on the closure of {x0 } × Cq in X and
therefore Cq is both open and closed in Cq# so Cq = Cq# . Hence, {x0 } × Cq is a
connected component of F(z0 , X) and the lemma is proved in this case.
Assume now u(q) = 0 and let {x0 } × Cq be the connected component of
F(z0 , X) that contains q. We will show that {x0 } × Cq ⊂ u−1 (0). Suppose there
exists a point q1 ∈ {x0 } × Cq \ u−1 (0) = ∅. By our previous reasoning, u would
assume a constant value c = 0 on the connected component {x0 } × Cq1 of F (z0 , X)
that contains q1 . Since q1 ∈ {x0 } × (Cq1 ∩ Cq ) we should have Cq1 = Cq and
consequently u(q) = c = 0, a contradiction. Hence, u vanishes identically on
F (z0 , X). 
At this point we will exploit the assumption that L is locally solvable in
degree one. By the geometric characterization of locally solvable structures of co-
rank one, we know that given any open neighborhood X of the origin there is
another open neighborhood Y ⊂ X of the origin such that, for every regular value
z0 ∈ C of Z : X −→ C, either F(z0 , Y ) is empty or else the homomorphism
H̃0 (F(z0 , Y ), C) −→ H̃0 (F (z0 , X), C) (∗)0
induced by the inclusion map F(z0 , Y ) ⊂ F(z0 , X) is identically zero. As men-
tioned at the end of Section 3, this implies that if q ∈ Y , z0 = Z(q) is a regular
value and Cq is the connected component of F (z0 , X) that contains q, it follows
that
F (z0 , Y ) = Y ∩ F(z0 , X) = Y ∩ Cq .
By Lemma 4.2 u is constant on Cq . This shows that u is constant on the regular
fibers F (z0 , Y ) of Z over Y but, using Sard’s theorem, a continuity argument
shows that u is constant on all fibers F(z0 , Y ), whether regular or not. Hence,
after restricting u to Y , we may write
u(x, t) = U ◦ Z(x, t), (x, t) ∈ Y , (4.2)
with U ∈ C (Z(Y )). Once U (x, y) has been defined, (4.2) will still hold if we
0

replace Y by a smaller neighborhood of the origin. Thus, redefining I and B as

I = (−a, a), B = {|t| ≤ T }, with 0 < a < 1, 0 < T < 1 conveniently small, we
may assume from the start that
u(x, t) = U ◦ Z(x, t), (x, t) ∈ X.
Shrinking I and B further if necessary we may also assume, recalling (4.1), that
1
|∇x,t ϕ(x, t)| ≤ for (x, t) ∈ X. (4.3)
24
Write
M (x) = sup ϕ(x, t), m(x) = inf ϕ(x, t)
|t|≤T |t|≤T
so 7 8
Z([−a, a] × B) = x + iy : |x| ≤ a, m(x) ≤ y ≤ M (x) .
198 J. Hounie

The functions M (x) and m(x) are continuous and we may write

N
{x ∈ (−a, a) : m(x) < M (x)} = Ij
j=1

where Ij ⊂ (−a, a) is an open interval for 1 ≤ j ≤ N ≤ ∞. We also write

/N
N = (−a, a) \ j=1 Ij and
Dj = {x + iy : x ∈ Ij , m(x) < y < M (x)}, 1 ≤ j ≤ N.

Case 2
/
We suppose now that 0 = Z(0, 0) ∈ j Dj , i.e., we will assume that 0 ∈ Ij for
some j (that we may take as j = 1) and m(0) < 0 = ϕ(0, 0) < M (0). If (0, 0) is
in the interior of u−1 (0), it is clear that Lu vanishes in a neighborhood of (0, 0)
as we wish to prove, so we may assume that u does not vanish identically in any
neighborhood of (0, 0). Hence, there are points q = (x0 , t0 ) ∈ I1 × B such that
. .
z0 = Z(q) = x0 + iϕ(x0 , t0 ) = x0 + iy0 ∈ D1 and u(q) = 0. Let {x0 } × Cq be
the connected component of F(z0 , I1 × B) that contains q. Cover Cq with a finite
number L of balls of radius δ > 0 centered at points of Cq and call ωδ the union
of these balls. Notice that any two points in ωδ can be joined by a polygonal line
γ of length |γ| ≤ (L + 2)δ. Thus, ωδ is a connected open set that contains Cq
and, since m(x0 ) < y0 < M (x0 ), there is no restriction in assuming that q has
been chosen so that ϕ(x0 , t) assumes on ωδ some values which are larger than
y0 as well as some values which are smaller than y0 . Indeed, consider a smooth
curve γ(s) : [0, 1] −→ B such that for some 0 < s1 < 1, ϕ(x0 , γ(0)) = m(x0 ),
ϕ(x0 , γ(s1 )) = y0 , ϕ(x0 , γ(1)) = M (x0 ) and ϕ(x0 , γ(s)) ≤ y0 for any 0 ≤ s ≤ s1 .
Having fixed γ(s), we may choose a largest s1 ∈ (0, 1) with that property. This
means that there are points s > s1 arbitrarily close to s1 such that ϕ(x0 , γ(s)) > y0 .
Let [s0 , s1 ] be the connected component of
{s ∈ [0, 1] : ϕ(x0 , γ(s)) = y0 }
that contains s1 (note that 0 < s0 ≤ s1 < 1). Set q = (x0 , γ(s0 )), q  = (x0 , γ(s1 ))
and notice that γ(s) ∈ Cq for s0 ≤ s ≤ s1 by connectedness so q  ∈ Cq . For any
ε > 0 there exist ε0 , ε1 ∈ (0, ε) such that γ(s0 − ε0 ) and γ(s1 + ε1 ) are points in
ωδ on which ϕ(x0 , t) takes values respectively smaller and larger than y0 .
Next, for small δ > 0, we consider the approximation operator Eτ u on (x0 −
.
δ, x0 +δ)×B = Iδ ×B (with initial trace taken at t = t0 ) and we wish to prove that
Eτ u converges to u uniformly on Iδ × ωδ , after choosing δ > 0 sufficiently small
to ensure that u does not vanish on Iδ × ωδ and therefore satisfies the equation
Lu = 0 there. This is proved almost exactly as Theorem 2.2. The main difference
is that here we do not want to shrink the ball B, so in order to prove the crucial
estimate (2.9) we use instead (4.3) to show that the oscillation of ϕ on ωδ is ≤ δ/6.
In fact, for |x − x0 | ≤ δ and t, t ∈ ωδ , we have
|ϕ(x , t) − ϕ(x , t )| ≤ |ϕ(x , t) − y0 | + |y0 − ϕ(x , t )|.
 A Radó Theorem for Locally Solvable Structures 199

Given t ∈ ωδ , there exists t• ∈ Cq such that |t − t• | < δ. Then

|x − x0 | + |t − t• |
|ϕ(x , t) − y0 | = |ϕ(x , t) − ϕ(x0 , t• )| ≤
24
δ
≤ .
12
Similarly, |y0 − ϕ(x , t )| ≤ δ/12, so
δ
|ϕ(x , t) − ϕ(x , t )| ≤ , |x − x0 | ≤ δ, t, t ∈ ωδ .
6
Thus, we obtain
|x − x |2 δ2 δ2
− |ϕ(x , t) − ϕ(x , t )|2 ≥ − ≥c>0
2 16 36
for |x − x0 | ≤ δ/4 and |x − x0 | ≥ δ/2. The arguments in the proof of Theorem
2.2 allow us to show that Eτ u → u uniformly on Iδ × ωδ . As a corollary, we find a
sequence of polynomials Pj (z) that converge uniformly to U on Z(Iδ × ωδ ) which
is a neighborhood of z0 because t → ϕ(x0 , t) maps ωδ onto an open interval that
contains y0 . Therefore we conclude that U (z) is holomorphic on a neighborhood
of z0 .
Summing up, we have proved that the continuous function U (z) is holomor-
phic on
D1 = {x + iy : x ∈ I1 , m(x) < y < M (x)}
except at the points where U vanishes. By the classical theorem of Radó, U is
holomorphic everywhere in D1 , in particular it is holomorphic in a neighborhood
of z = 0 and, since u = U ◦ Z, this implies that the equation Lu = 0 is satisfied in
a neighborhood of (x, t) = (0, 0).

Case 3
This is the general case and we make no restrictive assumption about the central
point p = (0, 0), in particular, any of the inequalities m(0) ≤ ϕ(0, 0) ≤ M (0) may
become an equality. It follows from the arguments in Case 2 that U is holomorphic
on
Dj = {x + iy : x ∈ Ij , m(x) < y < M (x)}, 1 ≤ j ≤ N.
This already shows that Lu = 0 on Z −1 (Dj ) which, in general, is a proper subset of
Ij ×B. To see that u is actually a homogeneous solution throughout Ij ×B we apply
Mergelyan’s theorem: for fixed j, there exists a sequence of polynomials Pk (z) that
converges uniformly to U on Dj . Thus, L(Pj ◦ Z) = 0 and Pj (Z(x, t)) → u(x, t)
uniformly on Ij ×
 B, so Lu = 0 on Ij × B for every 1 ≤ j ≤ N . Thus, we conclude
that Lu = 0 on (−a, a) \ N × B. The vector fields (2.4) may be written as
∂ ∂ iϕtj
Lj = + λj , λj = − , j = 1, . . . , n.
∂tj ∂x 1 + iϕx
200 J. Hounie

To complete the proof, we wish to show that


u(x, t)Ltj ψ(x, t) dxdt = 0
I×B

for any ψ(x, t) ∈ C0∞ (I × B). For each 0 <  < 1, choose ψ (x) ∈ C ∞ (−a, a) such
that
1. 0 ≤ ψ (x) ≤ 1;
2. ψ (x) ≡ 1 when dist (x, N ) ≥ 2 and ψ (x) ≡ 0 for dist (x, N ) ≤ ;
3. for some C > 0 independent of 0 <  < 1, |ψ (x)| ≤ C−1 .
 
Since u is a solution on I \ N × B,

0= u(x, t)Ltj (ψ (x)ψ(x, t)) dxdt
I×B
= u(x, t)ψ (x)Ltj (ψ)(x, t) dxdt
I×B

− u(x, t)λj (x, t)ψ(x, t)ψ (x) dxdt.
I×B

Observe that since λj (x, t) ≡ 0 for (x, t) ∈ N × B and ψ (x) is supported in the
set {x ∈ R :  ≤ dist (x, N ) ≤ 2},

lim h(x, t)λj (x, t)ψ(x, t)ψ (x) dxdt = 0,
→0 I×B

while
 
lim u(x, t)ψ (x)Lt ψ(x, t) dxdt = u(x, t)Lt ψ(x, t) dxdt.
→0 I×B I×B

It follows that I×B
u(x, t)Lt ψ(x, t) dxdt = 0 and hence Lu = 0 holds on I×B. 

5. An application to uniqueness
The Radó property can be used to give uniqueness in the Cauchy problem for
continuous solutions without requiring any regularity for the initial “surface”. Let
L be smooth locally integrable structure of co-rank one defined on an open subset
Ω of Rn+1 , n ≥ 1, and let U ⊂ Ω be open. We denote by ∂U the set boundary
points of U relative to Ω, i.e., p ∈ ∂U if and only if p ∈ Ω and for every ε > 0, the
ball B(p, ε) contains both points of U and points of Ω \ U . The orbit of L through
the point p is defined as the orbit of p in the sense of Sussmann [Su] with respect
to the set of real vector fields {Xα }, with Xα = Lα , where {Lα } is the set of
all local smooth sections of L (we refer to [T1], [T2] and [BCH, Ch 3] for more
information on orbits of locally integrable structures).
Definition 5.1. We say that ∂U is weakly noncharacteristic with respect to L if for
every point p ∈ ∂U the orbit of L through p intersects Ω \ U .
 A Radó Theorem for Locally Solvable Structures 201

Recall that if Σ = ∂U is a C 1 hypersurface, Σ is said to be noncharacteristic

at p ∈ Σ if L|p is not tangent to Σ for some local smooth section of L. This im-
plies that the orbit of L through p must exit U , so for regular surfaces the notion of
“noncharacteristic” at every point is stronger than that of “weakly noncharacter-
istic”. Similarly, if Σ is an orbit of L of dimension n that bounds some open set U ,
it will be a smooth hypersurface that fails to be noncharacteristic at every point
and also fails to be weakly noncharacteristic. On the other hand, it is easy to give
examples of a regular hypersurface Σ that bounds U and is characteristic precisely
at one point p while the orbit through p eventually exits U . In this case Σ will be
weakly noncharacteristic although it fails to be noncharacteristic at every point.

Theorem 5.2. Let L be a smooth locally integrable structure of co-rank one defined
on an open subset Ω of Rn+1 , n ≥ 1. Assume that L is locally solvable in degree
one in Ω and that ∂U is weakly noncharacteristic. If u is continuous on U ∪ ∂U ,
satisfies Lu = 0 on U in the weak sense and vanishes identically on ∂U , then there
is an open set V , ∂U ⊂ V ⊂ Ω such that u vanishes identically on V ∩ U .

Proof. The proof is standard. Define w ∈ C(Ω) by extending u as zero on Ω \ (U ∪

∂U ) (the continuity of w follows from the fact that u vanishes on ∂U ). Then w
is a Radó function and by Theorem 4.1 the equation Lw = 0 holds in Ω. By a
classical application of the Baouendi-Treves approximation theorem the support
.
of a homogeneous solution is L-invariant, i.e., S = supp w may be expressed as a
union of orbits of L in Ω and the same holds for its complement, Ω \ S ⊃ Ω \ U = ∅
(note that since ∂U is weakly noncharacteristic U is not dense in Ω). The fact that
∂U is weakly noncharacteristic implies that the union V of all the orbits of L that
intersect Ω\U is an open set that contains ∂U on which w vanishes identically. 

Example. Consider in R3 , where the coordinates are denoted by t1 , t2 , x, the func-

tion
Z(x, t) = x + i a(x)(t21 + t22 )/2.
Here a(x) is a smooth real function that is not real analytic at any point and
vanishes exactly once at x = 0. Then Z(x, t) is a global first integral of the system
of vector fields
∂ i t1 a(x) ∂
L1 = − 
∂t1 1 + ia (x)(t21 + t22 )/2 ∂x
∂ i t2 a(x) ∂
L2 = − 
∂t2 1 + ia (x)(t21 + t22 )/2 ∂x
which span a structure L of co-rank one. This structure is locally solvable in degree
one due to the fact that any nonempty fiber of Z over R3 , F (x0 + iy0 , R3 ), is either
a circle contained in the hyperplane x = x0 if x0 = 0 or the whole hyperplane x = 0
if x0 = 0, thus a connected set. If
U = {(x, t1 , t2 ) ∈ R3 : t31 > x}
202 J. Hounie

it follows that ∂U is weakly noncharacteristic with respect to L so theorem 5.2 can

be applied in this situation. The choice of a(x) also prevents the use of Holmgren’s
theorem even at noncharacteristic points.
Consider now a discrete set D ⊂ U such that every point in ∂U is an accumu-
lation point of D and set U1 = U \ D. We have that ∂U1 = ∂U ∪ D is not regular
but it is still weakly noncharacteristic, so if a continuous function u satisfies
L1 u = 0 on U1
L2 u = 0 on U1
u=0 on ∂U1
then u must vanish identically on a neighborhood V of ∂U1 and since R3 is the
union of three orbits of L, namely, x > 0, x = 0 and x < 0, it is apparent that
u vanishes identically. Notice that L1 and L2 are Mizohata type vector fields and
they are not locally solvable when considered individually, so uniqueness in the
Cauchy problem for this example does not follow from the results in [HT].

References
[A] H. Alexander, Removable sets for CR functions, Several complex variables (Stock-
holm, 1987/1988), Math. Notes, 38, Princeton Univ. Press, Princeton, NJ, (1993),
1–7.
[BT1] M.S. Baouendi and F. Treves, A property of the functions and distributions anni-
hilated by a locally integrable system of complex vector fields, Ann. of Math. 113
(1981), 387–421.
[BT2] M.S. Baouendi and F. Treves, A local constancy principle for the solutions of
certain overdetermined systems of first-order linear partial differential equations,
Math. Analysis and Applications, Part A, Advances in Math. Supplementary
Studies 7A (1981), 245–262.
[BCH] S. Berhanu, P. Cordaro and J. Hounie, An Introduction to Involutive Structures,
Cambridge University Press, 2007. 432 p.
[C1] E.M. Chirka, Radó’s theorem for CR-mappings of hypersurfaces, (Russian. Rus-
sian summary) Mat. Sb. 185 (1994), 6, 125–144; translation in Russian Acad. Sci.
Sb. Math. 82 (1995), 243–259.
[C2] E.M. Chirka, A generalization of Radó’s theorem, Ann. Polonici Math. 84, (2003),
109–112.
[CH] P.D. Cordaro and J. Hounie, Local solvability for a class of differential complexes,
Acta Math. 187 (2001), 191–212.
[CT] P.D. Cordaro and F. Treves, Homology and cohomology in hypo-analytic structures
of the hypersurface type, J. Geometric Analysis, 1 (1991), 39–70.
[HT] J. Hounie and J. Tavares, Radó’s theorem for locally solvable vector fields, Proc.
Amer. Math. Soc. 119 (1993), 829–836.
[MT] G. Mendoza and F. Treves Local solvability in a class of overdetermined systems
of linear PDE, Duke Math. J. 63 (1991), 355–377.
 A Radó Theorem for Locally Solvable Structures 203

[RS] J.-P. Rosay, and E. Stout, Radó’s theorem for C-R functions, Proc. Amer. Math.
Soc. 106 (1989), 1017–1026.
[St] E.L. Stout, A generalization of a theorem of Radó, Math. Ann. 177 (1968), 339–
340.
[Su] H.J. Sussmann, Orbits of families of vector fields and integrability of distributions,
Trans. Amer. Math. Soc. 180 (1973), 171–188.
[T1] F. Treves, Approximation and representation of functions and distributions an-
nihilated by a system of complex vector fields, École Polytech., Centre de Math.,
Palaiseau, France, (1981).
[T2] F. Treves, Hypo-analytic structures, Princeton University Press, (1992).
[T3] F. Treves, On the local solvability and local integrability of systems of vector fields,
Acta Math. 151, 1983, 1–38.

J. Hounie
Departamento de Matemática
Universidad Federal de São Carlos
São Carlos, SP, 13565-905, Brasil
e-mail: [email protected]
Complex Analysis
Trends in Mathematics, 205–211

c 2010 Springer Basel AG

Applications of a Parametric Oka Principle

for Liftings
Finnur Lárusson
Dedicated to Linda P. Rothschild

Abstract. A parametric Oka principle for liftings, recently proved by Forst-

nerič, provides many examples of holomorphic maps that are fibrations in
a model structure introduced in previous work of the author. We use this
to show that the basic Oka property is equivalent to the parametric Oka
property for a large class of manifolds. We introduce new versions of the
basic and parametric Oka properties and show, for example, that a complex
manifold X has the basic Oka property if and only if every holomorphic map
to X from a contractible submanifold of Cn extends holomorphically to Cn .
Mathematics Subject Classification (2000). Primary 32Q28. Secondary 18F20,
18G55, 32E10, 55U35.
Keywords. Stein manifold, Oka principle, Oka property, convex interpolation
property, subelliptic, model structure, fibration.

1. Introduction
In this note, which is really an addendum to the author’s papers [6] and [7], we
use a new parametric Oka principle for liftings, very recently proved by Franc
Forstnerič [4], to solve several problems left open in those papers. To make this
note self-contained would require a large amount of technical background. Instead,
we give a brief introduction and refer to the papers [2], [3], [6], [7] for more details.
The modern theory of the Oka principle began with Gromov’s seminal paper
[5] of 1989. Since then, researchers in Oka theory have studied more than a dozen
so-called Oka properties that a complex manifold X may or may not have. These
properties concern the task of deforming a continuous map f from a Stein manifold
S to X into a holomorphic map. If this can always be done so that under the
deformation f is kept fixed on a closed complex submanifold T of S on which f
is holomorphic, then X is said to have the basic Oka property with interpolation
206 F. Lárusson

(BOPI). Equivalently (this is not obvious), T may be taken to be a closed analytic

subvariety of a reduced Stein space S. If f can always be deformed to a holomorphic
map so that the deformed maps stay arbitrarily close to f on a holomorphically
convex compact subset K of S on which f is holomorphic and are holomorphic on
a common neighbourhood of K, then X is said to have the basic Oka property with
approximation (BOPA). If every holomorphic map to X from a compact convex
subset K of Cn can be approximated uniformly on K by entire maps Cn → X, then
X is said to have the convex approximation property (CAP), introduced in [3].
These properties all have parametric versions (POPI, POPA, PCAP) where instead
of a single map f we have a family of maps depending continuously on a parameter.
Some of the basic and parametric Oka properties have been extended from
complex manifolds to holomorphic maps (viewing a manifold as a constant map
from itself). For example, a holomorphic map f : X → Y is said to satisfy POPI
if for every Stein inclusion T → S (a Stein manifold S with a submanifold T ),
every finite polyhedron P with a subpolyhedron Q, and every continuous map
g : S × P → X such that the restriction g|S × Q is holomorphic along S, the
restriction g|T ×P is holomorphic along T , and the composition f ◦g is holomorphic
along S, there is a continuous map G : S × P × I → X, where I = [0, 1], such that:
1. G(·, ·, 0) = g,
2. G(·, ·, 1) : S × P → X is holomorphic along S,
3. G(·, ·, t) = g on S × Q and on T × P for all t ∈ I,
4. f ◦ G(·, ·, t) = f ◦ g on S × P for all t ∈ I.
Equivalently, Q → P may be taken to be any cofibration between cofibrant topo-
logical spaces, such as the inclusion of a subcomplex in a CW-complex, and the
existence of G can be replaced by the stronger statement that the inclusion into
the space, with the compact-open topology, of continuous maps h : S × P → X
with h = g on S × Q and on T × P and f ◦ h = f ◦ g on S × P of the subspace
of maps that are holomorphic along S is acyclic, that is, a weak homotopy equiv-
alence (see [6], §16). (Here, the notion of cofibrancy for topological spaces and
continuous maps is the stronger one that goes with Serre fibrations rather than
Hurewicz fibrations. We remind the reader that a Serre fibration between smooth
manifolds is a Hurewicz fibration, so we will simply call such a map a topological
fibration.)
In [6], the category of complex manifolds was embedded into the category of
prestacks on a certain simplicial site with a certain simplicial model structure such
that all Stein inclusions are cofibrations, and a holomorphic map is acyclic if and
only if it is topologically acyclic, and is a fibration if and only if it is a topological
fibration and satisfies POPI. It was known then that complex manifolds with
the geometric property of subellipticity satisfy POPI, but very few examples of
nonconstant holomorphic maps satisfying POPI were known, leaving some doubt
as to whether the model structure constructed in [6] is an appropriate homotopy-
theoretic framework for the Oka principle. This doubt is dispelled by Forstnerič’s
parametric Oka principle for liftings.
 Applications of a Parametric Oka Principle for Liftings 207

Forstnerič has proved that the basic Oka properties for manifolds are equiva-
lent, and that the parametric Oka properties for manifolds are equivalent ([2], see
also [7], Theorem 1; this is not yet known for maps), so we can refer to them as
the basic Oka property and the parametric Oka property, respectively. For Stein
manifolds, the basic Oka property is equivalent to the parametric Oka property
([7], Theorem 2). In [7] (the comment following Theorem 5), it was noted that the
equivalence of all the Oka properties could be extended to a much larger class of
manifolds, including for example all quasi-projective manifolds, if we had enough
examples of holomorphic maps satisfying POPI. This idea is carried out below.
It remains an open problem whether the basic Oka property is equivalent to the
parametric Oka property for all manifolds.
We conclude by introducing a new Oka property that we call the convex
interpolation property, with a basic version equivalent to the basic Oka property
and a parametric version equivalent to the parametric Oka property. In particu-
lar, we show that a complex manifold X has the basic Oka property if and only
if every holomorphic map to X from a contractible submanifold of Cn extends
holomorphically to Cn . This is based on the proof of Theorem 1 in [7].

2. The parametric Oka principle for liftings

Using the above definition of POPI for holomorphic maps, we can state Forstnerič’s
parametric Oka principle for liftings, in somewhat less than its full strength, as
follows.
Theorem 1 (Parametric Oka principle for liftings [4]). Let X and Y be complex
manifolds and f : X → Y be a holomorphic map which is either a subelliptic sub-
mersion or a holomorphic fibre bundle whose fibre has the parametric Oka property.
Then f has the parametric Oka property with interpolation.
The notion of a holomorphic submersion being subelliptic was introduced by
Forstnerič [1], generalising the concept of ellipticity due to Gromov [5]. Subelliptic-
ity is the weakest currently-known sufficient geometric condition for a holomorphic
map to satisfy POPI.
By a corollary of the main result of [6], Corollary 20, a holomorphic map
f is a fibration in the so-called intermediate model structure constructed in [6]
if and only if f is a topological fibration and satisfies POPI (and then f is a
submersion). In particular, considering the case when f is constant, a complex
manifold is fibrant if and only if it has the parametric Oka property. The following
result is therefore immediate.
Theorem 2.
(1) A subelliptic submersion is an intermediate fibration if and only if it is a
topological fibration.
(2) A holomorphic fibre bundle is an intermediate fibration if and only if its fibre
has the parametric Oka property.
208 F. Lárusson

Part (1) is a positive solution to Conjecture 21 in [6]. For the only-if direction
of (2), we simply take the pullback of the bundle by a map from a point into
the base of the bundle and use the fact that in any model category, a pullback
of a fibration is a fibration. As remarked in [6], (1) may be viewed as a new
manifestation of the Oka principle, saying that for holomorphic maps satisfying
the geometric condition of subellipticity, there is only a topological obstruction to
being a fibration in the holomorphic sense defined by the model structure in [6].
Theorem 2 provides an ample supply of intermediate fibrations.
A result similar to our next theorem appears in [4]. The analogous result for
the basic Oka property is Theorem 3 in [7].
Theorem 3. Let X and Y be complex manifolds and f : X → Y be a holomorphic
map which is an intermediate fibration.
(1) If Y satisfies the parametric Oka property, then so does X.
(2) If f is acyclic and X satisfies the parametric Oka property, then so does Y .
Proof. (1) This follows immediately from the fact that if the target of a fibration
in a model category is fibrant, so is the source.
(2) Let T → S be a Stein inclusion and Q → P an inclusion of parameter
spaces. Let h : S × P → Y be a continuous map such that the restriction h|S × Q is
holomorphic along S and the restriction h|T × P is holomorphic along T . We need
a lifting k of h by f with the same properties. Then the parametric Oka property
of X allows us to deform k to a continuous map S × P → X which is holomorphic
along S, keeping the restrictions to S × Q and T × P fixed. Finally, we compose
this deformation by f .
To obtain the lifting k, we first note that since f is an acyclic topological
fibration, h|T × Q has a continuous lifting, which, since f satisfies POPI, may
be deformed to a lifting which is holomorphic along T . We use the topological
cofibration T ×Q → S ×Q to extend this lifting to a continuous lifting S ×Q → X,
which may be deformed to a lifting which is holomorphic along S, keeping the
restriction to T × Q fixed. We do the same with S × Q replaced by T × P and get a
continuous lifting of h restricted to (S × Q) ∪ (T × P ) which is holomorphic along
S on S × Q and along T on T × P . Finally, we obtain k as a continuous extension
of this lifting, using the topological cofibration (S × Q) ∪ (T × P ) → S × P . 

3. Equivalence of the basic and the parametric Oka properties

Following [7], by a good map we mean a holomorphic map which is an acyclic
intermediate fibration, that is, a topological acyclic fibration satisfying POPI. We
call a complex manifold X good if it is the target, and hence the image, of a good
map from a Stein manifold. This map is then weakly universal in the sense that
every holomorphic map from a Stein manifold to X factors through it.
A Stein manifold is obviously good. As noted in [7], the class of good mani-
folds is closed under taking submanifolds, products, covering spaces, finite
 Applications of a Parametric Oka Principle for Liftings 209

branched covering spaces, and complements of analytic hypersurfaces. This does

not take us beyond the class of Stein manifolds. However, complex projective space,
and therefore every quasi-projective manifold, carries a holomorphic affine bundle
whose total space is Stein (in algebraic geometry this observation is called the
Jouanolou trick), and by Theorem 2, the bundle map is good. Therefore all quasi-
projective manifolds are good. (A quasi-projective manifold is a complex manifold
of the form Y \ Z, where Y is a projective variety and Z is a subvariety. We need
the fact, proved using blow-ups, that Y can be taken to be smooth and Z to be a
hypersurface.) The class of good manifolds thus appears to be quite large, but we
do not know whether every manifold, or even every domain in Cn , is good.
Theorem 4. A good manifold has the basic Oka property if and only if it has the
parametric Oka property.
Proof. Let S → X be a good map from a Stein manifold S to a complex manifold
X. If X has the basic Oka property, then so does S by [7], Theorem 3. Since S is
Stein, S is elliptic by [7], Theorem 2, so S has the parametric Oka property. By
Theorem 3, it follows that X has the parametric Oka property. 

4. The convex interpolation property

Let us call a submanifold T of Cn special if T is the graph of a proper holomorphic
embedding of a convex domain Ω in Ck , k ≥ 1, as a submanifold of Cn−k , that is,
T = {(x, ϕ(x)) ∈ Ck × Cn−k : x ∈ Ω},
where ϕ : Ω → Cn−k is a proper holomorphic embedding. We say that a complex
manifold X satisfies the convex interpolation property (CIP) if every holomorphic
map to X from a special submanifold T of Cn extends holomorphically to Cn , that
is, the restriction map O(Cn , X) → O(T, X) is surjective.
We say that X satisfies the parametric convex interpolation property (PCIP)
if whenever T is a special submanifold of Cn , the restriction map O(Cn , X) →
O(T, X) is an acyclic Serre fibration. (Since Cn and T are holomorphically con-
tractible, acyclicity is automatic; it is the fibration property that is at issue.)
More explicitly, X satisfies PCIP if whenever T is a special submanifold of Cn
and Q → P is an inclusion of parameter spaces, every continuous map f :
(Cn × Q) ∪ (T × P ) → X, such that f |Cn × Q is holomorphic along Cn and
f |T ×P → X is holomorphic along T , extends to a continuous map g : Cn ×P → X
which is holomorphic along Cn . The parameter space inclusions Q → P may range
over all cofibrations of topological spaces or, equivalently, over the generating cofi-
brations ∂Bn → Bn , n ≥ 0, where Bn is the closed unit ball in Rn (we take B0
to be a point and ∂B0 to be empty). Clearly, CIP is PCIP with P a point and Q
empty.
Lemma 1. A complex manifold has the parametric convex interpolation property if
and only if it has the parametric Oka property with interpolation for every inclusion
of a special submanifold into Cn .
210 F. Lárusson

Proof. Using the topological acyclic cofibration (Cn × Q) ∪ (T × P ) → Cn × P ,

we can extend a continuous map f : (Cn × Q) ∪ (T × P ) → X as in the definition
of PCIP to a continuous map g : Cn × P → X. POPI allows us to deform g to
a continuous map h : Cn × P → X which is holomorphic along Cn , keeping the
restriction to (Cn × Q) ∪ (T × P ) fixed, so h extends f .
Conversely, if h : Cn × P → X is a continuous map such that h|Cn × Q is
holomorphic along Cn and h|T × P is holomorphic along T , and g is an extension
of f = h|(Cn × Q) ∪ (T × P ) provided by PCIP, then the topological acyclic
cofibration
 n    
(C × Q) ∪ (T × P ) × I ∪ Cn × P × {0, 1} → Cn × P × I
provides a deformation of h to g which is constant on (Cn × Q) ∪ (T × P ). 

Theorem 5. A complex manifold has the convex interpolation property if and only if
it has the basic Oka property. A complex manifold has the parametric interpolation
property if and only if it has the parametric Oka property.
Proof. We prove the equivalence of the parametric properties. The equivalence of
the basic properties can be obtained by restricting the argument to the case when
P is a point and Q is empty. By Lemma 1, POPI, which is one of the equivalent
forms of the parametric Oka property by [2], Theorem 6.1, implies PCIP.
By [7], Theorem 1, POPI implies POPA (not only for manifolds but also for
maps). The old version of POPA used in [7] does not require the intermediate maps
to be holomorphic on a neighbourhood of the holomorphically convex compact
subset in question, only arbitrarily close to the initial map. This property is easily
seen to be equivalent to the current, ostensibly stronger version of POPA: see the
comment preceding Lemma 5.1 in [2].
The proof of Theorem 1 in [7] shows that to prove POPA for K convex in S =
Ck , that is, to prove PCAP, it suffices to have POPI for Stein inclusions T → Cn
associated to convex domains Ω in Ck as in the definition of a special submanifold.
Thus, by Lemma 1, PCIP implies PCAP, which is one of the equivalent forms of
the parametric Oka property, again by [2], Theorem 6.1. 

There are many alternative definitions of a submanifold of Cn being special

for which Theorem 5 still holds. For example, we could define special to mean
topologically contractible: this is the weakest definition that obviously works. We
could also define a submanifold of Cn to be special if it is biholomorphic to a
bounded convex domain in Ck , k < n. On the other hand, for the proof of Theorem
5 to go through, the class of special manifolds must contain T associated as above
to every element Ω in some basis of convex open neighbourhoods of every convex
compact subset K of Cn for every n ≥ 1, such that K is of the kind termed special
by Forstnerič (see [3], Section 1).

Acknowledgement
I am grateful to Franc Forstnerič for helpful discussions.
 Applications of a Parametric Oka Principle for Liftings 211

References
[1] F. Forstnerič. The Oka principle for sections of subelliptic submersions. Math. Z.
241 (2002) 527–551.
[2] F. Forstnerič. Extending holomorphic mappings from subvarieties in Stein manifolds.
Ann. Inst. Fourier (Grenoble) 55 (2005) 733–751.
[3] F. Forstnerič. Runge approximation on convex sets implies the Oka property. Ann.
of Math. (2) 163 (2006) 689–707.
[4] F. Forstnerič. Invariance of the parametric Oka property, in this volume.
[5] M. Gromov. Oka’s principle for holomorphic sections of elliptic bundles. J. Amer.
Math. Soc. 2 (1989) 851–897.
[6] F. Lárusson. Model structures and the Oka Principle. J. Pure Appl. Algebra 192
(2004) 203–223.
[7] F. Lárusson. Mapping cylinders and the Oka Principle. Indiana Univ. Math. J. 54
(2005) 1145–1159.

Finnur Lárusson
School of Mathematical Sciences
University of Adelaide
Adelaide, SA 5005, Australia
e-mail: [email protected]
Complex Analysis
Trends in Mathematics, 213–225

c 2010 Springer Basel AG

Stability of the Vanishing of the ∂ b-cohomology

Under Small Horizontal Perturbations of the
CR Structure in Compact Abstract q-concave
CR Manifolds
Christine Laurent-Thiébaut

To Linda for her sixtieth birthday

Abstract. We consider perturbations of CR structures which preserve the

complex tangent bundle. For a compact generic CR manifold its concavity
properties and hence the finiteness of some ∂ b -cohomology groups are also
preserved by such perturbations of the CR structure. Here we study the sta-
bility of the vanishing of these groups.

Mathematics Subject Classification (2000). 32V20.

Keywords. CR structure, homotopy formulas, tangential Cauchy Riemann
equation, vanishing theorems.

The tangential Cauchy-Riemann equation is one of the main tools in CR anal-

ysis and its properties are deeply related to the geometry of CR manifolds, in
particular the complex tangential directions are playing an important role. For
example it was noticed by Folland and Stein [3], when they studied the tangential
Cauchy-Riemann operator on the Heisenberg group and more generally on strictly
pseudoconvex real hypersurfaces of Cn , that one get better estimates in the com-
plex tangential directions. Therefore in the study of the stability properties for the
tangential Cauchy-Riemann equation under perturbations of the CR structure it
seems natural to consider perturbations which preserve the complex tangent vector
bundle. Such perturbations can be represented as graphs in the complex tangent
vector bundle over the original CR structure, they are defined by (0, 1)-forms with
values in the holomorphic tangent bundle. We call them horizontal perturbations.
We consider compact abstract CR manifolds and integrable perturbations of
their CR structure which preserve their complex tangent vector bundle. Since the
Levi form of a CR manifold depends only on its complex tangent vector bundle,
214 Ch. Laurent-Thiébaut

such perturbations will preserve the Levi form and hence the concavity properties
of the manifold which are closely related to the ∂ b -cohomology. For example it is
well known that, for q-concave compact CR manifolds of real dimension 2n−k and
CR-dimension n − k, the ∂ b -cohomology groups are finite dimensional in bidegree
(p, r), when 1 ≤ p ≤ n and 1 ≤ r ≤ q − 1 or n− k − q + 1 ≤ r ≤ n− k. Therefore, for
a q-concave compact CR manifold, this finiteness property is stable by horizontal
perturbations of the CR structure.
In this paper we are interested in the stability of the vanishing of the ∂ b -
cohomology groups after horizontal perturbations of the CR structure.
Let M = (M, H0,1 M) be an abstract compact CR manifold of class C ∞ , of real
dimension 2n − k and CR dimension n − k, and M , = (M, H, 0,1 M) another abstract
,
compact CR manifold such that H0,1 M is a smooth integrable horizontal pertur-
bation of H0,1 M, then H , 0,1 M is defined by a smooth form Φ ∈ C0,1 ∞
(M, H1,0 M).
We denote by ∂ b the tangential Cauchy-Riemann operator associated to the CR
Φ
structure H0,1 M and by ∂ b the tangential Cauchy-Riemann operator associated
to the CR structure H , 0,1 M.
The smooth ∂ b -cohomology groups on M in bidegree (0, r) and (n, r) are
defined for 1 ≤ r ≤ n − k by:
∞ ∞
H 0,r (M) = {f ∈ C0,r (M) | ∂ b f = 0}/∂ b (C0,r−1 (M))
and
∞ ∞
H n,r (M) = {f ∈ Cn,r (M) | ∂ b f = 0}/∂ b (Cn,r−1 (M)).
If f is a smooth differential form of degree r, 1 ≤ r ≤ n − k, on M, we denote
∞
by fr,0 its projection on the space Cr,0 (M) of (r, 0)-forms for the CR structure
H0,1 M. Note that if r ≥ n + 1 then fr,0 = 0.
Φ
The smooth ∂ b -cohomology groups on M , in bidegree (0, r) and (n, r) are
defined for 1 ≤ r ≤ n − k by:
, = {f ∈ C ∞ (M)
HΦ0,r (M) , | fr,0 = 0, ∂ Φ f = 0}/∂ Φ (C ∞ (M))
,
0,r b b 0,r−1

and
, = {f ∈ C ∞ (M)
HΦn,r (M) , | ∂ Φ f = 0}/∂ Φ (C ∞ (M)).
,
n,r b b n,r−1
In this paper the following stability result is proved:
Theorem 0.1. Assume M is q-concave, there exists then a sequence (δl )l∈N of pos-
itive real numbers such that, if Φl < δl for each l ∈ N,
, = 0 when 1 ≤ r ≤ q − 2,
(i) H p,r−p (M) = 0 for all 1 ≤ p ≤ r implies HΦ0,r (M)
in the abstract case, and when 1 ≤ r ≤ q − 1, if M is locally embeddable,
(ii) H n−p,r+p (M) = 0 for all 0 ≤ p ≤ n − k − r implies HΦn,r (M) , = 0 when
n − k − q + 1 ≤ r ≤ n − k.
We also prove the stability of the solvability of the tangential Cauchy-Rie-
mann equation with sharp anisotropic regularity (cf. Theorem 3.3).
 Stability of the Vanishing of the ∂ b -cohomology 215

Note that, when both CR manifolds M and M , are embeddable in the same
complex manifold (i.e., in the embedded case), a (0, r)-form f for the new CR
structure H , 0,1 M is also a (0, r)-form for the original CR structure H0,1 M and
hence the condition fr,0 = 0 in the definition of the cohomology groups HΦ0,r (M) ,
is automatically fulfilled. In that case, Polyakov proved in [7] global homotopy
formulas for a family of CR manifolds in small degrees which immediately imply
the stability of the vanishing of the ∂ b -cohomology groups of small degrees. In his
paper he does not need the perturbation to preserve the complex tangent vector
bundle, but his estimates are far to be sharp.
Moreover Polyakov [8] proved also that if a generically embedded compact
CR manifold M ⊂ X is at least 3-concave and satisfies H 0,1 (M, T  X|M ) = 0, then
small perturbations of the CR structure are still embeddable in the same manifold
X. From this result and the global homotopy formula from [7] one can derive
some stability results for the vanishing of ∂ b -cohomology groups of small degrees
without hypothesis on the embeddability a priori of the perturbed CR structure.
The main interest of our paper is that we do not assume the CR manifolds
M and M , to be embeddable. In the case where the original CR manifold M is
embeddable it covers the case where it is unknown if the perturbed CR structure
is embeddable in the same manifold as the original one, for example when the
manifold M is only 2-concave. Finally we also reach the case of the ∂ b -cohomology
groups of large degrees, which, even in the embedded case, cannot be deduced
from the works of Polyakov.
The main tool in the proof of the stability of the vanishing of the ∂ b -co-
homology groups is a fixed point theorem which is derived from global homotopy
formulas with sharp anisotropic estimates. Such formulas are proved in [9], in the
abstract case, by using the L2 theory for the b operator and in [5], in the locally
embeddable case, by first proving that the integral operators associated to the ker-
nels built in [1] satisfy sharp anisotropic estimates, which implies local homotopy
formulas with sharp anisotropic estimates, and then by using the globalization
method from [6] and [2].

1. CR structures
Let M be a C l -smooth, l ≥ 1, paracompact differential manifold, we denote by T M
the tangent bundle of M and by TC M = C ⊗ T M the complexified tangent bundle.
Definition 1.1. An almost CR structure on M is a subbundle H0,1 M of TC M such
that H0,1 M ∩ H0,1 M = {0}.
If the almost CR structure is integrable, i.e., for all Z, W ∈ Γ(M, H0,1 M)
then [Z, W ] ∈ Γ(M, H0,1 M), then it is called a CR structure.
If H0,1 M is a CR structure, the pair (M, H0,1 M) is called an abstract CR
manifold.
The CR dimension of M is defined by CR-dim M = rkC H0,1 M.
216 Ch. Laurent-Thiébaut

We set H1,0 M = H0,1 M and we denote by H 0,1 M the dual bundle (H0,1 M)∗
of H0,1 M. 6
Let Λ0,q M = q (H 0,1 M), then C0,q s
(M) = Γs (M, Λ0,q M) is called the space
of (0, q)-forms of class C , 0 ≤ s ≤ l, on M.
s

We define Λp,0 M as the space of forms of degree p that annihilate any p-

vector on M that has more than one factor contained in H0,1 M. Then Cp,q s
(M) =
C0,q (M, Λ M) is the space of (0, q)-forms of class C with values in Λ M.
s p,0 s p,0

If the almost CR structure is a CR structure, i.e., if it is integrable, and if

s ≥ 1, then we can define an operator
∂ b : C0,q
s
(M) → C0,q+1
s−1
(M), (1.1)
called the tangential Cauchy-Riemann operator, by setting ∂ b f = df|H0,1 M×···×H0,1 M .
It satisfies ∂ b ◦ ∂ b = 0.
Definition 1.2. Let (M, H0,1 M) be an abstract CR manifold, X be a complex
manifold and F : M → X be an embedding of class C l , then F is called a CR
embedding if dF (H0,1 M) is a subbundle of the bundle T0,1 X of the antiholomorphic
vector fields of X and dF (H0,1 M) = T0,1 X ∩ TC F (M).
Let F be a CR embedding of an abstract CR manifold M into a complex
manifold X and set M = F (M), then M is a CR manifold with the CR structure
H0,1 M = T0,1 X ∩ TC M .
Let U be a coordinate domain in X, then F|F −1 (U ) = (f1 , . . . , fN ), with
N = dimC X, and F is a CR embedding if and only if, for all 1 ≤ j ≤ N , ∂ b fj = 0.
A CR embedding is called generic if dimC X − rkC H0,1 M = codimR M.

Definition 1.3. An almost CR structure H , 0,1 M on M is said to be of finite distance

,
to a given CR structure H0,1 M if H0,1 M can be represented as a graph in TC M
over H0,1 M.
It is called an horizontal perturbation of the CR structure H0,1 M if it is of
finite distance to H0,1 M and moreover there exists Φ ∈ C0,1 (M, H1,0 M) such that
, 0,1 M = {W ∈ TC M | W = Z − Φ(Z), Z ∈ H0,1 M},
H (1.2)
, 0,1 M is a graph in HM = H1,0 M ⊕ H0,1 M over H0,1 M.
which means that H
Note that an horizontal perturbation of the original CR structure preserves
the complex tangent bundle HM.
Assume M is an abstract CR manifold and H , 0,1 M is an integrable horizontal
Φ
perturbation of the original CR structure H0,1 M on M. If ∂ b denotes the tangential
Cauchy-Riemann operator associated to the CR structure H , 0,1 M, then we have
Φ
∂ b = ∂ b − Φd = ∂ b − Φ∂b , (1.3)
where ∂ b is the tangential Cauchy-Riemann operator associated to the original CR
structure H0,1 M and ∂b involves only holomorphic tangent vector fields.
 Stability of the Vanishing of the ∂ b -cohomology 217

The annihilator H 0 M of HM = H1,0 M ⊕ H0,1 M in TC∗ M is called the char-

acteristic bundle of M. Given p ∈ M, ω ∈ Hp0 M and X, Y ∈ Hp M, we choose
 ∈ Γ(M, H 0 M) and X,
ω  Y ∈ Γ(M, HM) with ω p = ω, X p = X and Yp = Y .
Then d  Y ]). Therefore we can associate to each ω ∈ H 0 M an
ω (X, Y ) = −ω([X, p
hermitian form
Lω (X) = −iω([X, 
 X]) (1.4)
on Hp M. This is called the Levi form of M at ω ∈ Hp0 M.
In the study of the ∂ b -complex two important geometric conditions were
introduced for CR manifolds of real dimension 2n − k and CR-dimension n − k.
The first one by Kohn in the hypersurface case, k = 1, the condition Y(q), the
second one by Henkin in codimension k, k ≥ 1, the q-concavity.
An abstract CR manifold M of hypersurface type satisfies Kohn’s condition
Y (q) at a point p ∈ M for some 0 ≤ q ≤ n − 1, if the Levi form of M at p has at
least max(n − q, q + 1) eigenvalues of the same sign or at least min(n − q, q + 1)
eigenvalues of opposite signs.
An abstract CR manifold M is said to be q-concave at p ∈ M for some
0 ≤ q ≤ n − k, if the Levi form Lω at ω ∈ Hp0 M has at least q negative eigenvalues
on Hp M for every nonzero ω ∈ Hp0 M.
In [9] the condition Y(q) is extended to arbitrary codimension.
Definition 1.4. An abstract CR manifold is said to satisfy condition Y(q) for some
1 ≤ q ≤ n − k at p ∈ M if the Levi form Lω at ω ∈ Hp0 M has at least n − k − q + 1
positive eigenvalues or at least q+1 negative eigenvalues on Hp M for every nonzero
ω ∈ Hp0 M.
Note that in the hypersurface case, i.e., k = 1, this condition is equivalent to
the classical condition Y(q) of Kohn for hypersurfaces. Moreover, if M is q-concave
at p ∈ M, then q ≤ (n − k)/2 and condition Y(r) is satisfied at p ∈ M for any
0 ≤ r ≤ q − 1 and n − k − q + 1 ≤ r ≤ n − k.

2. Stability of vanishing theorems by horizontal perturbations

of the CR structure
Let (M, H0,1 M) be an abstract compact CR manifold of class C ∞ , of real dimension
2n−k and CR dimension n−k, and H , 0,1 M be an integrable horizontal perturbation
of H0,1 M. We denote by M the abstract CR manifold (M, H0,1 M) and by M , the
,
abstract CR manifold (M, H0,1 M).
, 0,1 M is an horizontal perturbation of H0,1 M, which means that H
Since H , 0,1 M
, ,
is a graph in HM = H1,0 M ⊕ H0,1 M over H0,1 M, the space H M = H1,0 M ⊕ H0,1 M ,
coincides with the space HM and consequently the two abstract CR manifolds
M and M , have the same characteristic bundle and hence the same Levi form.
This implies in particular that if M satisfies condition Y(q) at each point, then M ,
satisfies also condition Y(q) at each point.
218 Ch. Laurent-Thiébaut

It follows from the Hodge decomposition theorem and the results in [9] that if
M is an abstract compact CR manifold of class C ∞ which satisfies condition Y(q)
at each point, then the cohomology groups H p,q (M), 0 ≤ p ≤ n, are finite dimen-
sional. A natural question is then the stability by small horizontal perturbations
of the CR structure of the vanishing of these groups.
Let us consider a sequence (B l (M), l ∈ N) of Banach spaces with B l+1 (M) ⊂
B (M), which are invariant by horizontal perturbations of the CR structure of M
l

and such that if f ∈ B l (M), l ≥ 1, then XC f ∈ B l−1 (M) for all complex tangent
vector fields XC to M and there exists θ(l) ∈ N with θ(l + 1) ≥ θ(l) such that
f g ∈ B l (M) if f ∈ B l (M) and g ∈ C θ(l) (M). Such a sequence (B l (M), l ∈ N) will
be called a sequence of anisotropic spaces. We denote by Bp,r
l
(M) the space of (p, r)-
forms on M whose coefficients belong to B (M). Moreover we will say that these
l

Banach spaces are adapted to the ∂ b -equation in degree r ≥ 1 if, when H p,r (M) = 0,
0 ≤ p ≤ n, there exist linear continuous operators As , s = r, r + 1, from Bp,s 0
(M)
into Bp,s−1 (M) which are also continuous from Bp,s (M) into Bp,s−1 (M), l ∈ N, and
0 l l+1

moreover satisfy
f = ∂ b Ar f + Ar+1 ∂ b f , (2.1)
for f ∈ Bp,r
l
(M).
Theorem 2.1. Let M = (M, H0,1 M) be an abstract compact CR manifold of class
C ∞ , of real dimension 2n − k and CR dimension n − k, and M , = (M, H , 0,1 M)
another abstract compact CR manifold such that H , 0,1 M is an integrable horizontal
perturbation of H0,1 M. Let also (B l (M), l ∈ N) be a sequence of anisotropic Banach
θ(l)
spaces and q be an integer, 1 ≤ q ≤ (n − k)/2. Finally let Φ ∈ C0,1 (M, H1,0 M)
be the differential form which defines the tangential Cauchy-Riemann operator
Φ
∂ b = ∂ b − Φ∂b associated to the CR structure H , 0,1 M.
Assume H p,r−p
(M) = 0, for 1 ≤ p ≤ r and 1 ≤ r ≤ s1 (q), or H n−p,r+p (M) =
0, for 0 ≤ p ≤ n − k − r and s2 (q) ≤ r ≤ n − k and that the Banach spaces
(B l (M), l ∈ N) are adapted to the ∂ b -equation in degree r, 1 ≤ r ≤ s1 (q) or
s2 (q) ≤ r ≤ n−k. Then, for each l ∈ N, there exists δ > 0 such that, if Φθ(l) < δ,
φ
(i) for each ∂ b -closed form f in B0,r
l , 1 ≤ r ≤ s1 (q), such that the part of f
(M),
of bidegree (0, r) for the initial CR structure H0,1 M vanishes, there exists a
form u in B l+1 (M) , satisfying ∂ φ u = f ,
0,r−1 b
φ
(ii) for each ∂ b -closed form f in Bn,r
l , s2 (q) ≤ r ≤ n − k, there exists a form
(M),
u in Bn,r−1 (M)
l+1 , satisfying ∂ u = f .
φ
b

Remark 2.2. Note that if both M = (M, H0,1 M) and M , = (M, H, 0,1 M) are embed-
dable in the same complex manifold X, any r-form on the differential manifold M,
which represents a form of bidegree (0, r) for the CR structure H ,0,1 M represents
also a form of bidegree (0, r) for the CR structure H0,1 M. Hence the bidegree
hypothesis in (i) of Theorem 2.1 is automatically fulfilled.
 Stability of the Vanishing of the ∂ b -cohomology 219

Proof. Let f ∈ B0,r

l , be a (0, r)-form for the CR structure H
(M) , 0,1 M, 1 ≤ r ≤ s1 (q),
Φ
such that ∂ b f = 0, we want to solve the equation
Φ
∂ b u = f. (2.2)
r
The form f can be written p=0 fp,r−p , where the forms fp,r−p are of type
(p, r − p) for the CR structure H0,1 M. Then by considerations of bidegrees, the
Φ Φ
equation ∂ b f = 0 is equivalent to the family of equations ∂ b fp,r−p = 0, 0 ≤ p ≤ r.
r−1
Moreover, if u = s=0 us,r−1−s , where the forms us,r−1−s are of type (s, r −
1 − s) for the CR structure H0,1 M, is a solution of (2.2), then
Φ
∂ b up,r−1−p = fp,r−p ,
for 0 ≤ p ≤ r − 1, and
fr,0 = 0 .
Φ
Therefore a necessary condition on f for the solvability of (2.2) is that ∂ b f =
0 and fr,0 = 0, where fr,0 is the part of type (r, 0) of f for the CR structure H0,1 M,
and, to solve (2.2), we have to consider the equation
Φ
∂ b v = g, (2.3)
where g ∈ Bp,r−p
l
(M) is a (p, r − p)-form for the CR structure H0,1 M, 0 ≤ p ≤
Φ Φ
r − 1, which is ∂ b -closed. By definition of the operator ∂ b , this means solving
the equation ∂ b v = g + Φ∂b v. Consequently if v is a solution of (2.3), then
∂ b (g + Φ∂b v) = 0 and by (2.1)
∂ b (Ar−p (g + Φ∂b v)) = g + Φ∂b v.

Assume Φ is of class C θ(l) , then the map

Θ : Bp,r−1
l+1
(M) → Bp,r−1
l+1
(M)
v → Ar−p g + Ar−p (Φ∂b v) .
is continuous, and the fixed points of Θ are good candidates to be solutions of (2.3).
Let δ0 such that, if Φθ(l) < δ0 , then the norm of the bounded endomorphism
Ar−p ◦ Φ∂b of Bp,r−p−1
l+1
(M) is equal to 0 < 1. We shall prove that, if Φθ(l) < δ0 ,
Θ admits a unique fixed point.
Consider first the uniqueness of the fixed point. Assume v1 and v2 are two
fixed points of Θ, then
v1 = Θ(v1 ) = Ar−p g + Ar−p (Φ∂b v1 )
v2 = Θ(v2 ) = Ar−p g + Ar−p (Φ∂b v2 ).
This implies
 
v1 − v2 = Ar−p Φ∂b (v1 − v2 )
220 Ch. Laurent-Thiébaut

and, by the hypothesis on Φ,

v1 − v2 Bl+1 < v1 − v2 Bl+1 or v1 = v2
and hence v1 = v2 .
For the existence we proceed by iteration. We set v0 = Θ(0) = Ar−p (g) and,
for n ≥ 0, vn+1 = Θ(vn ). Then for n ≥ 0, we get
vn+1 − vn = Ar−p (Φ∂b (vn − vn−1 )).
Therefore, if Φθ(l) < δ0 , the sequence (vn )n∈N is a Cauchy sequence in the
Banach space Bp,r−1
l+1
(M) and hence converges to a form v, moreover by continuity
of the map Θ, v satisfies Θ(v) = v.
It remains to prove that v is a solution of (2.3). Since H p,r−p (M) = 0 for
1 ≤ p ≤ r, it follows from (2.1) and from the definition of the sequence (vn )n∈N
that
Φ
g − ∂ b vn+1 = Φ∂b (vn+1 − vn ) + Ar−p+1 ∂ b (g + Φ∂b vn )
and since
∂ b (g + Φ∂b vn ) = ∂ b g − ∂ b (∂ b − Φ∂b )vn
Φ
= ∂ b g − ∂ b (∂ b vn )
Φ Φ
= ∂ b g − (∂ b + Φ∂b )(∂ b vn )
Φ Φ
= ∂ b g − Φ∂b (∂ b vn ), since (∂ b )2 = 0
Φ Φ
= Φ∂b (g − ∂ b vn ), since ∂ b g = 0,
we get
Φ Φ
g − ∂ b vn+1 = Φ∂b (vn+1 − vn ) + Ar−p+1 (Φ∂b (g − ∂ b vn )). (2.4)
Note that since g ∈ Bp,r
l
(M) and Φ is of class C θ(l) , it follows from the
Φ
definition of the vn s that vn ∈ Bp,r
l+1
(M) and ∂ b vn ∈ Bp,r
l
(M) for all n ∈ N.
Thus by (2.4), we have the estimate
Φ Φ
g −∂ b vn+1 Bl ≤ Φ∂b (vn+1 −vn )Bl+1 +Ar−p+1 ◦Φ∂b g −∂ b vn Bl . (2.5)
Let δ such that if Φθ(l) < δ, then the maximum of the norm of the bounded
endomorphisms As ◦ Φ∂b , s = r − p, r − p + 1, of Bp,s−1
l
(M) is equal to  < 1.
Assume Φθ(l) < δ, then by induction we get
Φ Φ
g − ∂ b vn+1 Bl ≤ (n + 1)n+1Φ∂b v0 Bl+1 + n+1 g − ∂ b v0 Bl . (2.6)
But
Φ
g − ∂ b v0 = Φ∂b Ar−p g + Ar−p+1 (Φ∂b g)
and hence
Φ
g − ∂ b v0 Bl ≤ Φ∂b Ar−p gBl+1 + gBl .
This implies
Φ
g − ∂ b vn+1 Bl ≤ (n + 2)n+1 Φ∂b Ar−p gBl + n+2 gBl . (2.7)
 Stability of the Vanishing of the ∂ b -cohomology 221

Since  < 1, the right-hand side of (2.7) tends to zero, when n tends to infinity
Φ
and by continuity of the operator ∂ b from Bp,r−1
l+1
(M) into Bp,r
l
(M), the left-hand
Φ
side of (2.7) tends to g − ∂ b vBl , when n tends to infinity, which implies that v
is a solution of (2.3).
Now if f ∈ Bn,r
l , is an (n, r)-form for the CR structure H
(M) ,0,1 M, s2 (q) ≤ r ≤
Φ n−k−r
n − k, such that ∂ b f = 0, then the form f can be written p=0 fn−p,r+p , where
Φ
the forms fn−p,r+p are ∂ b -closed and of type (n − p, r + p) for the CR structure
H0,1 M. Then to solve the equation
Φ
∂ b u = f,
Φ
it is sufficient to solve the equation ∂ b v = g for g ∈ Bn−p,r+p
l
(M) and this can be
done in the same way as in the case of the small degrees, but using the vanishing
of the cohomology groups H n−p,p+r (M) for 0 ≤ p ≤ n − k − r and s2 (q) ≤ r ≤
n − k. 
Assume the horizontal perturbation of the original CR structure on M is
, the cohomology groups
smooth, i.e., Φ is of class C ∞ , then we can defined on M
, = {f ∈ C ∞ (M)
HΦo,r (M) , | fr,0 = 0, ∂ f = 0}/∂ (C ∞ (M))
Φ Φ
,
o,r b b o,r−1

and
, = {f ∈ C ∞ (M)
HΦn,r (M) , | ∂ Φ f = 0}/∂ Φ (C ∞ (M))
,
o,r b b n,r−1
for 1 ≤ r ≤ n − k.
Corollary 2.3. Under the hypotheses of Theorem 2.1, if the sequence (B l (M), l ∈
N) of anisotropic Banach spaces is such that ∩l∈N Bl (M) = C ∞ (M) and if the
horizontal perturbation of the original CR structure on M is smooth, there exists
a sequence (δl )l∈N of positive real numbers such that, if Φθ(l) < δl for each l ∈ N
, = 0, when 1 ≤ r ≤ s1 (q),
(i) H p,r−p (M) = 0, for all 1 ≤ p ≤ r, implies HΦo,r (M)
(ii) H n−p,r+p
(M) = 0, for all 0 ≤ p ≤ n − k − r, implies HΦn,r (M) , = 0, when
s2 (q) ≤ r ≤ n − k.
Proof. It is a direct consequence of the proof of Theorem 2.1 by the uniqueness of
the fixed point of Θ. 

3. Anisotropic spaces
In the previous section the main theorem is proved under the assumption of the ex-
istence of sequences of anisotropic spaces on abstract CR manifolds satisfying good
properties with respect to the tangential Cauchy-Riemann operator. We will make
precise Theorem 2.1 by considering some Sobolev and some Hölder anisotropic
spaces for which global homotopy formulas for the tangential Cauchy-Riemann
equation with good estimates hold under some geometrical conditions.
222 Ch. Laurent-Thiébaut

In this section M = (M, H0,1 M) denotes an abstract compact CR manifold

of class C ∞ , of real dimension 2n − k and CR dimension n − k.
Let us define some anisotropic Sobolev spaces of functions:
– S 0,p (M), 1 < p < ∞, is the set of Lploc functions on M.
1
– S 1,p (M), 1 < p < ∞, is the set of functions on M such that f ∈ W 2 ,p (M)
p
and XC f ∈ Lloc (M), for all complex tangent vector fields XC to M .
– S l,p (M), l ≥ 2, 1 < p < ∞, is the set of functions f such that Xf ∈ S l−2,p (M),
for all tangent vector fields X to M and XC f ∈ S l−1,p (M), for all complex
tangent vector fields XC to M .
The sequence (S l,p (M), l ∈ N) is a sequence of anisotropic spaces in the sense
of Section 2 with θ(l) = l + 1. Moreover ∩l∈N S l,p (M) = C ∞ (M).
The anisotropic Hölder space of forms S∗l,p (M), l ≥ 0, 1 < p < ∞, is then the
space of forms on M, whose coefficients are in S l,p (M).
We have now to see if the sequence (S l,p (M), l ∈ N) is adapted to the ∂ b -
equation for some degree r.
The L2 theory for b in abstract CR manifolds of arbitrary codimension is
developed in [9]. There it is proved that if M satisfies condition Y (r) the Hodge
decomposition theorem holds in degree r, which means that there exist a compact
operator Nb : L2p,r (M) → Dom(b ) and a continuous operator Hb : L2p,r (M) →
L2p,r (M) such that for any f ∈ L2p,r (M)
∗ ∗
f = ∂ b ∂ b Nb f + ∂ b ∂ b Nb f + Hb f. (3.1)
Moreover Hb vanishes on exact forms and if Nb is also defined on L2p,r+1(M)then
N b ∂ b = ∂ b Nb .
Therefore if M satisfy both conditions Y (r) and Y (r + 1) then (3.1) becomes
an homotopy formula and using the Sobolev and the anisotropic Sobolev estimates
in [9] (Theorems 3.3 and Corollary 1.3 (2)) we get the following result:
Proposition 3.1. If M is q-concave, the sequence (S l,p (M), l ∈ N) of anisotropic
spaces is adapted to the ∂ b -equation in degree r for 0 ≤ r ≤ q −2 and n−k −q +1 ≤
r ≤ n − k.
Let us define now some anisotropic Hölder spaces of functions:
– Aα (M), 0 < α < 1, is the set of continuous functions on M which are in
C α/2 (M).
– A1+α (M), 0 < α < 1, is the set of functions f such that f ∈ C (1+α)/2 (M) and
XC f ∈ C α/2 (M), for all complex tangent vector fields XC to M. Set
f Aα = f (1+α)/2 + sup XC f α/2. (3.2)
XC ≤1

– Al+α (M), l ≥ 2, 0 < α < 1, is the set of functions f of class C [l/2] such
that Xf ∈ Al−2+α (M), for all tangent vector fields X to M and XC f ∈
Al−1+α (M), for all complex tangent vector fields XC to M.
 Stability of the Vanishing of the ∂ b -cohomology 223

The sequence (Al+α (M), l ∈ N) is a sequence of anisotropic spaces in the

sense of Section 2 with θ(l) = l + 1. Moreover ∩l∈N Al+α (M) = C ∞ (M).
The anisotropic Hölder space of forms Al+α
∗ (M), l ≥ 0, 0 < α < 1, is then
the space of continuous forms on M, whose coefficients are in Al+α (M).
It remains to see if the sequence (Al+α (M), l ∈ N) is adapted to the ∂ b -
equation for some degrees r.
Assume M is locally embeddable and 1-concave. Then, by Proposition 3.1 in
[4], there exist a complex manifold X and a smooth generic embedding E : M →
M ⊂ X such that M is a smooth compact CR submanifold of X with the CR
structure H0,1 M = dE(H0,1 M) = TC M ∩ T0,1 . If E is a CR vector bundle over
M, by the 1-concavity of M and after an identification between M and M , the
CR bundle E can be extended to an holomorphic bundle in a neighborhood of
M , which we still denote by E. With these notations it follows from [5] that if
∞
M is q-concave, q ≥ 1, there exist finite-dimensional subspaces Hr of Zn,r (M, E),
∞
0 ≤ r ≤ q − 1 and n − k − q + 1 ≤ r ≤ n − k, where H0 = Zn,0 (M, E), continuous
linear operators
Ar : Cn,r
0
(M, E) → Cn,r−1
0
(M, E), 1 ≤ r ≤ q and n − k − q + 1 ≤ r ≤ n − k
and continuous linear projections
Pr : Cn,r
0
(M, E) → Cn,r
0
(M, E) , 0 ≤ r ≤ q − 1 and n − k − q + 1 ≤ r ≤ n − k ,
with
Im Pr = Hr , 0 ≤ r ≤ q − 1 and n − k − q + 1 ≤ r ≤ n − k , (3.3)
and
Cn,r
0
(M, E)∩∂ b Cn,r−1
0
(M, E) ⊆ Ker Pr , 1 ≤ r ≤ q−1 and n−k−q+1 ≤ r ≤ n−k,
(3.4)
such that:
(i) For all l ∈ N and 1 ≤ r ≤ q or n − k − q + 1 ≤ r ≤ n − k,

n,r (M, E)) ⊂ An,r−1 (M, E)

l+1α
Ar (Al+α

and Ar is continuous as an operator between Al+α n,r (M, E) and An,r−1 (M, E).
l+1+α

(ii) For all 0 ≤ r ≤ q − 1 or n − k − q + 1 ≤ r ≤ n − k and f ∈ Cn,r 0

(M, E)
with ∂ b f ∈ Cn,r+1 (M, E),
0

'
A1 ∂ b f if r = 0 ,
f − Pr f =
∂ b Ar f + Ar+1 ∂ b f if 1 ≤ r ≤ q − 1 or n − k − q + 1 ≤ r ≤ n − k .
(3.5)
This implies the following result
Proposition 3.2. If M is locally embeddable and q-concave the sequence Al+α (M),
l ∈ N of anisotropic spaces is adapted to the ∂ b -equation in degree r for 1 ≤ r ≤
q − 1 and n − k − q + 1 ≤ r ≤ n − k
224 Ch. Laurent-Thiébaut

Finally let us recall the definition of the anisotropic Hölder spaces Γl+α (M)
of Folland and Stein.
– Γα (M), 0 < α < 1, is the set of continuous functions in M such that if for
every x0 ∈ M
|f (γ(t) − f (x0 )|
sup <∞
γ(.) |t|α
for any complex tangent curve γ through x0 .
– Γl+α (M), l ≥ 1, 0 < α < 1, is the set of continuous functions in M such that
XC f ∈ Γl−1+α (M), for all complex tangent vector fields XC to M.
The spaces Γl+α (M) are subspaces of the spaces Al+α (M).
Note that by Corollary 1.3 (1) in [9] and Section 3 in [5], Propositions 3.1
and 3.2 hold also for the anisotropic Hölder spaces Γl+α (M) of Folland and Stein.
Let us summarize all this in connection with Section 2 in the next theorem.
Theorem 3.3. If M is q-concave,
(i) Theorem 2.1 holds for B l (M) = S l,p (M) with s1 (q) = q − 2 and s2 (q) =
n − k − q + 1 in the abstract case,
(ii) Theorem 2.1 holds for B l (M) = Al+α (M) with s1 (q) = q − 1 and s2 (q) =
n − k − q + 1 when M is locally embeddable
(iii) Theorem 2.1 holds for B l (M) = Γl+α (M) with s1 (q) = q − 2 in the abstract
case and s1 (q) = q − 1 when M is locally embeddable, and with s2 (q) =
n − k − q + 1 in both cases.
Since in all the three cases of Theorem 3.3 we have ∩l∈N Bl (M) = C ∞ (M),
Corollary 2.3 becomes
Corollary 3.4. Let M = (M, H0,1 M) be an abstract compact CR manifold of class
C ∞ , of real dimension 2n − k and CR dimension n − k, and M , = (M, H , 0,1 M)
, 0,1 M is an integrable horizontal
another abstract compact CR manifold such that H
∞
smooth perturbation of H0,1 M. Let Φ ∈ C0,1 (M, H1,0 M) be the differential form
Φ
which defines the tangential Cauchy-Riemann operator ∂ b = ∂ b − Φ∂b associated
to the CR structure H , 0,1 M. Assume M is q-concave, then there exists a sequence
(δl )l∈N of positive real numbers such that, if Φl < δl for each l ∈ N
, = 0, when 1 ≤ r ≤ q − 2
(i) H p,r−p (M) = 0, for all 1 ≤ p ≤ r, implies HΦ0,r (M)
in the abstract case and also for r = q − 1 if M is locally embeddable,
(ii) H n−p,r+p (M) = 0, for all 0 ≤ p ≤ n − k − r, implies HΦn,r (M) , = 0, when
n − k − q + 1 ≤ r ≤ n − k.
 Stability of the Vanishing of the ∂ b -cohomology 225

References
[1] M.Y. Barkatou and C. Laurent-Thiébaut, Estimations optimales pour l’opérateur de
Cauchy-Riemann tangentiel, Michigan Math. Journal 54 (2006), 545–586.
[2] T. Brönnle, C. Laurent-Thiébaut, and J. Leiterer, Global homotopy formulas on q-
concave CR manifolds for large degrees, preprint.
[3] G.B. Folland and E.M. Stein, Estimates for the ∂ b -complex and analysis on the
Heisenberg group, Comm. Pure Appl. Math. 27 (1974), 429–522.
[4] C.D. Hill and M. Nacinovich, Pseudoconcave CR manifolds, Complex analysis and
geometry, V. Ancona, E. Ballico, A. Silva, eds., Marcel Dekker, Inc., New York, 1996,
pp. 275–297.
[5] C. Laurent-Thiébaut, Poincaré lemma and global homotopy formulas with sharp
anisotropic Hölder estimates in q-concave CR manifolds, Prépublication de l’Institut
Fourier hal-00335229, arXiv 0810.5295 (2008), 1–15.
[6] C. Laurent-Thiébaut and J. Leiterer, Global homotopy formulas on q-concave CR
manifolds for small degrees, J. Geom. Anal. 18 (2008), 511–536.
[7] P.L. Polyakov, Global ∂ M -homotopy with C k estimates for a family of compact, reg-
ular q-pseudoconcave CR manifolds, Math. Zeitschrift 247 (2004), 813–862.
[8] , Versal embeddings of compact 3-pseudoconcave CR-submanifolds, Math.
Zeitschrift 248 (2004), 267–312.
[9] M.-C. Shaw and L. Wang, Hölder and Lp estimates for b on CR manifolds of
arbitrary codimension, Math. Ann. 331 (2005), 297–343.

Christine Laurent-Thiébaut
Université de Grenoble
Institut Fourier
UMR 5582 CNRS/UJF
BP 74
F-38402 St. Martin d’Hères Cedex, France
e-mail: [email protected]
Complex Analysis
Trends in Mathematics, 227–244

c 2010 Springer Basel AG

Coherent Sheaves and Cohesive Sheaves

László Lempert
To Linda Rothschild on her birthday

Abstract. We consider coherent and cohesive sheaves of O-modules over open

sets Ω ⊂ Cn . We prove that coherent sheaves, and certain other sheaves
derived from them, are cohesive; and conversely, certain sheaves derived from
cohesive sheaves are coherent. An important tool in all this, also proved here,
is that the sheaf of Banach space valued holomorphic germs is flat.
Mathematics Subject Classification (2000). 32C35, 32B05, 14F05, 13C.
Keywords. Coherent sheaves, cohesive sheaves, flat modules.

1. Introduction
The theory of coherent sheaves has been central to algebraic and analytic geome-
try in the past fifty years. By contrast, in infinite-dimensional analytic geometry
coherence is irrelevant, as most sheaves associated with infinite-dimensional com-
plex manifolds are not even finitely generated over the structure sheaf, let alone
coherent. In a recent paper with Patyi, [LP], we introduced the class of so-called
cohesive sheaves in Banach spaces, that seems to be the correct replacement of
coherent sheaves – we were certainly able to show that many sheaves that occur
in the subject are cohesive, and for cohesive sheaves Cartan’s Theorems A and
B hold. We will go over the definition of cohesive sheaves in Section 2, but for a
precise formulation of the results above the reader is advised to consult [LP].
While cohesive sheaves were designed to deal with infinite-dimensional prob-
lems, they make sense in finite-dimensional spaces as well, and there are reasons
to study them in this context, too. First, some natural sheaves even over finite-
dimensional manifolds are not finitely generated: for example the sheaf OE of
germs of holomorphic functions taking values in a fixed infinite-dimensional Ba-
nach space E is not. It is not quasicoherent, either (for this notion, see [Ha]), but

Research partially supported by NSF grant DMS0700281, the Mittag–Leffler Institute, and the
Clay Institute.
228 L. Lempert

it is cohesive. Second, a natural approach to study cohesive sheaves in infinite-

dimensional manifolds would be to restrict them to various finite-dimensional sub-
manifolds.
The issue to be addressed in this paper is the relationship between coherence
and cohesion in finite-dimensional spaces. Our main results are Theorems 4.3, 4.4,
and 4.1. Loosely speaking, the first says that coherent sheaves are cohesive, and
the second that they remain cohesive even after tensoring with the sheaf OF of
holomorphic germs valued in a Banach space F . A key element of the proof is that
OF is flat, Theorem 4.1. This latter is also relevant for the study of subvarieties.
On the other hand, Masagutov showed that OF is not free in general, see [Ms,
Corollary 1.4].
The results above suggest two problems, whose resolution has eluded us. First,
is the tensor product of a coherent sheaf with a cohesive sheaf itself cohesive? Of
course, one can also ask the more ambitious question whether the tensor product
of two cohesive sheaves is cohesive, but here one should definitely consider some
kind of “completed” tensor product, and it is part of the problem to find which
one. The second problem is whether a finitely generated cohesive sheaf is coherent.
If so, then coherent sheaves could be defined as cohesive sheaves of finite type. We
could only solve some related problems: according to Corollary 4.2, any finitely
generated subsheaf of OF is coherent; and cohesive subsheaves of coherent sheaves
are also coherent, Theorem 5.4.

2. Cohesive sheaves, an overview

In this Section we will review notions and theorems related to the theory of cohesive
sheaves, following [LP]. We assume the reader is familiar with very basic sheaf
theory. One good reference to what we need here – and much more – is [S]. Let
Ω ⊂ Cn be an open set and E a complex Banach space. A function f : Ω → E is
holomorphic if for each a ∈ Ω there is a linear map L : Cn → E such that
f (z) = f (a) + L(z − a) + oz − a, z → a.
This is equivalent to requiring that in each ball B ⊂ Ω centered at any a ∈ Ω
our
 f can bej represented as a locally uniformly convergent power series f (z) =
j cj (z − a) , with j = (j1 , . . . , jn ) a nonnegative multiindex and cj ∈ E. We
denote by OΩ E
or just OE the sheaf of holomorphic E-valued germs over Ω. In
particular, O = OC is a sheaf of rings, and OE is a sheaf of O-modules. Typically,
instead of a sheaf of O-modules we will just talk about O-modules.
Definition 2.1. The sheaves O E = OΩ
E
→ Ω are called plain sheaves.
Theorem 2.2 ([Bi, Theorem 4], [Bu, p. 331] or [L, Theorem 2.3]). If Ω ⊂ Cn is
pseudoconvex and q = 1, 2, . . ., then H q (Ω, OE ) = 0.
Given another Banach space F , we write Hom(E, F ) for the Banach space of
continuous linear maps E → F . If U ⊂ Ω is open, then any holomorphic Φ : U →
 Coherent Sheaves and Cohesive Sheaves 229

Hom(E, F ) induces a homomorphism ϕ : OE |U → OF |U , by associating with the

germ of a holomorphic e : V → E at ζ ∈ V ⊂ U the germ of the function z →
Φ(z)e(z), again at ζ. Such homomorphisms and their germs are called plain. The
sheaf of plain homomorphisms between OE and OF is denoted Homplain (OE , OF ).
If HomO (A, B) denotes the sheaf of O-homomorphisms between O-modules A and
B, then
Homplain (OE , OF ) ⊂ HomO (OE , OF ) (2.1)
is an O-submodule. In fact, Masagutov showed that the two sides in (2.1) are equal
unless n = 0, see [Ms, Theorem 1.1], but for the moment we do not need this. The
O(U )-module of sections Γ(U, Homplain (OE , OF )) is in one-to-one correspondence
with the O(U )-module Homplain (OE |U, OF |U ) of plain homomorphisms. Further,
Hom(E,F )
any germ Φ ∈ Oz induces a germ ϕ ∈ Homplain (OE , OF )z . As pointed out
in [LP, Section 2], the resulting map is an isomorphism
≈
OHom(E,F ) → Homplain (OE , OF ) (2.2)
of O-modules.
Definition 2.3. An analytic structure on an O-module A is the choice, for each
plain sheaf E, of a submodule Hom(E, A) ⊂ HomO (E, A), subject to
(i) if E, F are plain sheaves and ϕ ∈ Homplain (E, F )z for some z ∈ Ω, then
ϕ∗ Hom(F, A)z ⊂ Hom(E, A)z ; and
(ii) Hom(O, A) = HomO (O, A).
If A is endowed with an analytic structure, one says that A is an analytic
sheaf. The reader will realize that this is different from the traditional terminology,
where “analytic sheaves” and “O-modules” mean one and the same thing.
For example, one can endow a plain sheaf G with an analytic structure by
setting
Hom(E, G) = Homplain (E, G).
Unless stated otherwise, we will always consider plain sheaves endowed with this
analytic structure. – Any O-module A has two extremal analytic structures. The
maximal one is given by Hom(E, A) = HomO (E, A). In the  minimal structure,
Hommin (E, A) consists of germs α that can be written α = βj γj with
γj ∈ Homplain (E, O) and βj ∈ HomO (O, A), j = 1, . . . , k.
An O-homomorphism ϕ : A → B of O-modules induces a homomorphism
ϕ∗ : HomO (E, A) → HomO (E, B)
for E plain. When A, B are analytic sheaves, we say that ϕ is analytic if
ϕ∗ Hom(E, A) ⊂ Hom(E, B)
for all plain sheaves E. It is straightforward to check that if A and B themselves are
plain sheaves, then ϕ is analytic precisely when it is plain. We write Hom(A, B)
for the O(Ω)-module of analytic homomorphisms A → B and Hom(A, B) for the
sheaf of germs of analytic homomorphisms A|U → B|U , with U ⊂ Ω open. Again,
230 L. Lempert

one easily checks that, when A = E is plain, this new notation is consistent with
the one already in use. Further,
Hom(A, B) ≈ Γ(Ω, Hom(A, B)). (2.3)
Definition 2.4. Given an O-homomorphism ϕ : A → B of O-modules, any analytic
structure on B induces one on A by the formula
Hom(E, A) = ϕ−1
∗ Hom(E, B).

If ϕ is an epimorphism, then any analytic structure on A induces one on B by the

formula
Hom(E, B) = ϕ∗ Hom(E, A).
[LP, 3.4] explains this construction in the cases when ϕ is the inclusion of a
submodule A ⊂ B and when ϕ is the projection on a quotient B = A/C.
9 Ai , i ∈ I, of analytic sheaves, an analytic structure is induced
Given a family
on the sum A = Ai . For any plain E there is a natural homomorphism
:
HomO (E, Ai ) → HomO (E, A),
i

9 we define the analytic structure on A by letting Hom(E, A) be the image of

and
Hom(E, Ai ). With this definition, the inclusion maps Ai → A and the projec-
tions A → Ai are analytic.
Definition 2.5. A sequence A → B → C of analytic sheaves and homomorphisms
over Ω is said to be completely exact if for every plain sheaf E and every pseudo-
convex U ⊂ Ω the induced sequence
Hom(E|U, A|U ) → Hom(E|U, B|U ) → Hom(E|U, C|U )
is exact. A general sequence of analytic homomorphisms is completely exact if
every three-term subsequence is completely exact.
Definition 2.6. An infinite completely exact sequence
· · · → F 2 → F1 → A → 0 (2.4)
of analytic homomorphisms is called a complete resolution of A if each Fj is plain.
When Ω is finite dimensional, as in this paper, complete resolutions can be
defined more simply:
Theorem 2.7. Let
ϕ2 ϕ1 ϕ0
· · · → F2 −→ F1 −→ A −→ 0 (2.5)
be an infinite sequence of analytic homomorphisms over Ω ⊂ C , with each Fj
n

plain. If for each plain E over Ω the induced sequence

· · · → Hom(E, F2 ) → Hom(E, F1 ) → Hom(E, A) → 0 (2.6)
is exact, then (2.5) is completely exact.
 Coherent Sheaves and Cohesive Sheaves 231

Proof. Setting E = O in (2.6) we see that (2.5) is exact. Let Kj = Ker ϕj =

Im ϕj+1 , and endow it with the analytic structure induced by the embedding
ϕj
Kj → Fj , as in Definition 2.4. The exact sequence O → Kj → Fj −→ Kj−1 → 0
induces a sequence
0 → Hom(E, Kj ) → Hom(E, Fj ) → Hom(E, Kj−1 ) → 0, (2.7)
also exact since (2.6) was. Let U ⊂ Ω be pseudoconvex. Then in the long exact
sequence associated with (2.7)
· · · → H q (U, Hom(E, Fj )) → H q (U, Hom(E, Kj−1 )) →
→ H q+1 (U, Hom(E, Kj )) → H q+1 (U, Hom(E, Fj )) → · · · (2.8)
the first and last terms indicated vanish for q ≥ 1 by virtue of Theorem 2.2 and
(2.2). Hence the middle terms are isomorphic:

H q (U, Hom(E, Kj−1 )) ≈ H q+1 (U, Hom(E, Kj )) ≈ · · ·

· · · ≈ H q+n (U, Hom(E, Kj+n−1 )) ≈ 0.
Using this and (2.3), the first few terms of the sequence (2.8) are
0 → Hom(E|U, Kj |U ) → Hom(E|U, Fj |U ) → Hom(E|U, Kj−1 |U ) → 0.
The exactness of this latter implies · · · → Hom(E|U, F1 |U ) → Hom(E|U, A|U ) → 0
is exact, and so (2.5) is indeed completely exact. 
Definition 2.8. An analytic sheaf A over Ω ⊂ Cn is cohesive if each z ∈ Ω has a
neighborhood over which A has a complete resolution.
The simplest examples of cohesive sheaves are the plain sheaves, that have
complete resolutions of form · · · → 0 → 0 → E → E → 0. The main result of [LP]
implies the following generalization of Cartan’s Theorems A and B, see Theorem
2 of the Introduction there:
Theorem 2.9. Let A be a cohesive sheaf over a pseudoconvex Ω ⊂ Cn . Then
(a) A has a complete resolution over all of Ω;
(b) H q (Ω, A) = 0 for q ≥ 1.

3. Tensor products
Let R be a commutative ring with a unit and A, B two R-modules. Recall that
the tensor product A ⊗R B = A ⊗ B is the R-module freely generated by the set
A × B, modulo the submodule generated by elements of form
(ra + a , b) − r(a, b) − (a , b) and (a, rb + b ) − r(a, b) − (a, b ),
where r ∈ R, a, a ∈ A, and b, b ∈ B. The class of (a, b) ∈ A×B in A⊗B is denoted
a⊗b. Given homomorphisms α : A → A , β : B → B  of R-modules, α⊗β : A⊗B →
A ⊗ B  denotes the unique homomorphism satisfying (α ⊗ β)(a ⊗ b) = α(a) ⊗ β(b).
232 L. Lempert

A special case is the tensor product of Banach spaces A, B; here R = C.

The tensor product A ⊗ B is just a vector space, on which in general there are
several natural ways to introduce a norm. However, when dim A = k < ∞, all
those norms are equivalent, and turn A ⊗ B into a Banach space. For example, if a
basis a1 , . . . , ak of A is fixed, any v ∈ A⊗ B can be uniquely written v = aj ⊗ bj ,
with bj ∈ B. Then A ⊗ B with the norm
v = max bj B
j
⊕k
is isomorphic to B .
Similarly, if R is a sheaf of commutative unital rings over a topological space
Ω and A, B are R-modules, then the tensor product sheaf A ⊗R B = A ⊗ B can be
defined, see, e.g., [S]. The tensor product is itself a sheaf of R-modules, its stalks
(A ⊗ B)x are just the tensor products of Ax and Bx over Rx . Fix now an open
Ω ⊂ Cn , an O-module A, and an analytic sheaf B over Ω. An analytic structure
can be defined on A ⊗ B as follows. For any plain sheaf E there is a tautological
O-homomorphism
T = TE : A ⊗ Hom(E, B) → HomO (E, A ⊗ B), (3.1)
obtained by associating with a ∈ Aζ ,  ∈ Hom(E, B)ζ first a section ã of A over a
neighborhood U of ζ, such that ã(ζ) = a; then defining τ a ∈ HomO (B, A ⊗ B)ζ as
the germ of the homomorphism
Bz  b → ã(z) ⊗ b ∈ Az ⊗ Bz , z ∈ U;
and finally letting T (a ⊗ ) = τ .
a

Definition 3.1. The (tensor product) analytic structure on A ⊗ B is given by

Hom(E, A ⊗ B) = Im TE .
One quickly checks that this prescription indeed satisfies the axioms of an
analytic structure. Equivalently, one can define Hom(E, A ⊗ B) ⊂ HomO (E, A ⊗ B)
as the submodule spanned by germs of homomorphisms of the form
≈ α⊗β
E|U −→ O ⊗ E|U −−−−→ A ⊗ B|U,
where U ⊂ Ω is open, α : O|U → A|U and β : E|U → B|U are O-, resp. analytic
homomorphisms (and the first isomorphism is the canonical one). The following is
obvious.
Proposition 3.2. T in (3.1) is natural: if α : A → A and β : B → B  are O-,
resp. analytic homomorphisms, then T and the corresponding T  fit in a commu-
tative diagram

A ⊗ Hom(E, B)
α⊗β∗
/ A ⊗ Hom(E, B  )

T T
 (α⊗β)∗ 
HomO (E, A ⊗ B) / HomO (E, A ⊗ B  ).
 Coherent Sheaves and Cohesive Sheaves 233

Corollary 3.3. If α, β are as above, then α ⊗ β : A ⊗ B → A ⊗ B  is analytic.

Proposition 3.4. If A is an O-module and B an analytic sheaf, then the tensor
product analytic structure on A ⊗ O is the minimal one. Further, the map
B  b → 1 ⊗ b ∈ O ⊗ B
is an analytic isomorphism.
Both statements follow from inspecting the definitions.
Proposition 3.5. If A, Ai are O-modules and B, Bi are analytic sheaves, then the
obvious O-isomorphisms
: ≈
: : ≈
:
( Ai ) ⊗ B −→ (Ai ⊗ B), A ⊗ ( Bi ) −→ (A ⊗ Bi )
are in fact analytic isomorphisms.
This follows from Definition 3.1, upon taking into account the distributive
property of the tensor product of O-modules. Consider now a finitely generated
plain sheaf F = OF ≈ O ⊕ · · · ⊕ O, with dim F = k. By putting together Propo-
sitions 3.4 and 3.5 we obtain analytic isomorphisms
F ⊗ B ≈ (O ⊗ B) ⊕ · · · ⊕ (O ⊗ B) ≈ B ⊕ · · · ⊕ B.
When B = O B
is plain, this specializes to
⊕k
OF ⊗ OB ≈ OB ⊕ · · · ⊕ OB ≈ OB ≈ OF ⊗B . (3.2)
Later on we will need to know that inducing, in the sense of Definition 2.4, and
tensoring are compatible. Here we discuss the easy case, an immediate consequence
of the tensor product being a right exact functor; the difficult case will have to
wait until Section 6.
Proposition 3.6. Let ψ : A → A be an epimorphism of O-modules and B an an-
alytic sheaf. Then the tensor product analytic structure on A ⊗ B is induced (in
the sense of Definition 2.4) from the tensor product analytic structure on A ⊗ B
by the epimorphism ψ ⊗ idB : A ⊗ B → A ⊗ B.
Proof. We write A ⊗ B, A ⊗ B for the analytic sheaves endowed with the tensor
product structure. The claim means
(ψ ⊗ idB )∗ Hom(E, A ⊗ B) = Hom(E, A ⊗ B)
for every plain E. But this follows from Definition 3.1 if we take into account the
naturality of T (Proposition 3.2) and that
ψ ⊗ idHom(E,B) : A ⊗ Hom(E, B) → A ⊗ Hom(E, B)
is onto. 

In the sequel it will be important to know when T in (3.1) is injective. This

issue is somewhat subtle and depends on the analysis of Section 5.
234 L. Lempert

4. The main results

We fix an open set Ω ⊂ Cn . In the remainder of this paper all sheaves, unless
otherwise stated, will be over Ω.
Theorem 4.1. Let F be a plain sheaf, A ⊂ F finitely generated, and ζ ∈ Ω. Then on
some open U  ζ there is a finitely generated free subsheaf E ⊂ F |U that contains
A|U . In particular, plain sheaves are flat.
Recall that an O-module F is flat if for every exact sequence A → B → C of
O-modules the induced sequence A ⊗ F → B ⊗ F → C ⊗ F is exact.
Theorem 4.1 and Oka’s coherence theorem imply
Corollary 4.2. Finitely generated submodules of a plain sheaf are coherent.
Theorem 4.3. A coherent sheaf, endowed with its minimal analytic structure, is
cohesive.
Theorem 4.4. If A is a coherent sheaf and B is a plain sheaf, then A⊗B is cohesive.
Theorem 4.1 will be proved in Section 5, Theorems 4.3 and 4.4 in Section 7.

5. Preparation
The main result of this Section is the following. Throughout, Ω ⊂ Cn will be open.
Lemma 5.1. Let P, Q be Banach spaces, f : Ω → Hom(P, Q) holomorphic, and
ζ ∈ Ω.
(a) If dim P < ∞ then there are a finite-dimensional Q ⊂ Q, an open U  ζ,
and a holomorphic q : U → GL(Q) such that Im q(z)f (z) ⊂ Q for all z ∈ U .
(b) If dim Q < ∞ then there are a finite codimensional P  ⊂ P , an open U  ζ,
and a holomorphic p : U → GL(P ) such that P  ⊂ Ker f (z)p(z) for all z ∈ U .
The proof depends on various extensions of the Weierstrass Preparation The-
orem. That the road to coherence leads through the Preparation Theorem is, of
course, an old idea of Oka. Let A be a Banach algebra with unit 1, and let A× ⊂ A
denote the open set of invertible elements.
Lemma 5.2. Let f : Ω → A be holomorphic, 0 ∈ Ω, and d = 0, 1, 2, . . . such that
∂jf ∂ df
j
(0) = 0 f or j < d, and (0) ∈ A× .
∂z1 ∂z1d
Then on some open U  0 there is a holomorphic Φ : U → A× such that, writing
z = (z1 , z  )

d−1
Φ(z)f (z) = 1z1d + fj (z  )z1j , z ∈ U, (5.1)
j=0
and fj (0) = 0.
 Coherent Sheaves and Cohesive Sheaves 235

We refer the reader to [Hö, 6.1]. The proof of Weierstrass’ theorem given
there for the case A = C applies in this general setting as well.

Lemma 5.3. Let 0 ∈ Ω, E a Banach space, E ∗ its dual, g : Ω → E (resp. h : Ω →

E ∗ ) holomorphic functions such that


∂j g ∂j h
(0) = 0 resp. (0) = 0 , for some j. (5.2)
∂z1j ∂z1j
Then there are an open U  0, a holomorphic Φ : U → GL(E), and 0 = e ∈ E
(resp. 0 = e∗ ∈ E ∗ ), such that

d−1
Φ(z)g(z) = ez1d + gj (z  )z1j , z ∈ U,
j=0

 (5.3)

d−1
resp. h(z)Φ(z) = e∗ z1d + hj (z  )z1j ,
j=0

with some d = 0, 1, . . ., and gj (0) = 0, hj (0) = 0.

Proof. We will only prove for g, the proof for h is similar. The smallest j for
which (5.2) holds will be denoted d. Thus ∂ d g/∂z1d(0) = e = 0. Let V ⊂ E be a
closed subspace complementary to the line spanned by e, and define a holomorphic
f : Ω → Hom(E, E) by
f (z)(λe + v) = λg(z) + vz1d /d!, λ ∈ C, v ∈ V.
We apply Lemma 5.2 with the Banach algebra A = Hom(E, E); its invertibles
form A× = GL(E). As
∂j f ∂ df
(0) = 0 for j < d and (0) = idE ,
∂z1j ∂z1d
there are an open U  0 and Φ : U → GL(E) satisfying (5.1) and fj (0) = 0. Hence

d−1
Φ(z)g(z) = Φ(z)f (z)(e) = ez1d + fj (z  )(e)z1j ,
j=0

as claimed. 

Proof of Lemma 5.1. We will only prove (a), part (b) is proved similarly. The proof
will be by induction on n, the case n = 0 being trivial.
So assume the (n − 1)-dimensional case and consider Ω ⊂ Cn . Without loss
of generality we take ζ = 0. Suppose first dim P = 1, say, P = C, and let g(z) =
f (z)(1). Thus g : Ω → Q is holomorphic. When g ≡ 0 near 0, the claim is obvious;
otherwise we can choose coordinates so that ∂ j g/∂z1j (0) = 0 for some j. By Lemma
5.3 there is a holomorphic Φ : U → GL(Q) satisfying (5.3). We can assume U =
236 L. Lempert

U1 × Ω ⊂ C × Cn−1 . Consider the holomorphic function f  : Ω → Hom(Cd+1 , Q)

given by

d
f  (z  )(ξ0 , ξ1 , . . . , ξd ) = eξ0 + gj (z  )ξj .
1
By the inductive assumption, after shrinking U and Ω , there are a q  : U  → GL(Q)
and a finite-dimensional Q ⊂ Q so that Im q  (z  )f  (z  ) ⊂ Q for all z  ∈ U  . This
implies q  (z  )Φ(z)g(z) ∈ Q , and so with q(z) = q  (z  )Φ(z) indeed Im q(z)f (z) ⊂ Q .
To prove the claim for dim P > 1 we use induction once more, this time on
dim P . Assume the claim holds when dim P < k, and consider a k-dimensional P ,
k ≥ 2. Decompose P = P1 ⊕ P2 with dim P1 = 1. By what we have already proved,
there are an open U  0, a holomorphic q1 : U → GL(Q), and a finite-dimensional
Q1 ⊂ Q such that q1 (z)f (z)P1 ⊂ Q1 . Choose a closed complement Q2 ⊂ Q to Q1 ,
and with the projection π : Q1 ⊕ Q2 → Q2 let
f2 (z) = πq1 (z)f (z) ∈ Hom(P, Q2 ). (5.4)
As dim P2 = k − 1, by the inductive hypothesis there are a finite-dimensional
Q2 ⊂ Q2 and (after shrinking U ) a holomorphic q2 : U → GL(Q2 ) such that
q2 (z)f2 (z)P2 ⊂ Q2 . We extend q2 to q2 : U → GL(Q) by taking it to be the
identity on Q1 . Then q2 (z)f2 (z)P2 ⊂ Q2 and
q2 (z)q1 (z)f (z)P1 ⊂ Q1 . (5.5)
Further, (5.4) implies (q1 (z)f (z) − f2 (z))P ⊂ Q1 and so
q2 (z)q1 (z)f (z)P2 ⊂ q2 (z)Q1 + q2 (z)f2 (z)P2 ⊂ Q1 ⊕ Q2 . (5.6)
(5.5) and (5.6) show that q = q2 q1 and Q = Q1 ⊕ Q2 satisfy the requirements,
and the proof is complete. 

Proof of Theorem 4.1. Let F = OF and let A be generated by holomorphic

f1 , . . . , fk : Ω → F . These functions define a holomorphic f : Ω → Hom(Ck , F ) by

f (z)(ξ1 , . . . , ξk ) = ξj fj (z).
j

Choose a finite-dimensional Q ⊂ F , an open U  ζ, and a holomorphic q : U →

GL(F ) as in Lemma 5.1(a). Then E = q −1 OQ |U ⊂ F is finitely generated and
free; moreover, it contains the germs of each fj |U , hence also A|U .
As to flatness: it is known, and easy, that the direct limit of flat modules
is flat ([Mt, Appendix B]). As each stalk of F is the direct limit of its finitely
generated free submodules, it is flat. 

Here is another consequence of Lemma 5.1.

Theorem 5.4. Let A be a coherent sheaf and let B ⊂ A be a submodule. If there
are a plain sheaf OF = F and an O-epimorphism ϕ : F → B, then B is coherent.
In particular, cohesive subsheaves of A are coherent.
 Coherent Sheaves and Cohesive Sheaves 237

If F of the theorem is finitely generated, then so is B, and its coherence is

immediate from the definitions. For the proof of the general statement we need
the notion of depth. Recall that given an O-module A, the depth of a stalk Aζ is
0 if there is a submodule 0 = L ⊂ Aζ annihilated by the maximal ideal mζ ⊂ Oζ .
Otherwise depth Aζ > 0. (For the general notion of depth, see [Mt, p. 130]; the
version we use here is the one, e.g., in [Ms, Proposition 4.2], at least in the positive
dimensional case.)
Lemma 5.5. If A is a coherent sheaf, then
D = {z ∈ Ω : depth Az = 0}
is a discrete set.
Proof. Observe that, given a compact polydisc K ⊂ Ω, the O(K)-module Γ(K, A)
is finitely generated. Indeed, if 0 → A |K → O⊕p |K → A|K → 0 is an exact
sequence of O|K-modules, then H 1 (K, A ) = 0 implies that
O(K)⊕p ≈ Γ(K, O⊕p ) → Γ(K, A)
is surjective. We shall also need the fact that O(K) is Noetherian, see, e.g., [F].
As for the lemma, we can assume dim Ω > 0. If z ∈ D, there is a nonzero
submodule B ⊂ Az such that mz B = 0. Let B z denote the skyscraper sheaf over
Ω whose only nonzero stalk is B, at z. We do this construction for every z ∈ D.
With K ⊂ Ω a compact polydisc, the submodule

Γ(K, B z ) ⊂ Γ(K, A) (5.7)
z∈D∩K

is finitely generated. But Γ(K, B z ) = 0 consists of (certain) sections of A supported

at z. It follows that the sum in (5.7) is a direct sum, hence in fact a finite direct
sum. In other words, D ∩ K is finite for every compact polydisc K, and D must
be discrete. 
Proof of Theorem 5.4. We can suppose dim Ω > 0. First we assume that, in ad-
dition, depth Az > 0 for every z. Since coherence is a local property, and A is
locally finitely generated, we can assume that Ω is a ball, and there are a finitely
generated plain sheaf O E = E ≈ O ⊕ · · · ⊕ O and an epimorphism  : E → A. We
are precisely in the situation of Theorem 7.1 in [Ms]. By this theorem, ϕ factors
through : there is an O-homomorphism ψ : F → E such that ϕ = ψ. (Masagutov
in his proof of Theorem 7.1 relies on a result of the present paper, but the rea-
soning is not circular. What the proof of [Ms, Theorem 7.1] needs is our Theorem
4.3, whose proof is independent of Theorem 5.4 we are justifying here.) As an
O-homomorphism between plain sheaves, ψ is plain by [Ms, Theorem 1.1].
In view of Lemma 5.1(b), there are a finite codimensional F  ⊂ F and a

plain isomorphism ρ : F → F such that ψρ|OF = 0. If F  ⊂ F denotes a

(finite-dimensional) complement to F  , then ψρ(OF ) = ψρ(F ) = ψ(F ). Hence

ψρ(OF ) = ψ(F) = B is finitely generated; as a submodule of a coherent sheaf,
itself must be coherent.
238 L. Lempert

Now take an A whose depth is 0 at some z. In view of Lemma 5.5 we can

assume that there is a single such z. With
C = {a ∈ Az : mkz a = 0 for some k = 1, 2, . . .},
let C ⊂ A be the skyscraper sheaf over Ω whose only nonzero stalk is C, at z. As
C is finitely generated, C and A/C are coherent. Also, depth(A/C)ζ > 0 for every
ζ ∈ Ω. Therefore by the first part of the proof B/B ∩ C ⊂ A/C is coherent. Since
B ∩ C, supported at the single point z, is coherent, the Three Lemma implies B is
coherent, as claimed. 

6. Hom and ⊗
The main result of this section is the following. Let A be an O-module and B an
analytic sheaf. Recall that, given a plain sheaf E, in Section 3 we introduced a
tautological O-homomorphism
T = TE : A ⊗ Hom(E, B) → HomO (E, A ⊗ B), (6.1)
and Hom(E, A ⊗ B) was defined as the image of TE .

Theorem 6.1. If B is plain, then T is injective.

Suppose E = O E , B = OB are plain sheaves. If ζ ∈ Ω, a C-linear map

Hom(E,B)
S ζ : Aζ ⊗ Oζ → HomC (E, Aζ ⊗ OζB ) (6.2)
Hom(E,B)
can be defined as follows. Let a ∈ Aζ , Θ ∈ Oζ , then

S ζ (a ⊗ Θ)(e) = a ⊗ Θe, e ∈ E,
where on the right e is thought of as a constant germ ∈ OζE . The key to Theorem
6.1 is the following

Lemma 6.2. Let E, B be Banach spaces, ζ ∈ Ω, and let M be an Oζ -module. Then

the tautological homomorphism
Hom(E,B)
S : M ⊗ Oζ → HomC (E, M ⊗ OζB ) (6.3)

given by S(m ⊗ Θ)(e) = m ⊗ Θe, for e ∈ E, is injective.

As T was, S is also natural with respect to Oζ -homomorphisms M → N .

The claim of the lemma is obvious when M is free, for then tensor products M ⊗ L
are just direct sums of copies of L. The claim is also obvious when M is a direct
summand in a free module M  = M ⊕ N , as the tautological homomorphism for
M  decomposes into the direct sum of the tautological homomorphisms for M
and N .
 Coherent Sheaves and Cohesive Sheaves 239

Proof of Lemma 6.2. The proof is inspired by the proof of [Ms, Theorem 1.3]. The
heart of the matter is to prove when M is finitely generated. Let us write (Ln ) for
the statement of the lemma for M finitely generated and n = dim Ω; we prove it
by induction on n. (L0 ) is trivial, as Oζ ≈ C is a field and any module over it is
free. So assume (Ln−1 ) for some n ≥ 1, and prove (Ln ). We take ζ = 0.
Step 1◦ . First we verify (Ln ) with the additional assumption that gM = 0
with some 0 = g ∈ O0 . By Weierstrass’ preparation theorem we can take g to be
(the germ of) a Weierstrass polynomial of degree d ≥ 1 in the z1 variable. We

write z = (z1 , z  ) ∈ Cn , and O0 , O0F for the ring/module of the corresponding

germs in Cn−1 (here F is any Banach space). We embed O0 ⊂ O0 , O0F ⊂ O0F as
germs independent of z1 . This makes O0 -modules into O0 -modules. In the proof
tensor products both over O0 and O0 will occur; we keep writing ⊗ for the former
and will write ⊗ for the latter.
We claim that the O0 -homomorphism

i : M ⊗ O0F → M ⊗ O0F , i(m ⊗ f  ) = m ⊗ f  ,

is in fact an isomorphism. To verify it is surjective, consider m ⊗ f ∈ M ⊗ O0F . By

Weierstrass’ division theorem, valid for vector-valued functions as well (e.g., the
proof in [GuR, p. 70] carries over verbatim), f can be written

d−1

f = f0 g + fj z1j , f0 ∈ O0F , fj ∈ O0F .
j=0
 
Thus m ⊗ f = m ⊗ (f0 g + fj z1j ) = i( z1j m ⊗ fj ) is indeed in Im i. Further,

injectivity is clear if dim F = k < ∞, as M ⊗ O0F ≈ M ⊕k , M ⊗ O0F ≈ M ⊕k , and
i corresponds to the identity of M ⊕k . For a general F consider a finitely generated

submodule A ⊂ O0F . Lemma 5.1(a) implies that there are a neighborhood U of
0 ∈ Cn−1 , a finite-dimensional subspace G ⊂ F , and a holomorphic q : U → GL(F )
  
such that the automorphism ϕ of O0F induced by q maps A into O0G ⊂ O0F .
(The reasoning is the same as in the proof of Theorem 4.1.) If extended to C × U
independent of z1 , q also induces an automorphism ϕ of O0F , and i intertwines the

automorphisms idM ⊗ ϕ and idM ⊗ ϕ. Now i is injective between M ⊗ O0G and

M ⊗ O0G ⊂ M ⊗ O0F , because dim G < ∞. As the image of M ⊗ A in M ⊗ O0F


is contained in M ⊗ O0 , it follows that i is injective on this image. Since the
G

finitely generated A ⊂ O0F was arbitrary, i is indeed injective.
Applying this with F = Hom(E, B) and F = B, we obtain a commutative
diagram

Hom(E,B) ≈ / M ⊗ OHom(E,B)
M ⊗ O0 0

S S
 
 ≈ / HomC (E, M ⊗ OB ).
HomC (E, M ⊗ O0B ) 0
240 L. Lempert

Here S  is also a tautological homomorphism. Now M is finitely generated over

O0 /gO0 , and this latter is a finitely generated O0 -algebra by Weierstrass division.
It follows that M is finitely generated over O0 ; by the induction hypothesis S  is
injective, hence so must be S.
Step 2◦ . Now take an arbitrary finitely generated M . Let μ : L → M be an
epimorphism from a free finitely generated O0 -module L, and K = Ker μ. If the
exact sequence
λ μ
0→K → L→ M →0 (6.4)

splits, then M is a direct summand in L and, as said, the claim is immediate. The
point of the reasoning to follow is that, even if (6.4) does not split, it does split up
to torsion in the following sense: there are σ ∈ Hom(L, K) and 0 = g ∈ O0 such
that σλ : K → K is multiplication by g. To see this, let Q be the field of fractions
of O0 , and note that the induced linear map λQ : K ⊗ Q → L ⊗ Q of Q-vector
spaces has a left inverse τ . Clearing denominators in τ then yields the σ needed.
We denote the tautological homomorphisms (6.3) for K, L, M by SK , SL , SM .
Tensoring and Hom-ing (6.4) gives rise to a commutative diagram

Hom(E,B) o
λt
/ Hom(E,B) μt
/ M ⊗ OHom(E,B) /0
K ⊗ O0 σt
L ⊗ O0 0

SK SL SM
  
HomC (E, K ⊗ O0B ) o
λh
/ Hom (E, L ⊗ OB ) μh
/ HomC (E, M ⊗ OB ) /0
σh C 0 0

with exact rows. Here λt , λh , etc. just indicate homomorphisms induced on various
modules by λ, etc. Consider an element of Ker SM ; it is of form μt u, u ∈ L ⊗
Hom(E,B)
O0 . Then SL u ∈ Ker μh = Im λh . Let SL u = λh v. We compute

SL λt σt u = λh SK σt u = λh σh SL u = λh σh λh v = λh gv = SL gu.

Since L is free, SL is injective, so λt σt u = gu and gμt u = μt λt σt u = 0. We conclude

that gKer SM = 0. Let N ⊂ M denote the submodule of elements annihilated by
Hom(E,B)
g and, for brevity, set H = O0 , a flat module. Multiplication by g is a
monomorphism on M/N , so the same holds on M/N ⊗ H. The exact sequence

0 → N ⊗ H → M ⊗ H → M/N ⊗ H → 0

then shows that in M ⊗ H the kernel of multiplication by g is N ⊗ H. Therefore

N ⊗ H ⊃ Ker SM , and Ker SM ⊂ Ker SN . But gN = 0, so from Step 1◦ it follows
that Ker SN = 0, and again Ker SM = 0.
Step 3◦ . Having proved the lemma for finitely generated modules, consider
an arbitrary O0 -module M . The inclusion ι : N → M of a finitely generated
 Coherent Sheaves and Cohesive Sheaves 241

submodule induces a commutative diagram

N ⊗ O0
Hom(E,B) ιt
/ M ⊗ OHom(E,B)
0

SN S
 
HomC (E, N ⊗ O0B )
ιh
/ HomC (E, M ⊗ OB ),
0

with SN the tautological homomorphism for N . Flatness implies that ιt , ιh are

injective; as SN is also injective by what we have proved so far, S itself is injective
Hom(E,B)
on the range of ιt . As N varies, these ranges cover all of M ⊗ O0 , hence S
is indeed injective. 
Proof of Theorem 6.1. For ζ ∈ Ω we embed E → OζE as constant germs; this
induces a C-linear map
ρ : HomO (OE , A ⊗ OB )ζ → HomC (E, Aζ ⊗ OζB ).
It will suffice to show that if we restrict T to the stalk at ζ and compose it with
ρ, the resulting map
T ζ : Aζ ⊗ Hom(OE , OB )ζ → HomC (E, Aζ ⊗ OζB ),
given by T ζ (a ⊗ θ)(e) = a ⊗ θe, is injective. But, by the canonical isomorphism
Hom(E,B) ≈
Oζ → Hom(OE , OB )ζ , cf. (2.2), T ζ is injective precisely when S ζ of (6.2)
is; so that Lemma 6.2 finishes off the proof. 
Now we can return to the question how compatible are inducing in the sense
of Definition 2.4 and tensoring.
ϕ ψ
Lemma 6.3. If 0 → A → A → A → 0 is an exact sequence of O-modules and B
is a plain sheaf, then ϕ ⊗ idB , resp. ψ ⊗ idB , induce from A ⊗ B the tensor product
analytic structure on A ⊗ B, resp. A ⊗ B.
Proof. The case of A ⊗ B, in greater generality, is the content of Proposition 3.6.
Consider A ⊗ B. Meaning by A ⊗ B etc. the analytic sheaves endowed with the
tensor product structure, in light of Definition 2.4 we are to prove
Hom(E, A ⊗ B) = (ϕ ⊗ idB )−1
∗ Hom(E, A ⊗ B) (6.5)
for every plain E. Again using that B and Hom(E, B) are flat, from 0 → A →
A → A → 0 we obtain a commutative diagram with exact rows

0 / A ⊗ Hom(E, B) ϕt
/ A ⊗ Hom(E, B) ψt
/ A ⊗ Hom(E, B)

T T T 
  
0 / HomO (E, A ⊗ B) ϕh
/ HomO (E, A ⊗ B) ψh
/ HomO (E, A ⊗ B).

The vertical arrows are the respective tautological homomorphisms, and ϕt = ϕ ⊗

idHom(E,B) , ϕh = (ϕ⊗idB )∗ , etc. denote maps induced by ϕ, etc. From this diagram,
242 L. Lempert

the left-hand side of (6.5), Im T  , is clearly contained in ϕ−1

h Im T , i.e., in the right-
hand side. To show the converse, suppose  ∈ HomO (E, A ⊗ B) is in ϕ−1 h Im T ,
say, ϕh  = T u with u ∈ A ⊗ Hom(E, B). Then T  ψt u = ψh T u = ψh ϕh  = 0.
Since T  is injective by Theorem 6.1, ψt u = 0. It follows that u = ϕt v with some
v ∈ A ⊗ Hom(E, B), whence ϕh T  v = T ϕt v = T u = ϕh . As ϕh is also injective,
 = T  v; that is, ϕ−1 
h Im T ⊂ Im T , as needed. 

7. Coherence and cohesion

Proof of Theorems 4.3 and 4.4. We have to show that if A is a coherent sheaf and
B = OB plain then A⊗B is cohesive. This would imply that A⊗O is cohesive, and
in view of Proposition 3.4 that A ≈ A ⊗ O, with its minimal analytic structure, is
also cohesive.
We can cover Ω with open sets over each of which A has a resolution by
finitely generated free O-modules. We can assume that such a resolution
· · · → F 2 → F1 → A → 0
exists over all of Ω, and Fj = OFj , dim Fj < ∞. If E = OE is plain then
· · · → F2 ⊗ Hom(E, B) → F1 ⊗ Hom(E, B) → A ⊗ Hom(E, B) → 0
is also exact, Hom(E, B) ≈ OHom(E,B) being flat. By Theorem 6.1 this sequence is
isomorphic to
· · · → Hom(E, F2 ⊗ B) → Hom(E, F1 ⊗ B) → Hom(E, A ⊗ B) → 0,
which then must be exact. Here Fj ⊗ B ≈ OFj ⊗B analytically, cf. (3.2). Now
Theorem 2.7 applies. We conclude that
· · · → F 2 ⊗ B → F1 ⊗ B → A ⊗ B → 0
is completely exact, and A ⊗ B is indeed cohesive. 

8. Application. Complex analytic subspaces and subvarieties

The terminology in the subject indicated in the title is varied and occasionally
ambiguous, even in finite-dimensional complex geometry. Here we will use the
terms “complex subspace” and “subvariety” to mean different things. Following
[GrR], a complex subspace A of an open Ω ⊂ Cn is obtained from a coherent
submodule J ⊂# O. The support |A| of the sheaf O/J , endowed with the sheaf
of rings (O/J )#|A| = OA defines a ringed space, and the pair (|A|, OA ) is the
complex subspace in question.
For infinite-dimensional purposes this notion is definitely not adequate, and in
the setting of Banach spaces in [LP] we introduced a new notion that we called sub-
variety. Instead of coherent sheaves, they are defined in terms of cohesive sheaves,
furthermore, one has to specify a subsheaf J E ⊂ OE for each Banach space E,
(thought of as germs vanishing on the subvariety), not just one J ⊂ O. The reason
 Coherent Sheaves and Cohesive Sheaves 243

this definition was made was to delineate a class of subsets in Banach spaces that
arise in complex analytical questions, and can be studied using complex analysis.
At the same time, the definition makes sense in Cn as well, and it is natural to
ask how subvarieties and complex subspaces in Cn are related. Before answering
we have to go over the definition of subvarieties, following [LP].
An ideal system over Ω ⊂ Cn is the specification, for every Banach space E,
Hom(E,F )
of a submodule J E ⊂ OE , subject to the following: given z ∈ Ω, ϕ ∈ Oz ,
and e ∈ JzE , we have ϕe ∈ JzF .
Within an ideal system the support of OE /J E is the same for every E = (0),
and we call this set the support of the ideal system.
A subvariety S of Ω is given by an ideal system of cohesive subsheaves J E ⊂
O . The support of the ideal system is called the# support |S| of the subvariety,
E

and we endow it with the sheaves OSE = OE /J E #|S| of modules over OS = OSC .
The “functored space” (|S|, E → OSE ) is the subvariety S in question.

Theorem 8.1. There is a canonical way to associate a subvariety with a complex

subspace of Ω and vice versa.

Proof, or rather construction. Let i : J → O be the inclusion of a coherent sheaf

J that defines a complex subspace. The ideal system J E = J OE ⊂ OE then
gives rise to a subvariety, provided J E with the analytic structure inherited from
OE is cohesive. Consider the diagram
i⊗idOE
J ⊗ OE / O ⊗ OE

μ ≈
 
JE / OE .

Here the vertical arrow on the right, given by 1 ⊗ e → e, is an analytic isomor-

phism by Proposition 3.4. The vertical arrow μ on the left is determined by the
commutativity of the diagram; it is surjective. As O E is flat, i ⊗ idOE is injective,
therefore μ is an isomorphism. If J ⊗ OE is endowed with the analytic structure
induced by i ⊗ idOE , μ becomes an analytic isomorphism. On the other hand, this
induced structure of J ⊗ OE agrees with the tensor product analytic structure by
Lemma 6.3 (set A = J , A = O), hence it is cohesive by Theorem 4.4. The upshot
is that J E is indeed cohesive.
As to the converse, suppose J E is a cohesive ideal system defining a subva-
riety. Then J = J C ⊂ O is coherent by Theorem 5.4, and gives rise to a complex
subspace. 

Theorem 8.1 is clearly not the last word on the matter. First, it should be
decided whether the construction in the theorem is a bijection between subvarieties
and complex subspaces; second, the functoriality properties of the construction
should be investigated.
244 L. Lempert

References
[Bi] E. Bishop, Analytic functions with values in a Fréchet space, Pacific J. Math. 12
(1962), 1177–1192
[Bu] L. Bungart, Holomorphic functions with values in locally convex spaces and ap-
plications to integral formulas, Trans. Amer. Math. Soc 111 (1964), 317–344
[F] J. Frisch, Points de platitude d’un morphisme d’espaces analytiques complexes,
Invent. Math. 4 (1967), 118–138
[GrR] H. Grauert, R. Remmert, Theory of Stein spaces, Springer, Berlin, 1979
[GuR] R. Gunning, H. Rossi, Analytic functions of several complex variables, Prentice
Hall, Englewood Cliffs, N.J., 1965
[Ha] R. Harthshorne, Algebraic Geometry, Springer, New York, 1977
[Hö] L. Hörmander, An introduction to complex analysis in several variables, North
Holland, Amsterdam, 1973
[L] J. Leiterer, Banach coherent analytic Fréchet sheaves, Math. Nachr., 85 (1978),
91–109
[LP] L. Lempert, I. Patyi, Analytic sheaves in Banach spaces, Ann. Sci. École
Norm. Sup. 40 2007, 453–486
[Ms] V. Masagutov, Homomorphisms of infinitely generated sheaves, to appear in Arkiv
för Mat., arxiv:0811.1978
[Mt] H. Matsumura, Commutative ring theory, Cambridge University Press, Cam-
bridge, 1986
[S] J.P. Serre, Faisceaux algébriques cohérents, Ann. Math. 61 (1955), 197–278

László Lempert
Department of Mathematics
Purdue University
West Lafayette, IN 47907, USA
e-mail: [email protected]
Complex Analysis
Trends in Mathematics, 245–262

c 2010 Springer Basel AG

Characteristic Classes of the Boundary

of a Complex b-manifold
Gerardo A. Mendoza
Dedicated to Linda P. Rothschild

Abstract. We prove a classification theorem by cohomology classes for com-

pact Riemannian manifolds with a one-parameter group of isometries with-
out fixed points generalizing the classification of line bundles (more precisely,
their circle bundles) over compact manifolds by their first Chern class. We also
prove a classification theorem generalizing that of holomorphic line bundles
over compact complex manifold by the Picard group of the base for a subfam-
ily of manifolds with additional structure resembling that of circle bundles of
such holomorphic line bundles.
Mathematics Subject Classification (2000). Primary 32Q99; Secondary 58J10,
32V05.
Keywords. Complex manifolds, cohomology, characteristic class, Picard group,
b-tangent bundle.

1. Introduction
This work presents classification theorems for manifolds with additional structure
generalizing classical theorems concerning circle bundles of complex line bundles
over compact manifolds, both in the C ∞ and holomorphic categories.
In the next paragraphs we will briefly discuss the classical situations from the
perspective of this paper. Section 2 gives a context and serves as motivation for
the work. Sections 3 and 4 concern the classification theorems themselves. Finally,
Section 5 is devoted to an analysis of some aspects of the conditions in Section 4
that generalize the notion of holomorphic function.

The final version of this paper was written while the author was visiting the Department of
Mathematics at the University of São Paulo in São Carlos, Brazil, financed by a grant from
FAPESP. He gratefully acknowledges their hospitality.
246 G.A. Mendoza

It is well known that the isomorphism classes of smooth complex line bundles
over, say, a smooth compact manifold B are, in a natural way, in one to one corre-
spondence with the elements of the first cohomology group of B with coefficients in
the sheaf E ∗ of germs of smooth nonvanishing complex-valued functions on B, and
through the first Chern class, with the elements of the second cohomology group
of B with integral coefficients. In somewhat more generality, the construction of
the correspondence goes as follows. Fix a line bundle πE : E → B. If {Va }a∈A is a
sufficiently fine open cover of B, then for every line bundle πE  : E  → B there is a
−1
family of isomorphisms ha : EV a → EVa (where for instance EVa = πE (Va )) such
−1
that πE ◦ ha = πE  . The maps hab = ha ◦ hb are isomorphisms of EVa ∩Vb onto
itself such that πE ◦ hab = πE , therefore given by multiplication by nonvanishing
functions fab . These functions define a Čech 1-cocycle giving an element of the first
cohomology group of B with coefficients in E ∗ . This correspondence is a bijection
which in the case where E is the trivial line bundle is the one alluded to above.
Once the correspondence with the elements of the first cohomology group is
established, the first Chern class map arises through composition with the con-
necting homomorphism H 1 (B, E ∗ ) → H 2 (B, Z) in the long exact sequence in co-
homology associated with the short exact sequence
exp 2πi
0 → Z → E −−−−→ E ∗ → 0, (1.1)
in which Z is the sheaf of germs of locally constant Z-valued functions and E is the
sheaf of germs of smooth functions on B. The connecting map is an isomorphism
because E is a fine sheaf. Under this map, the class associated to the line bundle
E  as described in the previous paragraph goes to c1 (E  ) − c1 (E), the difference
of the standard first Chern classes of E  and E.
The version of the above classification generalized in Section 3 goes as follows.
Since the isomorphism classes of line bundles over B are in one to one correspon-
dence with the isomorphism classes of principal S 1 -bundles over B, the above
analysis also applies to such bundles. Fix a principal S 1 -bundle πN : N → B and
consider, for any other principal πN  : S 1 → N  and suitable open cover {Va }a∈A
of B, a family of equivariant diffeomorphisms ha : Ua → Ua (where Ua = π −1 (Va ))
such that πN ◦ ha = πN  |Ua . Then hab = ha ◦ h−1 b |Ua is an equivariant diffeo-
morphism from Ua ∩ Ub to itself such that πN ◦ hab = πN . From the collection
{hab } one can of course construct an isomorphic copy of N  . The family of germs
−1
of equivariant diffeomorphisms h : U → U (U = πN (V ), V ⊂ B open) such
that πN ◦ h = πN |U forms an abelian sheaf I (N ) over B. Elements of I ∞ (N )
∞

are locally represented by multiplication by functions e2πif , where f is a smooth

real-valued function on an open set of the base. This leads to an exact sequence
exp 2πi
0 → Z → C ∞ (B, R) −−−−→ I ∞ (N ) → 0, (1.2)
∞
similar to (1.1), in which C (B, R) is the sheaf of germs of smooth real-valued
functions on B. Here R is appears as the Lie algebra of S 1 . Since C ∞ (B, R) is
fine, again one has an isomorphism H 1 (B, I ∞ (N )) → H 2 (B, Z). In Section 3, the
S 1 -action is replaced by an R-action, the one-parameter group of diffeomorphisms
 Characteristic Classes of the Boundary of a Complex b-manifold 247

generated by a nonvanishing vector field T admitting a T -invariant Riemannian

metric. The orbits of the group need not be compact. The condition πN ◦h = πN |U
is replaced by the condition that for every p ∈ U , h(p) belongs to the closure of
the orbit of p, R is replaced by the Lie algebra g of a certain torus G, and Z by
the kernel of the exponential map exp : g → G.
Suppose now that B is a compact complex manifold and π : N → B is the
circle bundle of a holomorphic line bundle E → B. Implicit here is that N is the
set of unit vectors of E with respect to some Hermitian metric. In addition to the
infinitesimal generator T of the canonical S 1 -action on N one has the subbundle
V ⊂ CT N of the complexification of the tangent bundle of N whose elements are
precisely those whose image by π∗ lies in T 0,1 B. This is an involutive subbundle
having T as a section. The unit ball bundle of E has N as boundary, so N ,
as a hypersurface in the complex manifold E, is naturally a CR manifold. Its CR
structure K (as vectors of type (0, 1)) is also a subbundle of V, and in fact V = K ⊕
spanC T . The unique real one-form θ determined by the condition that it vanishes
on K and satisfies θ, T
= 1 is related to the complex and Hermitian structures
of E by the fact that iθ is the connection form of the Hermitian holomorphic
∗
connection of E. The restriction β of −iθ to V is of course a smooth section of V .
∗
62 V ∗is involutive, there is a differential operator from sections of V to sections
Since
of V . This operator, analogous but not equal, to the ∂ operator (or the ∂ b
operator), is denoted D and forms part of an elliptic complex. The form β is D-
closed: Dβ = 0. This condition reflects the fact that θ corresponds to a holomorphic
connection, so its curvature has vanishing (0, 2) component. Conversely, if N → B
is a circle bundle over a complex manifold, if V is defined as above, and if β is
∗
a smooth section of V such that β, T
= −i, then N is the circle bundle of a
holomorphic line bundle and β arises from the Hermitian holomorphic connection
as described. The equivariant diffeomorphisms hab : Ua ∩ Ub → Ua ∩ Ub arising
from a system of holomorphic transition functions satisfy dhab V = V.
In Section 4 we prove a classification result for general compact manifolds
together with a nonvanishing real vector field T admitting a T -invariant Riemann-
ian metric and an involutive subbundle V ⊂ CT N such that V ∩ V = spanC T and
∗
smooth sections β of V such that β, T
= −i and Dβ = 0. Again, the orbits
of T need not be compact. The objects T , V, β arise naturally on the boundary
of complex b-manifolds. These manifolds, and how the structure on the boundary
arises, are described in the next section.

2. Complex b-manifolds
Let M be a smooth manifold with boundary. The b-tangent bundle of M (Melrose
[2, 3]) is a smooth vector bundle bT M → M together with a smooth vector bundle
homomorphism
ev : bT M → T M
248 G.A. Mendoza

covering the identity such that the induced map

ev∗ : C ∞ (M; bT M) → C ∞ (M; T M)
is a C ∞ (M; R)-module isomorphism onto the submodule Ctan ∞
(M; T M) of vector
fields on M which are tangential to the boundary of M. The homomorphism ev
is an isomorphism over the interior of M, and its restriction to the boundary,
ev∂M : bT∂M M → T ∂M (2.1)
is surjective. Its kernel, a fortiori a rank-one bundle, is spanned by a canonical
section denoted r∂r ; r refers to any smooth defining function for ∂M in M, by
convention positive in the interior of M.
∞
Since Ctan (M, T M) is closed under Lie brackets, there is an induced Lie
∞
bracket on C (M; bT M), as well as on the space of smooth sections of the com-
plexification CbT M of bT M. So the notion of involutivity of a subbundle of CbT M
is well defined.
Definition 2.2. A complex b-structure on M is an involutive subbundle
b 0,1
T M ⊂ CbT M
such that
b 1,0
T M + bT 0,1 M = CbT M (2.3)
and
b 1,0
T M ∩ bT 0,1M = 0 (2.4)
with T M =
b 1,0 bT 0,1 M.
A complex b-manifold is a manifold with boundary to-
gether with a complex b-structure.
Thus a complex b-manifold is a complex manifold in the b-category. By the
Newlander-Nirenberg Theorem [8], the interior of M is a complex manifold.
Complex b-structures, more generally, CR b-structures, were introduced in
[4], some aspects of the boundary structure determined by a complex b-structure
were analyzed in [5], and families of examples were presented in [6]. An in-depth
study will appear in [7]. In particular, we showed in [5] that a complex b-structure
on M determines the following on each component N of the boundary of M:
(i) An involutive subbundle V ⊂ CT N with the property that V + V = CT N .
(ii) A real vector field T on N such that V ∩ V = spanC {T }.
Since V is involutive, there is a natural complex
6 q ∗ Dq 6q+1 ∗
· · · → C ∞ (N ; V ) −→ C ∞ (N ; V ) → ··· .
∗ ∗
(iii) A family β ⊂ C ∞ (N ; V ) of D-closed sections of V such that if β ∈ β , then
β, T
= a − i, a : N → R smooth (2.5)
∗
and an element β  ∈ C ∞ (N ; V ) belongs to β if and only if there is u : N → R
is smooth such that β − β  = Du.
 Characteristic Classes of the Boundary of a Complex b-manifold 249

It should be noted that V is not a CR structure, but rather an elliptic struc-

ture because of (i) above (see Treves [11, 12] for the general definition of elliptic
structure). However, for each β ∈ β , ker β ⊂ V is a locally integrable (i.e., locally
realizable) CR structure of hypersurface type. Local integrability is a consequence
of a result of Nirenberg [9] which in this case gives that in a neighborhood of any
point of N there are coordinates x1 , . . . , x2n , t such that V is locally spanned by
the vector fields
∂xi + i∂xi+n , i = 1, . . . , n, ∂t .
The functions z j = xj + ixj+n , j = 1, . . . , n, are annihilated by the elements of V,
hence also by the elements of ker β for any β ∈ β .
The datum of a manifold N , a vector subbundle V ⊂ CT N , a real vector
∗
field T , and a D-closed element β ∈ C ∞ (N ; V ) satisfying (i)–(iii) permits the
construction of a complex b-manifold whose boundary structure is the given one,
namely, let
M = [0, ∞) × N
with the fiber of bT 0,1 M at (r, p) given by
0,1
T(r,p) M = {v + β, v
r∂r : v ∈ V p },
see [5, Proposition 2.6].
The principal aim of this paper is to present a classification theorem of such
boundary structures assuming that N is compact and that there is a T -invariant
Riemannian metric g on N . Under these hypotheses for a given component of ∂M
one can show, [7], that there is an element β ∈ β such that (2.5) is improved to
β, T
= a − i, a : N → R constant. (2.6)
The number a is characteristic of the way the boundary structure is related to the
complex b-structure. One then obtains a new structure by defining β̃ = −i(a −
i)−1 β, which of course has the property that β̃, T
= −i (that is,  β̃, T
= 0),
and β̃β as the class of β̃ modulo D(C ∞ (N ; R)). The classification theorem in
Section 4 concerns structures for which there is an element β ∈ β such that
β̃, T
= −i, and the result of [7] alluded to above guarantees no loss of gen-
erality. From the perspective of this work, which is just the classification result,
the origin of the structure and the fact that there is β such that  β, T
is constant
are immaterial.

3. Classification by relative Chern classes

Let F be the family of pairs (N , T ) such that

1. N is a compact connected manifold without boundary;
2. T is a globally defined nowhere vanishing real vector field on N ; and
3. there is a T -invariant Riemannian metric on N .
250 G.A. Mendoza

The class F contains all pairs (N , T ) such that N is the circle bundle of a
Hermitian line bundle over a compact connected manifold and T is the infinitesimal
generator of the canonical S 1 -action on N .
We denote by at the group of diffeomorphisms generated by T , and by Op
the orbit of T through p. If (N , T ), (N  , T  ) ∈ F and h is a smooth map from an
open set of N  to one of N such that h∗ T  = T , then h is called equivariant.
Definition 3.1. Two elements (N , T ), (N  , T  ) ∈ F are globally equivalent (g-
equivalent) if there is an equivariant diffeomorphism h : N  → N . They are locally
equivalent (l-equivalent) if there are open covers {Ua }a∈A of N and {Ua }a∈A of N 
by T , resp. T  -invariant open sets and equivariant diffeomorphisms ha : Ua → Ua
for each a ∈ A such that
ha h−1
b (p) ∈ O p for every a, b ∈ A and p ∈ Ua ∩ Ub . (3.2)
Example 3.3. If N and N  are the respective circle bundles of Hermitian line
bundles E → B and E  → B  , and (N , T ) and (N  , T  ) locally equivalent, then
the base spaces are diffeomorphic. If (N , T ) and (N  , T  ) are globally equivalent
and h : N → N  is an equivariant diffeomorphism, then h induces a unitary line
bundle isomorphism covering a diffeomorphism of the base spaces.
If (N , T ) ∈ F define the relation p ∼ p if and only if p ∈ O p . The fact that
there is a T -invariant metric implies that this is a relation of equivalence (the key
fact being the implication p ∈ Op =⇒ p ∈ Op ). Let BN be the quotient space.
Then BN , the base space of (N , T ), is a Hausdorff space. The following lemma is
immediate in view of (3.2).
Lemma 3.4. Let (N , T ), (N  , T  ) ∈ F be locally equivalent. Then BN and BN  are
homeomorphic.
Henceforth we fix an element (N , T ) ∈ F, denote BN by B and let π :
N → B be the quotient map. We aim first at giving a classification modulo global
equivalence of the set of elements of F locally equivalent to (N , T ), by the elements
of H 2 (B, Z ) where Z is the sheaf of germs of locally constant functions on B with
values in a certain free, finite rank abelian group z to be described in a moment.
This classification is similar to the classification of circle bundles (or complex line
bundles) over B by their first Chern class.
Let Homeo(N ) be the group of homeomorphisms N → N with the compact-
open topology. The structure group of (N , T ) is the closure, to be denoted G,
of the subgroup {at : t ∈ R} ⊂ Homeo(N ). It is clearly an abelian group. Fix a
T -invariant metric. Then we may view {at : t ∈ R} as a subgroup of the group
of isometries of N with respect to the metric, a compact Lie group. Its closure is
therefore compact.
Lemma 3.5. The structure group G is a compact abelian Lie group acting on N by
smooth diffeomorphisms, and is a subgroup of the group of isometries of N with
respect to any T -invariant metric.
 Characteristic Classes of the Boundary of a Complex b-manifold 251

We will denote the action of an element g ∈ G on N by Ag , and by A :

G × N → N the map A(g, p) = Ag p. Let g be the Lie algebra of G, let exp : g → G
be the exponential map, and let z ⊂ g be its kernel.
Definition 3.6. Let (N , T ) ∈ F, let B be its base space, and let π : N → B be the
projection map. If V ⊂ B, let NV = π −1 (V ).
1. For each open set V ⊂ B let I ∞ (NV ) be the set of smooth T -equivariant
diffeomorphisms h : NV → NV such that h(p) ∈ Op for all p ∈ NV . It is
easy to see that I ∞ (NV ) is an abelian group under composition. The family
{I ∞ (NV )} with the obvious restriction maps forms a presheaf over B giving
an abelian sheaf I ∞ (N ) → B.
2. If V ⊂ B is open, let C ∞ (V, g) be the space of smooth functions NV → g
which are constant on the orbits of T . These spaces also form an abelian
presheaf. We let C ∞ (B, g) → B be the corresponding sheaf.
3. Finally, let Z → B be the sheaf of locally constant z-valued functions on B.
There is a natural map Exp : C ∞ (V, g) → I ∞ (NV ), namely, if f ∈ C ∞ (V, g),
let Exp(f )(p) = Aexp f (p) p. Since f (at p) = f (p),
Exp(f )(at p) = Aexp f (p) at p = at Aexp f (p) p.
So Exp(f ) is an equivariant diffeomorphism (its inverse is Exp(−f )). Thus
Exp(f ) ∈ I ∞ (NV ). With Exp we get a sheaf homomorphism
Exp : C ∞ (B, g) → I ∞ (N ),
and with the inclusion z → g, the sheaf homomorphism
ι : Z → C ∞ (B, g).
Proposition 3.7. The sequence
Exp
→ C ∞ (B, g) −−→ I ∞ (N ) → 0
ι
0→Z − (3.8)
is exact. The sheaf C ∞ (B, g) is a fine sheaf, so the long exact sequence in coho-
mology gives an isomorphism H 1 (B, I ∞ (N )) → H 2 (B, Z ) ≈ H 2 (B, Zd ) where d
is the dimension of G.
The proofs of this proposition and the next theorem are given below.
Theorem 3.9. Let (N , T ) ∈ F. There is a natural one-to-one correspondence be-
tween the elements of H 2 (B, Z ) and the g-equivalence classes of elements of F
which are locally equivalent to N .
If (N  , T  ) is locally equivalent to (N , T ) we write c1 (N  , N ) for the element
of H (B, Z) corresponding to (N  , T  ). The element c1 (N  , N ) should be regarded
2

as a relative Chern class.

Example 3.10. Suppose that B is a smooth manifold and π : N → B is the circle
bundle of a Hermitian line bundle E → B. Let E  → B be another Hermitian line
bundle with circle bundle π : N  → B. Let {Va }a∈A be an open cover of B such
252 G.A. Mendoza

that for each a, E|Va is isomorphic to E  |Va with unitary isomorphisms ha covering
the identity map on Va . Let Ua = NVa = π −1 (Va ), Ua = NV a . The ha restrict to
equivariant diffeomorphisms ha : Ua → Ua . Let
hab = ha ◦ h−1
b : U a ∩ U b → U a ∩ Ub .
The hab determine the element c1 (N  , N ) ∈ H 2 (B, Z). It is not hard to show that
c1 (N  , N ) = c1 (E  ) − c1 (E).
The proofs of Proposition 3.7 and Theorem 3.9 require some preparation. The
following theorem yields explicit information that will be useful in the development
of the theory.
Theorem 3.11. Suppose there is a T -invariant Riemannian metric on N . Then
there is a smooth embedding F : N → CN with image contained in the sphere
S 2N −1 , none of whose component functions vanishes to infinite order at any point,
such that for some positive numbers τj , j = 1, . . . N ,

F∗ T = i τj (wj ∂wj − w j ∂wj ) (3.12)
j
1
in the coordinates w , . . . , w N
of CN .
The proof relies on an idea originally due to Bochner [1], expressed in this
case by judiciously choosing enough functions F̃  that are at the same time eigen-
functions of T and eigenfunctions of the Laplacian with respect to some fixed
T -invariant Riemannian metric on N , and then using them to construct the map
F . The details are given in [7].
Note that the manifold S 2N −1 together with the vector field T  on S 2N −1
given by the expression on the right in (3.12) is an example of a pair in the class F .
The standard metric on S 2N −1 is T -invariant. Observe in passing that the orbits
of T  need not be compact.
Recall that G is the closure of {at} in the compact-open topology of
Homeo(N ). Fix a map F : N → S 2N −1 having the properties stated in Theo-
rem 3.11 and let T  again be the vector field on S 2N −1 given by the expression on
the right in (3.12). Let at be the one-parameter group generated by T  . The set
G0 = closure of {(eiτ1 t , . . . , eiτN t ) ∈ S 1 × · · · × S 1 : t ∈ R} (3.13)
is the structure group of the pair (S 2N −1 , T  ). For any w ∈ S 2N −1 , the closure of
the orbit of w by T  is
{(ω 1 w1 , . . . , ω N wN ) : (ω 1 , . . . , ω N ) ∈ G0 }.
Let
W = {p : F  (p) = 0,  = 1, . . . , N }.
Then
W is open and dense in N , if {tν }∞
ν=1 is such that atν p con-
verges for some p ∈ W , then {atν }∞ν=1 converges in the C
∞
(3.14)
topology, and if p ∈ W and Ag p = p, then g is the identity.
 Characteristic Classes of the Boundary of a Complex b-manifold 253

Indeed, if atν p converges, then so does atν F (p) since F is equivariant. Since F  (p) =
0 for each , {eiτ tν }∞ 
ν=1 converges for each . It follows that atν : S
2N −1
→ S 2N −1
∞ −1 
converges in the C topology, and then so does atν = F ◦ atν ◦ F .
Now, if g ∈ G, then there is a sequence {tν }∞ ν=1 such that atν → g in the
compact-open topology. In particular, if p ∈ W , then atν p converges, so atν → g
in the C ∞ topology.
The elements g = lim atν ∈ G and lim(eiτ1 tν , . . . , eiτN tν ) = ω ∈ G0 are
related by F (Ag p) = Aω F (p). If g ∈ G has the property that there is p ∈ W such
that Ag p = p, then Aω F (p) = F (p), so ω = I and Ag is the identity map. Thus,
tautologically, g is the identity element of G.
Let Jp denote the isotropy subgroup of G at p. We now show that the set
Nreg = {p ∈ N : Jp is trivial}.
has the properties listed for W in (3.14). One of the virtues of Nreg is that it is
independent of the auxiliary map F .
Lemma 3.15. The set Nreg is open, dense, and G-invariant. If p ∈ Nreg and
{tν }∞
ν=1 is a sequence such that atν p converges, then atν converges in G in the
C ∞ topology. If g ∈ G and Ag p = p for some p ∈ Nreg , then g is the identity.
Proof. The last property is the definition of Nreg . The G-invariance of Nreg is clear.
Fix some T -invariant metric on N . Let p0 ∈ N be arbitrary. Let π : HOp → Op0
0
be the orthogonal bundle of T Op0 in T N with respect to the metric, let
BOp0 ,ε = {v ∈ HOp : |v| < ε}
0

and let
UOp = exp(BOp0 ,ε )
0 ,ε

where, as usual, exp maps a sufficiently small vector v ∈ T N to the image of 1 by

the geodesic through v. Since Op0 is compact, there is ε > 0 such that for B =
BOp0 ,ε the map exp |B : B → N is a diffeomorphism onto its image U = UOp ,ε .
0
Since U = {p ∈ N : dist(p, Op0 ) < ε}, U is T -invariant. Let ρ = π ◦ (exp |B )−1 .
If g ∈ G, then Ag is an isometry, so ρAg = Ag ρ in U . It follows that if g ∈ Jp
with p ∈ U , ρ(p) = p0 , then g ∈ Jp0 . Consequently, if p0 ∈ Nreg , then U ⊂ Nreg .
Thus Nreg is open. Note in passing that if p0 ∈ Nreg , then ρ|Op : Op → O p0 is a
diffeomorphism.
The set W used in the proof of Lemma 3.5 is a subset of Nreg . Since W is
dense, so is Nreg .
Finally, let p0 ∈ Nreg and suppose that {tν }∞
ν=1 is a sequence such that atν p0
converges. Since W is dense in N , there is p ∈ UOp ,ε ∩ W . Since ρ|Op : Op → Op0
0
is a diffeomorphism that commutes with each at , atν p converges. Therefore atν
converges, by (3.14). 
Let Breg = π(Nreg ). One can show that Breg is a smooth manifold and that
Nreg → Breg is a principal G-bundle.
254 G.A. Mendoza

A variant of the proof of Lemma 3.5 gives:

Lemma 3.16. Let (N  , T  ) ∈ F be locally equivalent to (N , T ), and let GN , GN 
be the respective structure groups. Then there is a group isomorphism
ΨN ,N  : GN  → GN
such that for any open T -invariant set U  ⊂ N  and T -invariant set U ⊂ N , if


h : U  → U is an equivariant diffeomorphism, then

h ◦ Ag = AΨN ,N  g ◦ h.
Proof. Pick some equivariant diffeomorphism h : U  → U . If g  ∈ GN  , then there
is a sequence {tν } such that atν → g in the C ∞ topology. If
p ∈ U  ∩ Nreg

∩ h−1 (U ∩ Nreg ),
then the continuity and equivariance of h and the convergence of atν p give the
convergence of atν h(p), hence by Lemma 3.15 the convergence of atν to some
element ΨN ,N  g  . It is easy to verify that Ψ is a group isomorphism (the inverse
being ΨN  ,N ) independent of the auxiliary equivariant diffeomorphism used to
define it. 
The following lemma is the key component in the proof of the surjectivity of
Exp in the sequence (3.8)
Lemma 3.17. Let U ⊂ Rn be open, let f , g : U → C be smooth and such that
|f | = |g| and f , g are not flat at any point of U . Then there is a unique smooth
function ω : U → S 1 ⊂ C such that f = ωg.
Proof. On the open set V = {x ∈ U : f (x) = 0 and g(x) = 0} we have that
f /g is a smooth function with values in S 1 . So, since V is dense, ω is unique if it
exists. Let x0 ∈ U be arbitrary. Since neither of the functions f or g is flat at x0 ,
the Malgrange Preparation Theorem gives that there are coordinates (y1 , . . . , yn )
centered at x0 such that with y  = (y1 , . . . , yn−1 ) we have


k−1 k −1

 
qf = ynk + r (y )yn , qg= ynk + r (y  )yn ,
=0 =0

near x0 with smooth functions q, q  , r , r , and q(x0 ), q  (x0 ) = 0. The condition
k−1  k −1
|f | = |g| gives that the polynomials ynk + =0 r (y  )yn , ynk + =0 r (y  )yn have
the same roots for each fixed y  . Since f /g and g/f are both bounded on V and V
is dense, these roots appear with the same multiplicity in both polynomials, that
is, they are the same polynomial. Consequently qf = q  g and thus f = ωg near x0
with ω = q/q  smooth. 
Surjectivity of Exp in (3.8) is an immediate consequence of:
Lemma 3.18. Let V ⊂ B be open and let h ∈ I ∞ (NV ). For every x0 ∈ V there are
a neighborhood V  ⊂ V of x0 and f ∈ C ∞ (V  , g) such that h = Exp(f ) in NV  .
 Characteristic Classes of the Boundary of a Complex b-manifold 255

Proof. Let F : N → S 2N −1 be a map as in the proof of Theorem 3.11, let T 

be the vector field (3.12) and let at denote the associated one parameter group it
generates; recall that no component of F  vanishes to infinite order at any point
of N . Suppose h ∈ I ∞ (NV ). If p ∈ NV , then F (h(p)) is an element in the closure
of the orbit of F (p), so there is ω(p) ∈ G0 , the structure group of (S 2N −1 , T  ) (see
(3.13)) such that F (h(p)) = Aω(p) F (p); ω(p) need not be unique. Componentwise
this gives F  ◦ h = ω  F  . In particular |F  ◦ h| = |F  |. Since neither F  nor
F  ◦ h vanish to infinite order at any point, Lemma 3.17 gives that ω  is smooth.
Let g(p) ∈ G be the element that corresponds to ω(p) via the isomorphism of
Lemma 3.16. So g : NV → G is smooth and h(p) = Ag(p) p. Furthermore, g is
constant on the orbits of at . Indeed, on the one hand
h(at p) = Ag(at p) at p = atAg(at p) p
and on the other
h(at p) = at h(p) = at Ag(p) p,
so Ag(at p) p = Ag(p) p for any p ∈ NV . Then g(at p) = g(p) if p ∈ NV ∩ Nreg , so
g ◦ at = g by continuity of g and density of Nreg . Given p0 ∈ NV , let ρ : U → Op0
be a tubular neighborhood map as in the proof of Lemma 3.15, contained in NV .
Consider now the problem of lifting g : U → G to a map f : U → g,
g
exp

U /G
g

Since U is contractible to O p0 and g is constant on Op0 , g maps the fundamental

group of U to the identity element of the fundamental group of G. So there is
f : U → g such that exp ◦f = g|U . Let V  = π(U ), so NV  = U . If p ∈ U , then
exp(f (at p)) = g(at p) = g(p) = exp(f (p)) gives that exp(f (at p) − f (p)) is the
identity element of G for any t ∈ R. So t → f (at p) − f (p) is a continuous z-valued
function that vanishes at t = 0. Hence f (at p) = f (p) for every t ∈ R and p ∈ U .
Consequently f ∈ C ∞ (V, g) and h = Exp(f ) on NV  . 

The fact that C ∞ (B, g) → B is a fine sheaf will follow from existence of
partitions of unity in C ∞ (B, R) (recall that this is the space of smooth functions
on N that are constant on orbits of T ).
Let μ denote the normalized Haar measure of G.

Lemma 3.19. Suppose V ⊂ B is open. For f ∈ C ∞ (NV ) define Av f : NV → C by


Av f (p) = f (g · p)dμ(g).

Then Av f is smooth. If f ∈ Cc∞ (NV ), then Av f ∈ Cc∞ (NV ).

256 G.A. Mendoza

Proof. Let ρ : G × N → N be the canonical projection. If g ∈ C ∞ (G × NV ) then

ρ∗ (gμ) is smooth. If f ∈ C ∞ (NV ), then A∗ f ∈ C ∞ (G × NV ), so ρ∗ ((A∗ f )μ) is
smooth. But ρ∗ ((A∗ f )μ) = Av f . The proof of the last statement is also elementary.

As a consequence we have existence of smooth locally finite partitions of unity
for B.
Corollary 3.20. Let {Va } be an open cover of B. Then there is a family {χγ }γ∈Γ
of nonnegative functions χγ ∈ C ∞ (B) such that for every γ there is a(γ) such
that supp χγ ⊂ NVa(γ) , for every K  B the set {γ : supp χγ ∩ NK } is finite, and

γ χγ = 1.

A partition of unity as in the corollary gives a partition of unity for C ∞ (B, g),
so the latter is a fine sheaf.
Proof of Proposition 3.7. Let V ⊂ B be open. The map ι sends a locally constant
function ν : V → z to the element ν ◦ π of C ∞ (V, g), so ι is injective.
To see exactness at C ∞ (B; g), suppose ν : V → z is locally constant. Then
ν ◦ π ∈ C ∞ (V, g) and exp(ν ◦ π) is the identity. So ι(ν ◦ π) ∈ ker Exp. Suppose now
that f ∈ C ∞ (V, g) and Exp(f ) ∈ I ∞ (NV ) is the identity: Aexp f (p) p = p for all
p ∈ NV , in particular if p ∈ NV ∩ Nreg . By Lemma 3.15, exp f (p) is the identity
element of G when p ∈ NV ∩ Nreg . So f |Nreg ∩NV has values in z. Since f is smooth
and Nreg is dense, f has values in z, and since f is constant on fibers, there is a
locally constant function ν : V → z such that f = ν ◦ π.
Finally, Lemma 3.18 gives that Exp in (3.8) is surjective. 
Proof of Theorem 3.9. Let h ∈ H 1 (I ∞ (N )). We will, essentially by following the
proof of the corresponding statement for line bundles, show that there is a well-
defined g-equivalence class of elements associated with h. To construct a represen-
tative of this class, choose an open cover {Va }a∈A of B such that h is represented
by a cocycle {hab ∈ I ∞ (NVa ∩Vb )}. Thus haa is the identity map, hab = h−1ba , and
hab hbc hca = I. We then get a manifold
;
N = NVa /∼
a∈A
<
as for vector bundles, where ∼ is the relation of equivalence on a∈A NVa for
which p ∈ NVa and q ∈ NVb are equivalent if and only p, q ∈ NVa ∩Vb and p = hab q.
The set of equivalence classes of elements of NVa is an open set NV a ⊂ N  , and if
p ∈ NV a then there is a unique p ∈ NVa such that the class of p is p . This gives a
map ha : NV a → NVa for each a, a diffeomorphism. By the definition of the relation
of equivalence, if Va ∩ Vb = ∅ then ha h−1 b = hab on NVa ∩Vb . In particular (3.2) is
satisfied. Let Ta = dh−1  
a T . Then Tb = Ta on NVa ∩NVb , since dhab T = T . Thus N

 
admits a global nowhere vanishing vector field T , and by definition dha at = at ha
for each a ∈ A, i.e., ha is equivariant. In particular, the orbits of T  are compact.
By construction, (N  , T  ) is locally equivalent to (N , T ), and one can construct a
 Characteristic Classes of the Boundary of a Complex b-manifold 257

T  -invariant metric on N  using one such for N , the maps ha , and a partition of
unity as in Corollary 3.20. So (N  , T  ) ∈ F is locally equivalent to (N , T ).
If {h̃ab ∈ I ∞ (NVa ∩Vb )} also represents h, let (Ñ  , T̃  ) be the element of F
constructed as above with the h̃ab . We will show that N  is globally equivalent
to Ñ  . Let h̃a : ÑV a → NVa be maps such that h̃ab = h̃a h̃−1 b . Because {hab } and
{h̃ab } represent the same element h, there is, perhaps after passing to a refinement
of the cover {Va }, an element ga ∈ I ∞ (NVa ) for each a ∈ A such that
h̃ab = hab gagb−1 . (3.21)
Passing to a further refinement of the cover {Va } and using Lemma 3.18 we may
assume that for each a there is fa ∈ C ∞ (Va , g) such that ga = Exp fa . Let Ψ :
G → G be the isomorphism of Lemma 3.16. We have
ga ha (p ) = Aexp fa (ha (p )) ha (p ) = ha (AΨ−1 exp fa (ha (p )) p ), p ∈ NVa ,
in other words, ga ha = ha ga with ga (p ) = Exp(dΨ−1 (fa ◦ ha )). From (3.21) we
get
h̃a h̃−1 −1 −1   −1
b = ga ha hb gb = ha ga (hb gb )
so
h̃−1  −1 
b hb gb = h̃a ha ga .
That is, the maps h̃−1   
a ha ga : NVa → ÑVa , which are equivariant diffeomorphisms,
give a global map N  → Ñ  . So there is a well-defined g-equivalence class of
elements associated with h.
Conversely, given an element (N  , T  ) ∈ F which is locally equivalent to
(N , T ) by way of maps ha : Ua → Ua as in Definition 3.1 we get a cocy-
cle hab = ha h−1 ∞
b , hab ∈ I (NVa ∩Vb ). It is not hard to verify that the element
∞
h ∈ H (I (N )) constructed from this cocycle as above is globally equivalent to
1

(N  , T  ). 

4. Classification by a Picard group

We now present an analogue of the classification of holomorphic line bundles over
compact complex manifolds by the Picard group.
Let Fell be the set of triples (N , T , V) such that (N , T ) ∈ F and V ⊂ CT N
is an elliptic structure with V ∩ V = spanC T that admits a D-closed element
β ∈ C ∞ (N , V ∗ ) such that iT β = −i.
Definition 4.1. Two elements (N , T , V), (N  , T  , V  ) ∈ Fell will be said to be glob-
ally ell-equivalent if there is an equivariant diffeomorphism h : N  → N such
that h∗ V  = V. They are locally ell-equivalent if there are open covers {Ua}a∈A
of N and {Ua }a∈A of N  by T , resp. T  -invariant open sets and equivariant dif-
feomorphisms ha : Ua → Ua for each a ∈ A such that ha h−1 b satisfies (3.2) and
preserves V.
258 G.A. Mendoza

Fix (N , T , V) ∈ Fell . Let V ⊂ B be open. An element h ∈ I ∞ (NV ) preserves

V if h∗ (V NV ) ⊂ V NV .
1. For each open V ⊂ B let I V (NV ) be the subgroup of I ∞ (NV ) whose elements
h preserve V. The associated sheaf is I V (N ).
2. For each open V ⊂ B let C V (V, g) be the subspace of C ∞ (V, g) whose image
by Exp is I V (NV ). The associated sheaf is denoted C V (B, g).
Clearly, the sequence
0 → Z → C V (B, g) → I V (N ) → 0
is exact.
Theorem 4.2. Let (N , T , V) ∈ Fell . There is a natural one-to-one correspondence
between the elements of H 1 (B, I V (N ))) and the global ell-equivalence classes of
elements of Fell which are locally ell-equivalent to N .
Proof. Let h ∈ H 1 (I (N )), let {Va }a∈A be an open cover of B, and suppose that
the Čech cocycle {hab ∈ I V (NVa ∩Vb )} represents h. Let (N  , T  ) ∈ F be the
element constructed in the proof of Theorem 3.9 using {hab }, let ha : NV a → NVa
be the corresponding maps. Since ha h−1 
b preserves V for each a, b, N inherits an
  −1   
elliptic structure V , V = ha ∗ V, such that V ∩ V = spanC T .
The kernel K ⊂ V of β is a CR structure of CR codimension 1. Let θ be
the real one-form that vanishes on K ⊕ K and satisfies θ, T
= 1. Since Dβ = 0
and iT β is constant, iT dθ = 0. Let {χa}a∈A ⊂ C ∞ (B) be a locally finite partition
of unity subordinate to the cover {Va }, as provided by Corollary 3.20. Since only
finitely many of the χa are nonzero (because N is compact) there is no loss of
generality if we assume that already the index set for the partitionis the same as
that for the cover and that supp χa ⊂ NVa for each a. Let θ = χa h∗a θ. Then
   
ιT  θ = 1 and ιT  dθ = 0. It follows that the restriction iβ of θ to V is D-closed
which of course satisfies iiT  β  = 1. Thus (N  , T  , V  ) is locally ell-equivalent to
(N , T , V).
It is not hard to see that if (Ñ  , T̃  , Ṽ  ) ∈ Fell is another element locally
ell-equivalent to (N , T , V) with equivariant maps h̃a : ÑVa → NVa such that
hab = h̃a h̃−1      
b , then (Ñ , T̃ , Ṽ ) is globally ell-equivalent to (N , T , V ), nor that
every global ell-equivalence class of elements of Fell which are locally ell-equivalent
to N determines a unique element of H 1 (B, I V (N ))). 

5. The bundle V and the orbits of the structure group G

In order to analyze the condition that an element h ∈ I ∞ (NV ), V ⊂ B open,
preserves V we need to analyze the relation between the structure bundle V and
∗
the orbits of G. Let β be some D-closed section of V with ιT β = −i and let
K ⊂ V be its kernel. As in the proof of Theorem 4.2, let θ be the real one-form
that vanishes on K ⊕ K and satisfies θ, T
= 1. The complexification of the kernel
 Characteristic Classes of the Boundary of a Complex b-manifold 259

of θ, being the direct sum of subbundles K and K, determines an almost complex

structure J : ker θ → ker θ, v + iJv ∈ K if v ∈ ker θ.
Let g ∈ G and let {tν } be a sequence such that atν → g. If v ∈ V p , then
datν v ∈ V atν p , so dAg v ∈ V Ag p . Thus Exp(f ) preserves V whenever f : N → g is
constant.
The one-parameter subgroup t → at of G gives an element T̂ ∈ g. Let
Ŷ1 , . . . , Ŷd be a basis of g with Ŷd = T̂ with dual basis θ̂1 , . . . , θ̂d . Via the action
of G the Ŷj give smooth globally defined vector fields Y1 , . . . , Yd on N , Yd = T ,
#
d ##
Yj (p) = Exp(sŶj )(p).
ds #s=0
The vector fields Yj are tangent to the orbits of G and pointwise linearly indepen-
dent on Nreg but not necessarily so on Nsing = N \Nreg . Clearly [Yj , Yk ] = 0 for
all j, k. Let
Xk = Yk − θ, Yk
T , k = 1, . . . , d − 1,
so that Xk is a section of ker θ. Then JXk is defined and is another smooth section
of ker θ. Since the functions θ, Yk
are constant on orbits of G,
[Yj , Xk ] = 0, j = 1, . . . , d, k = 1, . . . , d − 1,
[Xj , Xk ] = 0, j, k = 1, . . . , d − 1.
The vector fields Xk + iJXk are sections of K and so are the sections
Exp(sŶj )∗ (Xk + iJXk ) = Exp(sŶj )∗ Xk + i Exp(sŶj )∗ JXk .
since Exp(sŶj ) preserves V for each s ∈ R. Thus J Exp(sŶj )∗ Xk = Exp(sŶj )∗ JXk ,
so
Exp(−sŶj )∗ J Exp(sŶj )∗ Xk = JXk .
It follows that [Yj , JXk ] = 0 for all k, j, and thus
[Xj , JXk ] = [Yj − θ, Yj
T , JXk ] = JXk θ, Yj
T .
The brackets
[Xj + iJXj , Xk + iJXk ] = −[JXj , JXk ] + i([Xj , JXk ] + [JXj , Xk ])
are sections of K, in particular of ker θ, so θ, [Xj , JXk ] + [JXj , Xk ]
= 0. But
[Xj , JXk ] + [JXj , Xk ] = (JXk θ, Yj
− JXj θ, Yk
)T ,
hence JXk θ, Yj
−JXj θ, Yk
= 0, that is, [Xj +iJXj , Xk +iJXk ] = −[JXj , JXk ].
Therefore [JXj , JXk ] = 0. Thus the vector fields
Y1 , . . . , Yd , JX1 , . . . , JXd−1 (5.1)
commute with each other.
For a real vector field W on N let exp(sW ) denote the one-parameter group
it determines. Define
F (t, s)(p) = exp(t1 Y1 ) ◦ · · · ◦ exp(td Yd ) ◦ exp(s1 JX1 ) ◦ · · · ◦ exp(sd−1 JXd−1 )(p)
260 G.A. Mendoza

for (t, s) ∈ Rd × Rd−1 . Then

Suss(p) = {F (t, s)(p) : (t, s) ∈ Rd × Rd−1 }
is the Sussmann orbit through p of the family (5.1), see Sussmann [10]; these orbits
are immersed submanifolds of N .
For any p ∈ N the intersection W Op = K ∩ CT Op is a CR structure on
O (which could be just the 0 bundle), pointwise spanned by linear combinations
pd−1 k d−1 k
k=1 b (Xk + iJXk ) such that k=1 b JXk is again tangent to O p :

d−1
d
bk JXk = a j Yj
k=1 j=1

j
with suitable a . Because the vector fields (5.1) commute, if this relation holds
at p, then it holds at every point of Suss(p) with constant aj , bk . This gives
F (t, s)∗ W Op = W OF (t,s)(p) for each s, t. Note in passing that WOp ⊕ W Op is
involutive, so
W Op = 0 if K is nondegenerate at p. (5.2)
Let dp = dim O p and let np be the CR codimension of W Op . Then the dimen-
sion of the span of the vector fields (5.1) at p is dp + np (so dim Suss(p) = dp + np ).
It also follows from the fact that the vector fields (5.1) commute that dim OF (t,s)(p)
is independent of t and s. Also K ∩ Suss(p) is a CR structure, necessarily of CR
codimension 1, spanned by the vector fields Xj + iJXj .
Given any p ∈ N we may choose the basis Ŷ1 , . . . , Ŷn−1 , Ŷd of g (Ŷd = T̂ ) so
that with mp = rank W Op ,
Xj = Yj , j = 1, . . . , d, JXj = Xj+mp , j = 1, . . . , mp
span{JXk : k = 2mp + 1, . . . , 2mp + np } ∩ T Op = 0, (5.3)
Yk = 0, k = 2mp + np + 1, . . . , d − 1.

holds along Op .
Suppose now that V ⊂ B is open and h = Exp(f ) with f ∈ C ∞ (V, g). Fix
some p0 ∈ NV . If v ∈ CTp0 N , then, with fp0 denoting the constant function
p → f (p0 )

d
h∗ v = Exp(fp0 )∗ v + v θ̂j , f
Yj
j=1
(5.4)

d
d−1
= Exp(fp0 )∗ v + θ, Yj
v θ̂ , f
T +
j
v θ̂ , f
Xj .
j

j=1 j=1

If d = 1, then
h∗ v = Exp(fp0 )∗ v + v θ̂1 , f
T for all v ∈ V p0 ,
 Characteristic Classes of the Boundary of a Complex b-manifold 261

so any h ∈ I ∞ (NV ) preserves V. Assume then that d > 1. The vector h∗ v in (5.4)
belongs to V for every v ∈ V p0 if and only if
1
d−1
v θ̂ j , f
(Xj − iJXj ),
2 j=1

the component of h∗ v in K according to the decomposition K⊕V of CT N , vanishes

for every such v. Using (5.4) with v = Xk gives

d
d−1
h∗ Xk = Exp(fp0 )∗ Xk + θ, Yj
Xk θ̂j , f
T + Xk θ̂j , f
Xj ,
j=1 j=1

d
d−1
h∗ JXk = Exp(fp0 )∗ JXk + θ, Yj
JXk θ̂j , f
T + JXk θ̂j , f
Xj .
j=1 j=1

Since Xk θ̂j , f
= 0 for all j and [Y , Xk ] = 0 for all , the first formula reduces to
h X = Xk . If h preserves V, this gives JXk = h∗ Exp(h)JXk , so we must have
∗ d k
j=1 θ, Yj
JXk θ̂ , f
= 0 and JXk θ̂ , f
= 0 for j = 1, . . . , d − 1, so θ̂ , f
is
j j j

constant on the Sussmann orbits of the family (5.1).

Let Zj = 12 (Xj − iJXj ). Assuming that the basis Ŷj satisfies (5.3) along Op0 ,
the condition that h preserves V is that
mp0 2mp0 +dp0

v( θ̂ , f
+ i θ̂
j j+mp0
, f
)Zj + v θ̂j , f
Zj = 0
j=1 j=2mp0 +1

along O p0 . This gives

D θ̂j , f
= −D θ̂j+mp0 , f
, j = 1, . . . , mp0 , D θ̂j , f
= 0, j = 2mp0 + 1, . . . np0
at p0 .
If this is the case, then

d
1
d−1
h∗ v = Exp(fp0 )∗ v + θ̂, Yj
v θ̂j , f
T + v θ̂j , f
(Xj + iJXj )
j=1
2 j=1

whenever v ∈ V p0 .
If W Op = 0 (for instance if Kp0 is nondegenerate, see (5.2)), then the con-
0

dition that h = Exp(f ) preserves V is that D θ̂j , f

= 0, j = 1, . . . , np0 . So if K is
nondegenerate on a dense subset of N , then D θ̂j , f
= 0, j = 1, . . . , d − 1 in NV .
Since the functions θ̂j , f
are real valued, they must be locally constant.
262 G.A. Mendoza

References
[1] Bochner, S., Analytic mapping of compact Riemann spaces into Euclidean space,
Duke Math. J. 3 (1937), 339–354
[2] Melrose, R.B., Transformation of boundary problems Acta Math. 147 (1981), 149–
236.
[3] , The Atiyah-Patodi-Singer index theorem, Research Notes in Mathematics,
A.K. Peters, Ltd., Wellesley, MA, 1993.
[4] Mendoza, G., Strictly pseudoconvex b-CR manifolds, Comm. Partial Differential
Equations 29 (2004) 1437–1503.
[5] , Boundary structure and cohomology of b-complex manifolds. In “Partial
Differential Equations and Inverse Problems”, C. Conca et al., eds., Contemp. Math.,
vol. 362 (2004), 303–320.
[6] Anisotropic blow-up and compactification, In “Recent Progress on some Prob-
lems in Several Complex Variables and Partial Differential Equations”, S. Berhanu
et al., eds., Contemp. Math., vol. 400 (2006) 173–187.
[7] The indicial complex of a b-complex manifold, in preparation.
[8] Newlander, A., Nirenberg, L., Complex analytic coordinates in almost complex man-
ifolds, Ann. of Math. 65 (1957), 391–404.
[9] L. Nirenberg, A complex Frobenius theorem, Seminar on analytic functions I, Prince-
ton, (1957) 172–189.
[10] Sussmann, H., Orbits of families of vector fields and integrability of distributions
Trans. Amer. Math. Soc. 180 (1973), 171–188.
[11] Treves, F., Hypoanalytic structures, in “Microlocal analysis”, M.S. Baouendi et al.,
eds., Contemp. Math., vol. 27, (1984), 23–44.
[12] Treves, F., Hypo-analytic structures. Local theory, Princeton Mathematical Series,
40, Princeton University Press, Princeton, NJ, 1992.

Gerardo A. Mendoza
Department of Mathematics
Temple University
Philadelphia, PA 19122, USA
e-mail: [email protected]
Complex Analysis
Trends in Mathematics, 263–278

c 2010 Springer Basel AG

Solvability of Planar Complex Vector Fields

with Applications to Deformation of Surfaces
Abdelhamid Meziani
Dedicated to Linda P. Rothschild

Abstract. Properties of solutions of a class of semilinear equations of the form

Lu = f (x, y, u), where L is a C-valued planar vector field is studied. Series
and integral representations are obtained in a tubular neighborhood of the
characteristic curve of L. An application to infinitesimal bendings of surfaces
with nonnegative curvature in R3 is given.

Mathematics Subject Classification (2000). Primary 35F05, 53A05; Secondary

30G20, 35C15.
Keywords. CR equation, normalization, asymptotic directions, infinitesimal
bending.

Introduction
This paper deals with the properties of the solutions of semiglobal equations related
to complex-valued vector fields in the plane as well as their applications to bending
of surfaces in R3 .
The typical vector field L considered here is smooth in an open set O ⊂ R2
and such that L and L are independent except along a simple closed curve Σ. The
vector field L is of infinite type along Σ and L ∧ L vanishes to first order on Σ (see
Section 2 for details). Properties of the solutions of the equation Lu = f (x, y, u)
are sought in a full neighborhood of the characteristic set Σ. Such equations are
then used to model infinitesimal bendings of surfaces with nonnegative curvature.
This work is motivated by results contained in the papers [1], [2], [3], [4], [5], [6],
[7], [9], [10], [21], [22].
Our approach to the study of the equation Lu = f in a tubular neighborhood
of Σ is to reduce it into the study of a CR equation with a punctual singularity.
Namely to an equation of the form zwz = f in a neighborhood of 0 ∈ C. For
this reason we start, in Section 1, by listing recent results about the behavior of
264 A. Meziani

the solutions of the CR equation. In Section 2, we recall the normal forms for
the vector field L. The simple normal form allows the transition to the singular
CR equation. In Section 3, the solutions in a neighborhood U of Σ, of the R-
linear equation Lu = Au + Bu are characterized. A uniqueness property is then
deduced. That is, we show that if u is a continuous solution that vanishes on
a sequence of points pk contained in a connected component U1 of U \Σ, and if
pk accumulates on Σ, then u ≡ 0 on U1 . In Section 4, Schauder’s Fixed Point
Theorem is used to study a semilinear equation. The last two sections deal with
bendings of surfaces. In Section 5, we reduce the equations for the bending fields
of a surface with nonnegative curvature into solvability of (complex) vector field
of asymptotic directions. In Section 6, we show the nonrigidity of surfaces with
positive curvature except at a flat point. In particular, if S is a germ of such a
surface, then for every  > 0, there exist surfaces S1 and S2 that are -close from
S, in the C k -topology, and such that S1 and S2 are isometric but not congruent.

1. A singular CR equation
We describe here recent results of the author [18] about the properties of solutions
of a CR equation with a punctual singularity (see also [12] [13] and [23]). These
properties will be used to study more general equations associated with complex
vector fields.
Let a(θ) and b(θ) be 2π-periodic, C-valued functions of class C k with k ≥ 2.
Consider the equation
∂w a(θ) b(θ)
= w+ w, (1.1)
∂z 2z 2z
where z = reiθ . We will assume throughout that the average of the function a is
real:  2π
1
a(θ)dθ ∈ R . (1.2)
2π 0
By using periodic ODEs, we construct basic solutions to (1.1).
Proposition 1.1. ([18]) There exists a sequence of real numbers
· · · < λ− − −
−1 ≤ λ−1 < λ0 ≤ λ0 < λ1 ≤ λ1 < · · ·
+ + +

with λ± ±
j −→ ±∞ as j −→ ±∞ and there exists a sequence of functions ψj (θ) ∈
±
C k+1 (S1 , C) such that wj± (r, θ) = rλj ψj± (θ) is a solution of (1.1) in R+ × S1 .
Furthermore, for each j ∈ Z, the rotation of ψj± is 2πj.
Remark 1.1 The λ± ±
j ’s and the ψj ’s are the eigenvalues and eigenfunctions of the
first-order ODE
iψ  (θ) = (a(θ) − λ)ψ(θ) + b(θ)ψ(θ).
Note that this differential equation has (locally) two R-independent solutions. λ± j0
are then the eigenvalues corresponding to periodic solutions with rotation 2πj0 . If
the ODE has two independent periodic solution, then λ− + − +
j0 = λj0 otherwise λj0 < λj0 .
 Solvability of Planar Complex Vector Fields 265

These basic solutions are then used to construct kernels from which a gener-
alized Cauchy formula is derived. More precisely, for ζ = ρeiα and z = reiθ , define
Ω1 (z, ζ) and Ω2 (z, ζ) by
⎧
λ±
⎪
⎪
⎪ −1 r j ±
⎪
⎪ ψj (θ)ψj± (α) if r < ρ
⎪
⎨ 2 λ± ≥0 ρ
j
Ω1 (z, ζ) =
⎪
λ±
⎪
⎪ 1 r j ±
⎪
⎪ ψj (θ)ψj± (α) if ρ < r
⎪ 2
⎩ ρ
±
λj <0
⎧
λ±
(1.3)
⎪
⎪
⎪ −1 r j ±
⎪
⎪ ψj (θ)ψj± (α) if r < ρ
⎪
⎨ 2 λ± ≥0 ρ
j

⎪ 1
r λj
Ω2 (z, ζ) = ±
⎪
⎪
⎪
⎪ 2 ψj± (θ)ψj± (α) if ρ < r
⎪
⎩ ρ
±
λj <0

By studying the asymptotic behavior of the spectral values λ± ±

j and of ψj as
j −→ ±∞, we can estimate the singularities # #of Ω1 and Ω2 . It is found (see
# z #ν ζ
[18]), that the singularity of Ω1 is of the form ## ## and that of Ω2 is of the
# #ν ζ z−ζ
#z # ζ
form ## ## log , where ν ∈ [0, 1) is the fractional part of the average of the
ζ ζ −z
coefficient a(θ).
Theorem 1.1. ([18]) Let U ⊂ C be an open and bounded set whose boundary ∂U
consists of finitely many simple closed curves. If w ∈ C 0 (U ) solves (1.1), then for
z ∈ U , we have
 $ %
−1 Ω1 (z, ζ) Ω2 (z, ζ)
w(z) = w(ζ)dζ − w(ζ)dζ . (1.4)
2πi ∂U ζ ζ
Remark 1.2 The spectral values λ± j give the possible orders at the singularity 0
of the zeros (if λ±
j > 0) or poles (if λ±
j < 0) the solutions of (1.1). In the generic
case, λ = 0 is not a spectral value. When λ = 0 is a spectral value, it gives rise
to a solution w(r, θ) = ψ(θ) of (1.1). This solution is bounded but not continuous,
unless it is constant. This latter case occurs only in the special case when the
coefficients a(θ) and b(θ) satisfy b(θ) = a(θ)eiθ0 for some constant θ0 .
Remark 1.3 The solutions of (1.1) admit series representations. More precisely, if
u is a continuous solution of (1.1) in the disc D(0, R), then
 
u(r, θ) = Cj− ψj− (θ) + Cj+ ψj+ (θ)
λ±
j ≥0

where Cj± ∈ R.
266 A. Meziani

For the nonhomogeneous equation we have the following representation.

Theorem 1.2. ([18]) Let U be as in Theorem 1.1, F ∈ Lp (U ) with p > 2/(1 − ν)
and suppose w ∈ C 0 (U ) satisfies the equation
∂w a(θ) b(θ)
= w+ w + F (z) . (1.5)
∂z 2z 2z
Then w has the representation
 $ %
−1 Ω1 (z, ζ) Ω2 (z, ζ)
w(z) = w(ζ)dζ − w(ζ)dζ +
2πi ∂U  
ζ ζ  (1.6)
1 Ω1 (z, ζ) Ω2 (z, ζ)
+ F (ζ) + F (ζ) dξdη.
π U ζ ζ
The second integral appearing in (1.6) is a particular solution of (1.5). It
defines an integral operator


1 Ω1 (z, ζ) Ω2 (z, ζ)
TR F (z) = F (ζ) + F (ζ) dξdη (1.7)
π U ζ ζ
It is shown in [18] that TR : Lp (U ) −→ C 0 (U ) is a compact operator if 0 is not a
spectral value. If 0 is a spectral value a slight modification of Ω1 and Ω2 (removing
in the sums (1.3) the contribution of the zero eigenvalue) gives a compact operator.
Furthermore, there is δ > 0 so that TR satisfies
|TR F (z)| ≤ CRδ ||F ||p , ∀z ∈ D(0, R), ∀f ∈ Lp (D(0, R)). (1.8)

2. Normal form of a class of vector fields

We reduce a complex vector field satisfying certain properties into a simple canon-
ical form. Let
∂ ∂
L = a(x, y) + b(x, y)
∂x ∂y
be a vector field with coefficients a, b ∈ C ∞ (O, C), where O ⊂ R2 is open. We
assume that L satisfies the following conditions
(i) L is elliptic everywhere in O except along a smooth simple closed curve
Σ ⊂ O. Thus, L and L are independent on O\Σ and dependent on Σ;
(ii) L is of infinite type on Σ. That is, L, L, and any Lie bracket [X1 , [X2 , [· · ·
[Xk−1 , Xk ] · · · ]]] are dependent on Σ, where each Xi is either L or L;
(iii) The bivector L ∧ L vanishes to first order along Σ.
The first condition means that for p ∈ O\Σ, we can find local coordinates (x, y),
∂
centered at p, in which L is a multiple of the CR operator . The second condition
∂z
means that for p ∈ Σ, there are local coordinates (x, y) around p, in which Σ is
∂ ∂
given by x = 0 and L is a multiple of a vector field + ic(x, y) where the
∂y ∂x
function c(x, y) satisfies c(0, y) = 0. The third condition means that c vanishes to
 Solvability of Planar Complex Vector Fields 267

first order (c(x, y) = xα(x, y) with α(0, 0) = 0). The following theorem contained
in [11] gives the canonical form for such vector fields.
Theorem 2.1. ([11]) Assume that L is as above and satisfies conditions (i), (ii),
and (iii). Then, there exist c ∈ R∗ +iR, open sets U ⊂ O, V ⊂ R×S1 , with Σ ⊂ U ,
{0} × S1 ⊂ V , and a diffeomorphism Φ : U −→ V such that


∂ ∂
Φ∗ L = m(r, θ) ic +r ,
∂θ ∂r
where (r, θ) are the coordinates in R × S1 and m is a nonvanishing function.
The number c is an invariant of the structure defined by L in O. More pre-
cisely, if ω = bdx − ady (the dual form of L), then dω = ω ∧ α for some 1-form α
and 
1 1
= α.
c 2πi Σ
Without loss of generality, we can assume that Re(c) > 0 (otherwise replacing
the coordinate θ by −θ will transform it to a vector field with the desired c). If
Imc = 0, then for every k ∈ Z+ , the diffeomorphism Φ in the theorem can be taken
to be of class C k (see [11] for details). When Imc = 0, then Φ is, in general, only
of class C 1 . However, it is proved in [8], that if c ∈ R\Q, then again, for every
k ∈ Z+ , the diffeomorphism Φ can be taken in C k . Note also that normal forms
for vector fields with higher order of vanishing along Σ are obtained in [15].
It follows from Theorem 2.1 that the study of the equation
Lu = g(u, x, y)
in a tubular neighborhood of the characteristic set Σ is reduced to the study of
the equation
Xu = f (u, r, θ)
in a neighborhood of the circle Σ0 = {0} × S1 in R × S1 , where X is the vector
field
∂ ∂
X = ic +r . (2.1)
∂θ ∂r
Throughout the remainder, and for simplicity, we will assume that c ∈ R+ .

3. Solvability for the vector field X

We use the singular CR equation (1.1) to describe the solutions for X in a neigh-
borhood of the characteristic set. Let a0 (θ), b0 (θ) ∈ C k (S1 , C) with k ≥ 2 and
such that the average of a0 is real. Consider the equation
Xu = (a0 (θ) + ra1 (r, θ))u + (b0 (θ) + rb1 (r, θ))u , (3.1)
where a1 and b1 are continuous functions defined in AR = (−R, R) × S . We will
1

use the notation

R = AR ∩ R × S
A+ + 1
and A− −
R = AR ∩ R × S .
1
268 A. Meziani

With (3.1), we associate the CR equation

∂w a0 (θ) b0 (θ)
= w+ w. (3.2)
∂z 2cz 2cz
±
Let rλj ψj± (θ) be the basic solutions of (3.2). We assume that 0 is not a spectral
value of (3.2). We have the following representation of the solutions of (3.1)
−
Theorem 3.1. Let u(r, θ) be a bounded solution of (3.1) in A+
R (resp. AR ). Then
±
there exists a spectral value λj > 0 such that
±
u(r, θ) = rcλj ψj± (θ)P (r, θ) , (3.3)
−
where P is a continuous function in A+ R (resp. AR ) with P (0, θ) = 0 for every θ.
±
Conversely, for every positive spectral value λj , there exists P as above so that u
given by (3.3) solves (3.1).
Proof. The function z = |r|c eiθ satisfies Xz = 0. Furthermore,
Φ± : R± × S1 −→ C\0; Φ± (r, θ) = z = |r|c eiθ
is a diffeomorphism. Since Xz = 2cz, then the pushforward of equation (3.1) via
Φ+ in A+ R into the punctured disc D(0, R )\0 in C is the CR equation
c

a 

0 c b0 c
wz = + |z| a1 (z) w + + |z| b1 (z) w , (3.4)
2cz 2cz
1
where a1 and b1 are bounded functions and where the exponent c is c = − 1. It
c
is shown in [13] that the solutions of equations (3.4) are similar to those of (3.2).
Thus each solution w of (3.4) can be written as w = w0 P for some solution w0
of (3.2) and some non vanishing continuous function P and vice versa. Now w0 is
similar to a basic solution (see Remark 1.3) and consequently, if u solves (3.1) in
− −
A+R (or in AR ), then w = u ◦ Φ (or w = u ◦ Φ ) solves (3.4) and the conclusion
+

of the theorem follows. 

Remark 3.1 If u is a bounded solution of (3.1) in AR , then necessarily u vanishes
on the circle Σ0 but the orders of vanishing from r > 0 and from r < 0 might be
different.
As a consequence of Theorem 3.1, we have the following uniqueness result.
−
Theorem 3.2. Let u be a bounded solution of (3.1) in A+R (resp. AR ). If u vanishes
−
on a sequence of points {pk }k ⊂ AR (resp. AR ) such that pk converges to a points
+

p0 ∈ Σ0 then u ≡ 0
Proof. It follows from Theorem 3.1, that it is enough to prove the uniqueness for
solutions of the model equation (3.2) that vanish on a sequence of points zk that
converges to 0 ∈ C. Let then w be such a solution of (3.2). We use the series
expansion of w (see Remark 1.3) to show that w ≡ 0. By contradiction, suppose
 Solvability of Planar Complex Vector Fields 269

that w ≡ 0, then there exists λ± ± −

j0 > 0 and dj0 ∈ R with dj0 = 0 or dj0 = 0 such that
+

λ− λ+

λ− +
w(z) = d−j0 r
j0
ψj−0 (θ) + d+
j0 r
j0
ψj+0 (θ) + (d−
j r
j ψ − (θ) + d+ r λj ψ + (θ))
j j j (3.5)
j>j0

Set zk = rk eiθk with rk → 0 and θk → θ0 (otherwise take a subsequence of θk ).

λ−
After replacing z by zk in (3.5) and dividing by rk j0 , we get
λ+ −λ− λ+ −λ−
0 = d− − +
j0 ψj0 (θk ) + dj0 rk
j0 j 0
ψj+0 (θk ) + o(rk j0 j 0
)
Now, we let k → ∞ to obtain
d− −
j0 ψj0 (θ0 ) = 0 if λ+ −
j0 > λj0 ; and
− −
dj0 ψj0 (θ0 ) + d+ +
j0 ψj0 (θ0 ) = 0 if λj0 = λ−
+
j0 .

In the first case, we get d− −

j0 = 0 since ψj0 is nowhere 0 (as a non trivial solution
of a first-order linear ODE). By repeating the argument we also find d+ j0 = 0. In
− + − +
the second case, we get dj0 = dj0 = 0 since, ψj0 and ψj0 are independent solutions
of the same linear homogeneous ODE (see Remark 1.1). In both cases, we get a
contradiction and so w ≡ 0. 

4. A semilinear equation
In this section we use Schauder’s Fixed Point Theorem and the operator T of sec-
tion 1 to construct non trivial solutions of a semilinear equation for the vector field
X. Let f (u, r, θ) be a function of class C k , with k ≥ 2, defined in a neighborhood
of {0} × {0} × S1 in C × R × S1 and such that
 2π
1 ∂f
f (0, r, θ) = 0 and (0, 0, θ)dθ ∈ R. (4.1)
2π 0 ∂u
Set
∂f ∂f
a0 (r, θ) = (0, r, θ) and b0 (r, θ) = (0, r, θ). (4.2)
∂u ∂u
With the equation
Xu = f (u, r, θ) (4.3)
we associate the CR equation
∂w a0 (θ) b0 (θ)
= w+ w (4.4)
∂z 2cz 2cz
We have the following theorem.
Theorem 4.1. Let f , a0 , and b0 be as above. Then there exists  > 0 such that for
±
every basic solution |z|λj ψj± (θ) of (4.4), with λ±
j > 0, there exists a nonvanishing
function P defined in A such that
±
u(r, θ) = |r|cλj ψj± (θ)P (r, θ) (4.5)
solves equation (4.3).
270 A. Meziani

Proof. We write f as
f (u, r, θ) = (a0 (θ) + ra1 (r, θ))u + (b0 (θ) + rb1 (r, θ))u + g(u, r, θ)
with g = o(|u|2 ) as u → 0. We seek a solution of (4.3) in the form
±
u(r, θ) = |r|cλj ψj± (θ) (1 + v(r, θ))
with v(0, θ) = 0. The function v must then satisfy the equation
ψj± (θ) ψj± (θ)
Xv = b0 (θ) (v − v) + ra1 (1 + v) + rb1 (1 + v) + h(v, r, θ) (4.6)
ψj± (θ) ψj± (θ)
with
g(w(1 + v), r, θ) ±
h(v, r, θ) = , and w = |r|cλj ψj± (θ).
w
2
Since g = o(|u| ), we can write h as
±
h(v, r, θ) = |r|cλj k(v, r, θ)
with k continuous and k = 0 for v = 0. The pushforward of equation (4.6), in say
A+
R , gives a singular semilinear CR equation of the form
∂w B0 (θ) ±
= (w − w) + |z|1/c−1 A(z) + |z|λj /c−1 F (w, z) (4.7)
∂z 2cz
ψj±
where B0 = b0 and A(z) is a bounded function near 0 ∈ C and where F is a
ψj±
bounded function, continuous in the variable w, and F (0, z) = 0. We are going to
prove that (4.7) has a continuous solution w with w(0) = 0.
First we need to verify that the average of B0 is a real number. Since the
function ψj± satisfies the ODE
dψj±
i = (a0 (θ) − λ± ± ±
j )ψj + b0 (θ)ψj ,
dθ
then
   
2π 2π ψj± 2π dψj± 2π
B0 dθ = b0 dθ =i − (a0 − λ±
j )dθ
0 0 ψj± 0 ψj± 0
is a real number because the average of a0 is real.
For each function s(z) ∈ C 0 (D(0, R)), consider the linear equation
∂w B0 (θ)
= (w − w) + G(s(z), z) (4.8)
∂z 2cz
with
±
G(s(z), z) = |z|1/c−1 A(z) + |z|λj /c−1 F (s(z), z).
We know from Section 1, that the function


1 Ω1 (z, ζ) Ω2 (z, ζ)
TR G(z) = G(s(ζ), ζ) + G(s(ζ), ζ) dξdη
π D(0,R) ζ ζ
 Solvability of Planar Complex Vector Fields 271

is a continuous solution of (4.8), satisfies TR G(0) = 0 and |TR G(z)| ≤ CRδ ||G||p ,
for some positive number δ. Consider the set

ΛK = {s ∈ C 0 (D(0, R)); s(0) = 0 and ||s||0 ≤ K}.

With the function G as defined above, there is a constant K̂ (depending on K)

such that ||G(s(z), z)||p ≤ K̂ for every s ∈ ΛK . Thus |TR G(z)| ≤ K̂Rδ for every
s ∈ ΛK . In particular, TR G ∈ ΛK if R is small enough. Since ΛK is convex and
compact (Ascoli-Arzelà Theorem), then by Schauder’s Fixed Point Theorem the
operator TR has a fixed point w which is a solution of (4.7). Consequently, by using
the pullbacks, we obtain desired solutions to the semilinear equation (4.3). 

5. Equations for the bending fields

In the remainder of this paper, we consider how complex vector fields can be used
to study deformations of surfaces in R3 with nonnegative curvature.
Let S ⊂ R3 be a C ∞ surface. Suppose that S is given by parametric equations:

S = {R(s, t) = (x(s, t), y(s, t), z(s, t)) ∈ R3 ; (s, t) ∈ O} ,

where x, y, and z are C ∞ functions defined in a domain O ⊂ R2 . An infinitesimal

bending of S of order n, of class C k , is a deformation surface S ⊂ R3 , with  ∈ R
a parameter, and given by a position vector

R (x, y) = R(x, y) + 2U 1 (x, y) + 22 U 2 (x, y) + · · · + 2n U n (x, y)

such that each bending field U j : O −→ R3 is of class C k and such that the first
fundamental form of S satisfies

dR2 = dR2 + o(n ) as  → 0. (5.1)

Thus S is an isometric approximation of S. The trivial bendings of S are those

generated by the rigid motions of R3 . In particular, for these motions, we have
U 1 = C × R(s, t) + D, where C, D, are constants in R3 , and where × denotes the
vector product in R3 . It should be mentioned right away that the existence and
characterization of nontrivial infinitesimal bendings for surfaces whose curvatures
change signs is not clear even in the local setting. We refer to [19] and the references
therein for an overview of problems related to these bendings. Some recent work
of the author about bendings of surfaces can be found in [14], [16], [17]. Note that
since,

dR2 = dR2 + 4dR · dU 1

 
n
j−1
+4  dR · dU +
j j
dU · dU
k j−k
+ o(n ),
j=1 k=2
272 A. Meziani

then condition (5.1) is equivalent to the system

dR · dU 1 = 0

j−1
(5.2)
dR · dU j = − dU k · dU j−k , j = 2, . . . , n.
k=1

Let E, F, G, and e, f, g be the coefficients of the first and second funda-

mental forms of S. The Gaussian curvature of S is therefore
eg − f 2
K= .
EG − F 2
The type of the system (5.2) depends on the sign of K. It is elliptic when K > 0;
hyperbolic when K < 0; and parabolic when K = 0. From now on, we will assume
that S has nonnegative curvature (K ≥ 0). Let L be the (complex) vector field of
asymptotic directions of S. That is,
∂ ∂
L= + σ(s, t) , (5.3)
∂s ∂t
where +
eg − f 2
f +i
σ=− (5.4)
g
is an asymptotic direction of S. Note that L is nondegenerate in the regions where
K > 0. With each bending field U j , we associate the complex-valued function
wj = LR · U j = mj + σnj , (5.5)
with mj = Rs · U j and nj = Rt · U j . The following proposition reformulates the
solvability of the U j in terms of the solvability for the wj ’s.
Proposition 5.1. If U 1 , . . . , U n solve (5.2) then the functions w1 , . . . , wn solve the
system
Lw1 = Aw1 + Bw1

j−1
(5.6)
Lwj = Awj + Bwj − LU k · LU j−k , j = 2, . . . , n.
k=1

where
(L2 R × LR) · (LR × LR) −(L2 R × LR) · (LR × LR)
A= , B= .
(LR × LR) · (LR × LR) (LR × LR) · (LR × LR)

Proof. A version of this proposition for the case n = 1 is contained in [16]. Thus we
need only to verify the nonhomogeneous equations for j ≥ 2. Calculations similar
to those used in [16] lead to the following system
Vsj + QVtj + M V = K j (5.7)
 Solvability of Planar Complex Vector Fields 273




mj 0 −e/g M11 M12
where V j = , Q= , M= ,
nj 1 −2f /g M21 M22
1 1
M11 = Rt · (Rss + Rtt ), M12 = − Rs · (Rss + Rtt )
g|Rs × Rt | g|Rs × Rt |
2
M21 = Rt [f (Rss + Rtt ) + gRst × (Rss + Rtt )]
g(e + g)|Rs × Rt |
2
M22 = Rs [−f (Rss + Rtt ) + gRst × (Rss + Rtt )]
g(e + g)|Rs × Rt |
and where
⎧
⎪
j−1
⎪
⎪ αj =− Usk · Usj−k
⎪
⎪
⎪
⎪

j  ⎪
⎪ k=1
⎨
j−1
α − γ j e/g
j
K = with βj =− Usk · Utj−k + Utk · Usj−k
β − 2γ f /g
j j
⎪
⎪
⎪
⎪ k=1
⎪
⎪
j−1
⎪
⎪
⎪
⎩ γ
j
=− Utk · Utj−k .
k=1
T
Note that the asymptotic

 direction σ is an eigenvalue of the matrix Q with
1
eigenvector E = . Multiply the system (5.7) by E T = (1, σ) and use the
σ
relation wj = E T V j to obtain
Lwj − Awj − Bwj = E T K j
with A and B as in the proposition (see [16] for details). It remains to verify that

j−1
ET K j = − LU k · LU j−k . We have
k=1

e + 2σf j
ET K j = αj + σβ j − γ = αj + σβ j + σ2 γ j
g

j−1
=− Usk · Usj−k + σ(Usk · Utj−k + Utk · Usj−k ) + σ 2 Utk · Utj−k
k=1

j−1
=− LU k · LU j−k
k=1

In the above relations we have used the fact that gσ 2 = −(e + 2f σ). 
Remark 5.1 Although Proposition 5.1 is stated for the vector field L given in (5.3),
a simple calculation shows that equation (5.3) remains valid if L is replaced by
 = mL and w is replaced by w
any multiple L  = mw.
Remark 5.2 In many situations when the degeneracy of the vector field L is not too
large, the solvability of (5.6) for the functions w1 , . . . , wn , leads to the determina-
tion of the bending fields U 1 , . . . , U n . This is the case for surfaces with a flat point
274 A. Meziani

(see [17] for the case of homogeneous surfaces). More precisely, if w1 is known and
vanishes at the flat point, then from the relations LR · U 1 = w1 , LR · U 1 = w1 , we
can solve for two components, say ξ 1 and η1 , of U 1 = (ξ 1 , η 1 , ζ 1 ) in terms of w1 ,
w1 , and ζ 1 . Then we can use dR · dU 1 = 0 to reduce it to an integrable system
for the component ζ 1 which can be solved by quadrature (Poincaré Lemma). Note
also that if w1 vanishes to order ν at 0, then U 1 can be chosen to vanish at 0 but
the order of vanishing is ν − 1. Once U 1 is known, U 2 can also be determined from
the knowledge of w2 and so on.

6. Local non rigidity of a class of surfaces

In this section we consider local deformation of surfaces with positive curvature
except at a flat point where both principal curvatures are zero. Assume that S ⊂ R3
is a C ∞ surface given as the graph of a function z(x, y). For δ > 0, let
Sδ = {R(x, y) = (x, y, z(x, y)) ∈ R3 ; x2 + y 2 ≤ δ}. (6.1)
We assume that the curvature K of S is positive except at the origin 0 and that
z vanishes to order m > 2 at 0. Thus,
z(x, y) = Pm (x, y) + z1 (x, y) (6.2)
where Pm is a polynomial of degree m and where z1 is a C ∞ function that vanishes
to order m at 0. The first theorem establishes the existence of surfaces that are
arbitrarily close to S and that are isometric but not congruent (they are not
obtained from one another by a rigid motion of R3 ).

Theorem 6.1. Let S be a surface given by (6.1) with K > 0 except at 0. For every
k ∈ Z+ and for every  > 0, there exist δ > 0 and surfaces Σ+ and Σ− of class
C k over the disc D(0, δ) such that
1. Σ+ and Σ− are -close to Sδ in the C k -topology;
2. Σ+ and Σ− are isometric but not congruent.

This theorem is a consequence of the next theorem about infinitesimal bend-

ings. In the statement of the following theorem, by generic surfaces we mean almost
all surfaces. The restriction to generic surfaces is not necessary to reach the con-
clusion but it is useful for the sake of simplicity in reducing the proof into the
setting of solvability of vector fields as in Section 3.

Theorem 6.2. Let n ∈ Z+ , k ∈ Z+ and let S be a generic surface given by (6.1) with
curvature K > 0 except at 0. Then there exists δ > 0 such that Sδ has nontrivial
infinitesimal bendings of order n and class C k .

Proof of Theorem 6.1. Given S satisfying the hypotheses of Theorem 6.1, we can
find another generic surface S as in Theorem 6.2 and such that S is (/2)-close to
 Solvability of Planar Complex Vector Fields 275

S in the C k -topology. Let St,δ be an infinitesimal bending of order 1 of class C k

of Sδ . Thus, St,δ is defined over D(0, δ) by the position vector
 y) + tU 1 (x, y),
t (x, y) = R(x,
R
where R  is the position vector of Sδ and U 1 is a nontrivial bending field. For
±t0 . Since
t0 > 0 define surfaces Σ± over the disc D(0, δ) by the position vectors R
dR · U 1 = 0, then
t2 = dR
dR −t
2
= dR2 + t20 dU 1 · dU 1 .
0 0

That is Σ+ and Σ− are isometric. They are not congruent because U 1 is nontrivial
(see Chapter 12 of [20]). Finally, if t0 is small enough, Σ± are within  from Sδ in
the C k topology. 

Proof of Theorem 6.2.. Let S be defined as the graph of z(x, y) as in (6.2), with
K > 0 except at 0. In polar coordinates x = ρ cos φ, y = ρ sin φ, the function z
becomes
z = ρm P (φ) + ρm+1 z2 (ρ, φ)
where P (φ) is a trigonometric polynomial of degree ≤ m. We can assume that
P > 0 (S is on one side of the tangent plane at 0). The condition on the curvature
means
m2 P 2 (φ) + mP (φ)P  (φ) − (m − 1)P 2 (φ) > 0 ∀φ. (6.3)
Introduce a new radius ρ1 by ρ1 = ρP (φ)1/m . The surface is given in the coordi-
nates (ρ1 , φ) by the position vector
m+1
R(ρ1 , φ) = (ρ1 eq(φ) cos φ, ρ1 eq(φ) cos φ, ρm
1 + ρ1 z3 (ρ1 , φ))
−1
where q(φ) = log P (φ). Condition (6.3) can be written for the function q as
m
1 + q  (φ) − q  (φ) > 0, ∀φ.
2

The coefficients of the second fundamental form of S are:

Rρ1 ρ1 · (Rρ1 × Rφ )
e = = m(m − 1)ρm−2 + o(ρm−2 ),
|Rρ1 × Rφ | 1 1

Rρ1 φ · (Rρ1 × Rφ )
f = = o(ρm−1 ),
|Rρ1 × Rφ | 1

Rφφ · (Rρ1 × Rφ ) 2 
= mρm1 (1 + q (φ) − q (φ)) + o(ρ1 ) ,
m
g =
|Rρ1 × Rφ |
and an asymptotic direction is
+ =
σ = −(f + i eg − f 2 ) = −imρm−1
1 (m − 1)(1 + q  2 − q  ) + o(ρm−1
1 ).
As vector field of asymptotic directions we can take
∂ ∂
L= + iρ1 A(φ, ρ1 ) ,
∂φ ∂ρ1
276 A. Meziani

where >
1 + q  2 − q
A(φ, ρ1 ) = + o(ρm−1 ).
m−1 1

From now on we will assume that the average A(φ, 0) is not a rational number.
That is,
>
 2π  2π
1 1 1 + q  2 (φ) − q  (φ)
c= A(φ, 0)dφ = dφ ∈ Q. (6.4)
2π 0 2π 0 m−1
This is the genericity condition that enables us to conjugate the vector field L to
a model vector field via C N -diffeomorphisms for any N ∈ Z+ (see section 2). We
take as a model vector field
∂ ∂
L0 = + icr .
∂θ ∂r
Furthermore, the diffeomorphism that realizes this conjugacy has the form
r = ρ1 + o(ρ1 ), θ = φ + β(φ) + o(ρ1 )

where β is 2π periodic and θ (φ) > 0. With respect to these new coordinates, the
position vector R has the form
R = (req(μ(θ)) cos μ(θ) + o(r), req(μ(θ)) sin μ(θ) + o(r), rm + o(rm )) ,
where μ(θ) is periodic and μ > 0.
Now, we construct the bending fields U 1 , . . . , U n . For this, we construct
1
w , . . . , wn , appropriate solutions of the system (5.6), and then deduce the U j ’s.
With respect to the coordinates (r, θ) and vector field L0 , the system for the wj ’s
has the form
L0 w1 = a(r, θ)w1 + b(r, θ)w1

j−1
(6.5)
L0 wj = a(r, θ)wj + b(r, θ)wj − L0 U k · L0 U j−k , j ≥ 2,
k=1
where


M  N − M N  i M  N  − M  N 
a(r, θ) = + c + + O(r2m−2 )
2(M  N − M N  ) 2
c(M  N − M N  ) 
M  N − M N  i M  N  − M  N 
b(r, θ) =−  
− 3c − + O(r2m−2 )
2(M N − M N ) 2 c(M  N − M N  )
with
M (θ) = eq(μ(θ)) cos μ(θ) , N (θ) = eq(μ(θ)) sin μ(θ) .
Note that
M  N − M N  μ (θ)
Re(a(θ, 0)) =  
= −q (μ(θ))μ (θ) + 
2(M N − M N ) 2μ (θ)
and its integral over [0, 2π] is zero. Thus the above system satisfies the sufficiency
conditions for solvability of Section 3. Given k ∈ Z+ , let w1 be a nontrivial solution
of the first equation of (6.5) such that w1 vanishes to an order ν > k along r = 0
 Solvability of Planar Complex Vector Fields 277

(see Theorem 3.1). To such w1 , corresponds a bending field U 1 of class C k that

vanishes to order ν − 1 on r = 0 (see Remark 5.2). With U 1 selected, the equation
for w2 has the nonhomogeneous term L0 U 1 · L0 U 1 which vanishes to order (ν − 1)2
on r = 0. Again a solution w2 of the equation in (6.5) can be found that vanishes
to an order ≥ ν on r = 0 giving rise to a bending field U 2 . This process can
be continued until we reach U n . In this way an infinitesimal bending of order n
of Sδ is achieved. That such a bending is nontrivial follows from the fact that
U 1 ≡ 0 but vanishes to an order (ν − 1) > 1 at 0 and hence cannot be of the form
 y) + B with A and B constants and is not therefore induced by a rigid
A × R(x,
motion of R3 . 

Remark 6.1 Theorem 6.2 is stated here for surfaces that satisfy condition (6.4). As
mentioned earlier this condition is sufficient but is not necessary for the existence of
the bending fields. Without appealing to condition (6.4), bendings of homogeneous
surfaces are characterized in [17].

References
[1] M.S. Baouendi and F. Treves: A property of the functions and distributions annihi-
lated by a locally integrable system of complex vector fields, Ann. of Math., Vol. 113,
387–421, (1981)
[2] M.S. Baouendi and F. Treves: A local constancy principle for the solutions of cer-
tain overdetermined systems of first-order linear partial differential equations, Math.
Analysis and Applications, Part A, Adv. in Math., Supplementary Studies, Vol. 7A,
245–262, (1981)
[3] A.P. Bergamasco, P. Cordaro and J. Hounie: Global properties of a class of vector
fields in the plane, J. Diff. Equations, Vol. 74, 179–199, (1988)
[4] A.P. Bergamasco and A. Meziani: Solvability near the characteristic set for a class
of planar vector fields of infinite type, Ann. Inst. Fourier, Vol. 55, 77–112, (2005)
[5] S. Berhanu and J. Hounie: Uniqueness for locally integrable solutions of overdeter-
mined systems, Duke Math. J., Vol. 105, 387–410, (2000)
[6] S. Berhanu, J. Hounie and P. Santiago: A generalized similarity principle for complex
vector fields and applications, Trans. Amer. Math. Soc., Vol. 353, 1661–1675, (2001)
[7] S. Berhanu and A. Meziani: Global properties of a class of planar vector fields of
infinite type, Comm. PDE, Vol. 22, 99–142, (1997)
[8] P. Cordaro and X. Gong: Normalization of complex-valued vector fields which degen-
erate along a real curve, Adv. Math., Vol. 184, 89–118, (2004)
[9] J. Hounie and P. Santiago: On the local solvability of semilinear equations, Comm.
PDE, Vol. 20, 1777–1789, (1995)
[10] A. Meziani: On the similarity principle for planar vector fields: applications to
second-order pde, J. Diff. Equations, Vol. 157, 1–19,(1999)
[11] A. Meziani: On planar elliptic structures with infinite type degeneracy, J. Funct.
Anal., Vol. 179, 333–373, (2001)
278 A. Meziani

[12] A. Meziani: Generalized CR equation with a punctual singularity: the model case,
Complex Variables, Vol. 48(6), 495–512, (2003)
[13] A. Meziani: Generalized CR equation with a singularity, Complex Variables, Vol.
48(9), 739–752, (2003)
[14] A. Meziani: Infinitesimal bending of homogeneous surfaces with nonnegative curva-
ture, Comm. Anal. Geom., Vol. 11, 697–719, (2003)
[15] A. Meziani: Elliptic planar vector fields with degeneracies, Trans. of the Amer. Math.
Soc., Vol. 357, 4225–4248, (2004)
[16] A. Meziani: Planar complex vector fields and infinitesimal bendings of surfaces with
nonnegative curvature, in: Contemp. Math., Vol. 400, Amer. Math. Soc., 189–202,
(2006)
[17] A. Meziani: Infinitesimal bendings of high orders for homogeneous surfaces with pos-
itive curvature and a flat point, J. Diff. Equations, Vol. 239, 16–37, (2007)
[18] A. Meziani: Representation of solutions of a singular CR equation in the plane,
Complex Variables and Elliptic Equations, Vol. 53(12), 1111–1130, (2008)
[19] I.K. Sabitov: Local Theory of Bending of Surfaces, Encyclopaedia Math. Sci., Vol. 48,
Springer-Verlag, (1992)
[20] M Spivak: A Comprehensive Introduction to Differential Geometry, Vol. 5, Publish
or Perish, Berkeley, (1975)
[21] F. Treves: Remarks about certain first-order linear PDE in two variables, Comm.
PDE, Vol. 5, 381–425, (1980)
[22] F. Treves: Hypo-analytic structures, Princeton University Press, (1992)
[23] Z.D. Usmanov: Generalized Cauchy-Riemann system with a singular point, Pitman
Monographs and Surveys in Pure and Applied Mathematics, Vol. 85, Longman Har-
low, (1997)

Abdelhamid Meziani
Department of Mathematics
Florida International University
Miami, Florida 33199, USA
e-mail: [email protected]
Complex Analysis
Trends in Mathematics, 279–300

c 2010 Springer Basel AG

On the Zariski Closure of a Germ

of Totally Geodesic Complex Submanifold
on a Subvariety of a Complex Hyperbolic
Space Form of Finite Volume
Ngaiming Mok
Dedicated to Linda Rothschild on the occasion of her 60th birthday

Abstract. Let X be a complex hyperbolic space form of finite volume, and

W ⊂ X be a complex-analytic subvariety. Let S ⊂ X be a locally closed
complex submanifold lying on W which is totally geodesic with respect to the
canonical Kähler-Einstein metric on X. We prove that the Zariski closure of
S in W is a totally geodesic subset. The latter implies that the Gauss map
on any complex-analytic subvariety W ⊂ X is generically finite unless W is
totally geodesic.
Mathematics Subject Classification (2000). 53C35; 53C55.
Keywords. complex hyperbolic space form, holomorphic projective connection,
complex geodesic, second fundamental form, holomorphic foliation, Gauss
map, Zariski closure, minimal compactification.

A fundamental object of study in projective geometry is the Gauss map. Griffiths-

Harris [GH] proved that the Gauss map of a non-linear projective submanifold
W ⊂ Pn is generically finite. By the work of Zak [Za], the Gauss map on W is in
fact a birational morphism onto its image. An elementary proof of the finiteness of
the Gauss map on W was given by Ein [Ei]. Regarding the geometry of the Gauss
map an analogous study on submanifolds of Abelian varieties was pursued in [Za].
In differential-geometric terms, generic finiteness of the Gauss map on a non-
linear projective submanifold W ⊂ Pn is the same as the vanishing of the kernel of
the projective second fundamental form at a general point. The projective second
fundamental form is on the one hand by definition determined by the canonical
holomorphic projective connection on the projective space, on the other hand it is

Partially supported by CERG 7034/04P of the Research Grants Council, Hong Kong.
280 N. Mok

the same as the restriction to (1,0)-vectors of the Riemannian second fundamental

form with respect to the Fubini-Study metric on the projective space. It is there-
fore natural to extend the study of Gauss maps on subvarieties to the context of
subvarieties of complex hyperbolic space forms of finite volume, i.e., subvarieties
of quotients of the complex unit ball B n by torsion-free lattices, either in terms of
the canonical holomorphic projective connection on B n ⊂ Pn embedded by means
of the Borel embedding, or in terms of the canonical Kähler-Einstein metric, not-
ing that the Riemannian connection of the canonical Kähler-Einstein metric is an
affine connection compatible with the canonical holomorphic projective connec-
tion on B n .
In the current article the first motivation is to examine projective geometry
in the dual situation of compact complex hyperbolic space forms. Specifically, we
will prove generic finiteness of the Gauss map mentioned above on subvarieties
W ⊂ X of compact complex hyperbolic space forms X. Here the Gauss map
is defined on regular points of π−1 (W ), where π : B n → X is the universal
covering map, and as such the Gauss map on W is defined only up to the action
of Γ, and the question of birationality is not very meaningful. A result on the
Gauss map on compact complex hyperbolic space forms analogous to the result
of Griffiths-Harris [GH] mentioned in the above is the statement that, given a
projective-algebraic submanifold W ⊂ X = B n /Γ which is not totally geodesic,
and taking W  ⊂ π −1 (W ) to be any irreducible component, the Gauss map on W 
is of maximal rank at a general point, or, equivalently, the kernel of the second
fundamental form vanishes at a general point. We prove in the current article
that this analogue does indeed hold true, and that furthermore, in contrast to
the case of projective subvarieties, it holds true more generally for any irreducible
complex-analytic subvariety W ⊂ X, without assuming that W is nonsingular,
with a proof that generalizes to the case of quasi-projective subvarieties of complex
hyperbolic space forms of finite volume. We leave unanswered the question whether
the analogue of finiteness of the Gauss map for the case of non-linear projective-
algebraic submanifolds W ⊂ X holds true, viz., whether every fiber of the Gauss
map on W  is necessarily discrete in the case where W is non-singular.

In the case of cocompact lattices Γ ⊂ Aut(B n ) the result already follows

from the study of homomorphic foliations in Cao-Mok [CM, 1990] arising from
kernels of the second fundamental form, but we give here a proof that applies
to arbitrary holomorphic foliations by complex geodesic submanifolds. Using the
latter proof, we are able to study the Zariski closure of a single totally geodesic
complex submanifold. For any subset E ⊂ W we denote by ZarW (E) the Zariski
closure of E in W . We show that, if the projective variety W ⊂ X = B n /Γ
admits a germ of complex geodesic submanifold S ⊂ W , then some open subset
V of ZarW (S) in the complex topology, V ∩ S = ∅, must admit a holomorphic
foliation F by complex geodesics such V ∩ S is saturated with respect to F .
Our stronger result on holomorphic foliations by complex geodesic submanifolds
implies that ZarW (S) must itself be a totally geodesic subset. For the proof of the
 Zariski Closure of a Germ of Totally Geodesic Complex Submanifold 281

implication we study varieties of tangents to complex geodesics on a subvariety

W ⊂ X = B n /Γ analogous to the notion of varieties of minimal rational tangents
on projective subvarieties uniruled by lines, a notion extensively studied in recent
years by Hwang and Mok (cf. Hwang [Hw] and Mok [Mk4]). In a certain sense, total
geodesy of the Zariski closure of a germ of complex geodesic submanifold on W
results from the algebraicity of varieties of tangents to complex geodesics on W and
the asymptotic vanishing of second fundamental forms on locally closed complex
submanifolds on B n swept out by complex geodesics (equivalently minimal disks).
This link between the study of subvarieties W of compact complex hyperbolic space
forms and those of projective subvarieties is in itself of independent interest, and it
points to an approach in the study of Zariski closures of totally geodesic complex
submanifolds on projective manifolds uniformized by bounded symmetric domains.
To prove our results for non-uniform lattices Γ ⊂ Aut(B n ) we make use
of the Satake-Borel-Baily compactification ([Sa, 1960], [BB, 1966]) in the case of
arithmetic lattices, and the compactification by Siu-Yau [SY, 1982] obtained by
differential-geometric means, together with the description of the compactification
as given in Mok [Mk3, 2009], in the case of non-arithmetic lattices.
Our result on the Zariski closure of a germ of complex geodesic submanifold
in the case of complex hyperbolic space forms is a special case of a circle of prob-
lems. In general, we are interested in the characterization of the Zariski closure of
a totally geodesic complex submanifold S on a quasi-projective subvariety W of
a compact or finite-volume quotient X of a bounded symmetric domain Ω. The
inclusion S ⊂ X is modeled on a pair (D, Ω), where D ⊂ Ω is a totally geodesic
complex submanifold (which is itself a bounded symmetric domain). The nature
and difficulty of the problem may depend on the pair (D, Ω). A characterization
of the Zariski closure relates the question of existence of germs of totally geodesic
complex submanifolds S ⊂ W ⊂ X to that of the global existence of certain types
of complex submanifolds ZarW (S) ⊂ W . For instance, taking Ω to be biholo-
morphically the Siegel upper half-plane Hg of genus g ≥ 2, X = Hg /Γ to be the
moduli space of principally polarized Abelian varieties (where Γ has some torsion),
and taking W to be the closure of the Schottky locus, S to be a totally geodesic
holomorphic curve, a characterization of ZarW (S) probably relates the question
of local existence of totally geodesic holomorphic curves to the question of global
existence of totally geodesic holomorphic curves and rank-1 holomorphic geodesic
subspaces, and hence to a conjecture of Oort’s (cf. Hain [Ha]). In such situations,
granted the characterization problem on ZarW (S) can be settled, a global non-
existence result may imply a non-existence result which is local with respect to
the complex topology, hence completely transcendental in nature.

1. Statement of the main result and background materials

(1.1) The Main Theorem on Zariski closures of germs of complex geodesic sub-
manifolds of complex hyperbolic space forms of finite volume. Let n ≥ 3 and
282 N. Mok

Γ ⊂ Aut(B n ) be a torsion-free lattice, so that X := B n /Γ is of finite volume with

respect to the canonical Kähler-Einstein metric. Let W ⊂ X be an irreducible
complex-analytic subvariety of complex dimension m. A simply connected open
subset U of the smooth locus Reg(W ) can be lifted to a locally closed complex
submanifold U  on B n , and on U we have the Gauss map which associates each
 
point y ∈ U to [Ty (U )] as a point in the Grassmann manifold of m-planes in Cn .
This way one defines a Gauss map on Reg(W ) which is well defined only modulo
the action of the image Φ of π1 (W ) in π1 (X) = Γ. When the Gauss map fails to
be of maximal rank we have an associated holomorphic foliation defined at gen-
eral points of W whose leaves are totally geodesic complex submanifolds. Partly
motivated by the study of the Gauss map on subvarieties of complex hyperbolic
space forms we are led to consider the Zariski closure of a single germ of totally
geodesic complex submanifold on W . We have
Main Theorem. Let n ≥ 2 and denote by B n ⊂ Cn the complex unit ball equipped
with the canonical Kähler-Einstein metric ds2Bn . Let Γ ⊂ Aut(B n ) be a torsion-
free lattice. Denote by X := B n /Γ the quotient manifold, of finite volume with
respect to the canonical Kähler-Einstein metric ds2X induced from ds2Bn . Denote
by X min the minimal compactification of X so that X min is a projective-algebraic
variety and X inherits the structure of a quasi-projective variety from X min . Let
W ⊂ X be an irreducible quasi-projective subvariety, and S ⊂ W be a locally closed
complex submanifold which is totally geodesic in X with respect to ds2X . Then, the
Zariski closure Z ⊂ W of S in W is a totally geodesic subset.
Here totally geodesic complex submanifolds of X are defined in terms of the
canonical Kähler-Einstein metric. Equivalently, they are defined in terms of the
canonical holomorphic projective connection on X which descends from B n ⊂ Pn
(cf. (1.3)). With the latter interpretation the projective second fundamental form
on a locally closed complex submanifold Z ⊂ X is holomorphic. Total geodesy of
Z means precisely the vanishing of the projective second fundamental form.
Consider X = B n /Γ, where Γ ⊂ Aut(B n ) is a non-uniform torsion-free lat-
tice. If Γ is arithmetic, we have the Satake-Borel-Baily compactification (Satake
[Sa], Borel-Baily [BB]). For the rank-1 bounded symmetric domain B n the set of
rational boundary components constitute a Γ-invariant subset Π of ∂B n , and Γ
acts on B n ∪ Π to give X min := (B n ∪ Π)/Γ, which consists of the union of X
and a finite number of points, to be called cusps, such that X min can be endowed
naturally the structure of a normal complex space.
(1.2) Description of Satake-Baily-Borel and Mumford compactifications for
X = B n /Γ. We recall briefly the Satake-Baily-Borel and Mumford compacti-
fication for X = B n /Γ in the case of a torsion-free non-uniform arithmetic sub-
group Γ ⊂ Aut (B n ) (For details cf. Mok [Mk4]). Let E ⊂ ∂B n be the set of
boundary points b such that for the normalizer Nb = {ν ∈ Aut (B n ) : ν(b) = b},
Γ ∩ Nb is an arithmetic subgroup. The points b ∈ E are the rational boundary
components in the sense of Satake [Sa] and Baily-Borel [BB]. Modulo the ac-
tion of Γ, the set A = E/Γ of equivalence classes is finite. Set-theoretically the
 Zariski Closure of a Germ of Totally Geodesic Complex Submanifold 283

Satake-Baily-Borel compactification X min of X is obtained by adjoining a finite

number of points, one for each α ∈ A. Fixing b ∈ E we consider the Siegel do-
main 7presentation Sn of B n obtained 8 via a Cayley transform which maps b to ∞,
Sn = (z  ; zn ) ∈ Cn : Imzn > z  2 . Identifying B n with Sn via the Cayley trans-
form, we write X = Sn /Γ. Writing z  = (z1 , . . . , zn−1); z = (z  ; zn ), the unipotent
radical of Wb ⊂ Nb is given by
4 5
Wb = ν ∈ Nb : ν(z  ; zn ) = (z  + a ; zn + 2i a · z  + ia 2 + t); a ∈ Cn−1 , t ∈ R ,
(1)
n−1
where a · z  = i=1 a i z i . Wb is nilpotent and U b := [Wb , Wb ] is the real 1-
parameter group of translations λt , t ∈ R, where λt (z  , z) = (z  , z + t). For a
rational boundary component b, Γ∩Wb ⊂ Wb is a lattice, and [Γ∩Wb , Γ∩Wb ] ⊂ Ub
must be nontrivial. Thus, Γ ∩ Ub ⊂ Ub ∼ = R must be a nontrivial discrete subgroup,
generated by some λτ ∈ Γ ∩ Ub . For any nonnegative integer N define
7 8
Sn(N ) = (z  ; zn ) ∈ Cn : Imzn > z  2 + N ⊂ Sn . (2)
Consider the holomorphic map Ψ : Cn−1 × C → Cn−1 × C given by
2πizn
Ψ(z  ; zn ) = (z  , e τ ) := (w ; wn ) ; w = (w1 , . . . , wn−1 ) ; (3)
which realizes Cn−1 × C as the universal covering space of Cn−1 × C∗ . Write
(N )
G = Ψ(Sn ) and, for any nonnegative integer N , write G(N ) = Ψ(Sn ). We have
7  28
,(N ) = (w ; wn ) ∈ C : |wn |2 < e −4πN
G τ
−4π
,=G
· e τ w  , G ,(0) . (4)
Γ ∩ Wb acts as a discrete group of automorphisms on Sn . With respect to this
action, any γ ∈ Γ ∩ Wb commutes with any element of Γ ∩ Ub , which is generated
by the translation λτ . Thus, Γ ∩ Ub ⊂ Γ ∩ Wb is a normal subgroup, and the action
of Γ ∩ Wb descends from Sn to Sn /(Γ ∩ Ub ) ∼ = Ψ(Sn ) = G. Thus, there is a group
homomorphism π : Γ∩Wb → Aut(G) such that Ψ◦ν = π(ν)◦Ψ for any ν ∈ Γ∩Wb .
More precisely, for ν ∈ Γ ∩ Wb of the form (1) where t = kτ , k ∈ Z, we have
 4π   2π  2 
π(ν)(w , wn ) = w + a , e− τ a ·w − τ a  · wn . (5)
Sn /(Γ ∩ Wb ) can be identified with G/π(Γ ∩ Wb ). Since the action of Wb on Sn
preserves ∂Sn , it follows readily from the definition of ν(z  ; zn ) that Wb preserves
the domains Sn , so that G(N ) ∼
(N ) (N )
= Sn /(Γ ∩ Ub ) is invariant under π(Γ ∩ Wb ).
Write G ,(N ) = G(N ) ∪ (Cn−1 × {0}) ⊂ Cn , G , (0) = G.
, G ,(N ) is the interior of the
closure of G (N ) n , Here π(Γ ∩ Wb ) acts
in C . The action of π(Γ ∩ Wb ) extends to G.
as a torsion-free discrete group of automorphisms of G. , Moreover, the action of
π(Γ ∩ Wb ) on Cn−1 × {0} is given by a lattice of translations Λb .
Denote the compact complex torus (Cn−1 × {0})/Λb by Tb . The Mumford
compactification X M of X is set-theoretically given by X M = X ) ()Tb ), the
(N )
disjoint union of compact complex tori being taken over A = E/Γ. Define Ωb =
G, (N ) /π(Γ ∩ Wb ) ⊃ G(N ) /π(Γ ∩ Wb ) ∼ (N )
= Sn /(Γ ∩ Wb ). Then the natural map
(N )
G(N ) /π(Γ∩Wb ) = Ωb −Tb → Sn /Γ = X is an open embedding for N sufficiently
284 N. Mok

large, say N ≥ N0 . The structure of X M as a complex manifold is defined by

(N )
taking Ωb , N ≥ N0 , as a fundamental system of neighborhood of Tb . From
the preceding description of X M one can equip the normal bundle Nb of each
(N )
compactifying divisor Tb in Ωb (N ≥ N0 ) with a Hermitian metric of strictly
negative curvature, thus showing that Tb can be blown down to a normal isolated
singularity by Grauert’s blowing-down criterion, which gives the Satake-Baily-
Borel (alias minimal) compactification X min . Tb is an Abelian variety since the
conormal bundle Nb is ample on Tb .
In Mok [Mk4] we showed that for a non-arithmetic torsion-free non-uniform
lattice Γ ⊂ Aut(B n ), the complex hyperbolic space form X = Ω/Γ admits a
Mumford compactification X M = X ) ()Tb ), with finitely many Abelian varieties
Tb = Cn /Λb , such that, by collapsing each of the finitely many Abelian varieties
Tb , X M blows down to a projective-algebraic variety X min with finitely many iso-
lated normal singularities. (As in the statement of the Main Theorem, identifying
X ⊂ X min as a Zariski open subset of the projective-algebraic variety X min we
will regard X as a quasi-projective manifold and speak of the latter structure as
the canonical quasi-projective structure.) The picture of a fundamental system of
(N )
neighborhoods Ωb of Tb , N ≥ N0 , in X M is exactly the same as in the arith-
metic case. For the topological structure of X, by the results of Siu-Yau [SY] using
Busemann functions, to start with we have a decomposition of X into the union
of a compact subset K ⊂ X and finitely many disjoint open sets, called ends,
(N )
which in the final analysis can be taken to be of the form Ωb for some N ≥ N0 .
Geometrically, each end is a priori associated to an equivalence class of geodesic
rays, where two geodesic rays are said to be equivalent if and only if they are at
(N )
a finite distance apart from each other. From the explicit description of Ωb as
given in the above, the space of geodesic rays in each end can be easily determined,
and we have

Lemma 1. Fix n ≥ 1 and let Γ ⊂ Aut(B n ) be a torsion-free non-uniform lattice,

X := Ω/Γ. Let X M be the Mumford compactification and Ωb = G
(N ) ,(N ) /π(Γ ∩ Wb )
be a neighborhood of a divisor Tb = C /Λb at infinity. Denote by p : G(N ) →
n

G(N ) /π(Γ ∩ Wb ) the canonical projection. Then, any geodesic ray λ : [0, ∞) → X
parametrized by arc-length on the end Ωb − Tb = G/π(Γ ∩ Wb ) ∼
(N ) (N )
= Sn /(Γ ∩ Wb )
must be of the form λ(s) = p (Ψ(ζ, a + Aiecs )) for some ζ ∈ Cn−1 , a ∈ R, some
constant c > 0 determined by the choice of the canonical Kähler-Einstein metric
(N )
on B n and for some sufficiently large constant A > 0 (so that λ(0) ∈ Sn ), where
2πiα
Ψ(ζ; α) = (ζ, e τ ), in which τ > 0 and the translation (z  ; z) → (z  , z + τ ) is the
generator of the infinite subgroup Γ ∩ Ub ⊂ Ub ∼= R.
(N )
Proof. A geodesic ray on an end Ωb − Tb must lift to a geodesic ray on the Siegel
domain Sn which converges to the infinity point ∞ of Sn . On the upper half-plane
H = {w > 0 : Im(w) > 0} a geodesic ray λ : [0, ∞) → H parametrized by arc-
length joining a point w0 ∈ H to infinity must be of the form μ(s) = u0 + iv0 ecs
 Zariski Closure of a Germ of Totally Geodesic Complex Submanifold 285

for some constant c > 0 determined by the Gaussian curvature of the Poincaré
metric ds2H chosen. In what follows a totally geodesic holomorphic curve on Sn
will also
7 be referred to as a complex8geodesic (cf. (1.3)). For the Siegel domain
Sn = (z  ; zn ) ∈ Cn : Imzn > z  2 , considered as a fibration over Cn−1 , the
fiber Fz over each z  ∈ Cn−1 is the translate of the upper half-plane by iz2.
Fz ⊂ Sn is a complex geodesic. It is then clear that any ν : [0, ∞) → Sn of the
form ν(s) = (ζ, a + Aiecs ), for ζ ∈ Cn−1 and for appropriate real constants a, c
and A, is a geodesic ray on Sn converging to ∞ (which corresponds to b ∈ ∂B n ).
Any complex geodesic S ⊂ Sn is the intersection of an affine line L := Cη + ξ with
Sn , and S is a disk on L unless the vector η is proportional to en = (0, . . . , 0; 1).
Any geodesic ray λ on Sn lies on a uniquely determined complex geodesic S. Thus,
λ(s) tends to the infinity point ∞ of Sn only if S is parallel in the Euclidean sense
to Cen , so that λ([0, ∞)) ⊂ Fz for some z  ∈ Cn−1 . But any geodesic ray on Fz
which converges to ∞ must be of the given form ν(s) = (ζ, a + Aiecs ), and the
proof of Lemma 1 is complete. 
(1.3) Holomorphic projective connections. For the discussion on holomor-
phic projective connections, we follow Gunning [Gu] and Mok [Mk2]. A holomor-
phic projective connection Π on an n-dimensional complex manifold X, n > 1,
consists of a covering U = {Uα } of coordinate open sets, with holomorphic coordi-
(α) (α)  
nates (z1 , . . . , zn ), together with holomorphic functions α Φkij 1≤i,j,k≤n on Uα

symmetric in i, j satisfying the trace condition k α Φkik = 0 for all i and satisfying
furthermore on Uαβ := Uα ∩ Uβ the transformation rule (†)
(α) $ (β) %
α k ∂zi
(α)
∂zj ∂z (β) ∂z
(α)
∂ 2 zk
β 
Φpq = Φij (β) (β) (α) + (α) (β) (β)
− δ k (αβ)
σ
p q − δ k (αβ)
σ
q p ,
i,j,k ∂zp ∂zq ∂zk  ∂zk ∂zp ∂zq

where the expression inside square brackets defines the Schwarzian derivative
S(fαβ ) of the holomorphic transformation given by z (α) = fαβ (z (β) ), in which
1 ∂
σp(αβ) = logJ(fαβ ) ,
n + 1 ∂zp(β)
 (α) 
∂zi
J(fαβ ) = det (β) being the Jacobian determinant of the holomorphic change
∂zp
of variables fαβ . Two holomorphic projective connections Π and Π on X are said
to be equivalent if and only if there exists a common refinement W = {Wγ } of the
respective open coverings such that for each Wγ the local expressions of Π and Π
agree with each other.  
We proceed to relate holomorphic projective connections α Φkij on a com-
plex manifold to affine connections. An affine connection on a holomorphic vector
bundle E is said to be a complex affine connection if and only if it is compatible
with the Cauchy-Riemann operator ∂ E on E, i.e., if and only if covariant differen-
tiation of smooth sections of E against (0,1)-vectors agree with ∂ E . If a complex
manifold X is endowed with a Kähler metric g, then the Riemannian connection of
(X, g) extends by change of scalars to the field of complex numbers to a connection
286 N. Mok

C C 1,0 0,1
on the complexified tangent bundle TX , with respect to which TX = TX ⊕ TX is
a decomposition into a direct sum of parallel smooth complex vector subbundles.
1,0
When we identify TX with the holomorphic tangent bundle TX the restriction of
1,0
the Riemannian connection to TX is then a complex affine connection on TX .
Given a complex manifold X and a complex affine connection ∇ on the
1,0
holomorphic tangent bundle TX = TX , by conjugation covariant differentiation is
0,1
also defined on T X = TX . Thus, ∇ induces a connection on the underlying real
manifold X. In what follows by an affine connection on a complex manifold X we
will mean a complex affine connection ∇ on the holomorphic tangent bundle TX .
We will say that ∇ is torsion-free to mean that the induced connection on the
underlying real manifold X is torsion-free.
Suppose now X is a complex
 k  manifold equipped with a holomorphic pro-
jective connection. Letting α Γ 
ij be an affine connection on X, we can define a
torsion-free affine connection ∇ on X with Riemann-Christoffel symbols
α k 1 k α  1 k α 
Γij = α Φkij + δi Γj + δj Γi . ()
n+1 n+1
 
We say that ∇ is an affine connection associated to Π. Two affine connections
∇ and ∇ on a complex manifold X are said to be projectively equivalent (cf.
Molzon-Mortensen [MM, §4]) if and only if there exists a smooth (1,0)-form ω
such that ∇ξ ζ − ∇ξ ζ = ω(ξ)ζ + ω(ζ)ξ for any smooth (1,0)-vector fields ξ and ζ on
an open set of X. With respect to two projectively equivalent affine connections
on the complex manifold X, for any complex submanifold S of X, the second
fundamental form of the holomorphic tangent bundle TS as a holomorphic vector
subbundle of TX |S are the same. In particular, the class of (locally closed) complex
geodesic submanifolds S of X are the same. We will say that a (locally closed)
complex submanifold S ⊂ X is geodesic with respect to the holomorphic projective
connection Π to mean that it is geodesic with respect to any affine connection
∇ associated to Π. A geodesic 1-dimensional complex submanifold will simply be
called a complex geodesic. Associated to a holomorphic projective connection there
is a holomorphic foliation F on PTX , called the tautological foliation, defined by
the lifting of complex geodesics. We have (cf. Mok [Mk, (2.1), Proposition 1])
Lemma 2. Let X be a complex manifold and π : PTX → X be its projectivized
holomorphic tangent bundle. Then, there is a canonical one-to-one correspondence
between the set of equivalence classes of holomorphic projective connections on X
and the set of holomorphic foliations F on PTX by tautological liftings of holomor-
phic curves.
From now on we will not distinguish between a holomorphic projective con-
nection and an equivalence class of holomorphic projective connections. Let X be
a complex manifold equipped with a holomorphic projective connection Π, ∇ be
any affine connection associated to Π by means of (), and S ⊂ X be a complex
submanifold. Then, the second fundamental form σ of TS in TX |S is independent
of the choice of ∇. We call σ the projective second fundamental form of S ⊂ X
 Zariski Closure of a Germ of Totally Geodesic Complex Submanifold 287

with respect to Π. Since locally we can always choose the flat background affine
connection it follows that the projective second fundamental form is holomorphic.
Consider now the situation where X is a complex hyperbolic space form, a
complex Euclidean space form, or the complex projective space. X is equipped
with a canonical Kähler metric g of constant negative resp. zero resp. positive
holomorphic sectional curvature. The universal covering space of X is the complex
unit ball B n resp. the complex Euclidean space Cn resp. the complex projective
space Pn (itself), equipped with the canonical Kähler-Einstein metric resp. the Eu-
clidean metric resp. the Fubini-Study metric. For Cn the family of affine lines leads
to a tautological foliation F0 on the projectivized tangent bundle, and g is associ-
ated to the flat holomorphic projective connection. In the case of Pn the projective
lines, which are closures of the affine lines in Cn ⊂ Pn , are totally geodesic with
respect to the Fubini-Study metric g. In the case of B n ⊂ Cn , the intersections
of affine lines with B n give precisely the minimal disks which are totally geodesic
with respect to the canonical Kähler-Einstein metric g. As a consequence, the
tautological foliation F on PTPn defined by the tautological liftings of projective
lines, which is invariant under the projective linear group Aut(Pn ) ∼= PGL(n + 1),
restricts to tautological foliations on Cn resp. B n , and they descend to quotients
Z of Cn resp. B n by torsion-free discrete groups of holomorphic isometries of Cn
resp. B n , which are in particular projective linear transformations. The holomor-
phic projective connection on Pn corresponding to F will be called the canonical
holomorphic projective connection. The same term will apply to holomorphic pro-
jective connections induced by the restriction of F to Cn and to B n and to the
tautological foliations induced on their quotient manifolds X as in the above. Re-
lating the canonical holomorphic projective connections to the canonical Kähler
metric g, we have the following result (cf. Mok [Mk2, (2.3), Lemma 2]).
Lemma 3. Let (X, g) be a complex hyperbolic space form, a complex Euclidean space
form, or the complex projective space equipped with the Fubini-Study metric. Then,
the Riemannian connection of the Kähler metric g induces an affine connection on
X (i.e., a complex affine connection on TX ) which is associated to the canonical
holomorphic projective connection on X. As a consequence, given any complex
submanifold S ⊂ X, the restriction to (1, 0)-vectors of the second fundamental
form on (S, g|S ) as a Kähler submanifold of (X, g) agrees with the projective second
fundamental form of S in X with respect to the canonical holomorphic projective
connection.
In the notation of Lemma 3, from now on by the second fundamental form
σ of S in X we will mean the projective second fundamental form of S in X
or equivalently the restriction to (1,0)-vectors of the second fundamental form of
(S, g|S ) in (X, g) as a Kähler submanifold. Denoting by h the Hermitian metric on
TX induced by the Kähler metric g, the latter agrees with the second fundamental
form of (TS , h|TS ) in (TX |S , h) as a Hermitian holomorphic vector subbundle.
We will make use of Lemma 3 to study locally closed submanifolds of complex
hyperbolic space forms admitting a holomorphic foliation by complex geodesic
288 N. Mok

submanifolds. The first examples of such submanifolds are given by level sets of
the Gauss map. By Lemma 3, it is sufficient to consider the second fundamental
form with respect to the flat connection in a Euclidean space, and we have the
following standard lemma. (For a proof cf. Mok [Mk1, (2.1), Lemma 2.1.3].)

Lemma 4. Let Ω ⊂ Cn be a domain and Z ⊂ Ω be a closed complex submanifold.

At z ∈ Z denote by σz : Tz (Z) × Tz (Z) → NZ|Ω,z the second fundamental form
with respect to the Euclidean flat connection ∇ on Ω. Denote by Ker(σz ) ⊂ Tz (Z)
the complex vector subspace of all η such that σz (τ, η) = 0 for any τ ∈ Tz (Z).
Suppose Ker(σz ) is of the same positive rank d on Z. Then, the distribution z →
Re(Ker(σz )) is integrable and the integral submanifolds are open subsets of d-
dimensional affine-linear subspaces.

With regard to the Gauss map, in the case of projective submanifolds we have
the following result of Griffiths-Harris [GH] according to which the Gauss map is
generically finite on a non-linear projective submanifold. Expressed in terms of the
second fundamental form, we have

Theorem (Griffiths-Harris [GH, (2.29)]). Let W ⊂ PN be a k-dimensional pro-

jective submanifold other than a projective linear subspace. For w ∈ W denote by
σw : Tw (W ) × Tw (W ) → NW |Pn ,w the second fundamental form in the sense of
projective geometry. Then, Ker(σw ) = 0 for a general point w ∈ W .

In Section 2 we will prove a result which includes the dual analogue of the
result above for complex submanifolds of compact complex hyperbolic space forms.
As will be seen, smoothness is not essential for the validity of the dual statement.
For the purpose of studying asymptotic behavior of the second fundamental
form on certain submanifolds of the complex unit ball B n we will need the following
standard fact about the canonical Kähler-Einstein metric ds2B n . We will normalize
the latter metric so that the minimal disks on B n are of constant holomorphic
sectional curvature −2. With this normalization, writing z = (z1 , . . . , zn ) for the
Euclidean coordinates on Cn , and denoting by  · the Euclidean norm, the Kähler
form ωn of ds2Bn is given by ωn = i∂∂(− log(1 − z2 )). We have

Lemma 5. Let n ≥ 1 and (B n , dsB n) be the complex unit n-ball equipped with the
canonical
 Kähler-Einstein
 metric of constant holomorphic sectional curvature −2.
Write gαβ 1≤α,β≤n for the expression of ds2Bn as Hermitian matrices in terms
of the Euclidean coordinates z = (z1 , . . . , zn ). Let t be a real number such that
0 ≤ t ≤ 1. Then, at (t, 0, . . . , 0) ∈ B n we have

1 1
g11 (t, 0, . . . , 0) = ; gαα (t, 0, . . . , 0) = √ for 2 ≤ α ≤ n ;
1 − t2 1 − t2
gβγ (t, 0, . . . , 0) = 0 for β = γ, 1 ≤ β, γ ≤ n .
 Zariski Closure of a Germ of Totally Geodesic Complex Submanifold 289

Proof. The automorphism group Aut(B n ) acts transitively on B n . Especially,

given 0 ≤ t ≤ 1 we have the automorphism Ψt = (ψt1 , . . . , ψtn ) on B n defined by
 √ √ 
z+t 1 − t2 z 2 1 − t2 zn
Ψt (z1 , z2 , . . . , zn ) = , ,..., , (1)
1 + tz 1 + tz 1 + tz

which maps 0 to (t, 0, . . . , 0). We have

∂ψt1 ∂ψtα +
(0) = 1 − t2 ; (0) = 1 − t2 for 2 ≤ α ≤ n ;
∂z1 ∂zα
∂ψtβ
(0) = 0 for β = γ, 1 ≤ β, γ ≤ n . (2)
∂zγ
Since the Kähler form ωn of ds2B n is defined by ωn = i∂∂(− log(1 − z2 ) we have
by direct computation gαβ (0) = δαβ , the Kronecker delta. Lemma 5 then follows
from the invariance of the canonical Kähler-Einstein metric under automorphisms,
as desired. 

Any point z ∈ B n is equivalent modulo a unitary transformation to a point

(t, 0, . . . , 0) where 0 ≤ t ≤ 1. If we write δ(z) = 1 − z on B n for the Euclidean
distance to the boundary ∂B n , then Lemma 5 says that, for a point z ∈ B n the
1
canonical Kähler-Einstein metric grows in the order δ(z) in the normal direction
and in the order √ 1
in the complex tangential directions.
δ(z)

2. Proof of the results

(2.1) Total geodesy of quasi-projective complex hyperbolic space forms holomorphic
foliated by complex geodesic submanifolds. Let X = B n /Γ be a complex hyper-
bolic space form, and denote by π : B n → X the universal covering map. For a
complex-analytic subvariety W ⊂ X we say that W is non-linear if and only if
an irreducible component (thus any irreducible component) W  of π −1 (W ) ⊂ B n
is not the intersection of B with a complex affine-linear subspace of Cn . Equiv-
n

alently, this means that W ⊂ X is not a totally geodesic subset. As one of our
first motivations we were aiming at proving generic finiteness of the Gauss map
for non-linear quasi-projective submanifolds of complex hyperbolic space forms
X = B n /Γ of finite volume. When the Gauss map on a locally closed complex
submanifold of B n fails to be generically finite, on some neighborhood of a general
point of the submanifold we obtain by Lemma 4 a holomorphic foliation whose
leaves are complex geodesic submanifolds, equivalently totally geodesic complex
submanifolds with respect to the canonical Kähler-Einstein metric. We consider
this more general situation and prove first of all the following result which in partic-
ular implies generic finiteness of the Gauss map for arbitrary non-linear irreducible
quasi-projective subvarieties of complex hyperbolic space forms.
290 N. Mok

Proposition 1. Let n ≥ 3, and Γ ⊂ Aut(B n ) be a torsion-free lattice, X := B n /Γ

be the quotient manifold equipped with the canonical structure as a quasi-projective
manifold. Let s, d be positive integers such that s+ d := m < n. Write π : B n → X
for the universal covering map, and let W ⊂ X be an irreducible m-dimensional
quasi-projective subvariety. Denote by W  an irreducible component of π −1 (W ).

Let x ∈ W be a smooth point, U be a neighborhood of x on the smooth locus of W ,
Z ⊂ U be an s-dimensional complex submanifold, and D ⊂ TU be an integrable
d-dimensional holomorphic foliation such that TZ and D are transversal to each
other on Z, i.e., Tz (U ) = Tz (Z) ⊕ Dz for every z ∈ Z, and such that the leaves on
U of the holomorphic foliation F defined by D are totally geodesic on B n . Then,
 ⊂ B n is itself totally geodesic in B n .
W
Proof. Assume first of all that W ⊂ X is compact. We will consider both U and
Z as germs of complex submanifolds of W at x, and thus they may be shrunk
whenever necessary. Write z = (z1 , . . . , zn ) for the Euclidean coordinates on Cn .
We choose now special holomorphic coordinates ζ = (ζ1 , . . . , ζm ) on U at x ∈ Z ⊂
U , as follows. Let (ζ1 , . . . , ζs ) be holomorphic coordinates on a neighborhood of
x in Z. We may choose (ζ1 , . . . , ζs ) to be 0 at the point x ∈ Z, and, shrinking Z
if necessary, assume that the holomorphic coordinates (ζ1 , . . . , ζs ) are everywhere
∼
=
defined on Z, giving a holomorphic embedding f : Δs −→ Z ⊂ U . Again shrinking
Z if necessary we may assume that there exist holomorphic D-valued vector fields
η1 , . . . , ηd on Z which are linearly independent everywhere on Z (hence spanning
the distribution D along Z). Write ζ = (ζ  , ζ  ), where ζ  := (ζ1 , . . . , ζs ) and
ζ  := (ζs+1 , . . . , ζm ). Define now F : Δs × Cd → Cn by
F (ζ  , ζ  ) = f (ζ  ) + ζs+1 η1 (ζ  ) + · · · + ζm ηd (ζ  ) . (1)
By assumption the leaves of the holomorphic foliation F defined by D are totally
geodesic on the unit ball B n . Since the totally geodesic complex submanifolds are
precisely intersections of B n with complex affine-linear subspaces of Cn , F (ζ) lies
on the smooth locus of W  for ζ belonging to some neighborhood of Δs × {0}. Now
F is a holomorphic immersion at 0 and hence everywhere on Δs × Cd excepting for
ζ belonging to some subvariety E  Δs × Cd . Choose now ξ = (ξ  , ξ  ) ∈ Δs × Cd
such that F is an immersion at ξ and such that F (ξ) := p ∈ ∂B n , p = F (ξ). Let G
be a neighborhood of ξ in Δs × Cd and V = B n (p; r) such that F |G : G → Cn is
a holomorphic embedding onto a complex submanifold S0 ⊂ V . Let σ denote the
second fundamental form of the complex submanifold S := S0 ∩B n ⊂ B n . To prove
Proposition 1 it is sufficient to show that σ vanishes identically on S := S0 ∩B n . By
Lemma 3 the second fundamental form σ : S 2 TS → NS|B n agrees with the second
fundamental form defined by the canonical holomorphic projective connection on
S, and that in turn agrees with the second fundamental form defined by the flat
Euclidean connection on Cn . The latter is however defined not just on S but also
on S0 ⊂ V . Thus, we have actually a holomorphic tensor σ0 : S 2 TS0 → NS0 |V such
that σ = σ0 |S . Denote by  ·  the norm on S 2 TS ⊗ NS|V ∩B n induced by ds2Bn .
We claim that, for any point q ∈ V ∩ ∂B n , σ(z) tends to 0 as z ∈ S tends to q.
 Zariski Closure of a Germ of Totally Geodesic Complex Submanifold 291

By means of the holomorphic embedding F |G : G → S ⊂ V identify σ0 with a

holomorphic section of S 2 TG ⊗ F ∗ TV /dF (TG ). For 1 ≤ k ≤ n write k := F ∗ ∂z∂k
and νk := k mod dF (TG ). Then, writing

n
σ0 (ζ) = k
σαβ (ζ) dζ α ⊗ dζ β ⊗ νk (ζ) , (2)
α,β=1

for ζ ∈ G ∩ F −1 (B n ) we have

n
# k #
σ(ζ) ≤ #σαβ (ζ)# dζ α  dζ β  νk (ζ) , (3)
α,β=1

where the norms  ·  are those obtained by pulling back the norms on TV and
on NS|V ∩B n . In what follows we will replace V by B n (p; r0 ) where 0 < r0 < r
k
and shrink G accordingly. Since the holomorphic functions σαβ (ζ) are defined on
a neighborhood of G, they are bounded on G and hence on G ∩ F −1 (B n ). From
Lemma 5 we have
+
dz k  ≤ C1 δ(F (ζ)) (4)
on B n ⊂ Cn . Given q ∈ V ∩ ∂B n , by means of the embedding F |G : G → V ⊂ Cn ,
the holomorphic coordinates (ζ1 , . . . , ζm ) on V can be completed to holomorphic
coordinates (ζ1 , . . . , ζm ; ζm+1 , . . . , ζn ) on a neighborhood of q in B n . Since F is a
holomorphic embedding on a neighborhood of G, expressing each dζ α , 1 ≤ α ≤ m,
in terms of dz k , 1 ≤ k ≤ n, it follows that
+
dζ α  ≤ C2 δ(F (ζ)) (5)

for some positive constant C2 independent of ζ ∈ G, where δ(z) = 1 − z on B n .

(Although the expression of dζ α , 1 ≤ α ≤ m, in terms of dz k , 1 ≤ k ≤ m, depends
on the choice of complementary coordinates (ζm+1 , . . . , ζn ), one can make use of
any choice of the latter locally for the estimates.) On the other hand, again by
Lemma 5 we have
(ζ) ≤ C3 δ(F (ζ)) (6)
for some positive constant C3 . By definition νk (ζ) ≤ k (ζ), and the estimates
(5) and (6) then yield
σ(ζ) ≤ C4 , (7)
for some positive constant C4 for ζ ∈ G ∩ F −1 (B n ). For the claim we need however
to show that σ(ζ) converges to 0 as z = F (ζ) converges to q ∈ V ∩ ∂B n . For
this purpose we will use a better estimate for νk (ζ). Write q = F (μ), μ = (μ , μ ).
Assume that F (0) = 0, that for 1 ≤ i ≤ d we have ηi (μ ) = (0, . . . , 0, 1, 0, . . . , 0)
with 1 in the ith position, and that F (μ) = (1, 0, . . . , 0). We consider now ζt =
292 N. Mok

(μ ; t, 0, . . . , 0), 0 < t < 1, as t approaches to 1. Writing qt = F (ζt ), we have

( (
( ∂ (
(
νk (ζt ) = k mod dF (TG ) = ( mod Tqt (V )(
∂zk (
( (
( ∂ ∂ ( +
≤( (
( ∂zk mod C ∂z1 ( ≤ C δ(qt ) . (8)

To prove the claim we have to consider the general situation of a point q ∈ V ∩∂B n ,
q = F (μ), and consider ζ ∈ (Δs × Cd ) ∩ F −1 (B n ) approaching q. Given such a
point ζ = (ζ  ; ζ  ) there exists an automorphism ϕ of B n such that ϕ(F (ζ  ; 0)) = 0,
ϕ(F (ζ)) = (t, 0, . . . , 0) for some t ∈ (0, 1). Moreover, replacing η1 (ζ  ), . . . , ηd (ζ  )
by a basis of the d-dimensional complex vector space spanned by these d linearly
independent vector fields, we may assume that ηi (ζ  ) = (0, . . . , 0, 1, 0, . . . , 0) with 1
in the ith position. The estimates in (8) then holds true for σ(ζ) at the expense of
introducing some constant which can be taken independent of ζ ∈ V ∩ B n (noting
that G and hence V have been shrunk). As ζ converges to q, t converges to 1 too,
and, replacing the estimate (6) for k (and hence for νk ) by the sharper estimate
(8) we have shown that σ(ζ) → 0 as ζ → q on + V ∩ B n , proving the claim.
(We have actually the uniform estimate σ(ζ) ≤ C (δ(F (z)) for some positive
constant C on V ∩ B n , but this estimate will not be needed in the sequel.)
We proceed now to prove Proposition 1 under the assumption that Γ ⊂
Aut(B n ) is cocompact. Pick any q ∈ V ∩∂B n . Choose a sequence of points zk ∈ V ∩
Bn ⊂ W  converging to q, and we have σ(zk ) → 0 as k → ∞. Since Γ ⊂ Aut(B n )
is cocompact, X is compact, so is W ⊂ X. W = W  /Φ for some discrete subgroup

Φ ⊂ Γ which stabilizes W as a set. Thus, there exists a compact subset K ⊂ W 
−1
and elements ϕk ∈ Φ such that xk := ϕk (zk ) ∈ K. Passing to a subsequence if
necessary we may assume that ϕ−1 k (zk ) converges to some point x ∈ K. Thus

σ(x) = lim σ(xk ) = lim σ(ϕ−1

k (zk )) = lim σ(zk ) = 0 . (9)
k→∞ k→∞ k→∞

Now if y ∈ W  is any point, for the distance function d(·, ·) on (B n , ds2 n ) we have
B
d(ϕk (y), zk ) = d(ϕk (y), ϕk (x)) = d(y, x). Since zk ∈ V ∩ ∂B n converges in Cn to
q ∈ V ∩ ∂B n , from the estimates of the metric ds2B n in Lemma 5 it follows readily
that ϕk (y) also converges to q, showing that σ(y) = lim σ(ϕk (y)) = 0. As
k→∞
a consequence σ vanishes identically on W  , implying that W  ⊂ B n is totally
geodesic, as desired.
It remains to consider the case X = B n /Γ, where Γ ⊂ Aut(B n ) is a torsion-
free non-uniform lattice, and W ⊂ X ⊂ X min is a quasi-projective subvariety. Re-
call that the minimal compactification of X is given by X min = X ) {Q1 , . . . , QN }
where Qj , 1 ≤ j ≤ N , are cusps at infinity. Renumbering the cusps if necessary,
the topological closure W of W ⊂ X min is given by W = W ∪ {Q1 , . . . , QM } for
some nonnegative integer M ≤ N . Pick q ∈ V ∩ ∂B n and let (zk )∞ k=1 be a sequence
of points on V ∩ B n such that zk converges to q. Recall that π : B n → X is the
universal covering map. Either one of the following alternatives occurs. (a) Passing
 Zariski Closure of a Germ of Totally Geodesic Complex Submanifold 293

to a subsequence if necessary π(zk ) converges to some point w ∈ W . (b) There

exists a cusp Q , 1 ≤  ≤ M , such that passing to a subsequence π(zk ) converges
in W to Q . In the case of Alternative (a), picking x ∈ B n such that π(x) = w,
by exactly the same argument as in the case of cocompact lattices Γ it follows
that the second fundamental form σ vanishes on W  and thus W ⊂ B n is totally
geodesic. It remains to treat Alternative (b). Without loss of generality we may
assume that π(zk ) already converges to the cusp Q .
We adopt essentially the notation in (1.2) on Mumford compactifications,
and modifications on the notation will be noted. Let now X M be the Mumford
compactification of X given by X M = X ) (T1 ) · · · ) TN ), where each Tj =
Cn−1 /Λj is an Abelian variety such that the canonical map ρ : X M → X min
collapses Tj to the cusp Qj . Here and henceforth we write Tj for Tbj , Λj for Λbj , etc.
For a (closed) subset A ⊂ X we will denote by AM the topological closure of A in
(N )
X M . Write Ωj for Ωj for some sufficiently large integer N , so that the canonical
projection μj : Ωj → Tj realizes Ωj as a disk bundle over Tj . Write Ω0j := Ωj − Tj .
Considering each π(zk ), 1 ≤ k < ∞, as a point in X ⊂ X M , replacing (zk ) by a
subsequence if necessary we may assume that zk converges to a point P ∈ T .
Recall that the fundamental group of the bundle Ω0 of puncture disks is a semi-
direct product Λ × Ψ of the lattice Λ ∼ = Z2(n−1) with an infinite cyclic subgroup
Ψ := Γ ∩ Ub ⊂ Ub = [Wb , Wb ]. Let D be a simply connected neighborhood
of P ∈ T , and define R := μ−1 ∼
 (D) − T = D × Δ diffeomorphically. Then,


π1 (R) is infinite cyclic. Without loss of generality we may assume that π(zk ) to be
contained in the same irreducible component E of R ∩ W . Consider the canonical
homomorphisms π1 (E) → π1 (R) → π1 (Ω0 ) → π1 (X) = Γ. By the description
of the Mumford compactification the homomorphisms Z ∼ = π1 (R) → π1 (Ω0 ) and
π1 (Ω ) → π1 (X) = Γ are injective. We claim that the image of π1 (E) in π1 (R) ∼
0
=Z
must be infinite cyclic. For the justification of the claim we argue by contradiction.
Supposing otherwise the image must be trivial, and E can be lifted in a univalent
way to a subset E ⊂W  ⊂ B n by a holomorphic map h : E → B n . Let E  be the

normalization of E, and E M be the normalization of E M ⊂ X M . Composing h on
the right with the normalization ν : E  → E we have h : E  → E.  Since E  ⊂ Bn

is bounded, by Riemann Extension Theorem the map h extends holomorphically
 
to h : E M → Cn . Suppose c ∈ E M − E  and h (c) = a ∈ B n . Since π(h(e)) = e
for any e ∈ E it follows that π(h (c)) = c ∈ W M − W , contradicting with the

definition of π : B n → X ⊂ X M . We have thus proven that h (E M ) ⊂ B n with

h (E M − E  ) ⊂ ∂B n , a plain contradiction to the Maximum Principle, proving by
contradiction that the image of π1 (E) → π1 (R)) is infinite cyclic, as claimed. As a
consequence of the claim, the image of π1 (E) in π1 (X) = Γ is also infinite cyclic.
Factoring through π1 (E) → π1 (W ) → π1 (X), and recalling that Φ is the image of
π1 (W ) in π1 (X), the image of π1 (E) in Φ is also infinite cyclic.
Recall that F : U → V ⊂ B n is a holomorphic embedding, ζ = (ζ  , ζ  ) ∈ U ,
q = F (ζ) ∈ V ∩ ∂B n , and assume that Alternative (b) occurs for any sequence
294 N. Mok

of points zk ∈ V ∩ B n converging to q. To simplify notations assume without

loss of generality that F (ζ  , 0) = 0, and define λ0 (t) = F (ζ  , tζ  ) for t ∈ [0, 1].
Then, re-parametrizing λ0 we have a geodesic ray λ(s), 0 ≤ s < ∞ with respect
to ds2B n parameterized by arc-length such that λ(s) converges to q ∈ ∂B n in Cn
as s → ∞. For the proof of Proposition 1, it remains to consider the situation
where Alternative (b) occurs for any choice of divergent sequence (zk ), zk = λ(sk )
with sk → ∞. When this occurs, without loss of generality we may assume that
π(λ([0, ∞))) ⊂ Ω0 , which is an end of X. Now by [(1.2), Lemma 1] all geodesic rays
in Ω0 can be explicitly described,
7 and, in terms of the unbounded8realization Sn of
Bn they lift as a set to (z01 , . . . , z0n−1 , w0 ) : w = u0 + iv, v ≥ v0 for some point
(z01 , . . . , z0n−1 ; u0 + iv0 ) ∈ Sn . We may choose E ⊂ π−1 (R) in the last paragraph
to contain the geodesic ray π(λ([0, ∞))) which converges to the point P ∈ T .
(N ) (N )
Now Ω0 = G /π(Γ ∩ Wb ) for the domain G ⊂ S n which is obtained as
a Cayley transform of B mapping some b ∈ ∂B to ∞. The inverse image
n n

π −1 (π(λ([0, ∞))) is necessarily a countable disjoint union of geodesic rays Ri ,

which is the image of a parametrized geodesic ray ρi : [0, ∞) → B n such that
lims→∞ ρi (s) := ai ∈ ∂B n . For any two of such geodesic rays Ri , Rj there exists
γij ∈ Γ such that Ri = γij (Rj ), hence γij (aj ) = ai . Now both b and q are end
points of such geodesic rays on B n and we conclude that b = γ(q) for some γ ∈ Γ.
In what follows without loss of generality we will assume that q is the same as b .
For q = b ∈ ∂B n , let χ : Sn → B n be the Cayley transform which maps
 (N ) 
the boundary ∂Sn (in Cn ) to ∂B n − {b }. Write χ G , := H . Denote by μ

a generator of the image of π1 (E) in π1 (X) = Γ. Recall that, with respect to
the unbounded realization of B n as the Siegel domain Sn , μ corresponds to an
element μ ∈ Ub ⊂ Wb , where Wb is the normalizer at b (corresponding to ∞
in the unbounded realization Sn ), and Ub is a 1-parameter group of translations.
Conjugating by the Cayley transform, Wb corresponds to Wb whose orbits are
horospheres with b ∈ ∂B n as its only boundary point. Thus for any z ∈ B n , μi (z)
converges to b as i → ∞. Since F is an immersion at ζ, for z ∈ W  , as has been
shown the norm σ(z) of the second fundamental form σ vanishes asymptotically
as z approaches q. From the invariance σ(z) = σ(μi (z)) and the convergence
of μi (z) to q it follows that σ(z) = 0. The same holds true for any smooth point
z  on W  . In fact μi (z  ) converges to b for any point z  ∈ B n as the distance
d(μ (z ), μi (z)) = d(z  , z) with respect to ds2B n is fixed (while the latter metric
i 

blows up in all directions as one approaches ∂B n ). As a consequence, in any event

the second fundamental form σ vanishes identically on W  , i.e., W
 ⊂ B n is totally
geodesic, as desired. The proof of Proposition 1 is complete. 

Remarks. For the argument at the beginning of the proof showing that V ∩ B n
is asymptotically totally geodesic at a general boundary point q ∈ V ∩ ∂B n with
respect to the canonical Kähler-Einstein metric there is another well-known argu-
ment which consists of calculating holomorphic sectional curvature asymptotically,
as for instance done in Cheng-Yau [CY]. More precisely, by direct computation
 Zariski Closure of a Germ of Totally Geodesic Complex Submanifold 295

it can be shown that for a strictly pseudoconvex domain with smooth bound-
ary, with a strictly plurisubharmonic function defining ϕ, the Kähler metric with
Kähler form i∂∂(− log(−ϕ)) is asymptotically of constant holomorphic sectional
curvature equal to −2, which in our situation is enough to imply the asymptotic
vanishing of the norm of the second fundamental form. Here we have chosen to give
an argument adapted to the geometry of our special situation where V ∩ ∂B n is
holomorphically foliated by complex geodesic submanifolds for two reasons. First
of all, it gives an interpretation of the asymptotic behavior of the second fundamen-
tal form which is not easily seen from the direct computation. Secondly, the set-up
of studying holomorphic foliations by complex geodesic submanifolds in which one
exploits the geometry of the Borel embedding B n ⊂ Pn may give a hint to ap-
proach the general question of characterizing Zariski closures of totally geodesic
complex submanifolds in the case of quotients of bounded symmetric domains.
As an immediate consequence of Proposition 1 and Lemma 4, we have the
following result on the Gauss map for finite-volume quotients of the complex unit
ball.
Theorem 1. Let n ≥ 2, and Γ ⊂ Aut(B n ) be a torsion-free lattice, X := B n /Γ.
Equipping X ⊂ X min with the structure of a quasi-projective manifold inherited
from the minimal compactification X min , let W ⊂ X be a quasi-projective subva-
riety. Denote by W0 ⊂ W the smooth locus of W . Let W0 ⊂ B n be an irreducible
−1
component (equivalently, a connected component) of π (W0 )). Then, the Gauss
0 ⊂ B n unless W ⊂ X is a totally
map is of maximal rank at a general point of W
geodesic subset.
Remarks. When W ⊂ X is projective, the total geodesy of W  ⊂ B n in Theorem
1 already follows from the last part of the proof of Cao-Mok [CM, Theorem 1]
(inclusive of Lemma 4.1 and the arguments
 thereafter). We summarize the argu-
ment there, as follows. B n , ds2B n is of constant Ricci curvature −(n + 1). An
m-dimensional locally closed complex submanifold S ⊂ B n is totally geodesic if
and only if it is of constant Ricci curvature  −(m+1).  In general,
 denoting
#  by RicB n
resp. RicS the Ricci curvature form of B n , ds2Bn resp. S, ds2Bn #S , and by ζ =
(ζ1 , . . . , ζm ) local holomorphic coordinates at x ∈ S, we have RicS = m+1 RicB n −ρ,
n n+1
α
where ρ(ζ) = α,β=1 ραβ (ζ)dζ dζ β . In the case of Theorem 1, where S = W  in the
notations of Proposition 1, the holomorphic distribution D is given by the kernel
of the second fundamental form σ, or equivalently by the kernel of the closed (1, 1)-
form ρ. Choosing the local holomorphic coordinates ζ = (ζ1 , . . . , ζs ; ζs+1 , . . . , ζm )
as in the proof of Proposition 1, from the vanishing of ρ|L for the restriction of ρ
to a local leaf of the holomorphic foliation F defined by D, it follows that ραα = 0
whenever α > s. From ρ ≥ 0 it follows that ραβ = 0 for all α > s and for all β (1 ≤
β ≤ m). Coupling with dρ = 0 one easily deduces that ραβ (ζ) = ραβ (ζ1 , . . . , ζs ),
so that ρ is completely determined by its restriction ρ|Z to Z ⊂ U , and the as-
ymptotic vanishing of ρ and hence of σ near boundary points of B n follows from
standard asymptotic estimates of the Kähler-Einstein metric ds2B n .
296 N. Mok

(2.2) Proof of the Main Theorem on Zariski closures of germs of complex geodesic
submanifolds. In the Main Theorem we consider quasi-projective subvarieties
W of complex hyperbolic space forms of finite volume. For the proof of the Main
Theorem first of all we relate the existence of a germ of complex geodesic subman-
ifold S on W with the existence of a holomorphic foliation by complex geodesics
defined on some neighborhood of S in its Zariski closure, as follows.

Proposition 2. Let n ≥ 3, and Γ ⊂ Aut(B n ) be a torsion-free lattice, X := B n /Γ,

which is endowed with the canonical quasi-projective structure. Let W ⊂ X be an
irreducible quasi-projective variety. Let S ⊂ W ⊂ X be a locally closed complex
geodesic submanifold of X lying on W . Then, there exists a quasi-projective sub-
manifold Z ⊂ W such that Z is smooth at a general point of S and such that
the following holds true. There is some subset V ⊂ Z which is open with respect
to the complex topology such that V is non-singular, V ∩ S = ∅, and there is a
holomorphic foliation H on V by complex geodesics such that for any y ∈ V ∩ S,
the leaf Ly of H passing through y lies on S.

Proof. Replacing W by the Zariski closure of S in W , without loss of generality

we may assume that S is Zariski dense in W . In particular, a general point of
S is a smooth point of W , otherwise S ⊂ Sing(W )  W , contradicting with
the Zariski density of S in W . With the latter assumption we are going to prove
Proposition 2 with Z = W . Let x ∈ W and α ∈ PTx (W ) be a non-zero tangent
vector. Denote by Sα the germ of complex geodesic at x such that Tx (Sα ) = Cα.
Define a subset A ⊂ PTX |W as follows. By definition a point [α] ∈ PTx (W )
belongs to A if and only if the germ Sα lies on W . We claim that the subset
A ⊂ PTX |W is complex-analytic. Let x0 ∈ S ⊂ W be a smooth point and U0
be a smooth and simply connected coordinate neighborhood of x in W which is
relatively compact in W . Recall that π : B n → X is the universal covering map.
Let U be a connected component of π −1 (U0 ) lying on W  and x ∈ U  be such
−1 
that π(x) = x0 . Define S = π (S) ∩ U ⊂ W , which is a complex geodesic


submanifold of B n . Identifying U with U0 , we use the Euclidean coordinates on

U as holomorphic coordinates on U0 . Shrinking U0 and hence U if necessary we
may assume that π −1 (W ) ∩ U ⊂ B n is defined as the common zero set of a finite
number of holomorphic functions f1 , . . . , fm on U . Then [α] ∈ A if and only if, in
∼
=
terms of Euclidean coordinates given by π|U : U −→ U0 , writing α = (α1 , . . . , αn )
we have fk (x1 +tα1 , . . . , xn +tαn ) = 0 for all k, 1 ≤ k ≤ m, and for any sufficiently
small complex number t. Consider only the subset G ⊂ TU consisting of non-zero
tangent vectors α of length < 1 with respect to ds2Bn . Varying t we have a family of
holomorphic functions defined on G whose common zero set descends to a subset
A ∩ PTU in PTU , showing that A ∩ PTU ⊂ PTU is a complex-analytic subvariety.
Since the base point x0 ∈ W is arbitrary, we have shown that A ⊂ PTX |W is a
complex-analytic subvariety.
Assume first of all that X is compact. Recall that S ⊂ W ⊂ X is a locally
closed complex geodesic submanifold. Obviously PTS ⊂ A. Let A1 ⊂ A be an
 Zariski Closure of a Germ of Totally Geodesic Complex Submanifold 297

irreducible component of A which contains PTS . Denote by λ : PTX → X the

canonical projection. Consider the subset W1 := λ(E1 ) ⊂ W , which contains S.
By the Proper Mapping Theorem, W1 ⊂ W is a subvariety. Since S ⊂ W1 ⊂ W
and S is Zariski dense in W we must have W1 = W . Thus W ⊂ X = B n /Γ is
an irreducible subvariety in X filled with complex geodesics, a situation which is
the dual analogue of the picture of an irreducible projective subvariety Y ⊂ B
uniruled by lines (cf. Hwang [Hw1] and Mok [Mk4]).
More precisely, let G be the Grassmannian of projective lines in Pn , G ∼ =
Gr(2, Cn+1 ), and K0 ⊂ G be the subset of projective lines  ⊂ Pn such that  ∩ B n
is non-empty, and K ⊂ K0 be the irreducible component which contains the set S
of projective lines  whose intersection with S  contains a non-empty open subset
of . Here B n ⊂ Cn ⊂ Pn gives at the same time the Harish-Chandra embedding
B n ⊂ Cn and the Borel embedding B n ⊂ Pn . Let ρ : U → G be the universal
family of projective lines on Pn , and ρ|K : ρ−1 (K) → K be the restriction of the
universal family to K. We will also write U|A := ρ−1 (A) for any subset A ⊂ G. By
means of the tangent map we identify the evaluation map μ : U → Pn canonically
with the total space of varieties of minimal rational tangents μ : PTPn → Pn . Thus,
μ associates each [α] ∈ PTx (Pn ) to its base point x. Denote by D ⊂ U|K the subset
defined by D := U|K ∩ μ−1 (B n ). Then, there exists some non-empty connected
open subset E ⊂ K containing S such that the image of μ (D ∩ U|E ) contains a
neighborhood U  ⊂ U of x in W .
The subgroup Φ ⊂ Γ acts canonically on U|K and the quotient U|K /Φ is
nothing other than A1 ⊂ PTX |W . Recall that A1 ⊃ PTS . Let A2 ⊂ A1 be the
Zariski closure of PTS in A1 . Again the image λ(A2 ) ⊂ W equals W by the
assumption that S is Zariski dense in W . Moreover, a general point [α] ∈ PTS ,
α ∈ Tx (S), is a smooth point of A2 and λ|A2 : A2 → W is a submersion at [α]. Fix
such a general point [α0 ] ∈ PTx (S) and let 0 be a germ of complex geodesic passing
through x such that Tx (0 ) = Cα0 . Let H ⊂ W be a locally closed hypersurface
passing through x such that α0 ∈ / Tx (H). Shrinking the hypersurface H if necessary
there exists a holomorphic vector field α(w) on H transversal at every point to
H such that α(x) = α0 and α(x ) ∈ Tx (S) for every x ∈ H ∩ S. Then, there is
an open neighborhood V of H admitting a holomorphic foliation H by complex
geodesics such that the leaf Lw passing through w ∈ H obeys Tw (Lw ) = Cα(w),
and such that, as x runs over S ∩H, the family of leaves Lx sweeps through V ∩S.
In particular, for y ∈ V ∩ S the leaf Ly lies on V ∩ S. This proves Proposition 2 in
the case where X is compact.
It remains to consider the case where Γ ⊂ Aut(B n ) is non-uniform, in which
case we will make use of the minimal compactification X ⊂ X min of Satake [Sa]
and Baily-Borel [BB]. When W ⊂ X = B n /Γ is compact the preceding arguments
go through without modification. At a cusp Q ∈ X min the notion of a complex
geodesic submanifold is undefined. In order to carry out the preceding arguments
when W ⊂ X is non-compact so that W contains some cusps Q ∈ X min we
have to work on the non-compact manifolds X and PTX . In the arithmetic case
298 N. Mok

by [Sa] and [BB] the holomorphic tangent bundle TX admits an extension to a

holomorphic vector bundle E defined on X min , and the same holds true for the
non-arithmetic case by the description of the ends of X as given in (1.2). For the
preceding arguments a priori Zariski closures on X ⊂ X min and on PTX ⊂ PE
have to be taken in the topology with respect to which the closed subsets are
complex-analytic subvarieties. We call the latter the analytic Zariski topology.
Nonetheless, since the minimal compactification X min of X is obtained by
adding a finite number of cusps, for any irreducible complex-analytic subvariety of
Y ⊂ X of positive dimension by Remmert-Stein Extension Theorem the topologi-
cal closure Y in X min a subvariety. On the other hand, since dim(PE − PTX ) > 0,
the analogous statement is not a priori true for PTX ⊂ PE. However, given any
complex-analytic subvariety Z ⊂ PTX , by the Proper Mapping Theorem its image
λ(Z) ⊂ X under the canonical projection map λ : PTX → X is a subvariety. Thus
B := λ(Z) ⊂ X ⊂ X min is quasi-projective. In the preceding arguments in which
one takes Zariski closure in the compact case, for the non-compact case it remains
the case that the subsets B ⊂ X on the base manifold are quasi-projective. In the
final steps of the arguments in which one obtains a subset A2 ⊂ PTX |W , A2 ⊃ PTS ,
such that a general point [α] ∈ A2 ∩ PTX |U over a neighborhood U of some x ∈ S
in W is non-singular and λ|A2 is a submersion at [α], A2 was used only to produce
a holomorphically foliated family of complex geodesics of X over some neighbor-
hood V of x ∈ S in W , and for that argument it is not necessary for A2 to be
quasi-projective. Thus, the arguments leading to the proof of Proposition 2 in the
compact case persist in the general case of W ⊂ X = B n /Γ for any torsion-free
lattice Γ ⊂ Aut(B n ), and the proof of Proposition 2 is complete. 
Finally, we are ready to deduce the Main Theorem from Proposition 2, as
follows.
Proof of the Main Theorem. Recall that X = Ω/Γ is a complex hyperbolic space
form of finite volume, W ⊂ X ⊂ X min is a quasi-projective subvariety, and S ⊂
X is a complex geodesic submanifold lying on W . As explained in the proof of
Proposition 2, the closure Z ⊂ X of S in W with respect to the analytic Zariski
topology is quasi-projective, so without loss of generality we may replace W by
its Zariski closure with respect to the usual Zariski topology and proceed under
that convention with proving that W is totally geodesic. By Proposition 2, there
exists some x ∈ S which is a non-singular point on W and some nonsingular open
neighborhood V of x in W which admits a holomorphic foliation F by complex
geodesics (such that the leaves of F passing through any y ∈ V ) lies on S ∩ V .
By Proposition 2, V ⊂ X is totally geodesic. As a consequence, W ⊂ X is totally
geodesic subset, i.e., the non-singular locus W0 of W is totally geodesic in X, as
desired. 
Remarks. In the proof of Proposition 2 we remarked that any irreducible complex-
analytic subvariety Z ⊂ X of positive dimension extends by Remmert-Stein Ex-
tension Theorem to a complex-analytic subvariety in X min . As a consequence, in
 Zariski Closure of a Germ of Totally Geodesic Complex Submanifold 299

the hypothesis of the Main Theorem, in place of assuming W ⊂ X to be an irre-

ducible quasi-projective subvariety we could have assumed that W ⊂ X is simply
an irreducible complex-analytic subvariety.

Acknowledgement
The author wishes to thank Jun-Muk Hwang for having raised questions on the
analogue of the Gauss map for subvarieties of complex hyperbolic space forms,
which was one of the original motivations for the author to formulate general
questions on the study of totally geodesic complex submanifolds on such subva-
rieties. The author’s interest in such and related questions was rekindled after
a recent joint work with Vincent Koziarz related to mappings between complex
hyperbolic space forms, and he wishes to thank the latter for his interest in the
circle of problems related to the current article as expressed during mutual visits at
the University of Hong Kong and Université de Nancy. The author also wishes to
thank the organizers of the Conference on Complex Analysis held in July 2008 at
the University of Fribourg in honor of Professor Linda Rothschild. He is delighted
to dedicate this article to Linda on this joyous occasion.

Added in proof
Most recently the author has noticed that a proof of Theorem 1 had already been
given in Hwang [Hw2].

References
[BB] Baily, W.L., Jr. and Borel, A.: Compactification of arithmetic quotients of
bounded symmetric domains, Ann. Math. 84 (1966), 442–528.
[CM] Cao, H.-D. and Mok, N.: Holomorphic immersions between compact hyperbolic
space forms, Invent. Math. 100 (1990), 49–61.
[CY] Cheng, S.Y. and Yau, S.T.: On the existence of a complete Kähler metric on
noncompact complex manifolds and the regularity of Fefferman’s equation, Comm.
Pure Appl. Math. 33 (1980), 507–544.
[Ei] Ein, L.: The ramification divisors for branched coverings Pn
k , Math. Ann 261
(1982), 483–485.
[GH] Griffiths, P. and Harris, J.: Algebraic geometry and local differential geometry,
Ann. Scient. Ec. Norm. Sup. 12 (1979), 355–452.
[Gu] Gunning R.: On Uniformization of Complex Manifolds, Mathematical Notes 22,
Princeton university, Princeton 1978.
[Ha] Hain, R.: Locally symmetric families of curves and Jacobians, in Moduli of curves
and abelian varieties, 91–108, Aspects Math., E33, Vieweg, Braunschweig 1999.
[Hw1] Hwang, J.-M.: Geometry of minimal rational curves on Fano manifolds, School on
Vanishing Theorems and Effective Results in Algebraic Geometry (Trieste 2000),
ICTP Lect. Notes, 6, Abdus Salam Int. Cent. Theoret. Phys., Trieste 2001, pp.
335–393.
300 N. Mok

[Hw2] Hwang, J.-M.: Varieties with degenerate Gauss mappings in complex hyperbolic
space forms, Intern. J. Math. 13 (2002), 209–216.
[Mk1] Mok, N.: G-structures on irreducible Hermitian symmetric spaces of rank ≥ 2
and deformation rigidity, Contemp. Math. 222 (1999), 81–107.
[Mk2] Mok, N.: On holomorphic immersions into Kähler manifolds of constant holo-
morphic sectional curvature, Science in China Ser. A 48 Supp. (2005), 123–145.
[Mk3] Mok, N.: Geometric structures on uniruled projective manifolds defined by their
varieties of minimal rational tangents, Proceedings of the Conference “Géometrie
différentielle, Physique mathématique, Mathématique et Société”, Astérisque 322
(2008), Volume II, 151–205, published by Société Mathématique de France, Paris.
[Mk4] Mok, N.: Projective-algebraicity of minimal compactifications of complex-
hyperbolic space forms of finite volume, Preprint 2009.
URL: http://hkumath.hku.hk/∼imr/IMRPreprintSeries/2009/IMR2009-4.pdf/.
[MM] Molzon, R. and Mortensen, K.: The Schwarzian derivative for maps between
manifolds with complex projective connections, Trans. AMS 348 (1996), 3015–
3036.
[Sa] Satake, I.: On compactifications of the quotient spaces for arithmetically defined
discontinuous groups, Ann. Math. 72 (1960), 555–580.
[SY] Siu, Y.-T. and Yau, S.-T.: Compactification of negatively curved complete
Kähler manifolds of finite volume, in Seminar on Differential Geometry, ed. Yau,
S.-T., Annals of Mathematics Studies, Vol. 102, pp. 363–380, Princeton Univ.
Press, Princeton, 1982.
[Za] Zak, F.L.: Tangents and secants of algebraic varieties, Translations of Mathe-
matical Monographs, Vol. 127, Amer. Math. Soc., Providence 1993.

Ngaiming Mok
The University of Hong Kong
Pokfulam
Hong Kong
e-mail: [email protected]
Complex Analysis
Trends in Mathematics, 301–306

c 2010 Springer Basel AG

The Large Time Asymptotics of the Entropy

Lei Ni

Abstract. In this note we supply the detailed proof of the entropy asymptotics
on manifolds with nonnegative Ricci curvature. We also discuss the possible
connections between the large time behavior of the entropy and the existence
of harmonic functions.

Mathematics Subject Classification (2000). 58G11.

Keywords. Heat kernel, entropy, harmonic functions.

In this note, due to several requests after the publication of [N2], as well as new
potential applications emerging, we supply the details of the computation on the
asymptotics of the entropy stated in [N1], Corollary 4.3, and Proposition 1.1 of
[N2]. It is my pleasure to contribute this paper to the birthday of Professor Roth-
schild.
In this note we assume that (M, g) is a complete Riemannian manifold of
dimension n with nonnegative Ricci curvature. We also further assume that it has
maximum volume growth. Namely

Vx (r)
lim > 0,
r→∞ rn

where Vx (r) is the volume of the ball of radius r centered at x. Let H(x, y, t) be
the heat kernel, or equivalently the minimum positive fundamental solution to the
∂
heat operator ∂t − Δ. For this note we fix x and write simply abbreviate H(x, y, t)
as H(y, t), and Vx (r) as V (r). Also we simply denote the distance between x and
y by r(y). Recall the definition of the Perelman’s entropy

 
W(H, t) := t|∇f |2 + f − n H dμ
M

where f = − log H − n
2log(4πt). Also recall the Nash entropy

n n
N (H, t) = − H log H dμ − log(4πt) − .
M 2 2
302 L. Ni

Let ν∞ be the cone angle at infinity which can be defined by

θ(r)
ν∞ := lim
r→∞ ωn

where θ(r) := Vr(r)

n , V (r) is the volume of the ball Bx (r) centered at x and ωn is

the volume of the unit ball in Rn .

Recall that in [N2], page 331, there we have proved that
lim W(H, t) = lim N (H, t).
t→∞ t→∞
Moreover, it was also proved in [N1] that M is of maximum volume growth if and
only if limt→∞ W(H, t) > −∞. The main purpose here is to supply the detail of
the following claim.
Theorem 1. For any x ∈ M ,
lim W(H, t) = lim N (H, t) = log ν∞ .
t→∞ t→∞
Besides that it is interesting to be able to detect the geometric information
such as the volume ratio ν∞ from the large time behavior of the entropy, there
appeared an extra indication of possible uses of such a result. In an very interesting
paper [Ka], Kaimanovich defined an entropy h(M ) on the space of so-called mini-
mal Martin boundary of the positive harmonic functions on the universal/a regular
covering space M of a compact Riemannian manifold. More precisely, let H(M ) be
the vector space with the seminorms uK = supK |u|, where K is a compact sub-
set. Let Kp = {u |u ∈ H(M ), u(p) = 1, u > 0}. This is a convex compact subset.
Define the minimal Martin boundary of M  by ∂ ∗ M
 = {u ∈ Kp , and u minimal}.
Here u > 0 is called minimal if for any nonnegative harmonic function h ≤ u, h
must be a multiple of u. This immediately implies the representation formula: for
any positive harmonic function f , there exists a Borel measure μf on ∂ ∗ M  such
that 
f (x) = u(x) dμf (u).

∂∗M
In particular, there exists a measure ν corresponding to f ≡ 1. Now u(x)dν(u)
 can be identified with the prob-
is also a probability measure. The points x ∈ M
∗
ability measure u(x)dν(u) on ∂ M . The so-called relative entropy is defined to
 u(x)
be φ(x, y) = − ∂ ∗ M log u(y) u(y) dν(u). By Jensen’s inequality it is easy to see
that φ(x, y) ≥ 0. The Kaimanovich’s entropy is defined by averaging φ(x, y) as
following: First check that

1
φ(x, y)H(τ, x, y) dμM (y)
τ M 

is a function independent of τ and also descends to M . Here H(τ, x, y) is the heat

kernel of M. Then define

 
)  1
h(M φ(x, y)H(τ, x, y) dμM
 (y) dμM (x).
M τ M 
 The Large Time Asymptotics of the Entropy 303

Here we normalize so that μM (x) is a probability measure on M . From the above

definition it was proved by Kaimanovich [Ka] (see also [W]) that
 

h(M ) = u(x)|∇ log u(x)|2 dν(u) dμ(x).
M 
M

Hence the positivity of h(M) implies the existence (in fact ampleness) of noncon-
stant positive harmonic functions on M . Most interestingly, the Theorem 2 of [Ka]
asserts that

 1
h(M ) = − lim H(t, x, y) log H(t, x, y) dμ(y)
t→∞ t M 

for any x. This is sensational since it is not hard to check for instance that when
λ(M), the bottom of the L2 -spectrum of the Laplace operator on M  is positive,
 
h(M ) ≥ 4λ(M ) [L]. In fact the above result of Kaimanovich plays a crucial role
in [W], where Xiaodong Wang solves a conjecture of Jiaping Wang on characteriz-
ing the universal cover of a compact Riemannian manifolds being the hyperbolic
space by the information on λ(M ) and the lower bound of Ricci curvature of M .
(See also [Mu] for the related work on Kähler manifolds.) Motivated by the above
discussions, mainly the entropy of Kaimanovich [Ka] and its above-mentioned con-
nection with the large time behaviors of the heat kernel and its direct implications
on the existence of harmonic functions, we propose the following problem.
Problem 1. Let M be a complete Riemannian manifold of positive sectional curva-
ture. When does M admit non-constant harmonic functions of polynomial growth?
Here a harmonic function u(x) is of polynomial growth if there exist positive
constants d, Cu such that
|u(y)| ≤ Cu (1 + r(y))d .
We conjecture that the necessary and sufficient condition is that limt→∞ N (H, t) >
−∞, namely M is of maximum volume growth. Note that for the corresponding
problem on holomorphic functions of polynomial growth, the sufficient part has
been solved by the author provided that the manifold is Kähler with bounded
non-negative bisectional curvature (cf.[N4]).
Now we devote the rest of the paper to the detailed proof of Theorem 1.
Under the above notation, let us first recall a result of Li, Tam and Wang [LTW].
A computation similar as below appeared in the earlier paper [N3], pages 935–936.
Theorem 2. [Li-Tam-Wang] Let (Mn , g) be a complete Riemannian manifold with
nonnegative Ricci curvature and maximum volume growth. For any δ > 0, the heat
kernel of (Mn , g) satisfies


ωn n 1 + 9δ 2
(4πt)− 2 exp − r (y) ≤ H(y, t)
θ(δr(y)) 4t


ωn n 1−δ 2
≤ (1 + C(n, θ∞ )(δ + β)) (4πt)− 2 exp − r (y) ,
θ∞ 4t
304 L. Ni

where θ∞ = limr→∞ θ(r),


−2n θx (r)
β := δ max 1− .
r≥(1−δ)r(y) θx (δ 2n+1 r)

Note that β is a function of r(y) and

lim β = 0.
r(y)→∞

Therefore, for any  > 0, there exists a B sufficiently large such that if
r(y) ≥ B we have
n  
ωn
θ∞
(1 − )(4πt)− 2 exp − 1+9δ
4t
r2 (y) ≤ H(y, t)
n  
≤ (1 + C(n, θ∞ )(δ + )) θω∞n (4πt)− 2 exp − 1−δ
4t
r2 (y) . (1)
We can also require that θ(δ 2n+1 r) ≤ (1 + )θ∞ .
The upper estimates: First by the lower estimate of Li-Tam-Wang,

 

ωn 1 + 9δ 2 n
N (H, t) ≤ − H log dμ + H r (y) dμ −
M θ(δr(y)) M 4t 2
n
= I + II − .
2
We shall estimate I and II below as in [N3]. Split
 B  ∞ 
 
ωn
I=− − H log dA ds = I1 + I2 .
0 B ∂B(s) θ(δr(y))
It is easy to see that
lim I1 ≤ 0.
t→∞
To compute II2 we now make use of the lower estimate in (1) to have that
 ∞


ωn −n 1 + 9δ 2 ωn
I2 ≤ −(1 − ) (4πt) 2 exp − s log dA ds
θ∞ B ∂B(s) 4t θ(δs)

  ∞

(1 + )θ∞ n 1 + 9δ 2 n−1
≤ log (1 − )nωn (4πt)− 2 exp − s s ds.
ωn B 4t
Here we have used that θ(δr(y)) ≤ θ∞ (1+) and the surface area of ∂B(s) satisfies
A(s) ≥ nθ∞ sn−1 . Computing the integral via the change of variable τ = 1+9δ
4t
s2
and taking t → ∞ we have that


(1 + )θ∞ −n/2
lim I2 ≤ log (1 − ) (1 + 9δ) .
t→∞ ωn
The estimate of II is very similar. Using the Gamma function identity
n n n
Γ( + 1) = Γ( )
2 2 2
we can have that
n
lim II ≤ (1 + )(1 + C(n, θ∞ )(δ + ))(1 + 9δ)(1 − δ)n/2+1 .
t→∞ 2
 The Large Time Asymptotics of the Entropy 305

Summarizing we have that


(1 + )θ∞ −n/2
lim N (H, t) ≤ log (1 − ) (1 + 9δ)
t→∞ ωn
n n
+(1 + )(1 + C(n, θ∞ )(δ + ))(1 + 9δ)(1 − δ)n/2+1 − .
2 2
Letting  → 0, then δ → 0 we have what
lim N (H, t) ≤ log ν∞ .
t→∞

For the lower estimate, we use the other inequality provided by Theorem 2,
mainly (1). First write

 $ %
1−δ ωn n
N (H, t) ≥ H r2 (y) dμ − log (1 + C(n, θ∞ )(δ + )) −
M 4t θ∞ 2
 
 $ %
1−δ ω n n
= + H r2 (y) dμ − log (1 + C(n, θ∞ )(δ + )) − .
r≤B r≥B 4t θ∞ 2
Similarly 

1−δ
lim H r2 (y) dμ → 0
t→∞ r≤B 4t
as t → ∞. On the other hand, using that A(r) ≥ nθ∞ rn−1 ,


1−δ
lim I3  H r2 (y) dμ
t→∞ r≥B 4t
 ∞

1 (1 + 9δ)r2 r2 n−1
≥ lim nωn (1 − )(1 − δ) exp − r dr.
t→∞ (4πt)n/2 B 4t 4t
The direct calculation shows that
 ∞

1 (1 + 9δ)r2 r2 n−1 1 n n
lim n/2
exp − r dr = Γ( + 1)(1 + 9δ)− 2 −1 .
t→∞ (4πt) B 4t 4t 2 2
2π n/2 π n/2
Using ωn = Γ( n = Γ( n , we finally have that
2 )n 2 +1)

n n
lim I3 ≥ (1 − )(1 − δ)(1 + 9δ)− 2 −1 .
t→∞ 2
The lower estimate
lim N (H, t) ≥ log ν∞
t→∞
follows after taking  → 0 and then δ → 0.

References
[Ka] V.A. Kaimanovich, Brownian motion and harmonic functions on covering man-
ifolds. An entropy approach. Soviet Math. Dokl. 33 (1986), 812–816.
[L] F. Ledrappier, Harmonic measures and Bowen-Margulis measures. Israel J. Math.
71(1990), 275–287.
306 L. Ni

[LTW] P. Li, L.-F. Tam and J. Wang, Sharp bounds for the Green’s function and the
heat kernel, Math. Res. Lett. 4 (1997), no. 4, 589–602.
[Mu] Munteanu, O., On a characterization of the complex hyperbolic space.
ArXiv:0802.0307.
[N1] L. Ni, The entropy formula for linear heat equation, J. Geom. Anal. 14(2004),
87–100.
[N2] L. Ni, Addenda to “The entropy formula for linear heat equation”, J. Geom.
Anal. 14(2004), 369–374.
[N3] L. Ni, A monotonicity formula on complete Kähler manifolds with nonnegative
bisectional curvature, J. Amer. Math. Soc. 17 (2004), 909–946.
[N4] L. Ni, Ancient solutions to Kähler-Ricci flow. Math. Res. Lett. 12 (2005), no. 5-6,
633–653.
[W] X.-D. Wang, Harmonic functions, entropy, and a characterization of the hyper-
bolic space, preprint.

Lei Ni
Department of Mathematics
University of California at San Diego
La Jolla, CA 92093, USA
e-mail: [email protected]
Complex Analysis
Trends in Mathematics, 307–320

c 2010 Springer Basel AG

The Closed Range Property for ∂ on Domains

with Pseudoconcave Boundary
Mei-Chi Shaw

Dedicated to Professor Linda P. Rothschild

¯
Abstract. In this paper we study the ∂-equation on domains with pseudocon-
cave boundary. When the domain is the annulus between two pseudoconvex
domains in Cn , we prove L2 existence theorems for ∂¯ for any ∂-closed
¯ (p, q)-
form with 1 ≤ q < n − 1. We also study the critical case when q = n − 1 on the
annulus Ω and show that the space of harmonic forms is infinite dimensional.
Some recent results and open problems on pseudoconcave domains in complex
projective spaces are also surveyed.

Mathematics Subject Classification (2000). 32W05, 35N15, 58J32.

¯
Keywords. Cauchy-Riemann equations, pseudoconcave domains, ∂-Neumann
operator.

1. Introduction
In the Hilbert space approach, the closed range property for an unbounded closed
operator characterizes the range of the operator. Thus it is important to know
whether the range of an unbounded operator is closed. When the unbounded op-
erator is the Cauchy-Riemann equation, the Hilbert space approach has been es-
tablished by the pioneering work of Kohn [Ko1] for strongly pseudoconvex domains
and by Hörmander [Hör1] for pseudoconvex domain in Cn or a Stein manifold. The
following L2 existence and regularity theorems for ∂¯ on pseudoconvex domains in
Cn (or a Stein manifold) are well known.
Theorem (Hörmander [Hör1]). Let Ω ⊂⊂ Cn be a pseudoconvex domain. For any
¯ = 0 in Ω, there exists
f ∈ L2(p,q) (Ω), where 0 ≤ p ≤ n and 1 ≤ q < n, such that ∂f

Partially supported by NSF grant.

308 M.-C. Shaw

¯ = f and
u ∈ L2(p,q−1) (Ω) satisfying ∂u
 
eδ 2
|u|2 ≤ |f |2
Ω q Ω
where δ is the diameter of Ω.
This implies that the range of ∂¯ is equal to the kernel of ∂,¯ which is closed
since ∂¯ is a closed operator. It also follows that the harmonic forms are trivial for
1 ≤ q ≤ n. Furthermore, if the boundary bΩ is smooth, we also have the following
global boundary regularity results for ∂. ¯

Theorem (Kohn [Ko2]). Let Ω ⊂⊂ Cn be a pseudoconvex domain with smooth

boundary bΩ. For any f ∈ W(p,q)
s
(Ω), where s ≥ 0, 0 ≤ p ≤ n and 1 ≤ q < n, such
¯
that ∂f = 0 in Ω, there exists u ∈ W(p,q)
s ¯ = f.
(Ω) satisfying ∂u
¯
In this paper we study the ∂-equation on domains with pseudoconcave bound-
ary. When the domain is the annulus between two pseudoconvex domains in Cn ,
the closed range property and boundary regularity for ∂¯ were established in the
author’s earlier work [Sh1] for 0 < q < n − 1 and n ≥ 3. In this paper, we will
study the critical case when q = n − 1 on the annulus Ω. In this case the space
of harmonic forms is infinite dimensional. We also show that in the case when
0 < q < n − 1, the space of harmonic forms is trivial. This improves the ear-
lier results in [Sh1] where only finite dimensionality for the harmonic forms has
been established. We first study the closed range property for the case when the
annulus is between two strictly pseudoconvex domains in Section 2. This simpler
case warrants special attention since it already illuminates the difference between
q < n − 1 and q = n − 1. Then we study the case when the annulus is between
two weakly pseudoconvex domains in Section 3. Special attention is given to the
case when n = 2 and q = 1. In Section 4 we survey some known existence and
regularity results for ∂¯ on pseudoconcave domains with Lipschitz domains in the
complex projective space CP n when n ≥ 3. The closed range property for ∂¯ for
(0, 1)-forms with L2 coefficients on pseudoconcave domains in CP n is still an open
problem (see Conjecture 1 and Conjecture 2 at the end of the paper). Very little
is known on the pseudoconcave domains in CP 2 .
The author would like to thank professor Lars Hörmander who first raised
this question to the author on the closed range property for ∂¯ on the annulus for
the critical degree. This paper is greatly inspired by his recent paper [Hör2]. She
would also like to thank professor Emil Straube for his comments on the proof of
Theorem 3.2.

¯
2. The ∂-equation on the annulus between two strictly
pseudoconvex domains in Cn
Let Ω ⊂⊂ Cn be the annulus domain Ω = Ω1 \ Ω2 between two strictly pseudocon-
vex domains Ω2 ⊂⊂ Ω1 with smooth boundary. In this section, we study the L2
 The Closed Range Property for ∂ 309

¯
existence for ∂¯ on Ω. We first prove the L2 existence theorem of the ∂-Neumann
operator for the easier case when q < n − 1. Let ∂¯∗ be the Hilbert space adjoint
¯ As before, we formulate the ∂-Laplacian
of ∂. ¯  = ∂¯∂¯∗ + ∂¯∗ ∂¯ and the harmonic
space of (p, q)-forms H(p,q) is defined as the kernel of .

Theorem 2.1. Let Ω ⊂⊂ Cn be the annulus domain Ω = Ω1 \ Ω2 between two

strictly pseudoconvex domains Ω2 ⊂⊂ Ω1 with smooth boundary.
¯
The ∂-Neumann operator N (p, q) exists and N (p, q) : L2(p,q) (Ω) → W(p,q)
1
(Ω),
where 0 ≤ p ≤ n, 1 ≤ q ≤ n − 2. The space of harmonic space H(p,q) is finite
dimensional.
Proof. Recall that condition Z(q) means that the Levi form has at least n − q
positive eigenvalues or q + 1 negative eigenvalues. From our assumption, the Levi
form for the boundary bΩ2 has n − 1 negative eigenvalues at each boundary point.
Thus it satisfies condition Z(q) for 0 ≤ q < n − 1. The Levi form on bΩ1 satisfies
condition Z(q) for 0 < q < n. Thus we have the estimates (see [Hör1], [FK] or
[CS])
¯ 2 + ∂¯∗ f 2 + f 2),
f 21 ≤ C(∂f ¯ ∩ Dom(∂¯∗ ).
f ∈ L2(p,q) (Ω) ∩ Dom(∂)
2

¯
This gives the existence of the ∂-Neumann operator N(p,q) : L2(p,q) (Ω) → W(p,q)
1
(Ω)
and the harmonic space H(p,q) is finite dimensional. 

Theorem 2.2. Let Ω be the same as in Theorem 2.1. For any ∂-closed ¯ f
1
2 ¯
∈ L(p,q) (Ω) ∩ Ker(∂), where 0 ≤ p ≤ n, 1 ≤ q ≤ n − 2, there exists u ∈ W(p,q−1) (Ω)
2

¯ = f . The space of harmonic (p, q)-forms H(p,q) is trivial when 1 ≤ q ≤

with ∂u
n − 2.
¯ and f ⊥ H(p,q) , there
Proof. From Theorem 2.1, for any f ∈ L2(p,q) (Ω) ∩ Ker(∂)
1
exists a u ∈ W(p,q−1)
2 ¯ = f.
(Ω) such that ∂u
To show that the harmonic forms are trivial, notice that from the regularity
¯
of the ∂-Neumann operator in Theorem 2.1, the harmonic forms are smooth up to
the boundary. Also to prove Theorem 2.2, it suffices to prove the a priori estimates
¯
for smooth ∂-closed forms f since they are dense.
2
Let N(p,q) ¯
denote the ∂-Neumann operator on the strongly pseudoconvex
¯ ∞
domain Ω2 . For any smooth ∂-closed f ∈ C(p,q) (Ω), we first extend f from Ω
˜
smoothly to f in Ω1 . We have that
∂¯f˜W −1 (Ω2 ) ≤ Cf˜L2(p,q) (Ω1 ) ≤ Cf L2(p,q) (Ω) .
(p,q+1)

¯ Ω2
Let v = −  ∂N ¯˜ ¯ ¯˜
(n−p,n−q−1)  ∂ f . We have ∂v = ∂ f in the distribution sense
in Ω1 if we extend v to be zero outside Ω2 since q < n − 1 (see Theorem 9.1.2 in
the book by Chen-Shaw [CS]). Also v satisfies
v 1 ≤ C∂¯f˜W −1 (Ω2 ) .
W − 2 (Ω2 ) (p,q+1)
310 M.-C. Shaw

Setting F = f˜ − v, we have that ∂F

¯ = 0 in Ω1 and F = f on Ω. Thus we
¯
have extended f as a ∂-closed form F in Ω1 and
F  −1
2
≤ Cf L2(p,q) (Ω) .
W(p,q) (Ω1 )

¯ with U ∈ W 12 (Ω1 ). Setting u = U |Ω , we have ∂u

Thus F = ∂U ¯ = f with

u 2
1 ≤ Cf L2(p,q) (Ω) .
W(p,q−1) (Ω)

¯
For general ∂-closed f ∈ L2(p,q) (Ω), we approximate f by smooth forms to obtain
1
a solution u ∈ W(p,q−1)
2 ¯ = f.
(Ω) satisfying ∂u
¯ = ∂¯∗ f = 0 in Ω. This means that f = ∂u
If f ∈ H(p,q) (Ω), we have ∂f ¯ and
¯ = (u, ∂¯∗ f ) = 0.
¯ ∂u)
f 2 = (∂u,
Thus we have H(p,q) (Ω) = {0}. 

Next we discuss the case when q = n−1, where condition Z(q) is not satisfied
on bΩ2 .
Theorem 2.3. Let Ω ⊂⊂ Cn be the annulus domain Ω = Ω1 \ Ω2 between two
strictly pseudoconvex domains Ω2 ⊂⊂ Ω1 with smooth boundary, where n ≥ 3.
For any 0 ≤ p ≤ n, the range of ∂¯ : L2(p,n−2) (Ω) → L2(p,n−1) (Ω) is closed and
¯
the ∂-Neumann ¯ ∩
operator N(p,n−1) exists on L2(p,n−1) (Ω). For any f ∈ Dom(∂)
∗ ⊥
Dom(∂¯ ) ∩ (H(p,n−1) ) , we have
¯ 1 2 + ∂¯∗ f2 2 ).
f 21 ≤ C(∂f (2.1)
2

Moreover, for any f ∈ L2(p,n−1) (Ω),

N(p,n−1) f  21 ≤ Cf ,
¯ (p,n−1) f  1 + ∂¯∗ N(p,n−1) f  1 ≤ Cf .
∂N 2 2

Proof. We note that for any domain, one can always solve ∂¯ for the top degree
and condition Z(n) is a void condition. When n ≥ 3, Ω satisfies both condition
¯
Z(n − 2) and Z(n), thus from Theorem (3.1.19) in [FK], the ∂-equation has closed
¯
range and ∂-Neumann operator exists for (p, n − 1)-forms. For f ∈ L2(p,n−1) (Ω),
we have
f = ∂¯∂¯∗ N(p,n−1) f + ∂¯∗ ∂N
¯ (p,n−1) f + H(p,n−1) f
where H(p,n−1) f is the projection onto the harmonic space H(p,n−1) .
We note that for any (p, n − 1)-form f ∈ Dom(∂) ¯ ∩ Dom(∂¯∗ ) ∩ (H(p,n−1) )⊥ ,
we write f = f1 + f2 with f1 ⊥ Ker(∂) ¯ and f2 ⊥ Ker(∂¯∗ ). Then ∂f¯ = ∂f¯ 1 and
∗ ∗
∂¯ f = ∂¯ f2 and we have (see Proposition 3.1.18 in [FK])
f = ∂¯∗ N(p,n) ∂f ¯ (p,n−2) ∂¯∗ f2 .
¯ 1 + ∂N (2.2)
 The Closed Range Property for ∂ 311

Using the regularity for N(p,n) and N(p,n−2) , we have

f 21 ≤ 2(∂¯∗ N(p,n) ∂f ¯ (p,n−2) ∂¯∗ f2 21 )
¯ 1 21 + ∂N
2 2 2

¯ 1 2 + ∂¯∗ f2 2 )
≤ C(∂f
¯ ∂f
= C((∂f, ¯ ) + (∂¯∗ f, ∂¯∗ f )). (2.3)
If we assume that, in addition, f is in Dom(), then
¯ ∂f
f 21 ≤ C((∂f, ¯ ) + (∂¯∗ f, ∂¯∗ f )) = C(f, f ) ≤ Cf f . (2.4)
2

Setting f = N(p,n−1) φ for some φ ∈ L2(p,n−1) (Ω), we have from (2.3) that
N φ21 ≤ CN φN φ ≤ C 2 φ2 . (2.5)
2
1
Thus the operator N(p,n−1) is a bounded operator from L2 to W . To show that 2

¯ (p,n−1) and ∂¯∗ N(p,n−1) is bounded from L2 to W 12 , we use the fact that (2.1)
∂N
holds for both q = n − 2 and q = n. Substituting f in (2.1) by ∂N ¯ (p,n−1) f and
¯∗
∂ N(p,n−1) f , then we have
∂¯∗ N f 21 ≤ C∂¯∂¯∗ N f 2 , f ∈ L2 (Ω),(p,n−1) (2.6)
2

¯ f 21 ≤ C∂¯∗ ∂N
∂N ¯ f 2 , f∈ L2(p,n−1) (Ω). (2.7)
2

Adding (2.6) and (2.7), we get for any f ∈ L2(p,n−1) (Ω),

∂¯∗ N f 21 + ∂N
¯ f 21 ≤ C(∂¯∂¯∗ N f 2 + ∂¯∗ ∂N
¯ f 2 )
2 2

≤ CN f 2 ≤ Cf 2 . (2.8)

This proves the theorem. 
Corollary 2.4. For any 0 ≤ p ≤ n and n ≥ 3, the range of ∂¯ : L2(p,n−2) (Ω) →
¯ = 0 and f ⊥ H(p,n−1) (Ω),
L2(p,n−1) (Ω) is closed. For any f ∈ L2(p,n−1) (Ω) with ∂f
1
there exists u ∈ W(p,n−2)
2 ¯ = f.
(Ω) satisfying ∂u
Corollary 2.5. The space of harmonic (p, n − 1)-forms H(p,n−1) is infinite dimen-
sional.
Proof. From Corollary 2.3 and Theorem 2.2, we have that the range of (p,n−1) (Ω)
is closed. The infinite dimensionality of the null space of (p,n−1) (Ω) is proved in
Theorem 3.1 in Hörmander [Hör2]. We will give another proof in Corollary 3.4 in
the next section. 
Remarks. (1) The case for the closed range property on the annulus domains when
n = 2 and q = 1 is more involved (see Theorem 3.3 in the next section).
(2) When the domain Ω is the annulus between two concentric balls or el-
lipsoids, the infinite-dimensional space H(p,n−1) has been computed explicitly by
Hörmander (see Theorem 2.2 in [Hör2]). For integral formula when 1 ≤ q < n − 1
in this case, see Section 3.5 in the book by Range [Ra] (see also the paper by
Hortmann [Hor]).
312 M.-C. Shaw

¯
3. The ∂-equation on the annulus between two weakly
pseudoconvex domains in Cn
We recall the following existence and estimates for ∂¯ in the annulus between two
pseudoconvex domains (see Shaw [Sh1])
Theorem 3.1. Let Ω ⊂⊂ Cn , n ≥ 3, be the annulus domain Ω = Ω1 \ Ω2 between
two pseudoconvex domains Ω1 and Ω2 with smooth boundary and Ω2 ⊂⊂ Ω1 . Let
Let φt be a smooth function which is equal to t|z|2 near bΩ1 and −t|z|2 near bΩ2
¯
where t > 0. Then for 0 ≤ p ≤ n and 1 ≤ q < n−1, the ∂-Neumann t
operator N(p,q)
with weights φt exists for sufficiently large t > 0 on L2(p,q) (Ω) and the harmonic
space H(p,q) (Ω) is finite dimensional. For any f ∈ L2(p,q) (Ω), we have
f = ∂¯∂¯t∗ N(p,q)
t
f + ∂¯t∗ ∂N
¯ t f + Ht f
(p,q) (p,q) (3.1)
t
where H(p,q) f is the projection of f into H(p,q) (Ω).
Furthermore, for each s ≥ 0, there exists Ts such that N(p,q) , ∂¯t∗ N(p,q)
t ¯ t
, ∂N(p,q)
and the weighted Bergman projection B(p,0)t
= I − ∂¯t∗ N(p,1)
t
∂¯ are exact regular on
s
W(p,q) (Ω) for t > Ts .
t
The existence and regularity of N(p,q) was proved in [Sh1]. The exact regular-
¯∗ t ¯ t
ity for the related operators ∂t N(p,q) , ∂N(p,q) and the weighted Bergman projection
¯ ∗ t ¯
Bt = I − ∂t N(p,1) ∂ are proved following the same arguments as in Theorem 6.1.4
¯
in [CS]. We will show that H(p,q) (Ω) = {0} using the following ∂-closed extension
of forms.
Theorem 3.2. Let Ω ⊂⊂ Cn be the same as in Theorem 3.1 with n ≥ 3. For any
f ∈ L2(p,q) (Ω), where 0 ≤ p ≤ n and 0 ≤ q < n − 1, such that ∂f ¯ = 0 in Ω, there
−1 ¯
exists F ∈ W(p,q) (Ω1 ) such that F |Ω = f and ∂F = 0 in Ω1 in the distribution
sense.
¯ = f in Ω.
If 1 ≤ q < n − 1, there exists u ∈ L2(p,q−1) (Ω) satisfying ∂u
Proof. From Theorem 3.1, we have that smooth ∂-closed ¯ forms are dense in
¯
L(p,q) (Ω) ∩ Ker(∂) (see Corollary 6.1.6 in [CS]). We may assume that f is smooth
2

and it suffices to prove a priori estimates. Let f˜ be the smooth extension of f so

that f˜|Ω = f . The rest of the proof is similar to the proof of Theorem 2.2. We will
only indicate the necessary changes. Let v ∈ W(p,q+1)1
(Ω2 ) and ṽ be an W 1 (Ω1 )
¯
extension of v to Ω1 and ṽ has compact support in Ω1 . Since f is ∂-closed on Ω,
we have
|(∂¯f˜, v)(Ω2 ) | = |(∂¯f˜, v)(Ω1 ) | = |(f˜, ϑṽ)(Ω1 ) |
≤ f˜ (Ω ) ϑṽ(Ω ) ≤ Cf˜(Ω ) vW 1 (Ω ) .
1 1 1 2

It follows that ∂¯f˜ is in W −1 (Ω2 ), defined as the dual of W 1 (Ω2 ) and

∂¯f˜W −1 (Ω2 ) ≤ Cf˜L2(p,q) (Ω1 ) .
(p,q+1)
 The Closed Range Property for ∂ 313

Ω2
Let N(n−p,n−q−1) ¯
be the weighted ∂-Neumann operator with weights t|z|2 for
some large t on Ω2 . Here we omit the dependence on t to avoid too many indices. It
Ω2
follows from Kohn [Ko2] that N(n−p,n−q−1) is exact regular on W s (Ω2 ) for each s >
0 if we choose sufficiently large t. We define T f˜ by T f˜ = −t ∂N
¯ Ω2 (−t ∂¯f˜)
(n−p,n−q−1)
2
on Ω2 , where t = e−t|z| is the Hodge star operator adjusted to the weighted
L2 space. From Theorem 9.1.2 in [CS], T f˜ satisfies ∂T ¯ f˜ = ∂¯f˜ in Ω1 in the
˜
distribution sense if we extend T f to be zero outside Ω2 . Furthermore, from the
¯
exact regularity of the weighted ∂-Neumann operator and ∂N¯ Ω2
(n−p,n−q−1) in the
1
dual of W (Ω2 ), we have that
˜ W −1 (Ω ) ≤ C∂¯f˜ −1
T f ≤ Cf˜L2(p,q) (Ω1 ) .
2 W (Ω2 )
(p,q+1)

Since T f˜ is in the dual of W 1 (Ω2 ), the extension of T f˜ by zero to Ω1 is continuous

from W −1 (Ω2 ) to W −1 (Ω1 ). Define
'
f, x ∈ Ω,
F = ˜ ˜ (3.2)
f − T f, x ∈ Ω2 .
−1 ¯
Then F ∈ W(p,q) (Ω1 ) and F is a ∂-closed extension of f .
¯ −1
It follows that F = ∂U for some U ∈ W(p,q−1) (Ω1 ) where we can take U to be
¯∗ Ω1
the canonical solution ∂t Nt F with respect to the weight t|z|2 with large t > 0.
It follows that U ∈ L2 (Ω1 , loc) from the interior regularity.
If 1 < q < n − 1, we can actually have that U ∈ L2 (Ω1 ) from the boundary
regularity for ∂¯ ⊕ ∂¯t∗ . Let ζ be a cut-off function which is supported in a tubular
neighborhood V of bΩ1 such that ζ = 1 in a neighborhood of bΩ1 . We first show
a priori estimates assuming U is in L2 (Ω1 ). Then we have from the Hörmander’s
weighted estimates
¯
ζU t(Ω1 ) ≤ C(∂(ζU )t(Ω1 ) + ∂¯t∗ (ζU )t(Ω1 ) ) < ∞ (3.3)
¯ = f and ∂¯∗ U = 0 are in L2 (Ω ∩ V ) and (∂ζ)U
since ∂U ¯ is in L2 (Ω1 ). The constant
t
in (3.3) depends only on the diameter of Ω1 . To pass from a priori estimates to the
real estimates, we approximate Ω1 from inside by strongly pseudoconvex domains
with smooth boundary. We refer the reader to the paper by Boas-Shaw [BS] or
Michel-Shaw [MS] (see also the proof of Theorem 4.4.1 in [CS]) for details.
When q = 1, we have to modify the solution. Let F1 = ∂(ζU ¯ ¯
) = ∂(ζ)U +
¯ ∈ L2 (Ω1 ). We write F = ∂(ζU
ζ ∂U ¯ ¯ − ζ)U ) = F1 + F2 . Let U1 = ∂¯∗ N Ω1 F1 .
) + ∂((1
Then U1 ∈ L2 (Ω1 ) and ∂U¯ 1 = F1 . Since F2 is a ∂-closed
¯ form with compact support
¯ 2 = F2 in Cn by convolution with the Bochner-Martinelli
in Ω1 , we can solve ∂U
¯
kernel (or solve the ∂-equation on a large ball containing Ω1 ). This gives that
U2 ∈ L2 (Cn ). Setting u = U1 + U2 and restricting u to Ω, we have u ∈ L2(p,0) (Ω)
¯ = f in Ω. Notice that the latter method can also be applied to the
satisfying ∂u
case when 1 ≤ q < n − 1. 
314 M.-C. Shaw

¯
For q = n − 1, there is an additional compatibility condition for the ∂-closed
extension of (p, n − 1)-forms.
Theorem 3.3. Let Ω ⊂⊂ Cn be the annulus domain Ω = Ω1 \ Ω2 between two
pseudoconvex domains Ω1 and Ω2 with smooth boundary and Ω2 ⊂⊂ Ω1 , n ≥ 2.
¯
For any ∂-closed f ∈ L2(p,n−1) (Ω), where 0 ≤ p ≤ n, the following conditions are
equivalent:
(1) There exists F ∈ W −1 ¯ = 0 in Ω1 in the
(Ω1 ) such that F |Ω = f and ∂F
(p,n−1)
distribution sense.
(2) The restriction of f to bΩ2 satisfies the compatibility condition

f ∧ φ = 0, φ ∈ W(n−p,0)
1 ¯
(Ω2 ) ∩ Ker(∂). (3.4)
bΩ2
¯ = f in Ω.
(3) There exists u ∈ L2(p,n−2) (Ω) satisfying ∂u
1
¯
Proof. We remark that any h in W 1 (Ω2 ) has a trace in W 2 (bΩ2 ) and any ∂-closed
(p, n − 1)-form with L2 (Ω) coefficients has a well-defined complex tangential trace
1
in W − 2 (bΩ2 ) (see, e.g., [CS2]). Thus the pairing between f and φ in (2) is well
defined.
¯
Since the weighted ∂-Neumann 2
operator N(p,1) on Ω2 with weights t|z|2 is
s
exact regular on W (Ω2 for sufficiently large t, the Bergman projection is bounded
s
from W(p,0) (Ω2 ) to itself for any s ≥ 0 (see Corollary 6.1.6 in [CS]). We have
that the space W(p,0)s ¯ is dense in L2 (Ω2 ) ∩ Ker(∂).
(Ω2 ) ∩ Ker(∂) ¯ In particular,
(p,0)
W(p,0) (Ω2 ) ∩ Ker(∂)
1 ¯ is infinite dimensional.
We first show that (3) implies (2). Since ∂f ¯ = 0 and f is in L2 (Ω), the
1
tangential part of f has a distribution trace fb ∈ W − 2 (bΩ) on the boundary.
Let u → u and ∂u ¯  → ∂u ¯ = f in L where u ∈ C 1 (Ω). Let ∂¯b be the
2

tangential Cauchy-Riemann equations induced by restricting ∂¯ to bΩ. For any

1
h ∈ W(n−p,0)
1
(Ω2 ) ∩ Ker(∂),¯ the restriction of h is in W 2
(n−p,0) (bΩ2 ).
¯
Let f ∈ L2(p,n−1) (Ω) be a ∂-closed form with ∂f ¯ = 0 in Ω. Suppose that f is
¯
∂-exact for some u ∈ L(p,n−2) (Ω). Let ζ be a cut-off function with ζ = 1 on bΩ2
2

and ζ = 0 on bΩ1 . Then f must satisfy the compatibility condition

   
f ∧h= ¯
∂(ζu) ∧ h = lim ¯  ) ∧ h = lim
∂(ζu ¯ = 0.
ζu ∧ ∂h
bΩ2 bΩ  bΩ  bΩ

This proves that (3) implies (2). To show that (2) implies (1), we will modify the
arguments in the proof of Theorem 3.2.
Using the same notation as before, we first approximate f by smooth forms
fν on Ω such that fν → f and ∂f ¯ ν → 0 in L2 (Ω). Since ∂f ¯ ν is top degree, we
¯ ν = ∂f
can always solve ∂g ¯ ν with smooth gν and gν → 0. Thus we may assume
¯
that f can be approximated by smooth ∂-closed forms fν in L2 (Ω) and denote the
smooth extension of fν by f˜ν . We define T f˜ν by T f˜ν = −∂N
¯ Ω2 ¯˜
(n−p,0) (∂ fν ) on Ω2 .
 The Closed Range Property for ∂ 315

From the proof of Theorem 9.1.3 in [CS], the space

¯ ν − B Ω2
¯ f˜ν = ∂f ¯˜
∂T (n−p,0) (∂ fν ).
Ω2
We claim that B(n−p,0) (∂¯f˜ν ) → B(n−p,0)
Ω2
(∂¯f˜) = 0. Let φ ∈ W(n−p,0)
1
(Ω2 ) ∩
¯
Ker(∂). Then from (2), we have
  
(φ, ∂¯f˜ν ) = φ ∧ ∂¯f˜ν = φ ∧ fν → φ ∧ f = 0.
Ω2 bΩ2 bΩ2
¯
From the regularity of the weighted ∂-Neumann 1
operator, we have W(n−p,0) (Ω2 ) ∩
¯ is dense in L
Ker(∂) 2
(Ω 2 ) ∩ Ker( ¯
∂) (see Corollary 6.1.6 in [CS]). This gives
(n−p,0)
¯ ˜ ¯ ˜
that ∂T f = ∂ f in Ω2 since B Ω2 ¯ ˜ ¯ ˜ ¯ ˜
(∂ f ) = 0. Furthermore, ∂T f = ∂ f in Ω1 in
(n−p,0)
the distribution sense if we extend T f˜ to be zero outside Ω2 .
Define F similarly as before,
'
f, x ∈ Ω,
F = ˜
f − T f˜, x ∈ Ω2 .
−1 ¯
Then F ∈ W(p,q) (Ω1 ) and F is a ∂-closed extension of f . This proves that (2)
implies (1).
¯ for some U ∈ L2
To show that (1) implies (3), one can solve F = ∂U (p,q−1) (Ω1 ).
¯
Let u = U on Ω, we have u ∈ L(p,q−1) (Ω) satisfying ∂u = f in Ω. Thus (1) im-
2

plies (3). 

Corollary 3.4. Let Ω be the same as in Theorem 3.3. Then ∂¯ has closed range
¯
in L2(p,n−1) (Ω) and the ∂-Neumann operator N(p,n−1) exists on L2(p,n−1) (Ω). The
space of harmonic (p, n − 1)-forms H(p,n−1) is of infinite dimension.

Proof. That ∂¯ has closed range follows from Condition (2) in Theorem 3.3. The
1
Bergman space W(n−p,0) ¯
(Ω2 ) is infinite dimensional. Each will yield a ∂-closed
¯
(p, n − 1)-form F on Ω which is not ∂-exact. Since the space H(p,n−1) (Ω) is isomor-
¯
phic to the quotient space of L2 ∂-closed (p, n − 1)-forms over the closed subspace
¯
space of ∂-exact forms, we conclude that H(p,n−1) (Ω) is infinite dimensional. 

We summarize the closed range property and regularity for ∂¯ on the annulus
between two pseudoconvex domains in Cn in the following theorem.
Theorem 3.5. Let Ω ⊂⊂ Cn be the annulus domain Ω = Ω1 \ Ω2 between two
pseudoconvex domains Ω1 and Ω2 with smooth boundary and Ω2 ⊂⊂ Ω1 . Then the
¯
∂-Neumann operator N(p,q) exists on L2(p,q) (Ω) for 0 ≤ p ≤ n and 1 ≤ q ≤ n − 1.
For any f ∈ L2(p,q) (Ω), we have

f = ∂¯∂¯∗ N(p,q) f + ∂¯∗ ∂N

¯ (p,q) f, 1 ≤ q ≤ n − 2.

f = ∂¯∂¯∗ N(p,n−1) f + ∂¯∗ ∂N

¯ (p,n−1) f + H(p,n−1) f, q =n−1
316 M.-C. Shaw

where H(p,n−1) is the projection operator onto the harmonic space H(p,n−1) (Ω)
which is infinite dimensional.
Suppose that f ∈ W(p,q)
s
(Ω), where s ≥ 0 and 1 ≤ q ≤ n − 1. We assume that
¯
∂f = 0 in Ω for q ≤ n − 1 and if q = n − 1, we assume furthermore that f satisfies
the condition 
f ∧ φ = 0, φ ∈ W(n−p,0)
1 ¯
(Ω2 ) ∩ Ker(∂).
bΩ2
Then there exists u ∈ W(p,q)
s ¯ = f.
(Ω) satisfying ∂u
Proof. Since ∂¯ has closed range in L2(p,q) (Ω) for all degrees, we have that the ∂-¯
Neumann operator exists (without weights). The proof is exactly the same as the
proof of Theorem 4.4.1 in [CS]. The regularity for ∂¯ follows from Theorem 3.1 for
q < n − 1 and the earlier work of [Sh1]. When q = n − 1, we can trace the proof of
¯
Theorem 3.3 to see that there exists a ∂-closed form F ∈ W(p,q)
s−1
(Ω1 ) which is equal
to f on Ω. Thus one can find a solution u ∈ W(p,n−2) (Ω) satisfying ∂u
s ¯ = f. 
Remark: All the results in this section can be extended to annulus between pseu-
doconvex domains in a Stein manifold with trivial modification.

¯
4. The ∂-equation on weakly pseudoconcave domains in CP n
Much of the results in Section 2 can be applied to the strongly pseudoconcave
domains or complements of finite type pseudoconvex domains in CP n without
¯
much change. For the ∂-equation on a weakly pseudoconcave domain in CP n , we
cannot use the weight function methods used in Section 3 since CP n is not Stein.
We have the following results obtained in the recent papers [CSW] and [CS2] for
pseudoconcave domains in CP n Lipschitz boundary. Related results for ∂¯ on the
pseudoconcave domains in CP n , see the paper by Henkin-Iordan [HI].
We recall that a domain is called Lipschitz if the boundary is locally the
graph of a Lipschitz function. For some basic properties of Lipschitz domains, see
the preliminaries in [Sh4].
Theorem 4.1. Let Ω+ be a pseudoconcave domain in CP n with Lipschitz boundary,
where n ≥ 3. For any f ∈ W(p,q)
1+
(Ω+ ), where 0 ≤ p ≤ n, 1 ≤ q < n − 1, p = q and
¯ = 0 in Ω+ , there exists u ∈ W 1+ (Ω+ ) with ∂u
0 <  < 12 , such that ∂f ¯ = f in
(p,q−1)
+ + 2
Ω . If bΩ is C , the statement is also true for  = 0.
¯
The proof of Theorem 4.1 depends on the ∂-closed extension of forms, which
¯
in term depends on the following ∂-Cauchy problem.
Proposition 4.2. Let Ω ⊂⊂ CP n be a pseudoconvex domain with Lipschitz bound-
ary, n ≥ 3. Suppose that f ∈ L2(p,q) (δ −t , Ω) for some t > 0, where 0 ≤ p ≤ n and
1 ≤ q < n. Assuming that ∂f ¯ = 0 in CP n with f = 0 outside Ω, then there exists
−t ¯ t = f in the distribution
ut ∈ L(p,q−1) (δ , Ω) with ut = 0 outside Ω satisfying ∂u
2

sense in CP . If bΩ is C , the statement is also true for t = 0.

n 2
 The Closed Range Property for ∂ 317

¯
Proof. Following Takeuchi (see [Ta] or [CS1]), the weighted ∂-Neumann operators
Nt exists for forms in L2(n−p,n−q) (δ t , Ω). Let (t) denote the Hodge-star operator
with respect to the weighted norm L2 (δ t , Ω). Then
(t) = δ t  = δ t
where  is the Hodge star operator with the unweighted L2 norm. Since f ∈
L2(p,q) (δ −t , Ω), we have that (−t) f ∈ L2(p,q) (δ t , Ω). Let ut be defined by
¯ t (−t) f .
ut = − (t) ∂N (4.1)
−t ∗
¯ t (−t) f is in Dom(∂¯ ) ⊂ L
Then ut ∈ L(p,q−1) (δ , Ω), since ∂N
2
t
2 t
(n−p,n−q+1) (δ , Ω).
Since ∂t = δ ϑδ = − (−t) ∂(t) , we have (−t) f ∈ Dom(∂t ) and ∂¯t∗ (−t) f = 0
¯∗ −t t ¯ ¯∗

in Ω. This gives
∂¯t∗ Nt (−t) f = Nt ∂¯t∗ (−t) f = 0. (4.2)
¯
From (3.2), we have ∂ut = f
First notice that (−t) (−1)p+q ∂N¯ t (−t) f = ∂N¯ t (−t) f ∈ Dom(∂¯∗ ). We also
t
¯∗ p+q
have ∂t (−t) u = (−1) (−t) f in Ω. Extending ut to be zero outside Ω, one can
¯ t = f in CP n using that the boundary is Lipschitz.
show that ∂u
+ ¯
If bΩ is C 2 , the ∂-Neumann operator exists without weights. The above
arguments can be applied to the case when t = 0. For details, see [CSW] and
[CS2]. 
Proposition 4.3. Let Ω ⊂⊂ CP n be a pseudoconvex domain with Lipschitz boundary
and let Ω+ = CP n \ Ω. For any f ∈ W(p,q) 1+
(Ω+ ), where 0 ≤ p ≤ n, 0 ≤ q < n − 1
and 0 <  < 12 , such that ∂f ¯ = 0 in Ω+ , there exists F ∈ W  (CP n ) with
(p,q)
¯
F |Ω+ = f and ∂F = 0 in CP n in the distribution sense.
If bΩ is C 2 , the statement is also true for  = 0.
Proof. Since Ω has Lipschitz boundary, there exists a bounded extension operator
from W s (Ω+ ) to W s (CP n ) for all s ≥ 0 (see, e.g., [Gr]). Let f˜ ∈ W(p,q)
1+
(CP n )
be the extension of f so that f˜|Ω+ = f with f˜W 1+ (CP n ) ≤ Cf W 1+ (Ω+ ) .
Furthermore, we can choose an extension such that ∂¯f˜ ∈ W  (Ω) ∩ L2 (δ −2 , Ω).
We define T f˜ by T f˜ = −(2)∂N ¯ 2 ((−2) ∂¯f˜) in Ω. From Proposition 4.2,
2 −2
we have that T f ∈ L (δ , Ω). But for a Lipschitz domain, we have that T f˜ ∈
˜
L2 (δ −2 , Ω) is comparable to W  (Ω) when 0 <  < 12 . This gives that T f˜ ∈ W  (Ω)
and T f˜ satisfies ∂T ¯ f˜ = ∂¯f˜ in CP n in the distribution sense if we extend T f˜ to
be zero outside Ω.
Since 0 <  < 12 , the extension by 0 outside Ω is a continuous operator from
W  (Ω) to W s (CP n ) (see, e.g., [Gr]). Thus we have T f˜ ∈ W  (CP n ).
Define ' +
f, x∈Ω ,
F =
f˜ − T f˜, x ∈ Ω.
Then F ∈ W  (CP n ) and F is a ∂-closed¯ extension of f . 
(p,q)
318 M.-C. Shaw

From Proposition 4.3, Theorem 4.1 follows easily.

A Lipschitz (or C 1 ) hypersurface is said to be Levi-flat if it is locally foliated
by complex manifolds of complex dimension n − 1. A C 2 hypersurface M is called
Levi-flat if its Levi-form vanishes on M . Any C k Levi-flat hypersurface, k ≥ 2 is
locally foliated by complex manifolds of complex dimension n − 1. The foliation is
of class C k if the hypersurface is of class C k , k ≥ 2 (see Barrett-Fornaess [BF]).
The proof in [BF] also gives that if a real C 1 hypersurface admits a continuous
foliation by complex manifolds, then the foliation is actually C 1 . One of the main
¯
applications of the ∂-equation on pseudoconcave domains in CP n is the following
result (see [CS2]).

Theorem 4.4. There exist no Lipschitz Levi-flat hypersurfaces in CP n for n ≥ 3.

It was first proved in Lins-Neto [LNe] that there exist no real analytic Levi-flat
hypersurfaces in CP n with n ≥ 3. Siu [Si] proved the nonexistence of smooth (or
3n
2
+ 7) Levi-flat hypersurfaces in CP n with n ≥ 3. The proof for the nonexistence
of C 2 Levi-flat hypersurfaces in CP 2 in [CSW] is still incomplete. The main missing
ingredient is the lack of closed range property for ∂¯ on pseudoconcave domains.
In the case for an annulus domains in Cn , notice that we have used the regularity
¯
of the weighted ∂-Neumann operator on pseudoconvex domains proved by Kohn
in the proof of Theorem 3.3. We end the section with the following three open
questions.

Conjecture 1. Let Ω+ ⊂⊂ CP n be a pseudoconcave domain with C 2 -smooth bound-

ary (or Lipschitz) bΩ+ , n ≥ 2. Then ∂¯ : L2(p,q−1) (Ω)+ → L2(p,q) (Ω)+ has closed
range for 0 ≤ p ≤ n and 1 ≤ q ≤ n − 1.

Conjecture 2. Let Ω+ be a C 2 pseudoconcave domain in CP n , n ≥ 2. For any

0 ≤ p ≤ n, the space of harmonic (p, n − 1)-forms H(p,n−1) is infinite dimensional
and for any f ∈ Dom(∂) ¯ ∩ Dom(∂¯∗ ) ∩ (H(p,n−1) )⊥ , we have

¯ 1 2 + ∂¯∗ f2 2 ).
f 2 ≤ C(∂f

Both Conjecture 1 and Conjecture 2 will imply the nonexistence of Levi-flat

hypersurfaces in CP 2 .

Conjecture 3. Let Ω be a pseudoconvex domain in CP n with C 2 boundary, where

n ≥ 2. Then the range of ∂¯b : L2(p,q−1) (bΩ) → L2(p,q) (bΩ) is closed in the L2(p,q) (bΩ)
space for all 0 ≤ p ≤ n and 1 ≤ q ≤ n − 1.

When Ω is a smooth pseudoconvex domain in Cn , Conjecture 3 is proved in

Shaw [Sh2], Boas-Shaw [BS] and Kohn [Ko3] (see also Chapter 9 in Chen-Shaw
[CS] and also Harrington [Ha] for C 1 pseudoconvex boundary). If Ω is Lipschitz
pseudoconvex in Cn and we assume that there exists a plurisubharmonic defining
function in a neighborhood of Ω, L2 existence for ∂¯b has been established in [Sh3].
 The Closed Range Property for ∂ 319

References
[BF] Barrett, D.E. and Fornnaess, J.E., On the smoothness of Levi-foliations, Publ.
Mat. 32 (1988), 171–177.
[BS] Boas, H.P. and Shaw, M.-C., Sobolev Estimates for the Lewy Operator on Weakly
pseudo-convex boundaries, Math. Annalen 274 (1986), 221–231.
[BSt] Boas, H.P., and Straube, E.J., Sobolev estimates for the ∂-Neumann operator
on domains in Cn admitting a defining function that is plurisubharmonic on the
boundary, Math. Zeit. 206 (1991), 81–88.
[CS1] Cao, J. and Shaw, M.-C., A new proof of the Takeuchi theorem, Proceedings of
the Conference “CR Geometry and PDEs” Trento, Italy, Interdisplinare di Math-
ematica 4 (2005), 65–72.
¯
[CS2] Cao, J. and Shaw, M.-C., The ∂-Cauchy problem and nonexistence of Lipschitz
Levi-flat hypersurfaces in CP n with n ≥ 3, Math. Zeit. 256 (2007), 175–192.
[CS] Chen, S.-C. and Shaw, M.-C., Partial Differential Equations in Several Com-
plex Variables, American Math. Society-International Press, Studies in Advanced
Mathematics, Volume 19, Providence, R.I. 2001.
¯
[CSW] Cao, J., Shaw, M.-C. and Wang L., Estimates for the ∂-Neumann problem and
nonexistence of Levi-flat hypersurfaces in CP n , published online, April 27, Math.
Zeit. (2004) Erratum, Math. Zeit. (to appear).
[FK] Folland, G.B. and Kohn, J.J., The Neumann Problem for the Cauchy-Riemann
Complex, Ann. Math. Studies 75, Princeton University Press, Princeton, N.J.,
1972.
[Gr] Grisvard, P., Elliptic Problems in Nonsmooth Domains Pitman, Boston, 1985.
[Ha] Harrington, P.S., Sobolev Estimates for the Cauchy-Riemann Complex on C 1
Pseudoconvex Domains, Math. Z. (to appear).
[HI] Henkin, G.M. and Iordan, A., Regularity of ∂¯ on pseudoconcave compacts and
applications, Asian J. Math. 4 (2000), 855–884, (see also Erratum: Asian J. Math.,
vol 7, (2003) No. 1, pp. 147–148).
[Hör1] Hörmander, L., L2 estimates and existence theorems for the ∂¯ operator, Acta
Math. 113 (1965), 89–152.
[Hör2] Hörmander, L., The null space of the ∂-Neumann operator, Ann. Inst. Fourier
(Grenoble) 54 (2004), 1305–1369.
¯
[Hor] Hortmann, M., Über die Lösbarkeit der ∂-Gleichung auf Ringgebieten mit Hilfe
von Lp -, C k - und D-stetigen Integraloperatoren. Math. Ann. 223 (1976), no. 2,
139–156.
[Ko1] Kohn, J.J., Harmonic integrals on strongly pseudoconvex manifolds, I, Ann. of
Math. 78 (1963), 112–148.
[Ko2] Kohn, J.J., Global regularity for ∂ on weakly pseudoconvex manifolds, Trans. Amer.
Math. Soc. 181 (1973), 273–292.
[Ko3] Kohn, J.J., The range of the tangential Cauchy-Riemann operator, Duke Math.
Journ. 53 (1986), 525–545.
[KoR] Kohn, J.J., and Rossi, H., On the extension of holomorphic functions from the
boundary of a complex manifold, Ann. Math. 81 (1965), 451–472.
320 M.-C. Shaw

[LNe] Lins Neto, A., A note on projective Levi flats and minimal sets of algebraic folia-
tions, Ann. Inst. Fourier 49 (1999), 1369–1385.
¯
[MS] Michel, J., and Shaw, M.-C., Subelliptic estimates for the ∂-Neumann operator on
piecewise smooth strictly pseudoconvex domains, Duke Math. J. 93 (1998), 115–
128.
[Ra] Range, R.M., Holomorphic Functions and Integral Representations in Several
Complex Variables, Graduate Texts in Math., Vol. 108, Springer-Verlag, N.Y.,
1986.
[Sh1] Shaw, M.-C., Global solvability and regularity for ∂ on an annulus between two
weakly pseudoconvex domains, Trans. Amer. Math. Soc. 291 (1985), 255–267.
[Sh2] Shaw, M.-C., L2 estimates and existence theorems for the tangential Cauchy-
Riemann complex., Invent. Math. 82 (1985), 133–150.
[Sh3] Shaw, M.-C., L2 estimates and existence theorems for ∂ b on Lipschitz boundaries,
Math. Zeit. 244 (2003), 91–123.
[Sh4] Shaw, M.-C., Boundary value problems on Lipschitz domains in Rn or Cn ,
Comtemporary Mathematics, Geometric Analysis of PDE and Several Complex
Variables: dedicated to F. Treves 368 (2005), 375–404.
[Si] Siu, Y.-T., Nonexistence of smooth Levi-flat hypersurfaces in complex projective
spaces of dimension ≥ 3, Ann. Math. 151 (2000), 1217–1243.
[Ta] Takeuchi, A., Domaines pseudoconvexes infinis et la métrique riemannienne dans
un espace projectif, J. Math. Soc. Japan 16 (1964), 159–181.

Mei-Chi Shaw
Department of Mathematics
University of Notre Dame
Notre Dame, IN 46556, USA
e-mail: [email protected]
Complex Analysis
Trends in Mathematics, 321–340

c 2010 Springer Basel AG

New Normal Forms for

Levi-nondegenerate Hypersurfaces
Dmitri Zaitsev

Dedicated to Linda Preiss Rothschild on the occasion of her birthday

Abstract. In this paper we construct a large class of new normal forms for
Levi-nondegenerate real hypersurfaces in complex spaces. We adopt a general
approach illustrating why these normal forms are natural and which role is
played by the celebrated Chern-Moser normal form. The latter appears in
our class as the one with the ”maximum normalization” in the lowest degree.
However, there are other natural normal forms, even with normalization con-
ditions for the terms of the same degree. Some of these forms do not involve
the cube of the trace operator and, in that sense, are simplier than the one by
Chern-Moser. We have attempted to give a complete and self-contained ex-
position (including proofs of well-known results about trace decompositions)
that should be accessible to graduate students.

Mathematics Subject Classification (2000). 32V05.

Keywords. Chern-Moser, normal form, Levi form.

1. Introduction
In this paper we construct a large class of new normal forms for Levi-nondegenerate
real hypersurfaces in complex spaces. We adopt a general approach illustrating why
these normal forms are natural and which role is played by the celebrated Chern-
Moser normal form [CM74]. The latter appears in our class as the one with the
“maximum normalization” in the lowest degree. However, there are other natural
normal forms, even with normalization conditions for the terms of the same degree.
Some of these forms do not involve the cube of the trace operator and, in that sense,
are simpler than the one by Chern-Moser. We have attempted to give a complete
and self-contained exposition (including proofs of well-known results about trace
decompositions) that should be accessible to graduate students.
322 D. Zaitsev

All normal forms here are formal, i.e., at the level of formal power series.
This is sufficient for most purposes such as constructing invariants or solving the
local equivalence problem for real-analytic hypersurfaces. In fact, a formal equiv-
alence map between Levi-nondegenerate real-analytic hypersurfaces is automati-
cally convergent and is therefore a local biholomorphic map. This is a special case
of an important result of Baouendi-Ebenfelt-Rothschild [BER00a] and can also be
obtained from the Chern-Moser theory [CM74]. The reader is referred for more
details to an excellent survey [BER00b]. We also refer to normal forms for Levi-
degenerate hypersurfaces [E98a, E98b, Ko05], for real CR submanifolds of higher
codimension [ES95, SS03], for submanifolds with CR singularities [MW83, HY08],
and for non-integrable Levi-nondegenerate hypersurface type CR structures [Z08].
Throughout the paper we consider a real-analytic hypersurface M in Cn+1
passing through 0 and locally given by an equation
Im w = ϕ(z, z̄, Re w), (z, w) ∈ Cn × C, (1.1)
where we think of ϕ as a power series in the components of z, z̄ ∈ C and u =
n

Re w ∈ R. If the given hypersurface is merely smooth, one can still consider the
Taylor series of its defining equation, which is a formal power series. This motivates
the notion of a formal hypersurface, i.e., the one given (in suitable coordinates) by
(1.1) with ϕ being a formal power series.
In order to describe the normal forms, consider the expansion

ϕ(z, z̄, u) = ϕkml (z, z̄)ul , (1.2)
k,l,m

where each ϕkml (z, z̄) is a bihomogeneous polynomial in (z, z̄) of bidegree (k, m),
i.e., ϕkml (tz, sz̄) = tk sm ϕkml (z, z̄) for t, s ∈ R. The property that ϕ is real is
equivalent to the reality conditions
ϕkml (z, z̄) = ϕmkl (z, z̄). (1.3)
The property that M passes through 0 corresponds to ϕ having no constant terms.
Furthermore, after a complex-linear transformation of Cn+1 , one may assume that
ϕ also has no linear terms, which will be our assumption from now on. The Levi
form now corresponds to ϕ11 (z, z̄), the only lowest order term that cannot be
eliminated after a biholomorphic change of coordinates. Following [CM74], we
write ϕ11 (z, z̄) = z, z
, which is a hermitian form in view of (1.3). If the Levi
form is nondegenerate, after a further complex-linear change of coordinates, we
may assume that
n
z, z
= εj zj z̄j , εj = ±1. (1.4)
j=1
An important role in the normal forms is played by the trace operator associated
with z, z
, which is the second-order differential operator given by

n
∂2
tr := εj . (1.5)
j=1
∂zj ∂ z̄j
 New Normal Forms for Levi-nondegenerate Hypersurfaces 323

In particular, for n = 1 we have

ϕkml (z, z̄) = ckml z k z̄ m ,
'
k m kmckml z k−1 z̄ m−1 , min(k, m) ≥ 1,
trϕkml = tr(ckml z z̄ ) = (1.6)
0, min(k, m) = 0.
We now consider the following normalization conditions.
1. For every k ≥ 2 and l ≥ 1, choose distinct integers 0 ≤ m, m ≤ l with m ≥ 1,
m = m and consider the conditions

trm−1 ϕk+m,m,l−m = 0, trm ϕk+m ,m ,l−m = 0.
2. For every k ≥ 2, consider the condition
ϕk00 = 0.
3. For every l ≥ 2, choose pairwise distinct integers 0 ≤ m, m , m ≤ l with
m ≥ 1, of which the nonzero ones are not of the same parity (i.e., neither all
are even nor all are odd) and consider the conditions
 
trm−1 ϕm+1,m,l−m = 0, trm ϕm +1,m ,l−m = 0, trm ϕm +1,m ,l−m = 0.
4. For every l ≥ 3, choose distinct even integers 0 ≤ m, m ≤ l with m ≥ 1 and
disjoint odd integers 0 ≤ m,
 m  ≤ l and consider the conditions

trm−1 ϕm,m,l−m = 0, trm ϕm ,m ,l−m = 0,

 
trm−1 ϕm,
 m,l−
  = 0,
m trm ϕm
  ,m   = 0.
  ,l−m

5. Consider the conditions

ϕ101 = 0, ϕ210 = 0, ϕ002 = 0, ϕ111 = 0, trϕ220 = 0.
In view of (1.5), in case n = 1 all traces can be omitted. The multi-indices
(k, m, l) of ϕkml involved in different conditions (1)–(5) are all disjoint. They are
located on parallel lines in the direction of the vector (1, 1, −1). In fact, (1) corre-
sponds to the lines through (k + l, l, 0) with k ≥ 2, l ≥ 1, whereas (2) corresponds
to the same lines with l = 0 containing only one triple (k, 0, 0) with nonnegative
components. Condition (3) corresponds to the lines through (k + l, l, 0) for k = 1,
l ≥ 2, whereas (4) corresponds to the same lines for k = 0 and l ≥ 3. Finally (5)
involves all 5 coefficients that correspond to the lines through (2, 1, 0) and (2, 2, 0).
The following is the main result concerning the above normal forms.
Theorem 1.1. Every (formal) real hypersurface M through 0 admits a local (formal)
biholomorphic transformation h preserving 0 into each of the normal forms given
by (1)–(5). If M is of the form (1.1) with ϕ having no constant and linear terms
and satisfying (1.4), the corresponding transformation h = (f, g) is unique provided
it is normalized as follows:
fz = id, fw = 0, gz = 0, gw = 1, Re gz2 = 0, (1.7)
where the subscripts denote the derivatives taken at the origin.
324 D. Zaitsev

The Chern-Moser normal form corresponds to a choice of the coefficients of

the lowest degree that may appear in (1)–(5). In fact, we choose m = 1, m = 0 in
(1), m = 1, m = 0, m = 2 in (2), and m = 2, m = 0, m   = 3 in (4) and
 = 1, m
obtain the familiar Chern-Moser normal form
ϕk0l = 0, ϕk1l = 0, trϕ22l = 0, tr2 ϕ32l = 0, tr3 ϕ33l = 0, (1.8)
where (k, l) = (1, 0) in the second equation.
However, we also obtain other normal forms involving the same coefficients
ϕkml . Namely, we can exchange m   in (4), i.e., choose m
 and m   = 1, which
 = 3, m
leads to the normalization
ϕk0l = 0, ϕk1l = 0, trϕ11l = 0, ϕ111 = 0,
(1.9)
trϕ22l = 0, tr2 ϕ32l = 0, tr2 ϕ33l = 0,
where k ≥ 2 in the second and l ≥ 2 in the third equation. This normal form
is simpler than (1.8) in the sense that it only involves the trace operator and its
square rather than its cube. A comparison of (1.8) and (1.9) shows that the first
set of conditions has more equations for ϕ11l , whereas the second set has more
equations for ϕ33l . Thus we can say that the Chern-Moser normal form has the
“maximum normalization” in the lowest degree.
Alternatively, we can exchange m and m in (2), i.e., choose m = 2, m = 0,

m = 1, which leads to the normalization
ϕk0l = 0, ϕk1l = 0, trϕ21l = 0, ϕ210 = 0,
(1.10)
trϕ22l = 0, trϕ32l = 0, tr3 ϕ33l = 0,
where k = 1, l ≥ 1 or k ≥ 3 in the second and l ≥ 1 in the third equations. Again,
comparing with (1.8), we see that the latter has more equations for ϕ21l and less for
ϕ32l , i.e., again the Chern-Moser normal form has the “maximum normalization”
in the lowest degree.
Finally, we can combine both changes leading to (1.9) and (1.10), i.e., choose
m = 2, m = 0, m = 1 in (2) and m  = 3, m   = 1 in (4) and obtain yet another
normal form involving the same terms ϕkml . We leave it to the reader to write the
explicit equations.
Thus we have 4 normal forms involving the same terms as the one by Chern-
Moser. We now describe a completely different normal form, which also has certain
extremality property. Roughly speaking, in those 4 normal forms, we have chosen
the multi-indices with the smallest first two components on each line of indices in
(1), (3), (4). We now choose the multi-indices with the smallest last component,
i.e., those at the other end of each line. For instance, we can choose m = l, m = l−1
in (1), m = l, m = l − 1, m = l − 2 in (3) and m = l, m = l − 2, m  = l − 1,
m  = l − 3 or m   = l − 2, m = l − 1, m = l − 3 in (4), depending on the
 = l, m
parity of l. We obtain
trl−1 ϕkl0 = 0, ϕk00 = 0, trl ϕkl1 = 0, trl−1 ϕll1 = 0,
(1.11)
trl ϕl+1,l,2 = 0, trl ϕll2 = 0, trl ϕll3 = 0,
 New Normal Forms for Levi-nondegenerate Hypersurfaces 325

where k ≥ l ≥ 1, (k, l) = (1, 1) in the first, k ≥ l + 1 in the third, and l ≥ 1 in

the forth equations. This normal form is distinguished by the property that it has
the “maximum normalization” of the terms ϕkml with the lowest index l. In fact,
(1.11) only involves ϕkml with l = 0, 1, 2, 3. In particular, most harmonic terms
ϕk0l are not eliminated in contrast to the normal forms (1.8)–(1.10). In case n = 2
all traces can be removed, in particular, all terms ϕkml with l = 0, except the Levi
form, are eliminated, i.e., ϕ(z, z̄, u) = z, z
+ O(|u|).
Finally, we consider another interesting normal form that in some sense mixes
the one by Chern-Moser with the one in (1.11) (or, more precisely, the ones in (1.9)
and (1.11)). Here the multi-indices on both ends of the lines are involved. This
normal form corresponds to the choice of m = l, m = 0 in (1), m = l, m = 0,
m = l − 1 in (3) and m = l, m = 1, m  = l − 1, m  = 0 if l is even and m
 = l,
 
m = 1, m = l − 1, m = 0 if l is odd. This leads to the normalization conditions
ϕk0l = 0, trl−1 ϕkl0 = 0, trl ϕl+1,l,1 = 0, trl−1 ϕll1 = 0, trϕ11l = 0, (1.12)
where k ≥ l ≥ 1, (k, l) = (1, 1) in the second, l ≥ 1 in the forth, and l ≥ 2 in the
fifth equation. A remarkable feature of this normal form that distinguishes it from
the previous ones, including the one by Chern-Moser, is that it only involves ϕkml
with min(k, m, l) ≤ 1.
The rest of the paper is now devoted to the proof of Theorem 1.1, where
we also explain how the normalization conditions (1)–(5) arise and why they are
natural.

2. Transformation rule and its expansion

As before, we shall consider a real-analytic hypersurface M in Cn+1 through 0,
locally given by (1.1) with ϕ having no constant and linear terms. To the hyper-
surface M (or more precisely to the germ (M, 0)) we apply a local biholomorphic
transformation (z, w) → (f (z, w), g(z, w)) preserving 0 and transforming it into
another germ (M  , 0) of a real-analytic hypersurface in Cn+1 , still given by an
equation
Im w = ϕ (z  , z̄  , Re w ), (2.1)
where ϕ has no constant and linear terms. We consider (multi)homogeneous power
series expansions

f (z, w) = fkl (z)wl , g(z, w) = gkl (z)wl ,
(2.2)
ϕ(z, z̄, u) = ϕkml (z, z̄)ul , ϕ (z  , z̄  , u ) = ϕkml (z  , z̄  )ul ,
where fkl (z) and gkl (z) are homogeneous polynomials in z ∈ Cn of degree k and
ϕkml (z, z̄) and ϕkml (z  , z̄  ) are bihomogeneous polynomials in (z, z̄) ∈ Cn × Cn
and (z  , z̄  ) ∈ Cn × Cn respectively of bidegree (k, l). Furthermore, fkl and gkl are
abitrary whereas ϕkml (z, z̄) and ϕkml (z  , z̄  ) satisfies the reality condition (1.3)
which are equivalent to ϕ and ϕ being real valued.
326 D. Zaitsev

The fact that the map (z, w) → (f (z, w), g(z, w)) transforms (M, 0) into
(M  , 0) can be expressed by the equation

Im g(z, u + iϕ(z, z̄, u))

 
= ϕ f (z, u + iϕ(z, z̄, u)), f (z, u + iϕ(z, z̄, u)), Re g(z, u + iϕ(z, z̄, u)) . (2.3)

We use (2.2) to expand both sides of (2.3):

  l 
Im g(z, u + iϕ(z, z̄, u)) = Im gkl (z) u + i ϕjhm (z, z̄)um , (2.4)

 
ϕ f (z, u + iϕ(z, z̄, u)), f (z, u + iϕ(z, z̄, u)), Re g(z, u + iϕ(z, z̄, u))
  b  d 
= ϕklm fab (z) u+i ϕjhr (z, z̄)ur , fcd (z) u + i ϕjhr (z, z̄)ur
   t m
× Re gst (z) u + i ϕjhr (z, z̄)ur . (2.5)

Since both ϕ and ϕ have vanishing linear terms, collecting the linear terms
in (2.4) and (2.5) and substituting into (2.3) we obtain

g10 = 0, Im g01 = 0. (2.6)

Conditions (2.6) express the fact that the map (f, g) sends the normalized tangent
space T0 M = Cnz × Ru into the normalized tangent space T0 M  = Cnz × Ru (where
u = Re w ). The first condition in (2.6) implies that the Jacobian matrix of (f, g)
at 0 is block-triangular and hence its invertibility is equivalent to the invertibility
of both diagonal blocks f10 = fz (0) and g01 = gw (0).

3. Partial normalization in general case

Our first goal is to obtain a general normalization procedure that works for all
series ϕ without any nondegeneracy assumption. The key starting point consists
of identifying non-vanishing factors in (2.4) and (2.5). These are f10 and g01 . All
other factors may vanish. Then we look for terms in the expansions of (2.4) and
(2.5) involving at most one factor that may vanish. These are

Im gkl (z)ul , g01 ϕkml (z, z̄)ul , ϕkml (f10 (z), f10 (z))(g01 u)l , (3.1)

where we used the reality of g01 and (1.3) and have dropped the argument z for g01
since the latter is a constant. The first term in (3.1) has only gkl which may vanish,
the second ϕklm and the third ϕklm . Other summands have more than one entry
(term) that may vanish. The terms (3.1) play a crucial role in the normalization
and are called here the “good” terms. The other terms are called the “bad” terms.
Good terms can be used to obtain a partial normalization of M as follows.
 New Normal Forms for Levi-nondegenerate Hypersurfaces 327

Consider in (2.3) the terms of multi-degree (k, 0, l) in (z, z̄, u), where the first
term from (3.1) appears. We obtain
1
gkl (z)ul + g01 ϕk0l (z, z̄)ul = ϕk0l (f10 (z), f10 (z))(g01 u)l + · · · , k > 0,
2i (3.2)
Im g0l ul + g01 ϕ00l ul = ϕ00l (g01 u)l + · · · ,
where the dots stand for the “bad” terms. If no dots were present, one can suitably
choose gkl in the first equation and Im g0l in the second to obtain the normalization
ϕk0l (z) = 0. (3.3)
In presence of the “bad” terms, an induction argument is used. In fact, an inspec-
tion of the expansions of (2.4) and (2.5) shows that the terms in (3.2) included in
the dots in (3.2), involve other coefficients gst (z) only of order s + t less than k + l.
Thus the normalization (3.3) can be obtained by induction on the order k + l. Fur-
thermore, the expansion terms gkl (z) for k > 0 and Im g0l are uniquely determined
by this normalization. However, the infinitely many terms fkl (z) and the real parts
Re g0l are not determined and act as free parameters. For every choice of those
parameters, one obtains a germ (M  , 0) which is biholomorphically equivalent to
(M, 0) and satisfies the normalization (3.3). Thus this normalization is “partial”.
Normalization (3.3) is the well-known elimination of the so-called harmonic terms
and works along the same lines also when M is of higher codimension, i.e., when
w is a vector.

4. Levi-nondegenerate case
The good terms in (3.1) were not enough to determine fkl (z) and Re g0l . Thus
we need more good terms for a complete normal form. These new “good” terms
must be obtained from the expansion of (2.5) since fkl (z) do not appear in (2.4).
Thus we need a nonvanishing property for some ϕkml . The well-known lowest-
order invariant of (M, 0) is the Levi form ϕ110 (z, z̄) which we shall write following
[CM74] as z, z
. In view of (1.3), z, z
is a hermitian form. We assume it to be
complex-linear in the first and complex-antilinear in the second argument. The
basic assumption is now that the Levi form is nondegenerate.
Once the class of all M is restricted to Levi-nondgenerate ones, we obtain
further “good” terms involving the new nonvanishing factor z, z
. The good terms
in both (2.5) and (2.4) come now from the expansion of
   
Im gkl (z)(u + i z, z
)l , 2Re fkl (z), f10 (z)
(u + i z, z
)l ,
(4.1)
g01 ϕkml (z, z̄)ul , ϕkml (f10 (z), f10 (z))(g01 u)l .
This time each of the coefficients gkl and fkl appears in some good term and thus
can be potentially uniquely determined. The linear coefficients g01 and f10 (z) play
a special role. They appear by themselves in the good terms Im g01 , Re g01 z, z

328 D. Zaitsev

and f10 (z), f10 (z)

in the expansions of (2.3) in multidegrees (0, 0, 1) and (1, 1, 0).
Equating terms of those multidegrees, we obtain

Im g01 = 0, Re g01 z, z
= f10 (z), f10 (z)
, (4.2)

where the first condition already appeared in (2.6) and the second expresses the
invariance of the Levi form (i.e., its transformation rule as tensor). The restrictions
(4.2) describe all possible values of g01 and f10 (z), which form precisely the group
G0 of all linear automorphism of the hyperquadric Im w = z, z
. In order to
study the action by more general biholomorphic maps (z, w) → (f (z, w), g(z, w))
satisfying (4.2), it is convenient to write a general map as a composition of one
from G0 and one satisfying

g01 = 1, f10 (z) = z, (4.3)

and study their actions separately. Since the action by the linear group G0 is
easy, we shall in the sequel consider maps (f, g) satisfying (4.3), unless specified
otherwise.
Since the Levi form z, z
is of bidegree (1, 1) in (z, z̄), we conclude from the
binomial expansion of the powers in the first line of (4.1) that every coefficient
gkl contributes to the good terms in multidegrees (k + m, m, l − m) and (m, k +
m, l − m) in (z, z̄, u) for all possible 0 ≤ m ≤ l. The latters are the integral points
with nonnegative components of the lines passing through the points (k, 0, l) and
(0, k, l) in the direction (1, 1, −1). Similarly, every coefficient fkl contributes to
the multidegrees (k + m, m + 1, l − m) and (m + 1, k + m, l − m) for all possible
0 ≤ m ≤ l, corresponding to the lines passing through (k, 1, l) and (1, k, l) in the
same direction (1, 1, −1). For convenience, we shall allow one or both of k, l being
negative, in which case the corresponding terms are assumed to be zero. We see
that good terms with gkl for k > 0 appear in two lines, whereas for g0l both lines
coincide with that through (0, 0, l). Similarly, each fkl with k > 1 or k = 0 appears
in two lines, whereas for f1l both lines coincide with that through (1, 1, l). Thus
the lines through (0, 0, l) are special as well as the lines through (1, 0, l) and (0, 1, l)
next to it. The latter lines contain good terms with f0l , f2,l−1 and g1l . Each other
line contains good terms with precisely one fkl and one gkl .
Thus we treat those groups of lines separately. Collecting in (2.3) terms of
multi-degree (k + m, m, l − m) in (z, z̄, u) for k ≥ 2 we obtain


1 l
gkl (z)ul−m (i z, z
)m + ϕk+m,m,l−m (z, z̄)ul−m
2i m


l−1
= fk+1,l−1 (z), z
ul−m (i z, z
)m−1 + ϕk+m,m,l−m (z, z̄)ul−m + · · · ,
m−1
(4.4)

where as before the dots stand

 l−1 for
 all bad terms. Note that due to our convention,
for m = 0, the term with m−1 = 0 is not present. Similarly we collect terms of
 New Normal Forms for Levi-nondegenerate Hypersurfaces 329

multi-degree (m + 1, m, l − m), this time we obtain two different terms with fab :


1 l
g1l (z)ul−m (i z, z
)m + ϕm+1,m,l−m (z, z̄)ul−m
2i m



l−1 l−1
= f2,l−1 (z), z
u l−m
(i z, z
)m−1
+ z, f0l
ul−m (−i z, z
)m
m−1 m−1
+ ϕm+1,m,l−m (z, z̄)ul−m + · · · , (4.5)
where we have dropped the argument z from f0l since the latter is a constant.
Finally, for the terms of multidegree (m, m, l − m), we have


l  
Im g0l ul−m (i z, z
)m + ϕm,m,l−m (z, z̄)ul−m
m


l−1  
=2 Re f1,l−1 (z), z
ul−m (i z, z
)m−1 + ϕm,m,l−m (z, z̄)ul−m + · · · .
m−1
(4.6)
Since both sides of (2.3) are real, its multihomogeneous part of a multi-
degree (a, b, c) is conjugate to that of multi-degree (b, a, c). Hence the system of all
equations in (4.4)–(4.6) is equivalent to (2.3), i.e., to the property that the map
(z, w) → (f (z, w), g(z, w)) sends (M, 0) into (M  , 0).

5. Weight estimates
In order to set up an induction similar to that of Section 3, we have to estimate
the degrees of fkl and gkl appearing in the bad terms in (4.4)–(4.6) and compare
it with the degrees of the good terms. However, different good terms with gkl (see
(4.1)) do not have the same degree but rather have the same weight k + 2l, where
the weight of z and z̄ is 1 and the weight of u is 2. Similarly, different good terms
with fkl have the same weight k + 2l + 1. Hence this weight is more suitable to
separate good terms from bad ones. Thus, instead of proceeding by induction on
the degree as in Section 3, we proceed by induction on the weight.
We first inspect the weights of the bad terms in the expansions of (2.4) and
(2.5). Recall that both ϕ and ϕ have no constant or linear terms. Hence the weight
of ϕjhm (z, z̄)um is greater than 2 unless (j, h, m) ∈ {(1, 1, 0), (2, 0, 0), (0, 2, 0)}. In
particular, the expansion of
gkl (z)(u + i z, z
+ iϕ200 (z, z̄) + iϕ020 (z, z̄))l
contains bad terms of the same weight k + 2l as “good” terms. The latter fact
is not suitable for setting up the induction on the weight. This problem is solved
by initial prenormalization of (M, 0) as follows. According to Section 3, one can
always eliminate harmonic terms from the expansion of ϕ. For our purposes, it
will suffice to eliminate ϕ200 (and hence ϕ020 in view of (1.3)). Thus in the sequel,
we shall assume that ϕ200 , ϕ020 , ϕ200 , ϕ020 are all zero.
330 D. Zaitsev

With that assumption in mind, coming back to (2.4), we see that the weight
of ϕjhm (z, z̄)um is always greater than 2 unless we have (j, h, m) = (1, 1, 0),that
is ϕjhm (z, z̄)um = z, z
. Since any bad term in the expansion of gkl (z) u +
 l
i ϕjhm (z, z̄)um contains at least one factor ϕjhm (z, z̄)um with (j, h, m) =
(1, 1, 0), its weight is greater than k + 2l. Since the weights of all terms (including
bad ones) in (4.4)–(4.6) are k + 2l, the bad terms there coming from (2.4) can only
contain gab (z) or gab (z) with weight a + 2b < k + 2l, which is now suitable for
our induction. Inspecting now the terms with gst (z) and gst (z) in the expansion of
(2.5), we see that their weights must be greater than s + 2t. Hence, the bad terms
in (4.4)–(4.6) coming from (2.5) can only contain gab or its conjugate with weight
a + 2b < k + 2l.
Similarly, we inspect “bad” terms containing fab . This time we only need to
look at the expansion of (2.5). A term in the expansion of
 b
fab (z) u + i ϕjhm (z, z̄)um (5.1)
is of weight greater than a + 2b unless it appears in the expansion of
 b
fab (z) u + i z, z̄
. (5.2)

Keeping in mind that ϕ has no linear terms, we conclude that a bad term in the
expansion of (2.5) containing fab is always of weight greater than a + 2b + 1. The
same holds for bad terms containing fab . Hence, a bad term in (4.4)–(4.6) can only
contain fab or its conjugate with weight a + 2b < k + 2l − 1.
Summarizing, we obtain that bad terms in (4.4)–(4.6) can only contain gab
or its conjugate with weight a + 2b < k + 2l and fab or its conjugate with weight
a + 2b < k + 2l − 1. On the other hand, the good terms contain gab (or Im gab ) of
weight precisely k + 2l and fab of weight precisely 2k + l − 1. Thus we may assume
by induction on the weight that all terms denoted by dots in (4.4)–(4.6) are fixed
and proceed by normalizing the good terms there.

6. Trace decompositions
The good terms involving gkl (z) appear as products of the latters and a power of
the Levi form z, z
. As gkl (z) varies, these products
gkl (z) z, z
s (6.1)
form a vector subspace of the space of all bihomogeneous polynomials in (z, z̄) of
the corresponding bidigree (k + s, s). In order to normalize such a product, we
need to construct a complementary space to the space of all products (6.1). The
latter is done by using the so-called trace decompositions described as follows.
Since the Levi form z, z
is assumed to be nondegenerate, we can choose
coordinates z = (z1 , . . . , zn ) such that (1.4) is satisfied and consider the associ-
ated trace operator (1.5). The rest of this section is devoted to the proof of the
 New Normal Forms for Levi-nondegenerate Hypersurfaces 331

following well-known trace decompositions (see [F17, S89, ES95] for more general
decomposition results):
Proposition 6.1. For every polynomial P (z, z̄), there exist unique polynomials
Q(z, z̄) and R(z, z̄) such that
P (z, z̄) = Q(z, z̄) z, z
s + R(z, z̄), trs R = 0. (6.2)
Taking bihomogeneous components of all terms in (6.2) and using the unique-
ness we obtain:
Lemma 6.2. If P in Proposition 6.1 is bihomogeneous in (z, z̄), so are Q and R.
Since the trace operator is real (maps real functions into real ones), we can
take real parts of both sides in (6.2) and use the uniqueness to obtain:
Lemma 6.3. If P in Proposition 6.1 is real, so are Q and R.
We begin the proof of Proposition 6.1 with the following elementary lemma.
Lemma 6.4. Let P (z, z̄) be a bihomogeneous polynomial of bidegree (p, q). Then

Pzj (z, z̄)zj = pP (z, z̄), Pz̄j (z, z̄)z̄j = qP (z, z̄), (6.3)
j j
p q
Proof. By the assumption, P (sz, tz̄) = s t P (z, z̄). Differentiating in s for s = t =
1 we obtain the first identity in (6.3). Similarly, differentiating in t for s = t = 1
we obtain the second identity. 
The following is the key lemma in the proof of Proposition 6.1.
Lemma 6.5. Let P (z, z̄) be a bihomogeneous polynomial of bidegree (p, q). Then
   
tr P (z, z̄) z, z
= (n + p + q)P (z, z̄) + trP (z, z̄) z, z
. (6.4)
Proof. By straightforward calculations, we have


  ∂2
tr P (z, z̄) z, z
= εj P (z, z̄) εs zs z̄s
∂zj ∂ z̄j s
j


∂2
= εj P (z, z̄) εs zs z̄s + εj Pzj (z, z̄)εj zj
j
∂zj ∂ z̄j s j

+ εj Pz̄j (z, z̄)εj z̄j + P (z, z̄) ε2j . (6.5)
j j

Using (6.3) we obtain the right-hand side of (6.4) as desired. 

Proof of Proposition 6.1 for s = 1. We begin by proving the uniqueness of the de-
composition (6.2). Let P (z, z̄) be bihomogeneous in (z, z̄) of bidegree (p, q) and
suppose that
P (z, z̄) = Q(z, z̄) z, z
+ R(z, z̄), trR = 0. (6.6)
Applying k ≥ 1 times tr to both sides of (6.6) and using (6.4) we obtain, by
induction on k,
 
trk P (z, z̄) = ck trk−1 Q(z, z̄) + trk Q(z, z̄) z, z
, (6.7)
332 D. Zaitsev

where ck are positive integers depending only on n, p, q and satisfying

c1 = n + p + q − 2, ck+1 = ck + n + p + q − 2k − 2. (6.8)
k0
Since applying tr decreases both degrees in z and z̄ by 1, one has tr Q = 0 for
k0 := min(p, q). Hence trk0 −1 Q is uniquely determined from (6.7) for k = k0 . Then
going backwards through the identities (6.7) for k = k0 − 1, k0 − 2, . . . , 1, we see
that each trk−1 Q is uniquely determined including tr0 Q = Q. Then R is uniquely
determined by (6.6) proving the uniqueness part of Proposition 6.1 for s = 1.
To prove the existence, consider the equations
trk P (z, z̄) = ck Qk−1 (z, z̄) + Qk (z, z̄) z, z
, k = 1, . . . , k0 , (6.9)
obtained from (6.7) by replacing each trk Q(z, z̄) with an indeterminant polynomial
Qk (z, z̄), where Qk0 = 0 for the bidegree reason. Hence the last equation for k = k0
reads trk0 P = ck0 Qk0 −1 , which we can solve for Qk0 −1 . Going backwards through
the equations (6.9) for k = k0 − 1, k0 − 2, . . . , 1, as before, we can solve the system
(6.9) uniquely for Qk0 −2 , . . . , Q0 . We claim that
Qk = trQk−1 , k = 1, . . . , k0 . (6.10)
Indeed, (6.10) clearly holds for k = k0 for bidegree reason. Suppose (6.10) holds
for k > k1 . Applying tr to both sides of (6.9) for k = k1 , and using (6.4) we obtain
 
trk1 +1 P (z, z̄) = ck1 trQk1 −1 (z, z̄)+ (n+ p+ q − 2k − 2)Qk1 (z, z̄)+ trQk1 (z, z̄) z, z
,
(6.11)
which we compare to (6.9) for k = k1 + 1:
 
trk1 +1 P (z, z̄) = ck1 +1 Qk1 (z, z̄) + trQk1 (z, z̄) z, z
, (6.12)
where we have used (6.10) for k = k1 +1. Using (6.8), we immediately obtain (6.10)
for k = k1 . Thus (6.10) holds for all k by induction. In particular, substituting
into (6.9) for k = 1, we obtain
   
trP (z, z̄) = c1 Q0 (z, z̄) + trQ0 (z, z̄) z, z
= tr Q0 (z, z̄) z, z
, (6.13)
where we have used (6.4) and (6.8). Thus we can take Q(z, z̄) := Q0 (z, z̄) and
R(z, z̄) := P (z, z̄) − Q0 (z, z̄) z, z
to satisfy (6.6), proving the existence part of
Proposition 6.1 for s = 1. 
In Proposition 6.1 in general case we shall use the following lemma.
Lemma 6.6. Assume that a polynomial P (z, z̄) satisfies trP = 0. Then
 
trs P (z, z̄) z, z
s−1 = 0 (6.14)
for any s ≥ 1.
Proof. Using (6.4) for P (z, z̄) replaced with P (z, z̄) z, z
s−1 we obtain
   
tr P (z, z̄) z, z
s = tr P (z, z̄) z, z
s−1 z, z

  
= cs P (z, z̄) z, z
s−1 + tr P (z, z̄) z, z
s−1 z, z
(6.15)
 New Normal Forms for Levi-nondegenerate Hypersurfaces 333
 
for suitable integer cs depending on s. Replacing tr P (z, z̄) z, z
s−1 with the
right-hand side of (6.15) for s replaced with s − 1 and continuing the process we
obtain
 
tr P (z, z̄) z, z
s = cs P (z, z̄) z, z
s−1 , (6.16)
where cs is another integer depending on s. Now (6.14) can be proved directly by
induction on s using (6.16). 

Proof of Proposition 6.1 in the general case. We prove the statement by induction
on s. Since it has been proved for s = 1, it remains to prove the induction step.
Suppose for given P we have a decomposition (6.2). Applying Proposition 6.1 for
s = 1 to Q, we also obtain
Q(z, z̄) = Q (z, z̄) z, z
+ R (z, z̄), trR = 0. (6.17)
Substituting into (6.2), we obtain
P (z, z̄) = Q (z, z̄) z, z
s+1 + R (z, z̄) z, z
s + R(z, z̄), trR = 0.
trs R = 0,
  (6.18)
Furthermore, trs+1 R (z, z̄) z, z
s = 0 by Lemma 6.6 and therefore we obtain a
decomposition
P (z, z̄) = Q (z, z̄) z, z
s+1 + R (z, z̄), trs+1 R = 0, (6.19)
as desired with R (z, z̄) := R (z, z̄) z, z
s + R(z, z̄). This proves the existence part.
Clearly it suffices to prove the uniqueness for P = 0. Assume it holds for s
and that there is another decomposition
 z̄) z, z
s+1 + R(z,
0 = Q(z,  z̄),  = 0.
trs+1 R (6.20)
By Proposition 6.1 for s = 1, we can write
=Q
R   (z, z̄) z, z
+ R
 (z, z̄),  = 0.
trR (6.21)
Substitution into (6.20) yields
 
  (z, z̄) z, z
+ R
 z̄) z, z
s + Q
0 = Q(z,   (z, z̄),  = 0.
trR (6.22)
  = 0 and
Then the uniqueness for s = 1 implies R
Q(z,   (z, z̄) = 0.
 z̄) z, z
s + Q (6.23)
Applying trs+1 to both sides of (6.21) we obtain
  
trs+1 Q (z, z̄) z, z
= 0. (6.24)
  (z, z̄) z, z
, Q replaced by Q
Now using the identities (6.7) for P (z, z̄) := Q   and
s 
k = s + 1, . . . , k0 with k0 as chosen there, we conclude that tr Q = 0. Then
the uniqueness in (6.23) implies Q  = 0. Hence R  = 0 by (6.20) and the proof is
complete. 
334 D. Zaitsev

7. Normalizations
We now proceed with normalization of the equation for M  , i.e., of the terms
ϕklm (z, z̄). As noted at the end of §5, we may assume by induction on the weight
that all “bad” terms denoted by dots in (4.4)–(4.6) are fixed. As explained in §5,
every coefficient gkl (resp. fkl ) contributes to good terms corresponding to the lines
through (k, 0, l), (0, k, l) (resp. (k, 1, l), (1, k, l)) in the same direction (1, 1, −1).

7.1. Normalization for k ≥ 2

We begin by analysing the line through (k, 0, l) for k ≥ 2, corresponding to the
multi-degrees (k + m, m, l − m), 0 ≤ m ≤ l, for which we have the equation (4.4).
We rewrite this equation as

ϕk+m,m,l−m (z, z̄)



1 l l−1
= gkl (z)(i z, z
) −
m
fk+1,l−1 (z), z
(i z, z
)m−1 + · · · , (7.1)
2i m m−1
Where we have included the given term ϕk+m,m,l−m (z, z̄) in the dots. Our goal
is to write normalization conditions for ϕk+m,m,l−m (z, z̄) that uniquely determine
gkl and fk+1,l−1 . If m = 0, the term with fk+1,l−1 is not present. Hence we have
to consider an identity (7.1) with m ≥ 1. Then the sum of the terms involving
gkl and fk+1,l−1 is a multiple of z, z
m−1 . Thus, by varying gkl and fk+1,l−1 , we
may expect to normalize ϕk+m,m,l−2m (z, z̄) to be in the complement of the space
of polynomials of the form P (z, z̄) z, z
m−1 . The suitable decomposition is given
by Proposition 6.1:

ϕk+m,m,l−m (z, z̄) = Q(z, z̄) z, z

m−1 + R(z, z̄), trm−1 R = 0. (7.2)

Using similar decompositions for other terms in (7.1) and equating the factors of
z, z
m−1 , we obtain



1 l l−1
Q(z, z̄) = gkl (z)im z, z
− fk+1,l−1 (z), z
im−1 + · · · , (7.3)
2i m m−1
where, as before, the dots stand for the terms that have been fixed. Since
fk+1,l−1 (z) is free and the form z, z
is nondegenerate, fk+1,l−1 (z), z
is a free
bihomogeneous polynomial of the corresponding bidegree. Hence we can normalize
Q to be zero. In view of (7.2), Q = 0 is equivalent to the normalization condition

trm−1 ϕk+m,m,l−m = 0. (7.4)

Putting Q = 0 in (7.3), we can now uniquely solve this equation for fk+1,l−1 (z), z

in the form


−1
1 l l−1
fk+1,l−1 (z), z
= gkl (z) z, z
+ · · · , (7.5)
2 m m−1
 New Normal Forms for Levi-nondegenerate Hypersurfaces 335

from where fk+1,l−1 (z) is uniquely determined. It remains to determine gkl , for
which we substitute (7.5) into an identity (7.1) with m replaced by m = m:

ϕk+m ,m ,l−m (z, z̄)




−1 
1 l l−1 l l−1 
= − gkl (z)(i z, z
)m + · · · , (7.6)
2i m m − 1 m m − 1

where we have assumed m ≥ 1. If m = 0, the term with fk+1,l−1 does not occur
and hence no substitution is needed. In the latter case, the coefficient in front of
gkl (z) is clearly nonzero. Otherwise, for m ≥ 1, that coefficient is equal, up to a
nonzero factor, to the determinant
#   l−1  ## ## #
# l l! (l−1)! #
# m  − m−1 # # m!(l−m)! (m−1)!(l−m)! #
# l l−1 # = # #
# m − m −1 # # m !(l−m
l!
 )!
(l−1)!
(m −1)!(l−m )!
#
# #
l!(l − 1)! #1
# m ##
= = 0. (7.7)
m!(l − m)!m !(l − m )! #1
  m #

Hence the coefficient in front of gkl (z) in (7.6) is nonzero in any case. There-
fore, using Proposition 6.1 as above we see that we can obtain the normalization
condition

trm ϕk+m ,m ,l−m = 0, (7.8)
which determines uniquely gkl (z) and hence fk+1,l−1 (z) in view of (7.5).
Summarizing, for each m ≥ 1 and m = m, we obtain the normalization
conditions

trm−1 ϕk+m,m,l−m = 0, trm ϕk+m ,m ,l−m = 0, (7.9)
which determine uniquely gkl (z) and fk+1,l−1 (z). Such a choice of m and m is
always possible unless l = 0. In the latter case, the coefficient fk+1,−1 (z) = 0
is not present and gk0 (z) is uniquely determined by the normalization condition
ϕk00 = 0 corresponding to m = 0. That is, for l = 0, we only have the second
condition in (7.9) with m = 0.
Since the degree of ϕk+m,m,l−m is k + l + m, we obtain the lowest possible
degrees in (7.9) for m = 0, m = 1, which corresponds to the normalization

ϕk+1,1,l−1 = 0, ϕk0l = 0, (7.10)

which is the part of the Chern-Moser normal form [CM74].

7.2. Normalization for k = 1

We next analyze the line through (1, 0, l) corresponding to the multi-degrees (m +
1, m, l − m), 0 ≤ m ≤ l, for which we have the equation (4.5). As before, we rewrite
336 D. Zaitsev

this equation as


1 l
ϕm+1,m,l−m (z, z̄) = g1l (z)(i z, z
)m
2i m



l−1 l−1
− f2,l−1 (z), z
(i z, z
)m−1 − z, f0l
(−i z, z
)m + · · · .
m−1 m−1
(7.11)
Arguing as before, we consider m ≥ 1 and decompose
ϕm+1,m,l−m (z, z̄) = Q(z, z̄) z, z
m−1 + R(z, z̄), trm−1 R = 0. (7.12)
Decomposing similarly the other terms in (7.11) and equating the factors of
z, z
m−1 , we obtain


1 l
Q(z, z̄) = g1l (z)im z, z

2i m



l−1 l−1
− f2,l−1 (z), z
im−1 − z, f0l
(−i)m z, z
+ · · · . (7.13)
m−1 m−1
Since f2,l−1 (z) is free and the form z, z
is nondegenerate, f2,l−1 (z), z
is also
free and hence we can choose it suitably to obtain Q = 0, which is equivalent to
the normalization condition
trm−1 ϕm+1,m,l−m = 0. (7.14)
Putting Q = 0 into (7.13), we solve it uniquely for f2,l−1 (z), z
, which, in turn,
determines uniquely f2,l−1 (z). Arguing as in §7.1, we substitute the obtained ex-
pression for f2,l−1 (z), z
into identities (7.13) with m replaced by m = m. The
result can be written as
  
ϕm +1,m ,l−m (z, z̄) = cm g1l (z) + dm z, f0l
z, z
m + · · · (7.15)
with suitable coefficients cm , dm . Then as in §7.1 we see that we can obtain the
normalization
 
trm ϕm +1,m ,l−m = 0, trm ϕm +1,m ,l−m = 0 (7.16)
for any pair (m , m ) such that
# #
# cm  dm ##
# = 0. (7.17)
#cm dm #
Furthermore, assuming (7.17), both g1l (z) and f0l are uniquely determined by
(7.16), which, in turn, determine f2,l−1 (z) in view of (7.13) with Q = 0. Thus
it remains to choose m , m satisfying (7.17). Inspecting the construction of the
coefficients cm , dm , it is straightforward to see that the determinant in (7.17) is
equal, up to a nonzero multiple, to the determinant
#    l−1  m−1  l−1  #
# 1 l im − m−1 − m−1 (−i)m ##
# 2i m  
i
 m −1  
# 1 l m m #
# 2i m i − ml−1 −1 i − ml−1 −1 (−i) # (7.18)
# 1  l  m  l−1  m −1  l−1  #
m #
# 2i m i − m −1 i − m −1 (−i)
 New Normal Forms for Levi-nondegenerate Hypersurfaces 337

consisting of the coefficients in (7.13). In fact one can see that the matrix in (7.17)
is obtained as a block in (7.19) by elementary row operations. The determinant
(7.18) is equal, up to a nonzero multiple, to
# #
#1 m (−1)m m ## ##  #
# m  #
#1 m (−1)m m # = # m − m (−1)  m − (−1) m #
m
# # #m − m (−1) m − (−1) m#
 m  m
#1 m (−1)m m #
#  #
# m − m (−1)m −m m − m #
= ± ##  # . (7.19)
m − m (−1)m −m m − m#


In case both m − m and m − m are odd, the latter determinant is equal to
(m −m)(−m −m)−(m −m)(−m −m) = 2m m−2mm = 2m(m −m), (7.20)
which is nonzero by the construction. In case m − m is even and m − m is odd,
the last determinant in (7.19) is equal, up to a sign, to
(m −m)(−m −m)−(m −m)(m −m) = 2m m−2m m = 2m (m−m ), (7.21)
which is nonzero provided m = 0. Similarly, in case m − m is odd and m − m is
even, the determinant is nonzero provided m = 0. On the other hand, if m = 0,
the determinant (7.19) is nonzero if m −m is odd and similarly, if m = 0, the deter-
minant (7.19) is nonzero if m −m is odd. In all other cases, the determinant is zero.
Summarizing we conclude that (7.19) is nonzero and hence (7.17) holds when-
ever m, m , m are disjoint and the nonzero ones among them are not of the same
parity. For such a choice of m, m , m with m ≥ 1, we have the normalization
conditions
 
trm−1 ϕm+1,m,l−m = 0, trm ϕm +1,m ,l−m = 0, trm ϕm +1,m ,l−m = 0,
(7.22)
that determine uniquely g1l (z), f2,l−1 (z) and f0l . Such a choice of m, m , m is
always possible unless l ∈ {0, 1}. In case l = 0, all terms in (7.13) are already zero.
If l = 1, the only choice is m = 1 and m = 0 leaving no space for m . In that case
we still obtain the normalization
ϕ101 = 0, ϕ210 = 0, (7.23)
which, however, does not determine g11 (z), f20 (z) and f01 uniquely. Instead, we
regard f01 as a free parameter and then (7.23) determine uniquely g11 (z) and
f20 (z). The free parameter f01 corresponds to the choice of a ∈ Cn in the following
group of automorphisms of the quadric Im w = z, z
:
(z + aw, w)
(z, w) → . (7.24)
1 − 2i z, a
− i a, a
w
To obtain the lowest possible degrees in (7.22), we have to choose m = 1,
m = 0, m = 0, in which case (7.22) gives
ϕ10l = 0, ϕ2,1,l−1 = 0, tr2 ϕ3,2,l−2 = 0, (7.25)
which is precisely a part of the Chern-Moser normal form [CM74].
338 D. Zaitsev

7.3. Normalization for k = 0

It remains to analyze the line through (0, 0, l) corresponding to the multi-degrees
(m, m, l − m), 0 ≤ m ≤ l, for which we have the equation (4.6). Similarly to the
above, we rewrite this equation as
ϕm,m,l−m (z, z̄)



l   l−1  
= Im g0l (i z, z
)m − 2 Re f1,l−1 (z), z
(i z, z
)m−1 + · · · .
m m−1
(7.26)
In view of the presence of the power im inside real and imaginary parts, we have
to deal separately with m being even and odd.
We first assume m to be even. In that case (7.26) can be rewritten as
ϕm,m,l−m (z, z̄)



l l−1
= (Im g0l )(i z, z
)m − 2i (Im f1,l−1 (z), z
)(i z, z
)m−1 + · · · .
m m−1
(7.27)
Arguing as above we can obtain the normalization
trm−1 ϕm,m,l−m = 0, (7.28)
implying an equation for f1,l−1 (z) and g0l which can be solved for Im f1,l−1 (z), z

in the form

−1

1 l−1 l
Im f1,l−1 (z), z
= (Im g0l ) z, z
+ · · · . (7.29)
2 m−1 m
The latter expression is to be substituted into another equation (7.27) with m
replaced by m (still even). As above, we obtain the normalization

trm−1 ϕm,m,l−m = 0, trm ϕm ,m ,l−m = 0, (7.30)
provided # 
# l  l−1  ##
# m  −2 m−1 #
# l # = 0, (7.31)
# m  −1 #
−2 ml−1
which always holds in view of (7.7). Hence Im g0l is uniquely determined by (7.30)
and therefore also Im f1,l−1 (z), z
.
Now assume m is odd. In that case (7.26) becomes
ϕm,m,l−m (z, z̄)



l l−1
= (Re g0l )i m−1
z, z
− 2
m
(Re f1,l−1 (z), z
)(i z, z
)m−1 + · · · .
m m−1
(7.32)
Then the above argument yields the normalization

trm−1 ϕm,m,l−m = 0, trm ϕm ,m ,l−m = 0, (7.33)
 New Normal Forms for Levi-nondegenerate Hypersurfaces 339

where this time both m and m are odd. This time (7.33) determines uniquely
both Re g0l and Re f1,l−1 (z), z
.
Summarizing, we obtain the normalization

trm−1 ϕm,m,l−m = 0, trm ϕm ,m ,l−m = 0,


trm−1 ϕm,
 m,l−
  = 0,
m

trm ϕm
  ,m   = 0,
  ,l−m (7.34)
where m ≥ 1, m = m, m   = m,
 both m, m are even and both m,  m  are odd
(note that automatically m  ≥ 1). The choice of such numbers m, m , m,  m  is
always possible unless l ∈ {0, 1, 2}. If l = 0, all terms in (7.26) are zero. If l = 1,
we obtain a valid identity since we have assumed g01 = 1, f01 = id. Finally, for
l = 2, we can choose m = 2, m = 0 for the even part and m  = 1 for the odd part
  . We obtain the normalization
but there is no place for m
ϕ002 = 0, ϕ111 = 0, trϕ220 = 0, (7.35)
which determines uniquely Im g02 (z) and f11 (z), whereas Re g02 (z) is a free pa-
rameter. The latter corresponds to the choice of r ∈ R in the following group of
automorphisms of the quadric Im w = z, z
:
(z, w)
(z, w) → . (7.36)
1 − rw
To obtain the lowest possible degrees in (7.34), we can choose m = 2, m = 0,
  = 3, which leads to the normalization
 = 1, m
m
ϕ00l = 0, ϕ1,1,l−1 = 0, trϕ2,2,l−2 = 0, tr3 ϕ3,3,l−3 = 0, (7.37)
which is precisely a part of Chern-Moser normal form [CM74]. However, there is
another choice in the lowest degree, namely m = 2, m = 0, m   = 1, in
 = 3, m
which case the normalization reads
ϕ00l = 0, trϕ1,1,l−1 = 0, trϕ2,2,l−2 = 0, tr2 ϕ3,3,l−3 = 0. (7.38)
Comparing (7.37) and (7.38) we can say that the Chern-Moser normalization (7.37)
has more equations for ϕ1,1,l−1 and less equations for ϕ3,3,l−3 . In a sense, the
Chern-Moser normalization corresponds to the maximum conditions in the lowest
possible degree.
Summarizing the results of this section, we obtain the proof of Theorem 1.1.

References
[BER00a] Baouendi, M.S.; Ebenfelt, P.; Rothschild, L.P. – Convergence and finite deter-
mination of formal CR mappings. J. Amer. Math. Soc. 13 (2000), 697–723.
[BER00b] Baouendi, M.S.; Ebenfelt, P.; Rothschild, L.P. – Local geometric properties of
real submanifolds in complex space. Bull. Amer. Math. Soc. (N.S.) 37 (2000),
no. 3, 309–336.
[CM74] Chern, S.S; Moser, J.K. – Real hypersurfaces in complex manifolds. Acta Math.
133 (1974), 219–271.
340 D. Zaitsev

[E98a] Ebenfelt, P. – Normal forms and biholomorphic equivalence of real hypersur-

faces in C3 . Indiana Univ. Math. J. 42 (1998), 311–366.
[E98b] Ebenfelt, P. – New invariant tensors in CR structures and a normal form
for real hypersurfaces at a generic Levi degeneracy, J. Differential Geom. 50
(1998), 207–247.
[ES95] Ebenfelt, E.; Shapiro, H.S. – The mixed Cauchy problem for holomorphic
partial differential operators. J. Analyse Math. 65 (1995), 237–295.
[F17] Fischer, E. – Über die Differenziationsprozesse der Algebra. J. für Math. 148
(1917), 1–7.
[HY08] Huang, X.; Yin, W. – A Bishop surface with a vanishing Bishop invariant,
Invent. Math., to appear.
[Ko05] Kolar, M. – Normal forms for hypersurfaces of finite type in C2 . Math. Res.
Lett. 12 (2005), no. 5-6, 897–910.
[MW83] Moser, J.K.; Webster, S.M. – Normal forms for real surfaces in C2 near complex
tangents and hyperbolic surface transformations. Acta Math. 150 (1983), no.
3-4, 255–296.
[SS03] Schmalz, G.; Spiro, A. – Normal systems of coordinates on manifolds of Chern-
Moser type. J. Korean Math. Soc. 40 (2003), no. 3, 461–486.
[S89] Shapiro, H.S. – An algebraic theorem of E. Fischer, and the holomorphic Gour-
sat problem. Bull. London Math. Soc. 21 (1989), 513–537.
[Z08] Zaitsev, D. – Normal forms for non-integrable CR structures. Preprint.
http://arxiv.org/abs/0812.1104.

Dmitri Zaitsev
D. Zaitsev
School of Mathematics
Trinity College Dublin
Dublin 2, Ireland
e-mail: [email protected]