On Parity Unimodality of $q$-Catalan Polynomials

A polynomial $A(q)=\sum_{i=0}^n a_iq^i$ is said to be unimodal if $a_0\le a_1\le \cdots \le a_k\ge a_{k+1} \ge \cdots \ge a_n$. We investigate the unimodality of rational $q$-Catalan polynomials, which is defined to be $C_{m,n}(q)= \frac{1}{[n+m]} \left[ m+n \atop n\right]$ for a coprime pair of positive integers $(m,n)$. We conjecture that they are unimodal with respect to parity, or equivalently, $(1+q)C_{m+n}(q)$ is unimodal. By using generating functions and the constant term method, we verify our conjecture for $m\le 5$ in a straightforward way.


Introduction
We will consider the unimodality of some symmetric polynomials. A sequence a 0 , a 1 , . . . , a n is said to be symmetric if a i = a n−i for all i. It is said to be unimodal if there is a k such that a 0 ≤ a 1 ≤ · · · ≤ a k ≥ a k+1 ≥ · · · ≥ a n . It is said to be unimodal with respect to parity if a 0 , a 2 , . . . and a 1 , a 3 , . . . are both unimodal. A polynomial P (q) = a 0 + a 1 q + · · · + a n q n of degree n is said to be symmetric (resp., unimodal) if its coefficient sequence a 0 , a 1 , . . . , a n is symmetric (resp., unimodal).
Stanley gave a nice survey [22] on various methods for showing that a sequence is unimodal (or log-concave which we will not discuss here). A classical example is the following. This important result has many proofs. See [4,6,7,17,19,22,29,30]. It is an outstanding open question to find an explicit order matching proof for the unimodality. Though the Gaussian polynomials have been extensively studied, the closely related q-Catalan polynomials are less studied.
Conjecture 2. The q-Catalan polynomials C n (q) are unimodal with respect to parity.
We find that Conjecture 2 can be extended for rational q-Catalan polynomials. For a pair (m, n) of positive integers, define When m and n are coprime to each other, i.e., gcd(m, n) = 1, C m,n (q) is known to be in N[q] (a polynomial with nonnegative coefficients), and is called the (m, n) rational q-Catalan polynomials (or q-Catalan numbers). See, e.g., [13].
Conjecture 3. For a coprime pair of positive integers (m, n), the (m, n)-rational q-Catalan polynomials C m,n (q) are unimodal with respect to parity.
Conjecture 4. For a pair of positive integers (m, n), the polynomialC m,n (q) is unimodal with respect to parity.
Conjecture 4 includes Conjecture 3 as a special case, sinceC m,n (q) reduces to C m,n (q) when gcd(m, n) = 1. We state the latter separately because the combinatorial meaning of C m,n (q) is much more elegant as we will explain later in Section 5. Conjecture 3 includes Conjecture 2 as a special case, since C n+1,n (q) = C n (q). We state the latter separately because C n (q) has a different combinatorial interpretation. See Section 5.
We have verified Conjecture 4 for m, n ≤ 180 by Maple. Observe thatC m,km (q) = [m]C m,km = km+m−1 km q , which by Theorem 1 is indeed unimodal. We will prove this conjecture for m ≤ 5. Our method is to compute the corresponding generating functions by means of the constant term method. It turns out that for m ≤ 5, the positivity is transparent in view of their generating functions. This paper is organized as follows. Section 1 is this introduction. Section 2 introduces the basic concepts and idea for our proof. The unimodality conjectures are translated by using their generating functions. Section 3 tries direct computation, which already becomes complicated for m = 4. Section 4 uses the constant term method to compute the corresponding generating functions, whose positivity is transparent and hence proves Conjecture 3 for m ≤ 5. In Section 5, we discuss possible representation approach for settling these conjectures. We also introduce the combinatorial interpretations of q-Catalan polynomials.

Preliminary
A Laurent polynomial L(q) is said to be symmetric if L(q −1 ) = L(q), and is said to be anti-symmetric if L(q −1 ) = −L(q).
Then the symmetry of P (q) (i.e., P (q) = q n P (q −1 )) is transformed to the more natural symmetry of the Laurent polynomial N P (q). The following properties are easy to verify and will be used without mentioning: (1) For polynomials P (q) and Q(q), we have N (P (q)Q(q)) = N (P (q))N (Q(q)).
(2) The product of two symmetric Laurent polynomials is still symmetric.
We will also use the following linear operators on Laurent polynomials in Q[q, The following lemma is transparent.
Lemma 5. Suppose P (q) is a symmetric polynomial of degree n. Then (2) P (q) is unimodal with respect to parity if and only if Thus Conjecture 3 can be restated as follows.
Conjecture 3 a . For a coprime pair of positive integers (m, n), the polynomial has nonnegative integer coefficients.
Our point is that it is usually easier to consider generating functions. Let Note that the coefficients in x are not always Laurent polynomials in q. We take F m (x, q) as an element in Q((q))[[x]], the ring of power series in x with coefficients Laurent series in q.
For integers m > r ≥ 0, let X m,r be the linear operator acting on Q(q) [[x]] by X m,r n≥0 a n (q)x n = k≥0 a km+r (q)x k . (extracting the terms with special exponents) When gcd(m, r) = 1, Then Conjecture 3 a is transformed as follows.
is a power series in x with coefficients in N[q].
The case gcd(m, r) = d is a little complicated. We need to consider the generating function . Thus Conjecture 4 can be transformed as follows.
is a power series in x with coefficients in N[q].
We remark that is a power series in x with coefficient Laurent series in q, so we need to extend the PT q operator. See Section 4.
To our surprise, F m (x, q) has a product formula as follows.
Proposition 6. For any positive integer m, we have Proof. The proposition is indeed a consequence of the following well-known identity. See, e.g., [23, pp. 272]. .
We can get

Direct computation
Conjecture 3 can be verified directly for m = 3, but already becomes complicated for m = 4.
3.1. The case m = 3. In this case, we have the following explicit expansion. (1) Consequently, Conjecture 3 holds true for m = 3.

3.2.
The case m = 4. This case is already complicated. We have We can have explicit polynomial representation, but that will not help to prove our conjecture. For instance, if n = 12k + 1, then Now it is unclear why its coefficients in q r is negative for r ≤ 3n−1 2 .

The generating function method
4.1. Basic idea. We illustrate the idea by redoing the case m = 3, n = 3k +1. Consider the generating function By taking partial fraction decompositions in q, we obtain: .
When expanding as a power series in x, the first term has only negative powers in q and the second term has only positive powers in q: Thus by extracting positive powers in q, we obtain Conjecture 3 b thus holds for the case (m, r) = (3, 1).
Extracting positive powers in q of a general class of rational series can be done systematically by the constant term method. The resulting rational function turns out to be trivially positive for m ≤ 5.

4.2.
The constant term method. Constant term extraction or residue computation has a long history. See, e.g., [26] for further references. The fundamental problem we are concerned here is to compute the constant term of in a set of variables of a formal series in the field of iterated Laurent series K = Q((x n )) · · · ((x 1 )), which is called the working field. The reader is referred to [9] for the original development of the field of iterated Laurent series. Here we only recall that K defines a total ordering 0 < x 1 < x 2 < · · · < x n < 1 on the variables (more precisely, a total group order on its monomials), which can be formally treated as 0 < < x 1 < < x 2 < < · · · < < x n < < 1. Every rational function has a unique series expansion in K. We will focus on the class of Elliott-rational functions, which are rational functions whose denominators are the product of binomials. It is known that the constant term of an Elliott-rational function is still an Elliott-rational function. Efficient algorithms have been developed to evaluate this type of constant terms, such as the Omega Mathematica package [2,3], Ell Maple package [27] developed from Algebraic Combinatorics. See [25] for further references on algorithmic development from Computational Geometry and Algebraic Combinatorics.
We will use the first author's (updated) Ell2 Maple package. We use a list xvar = [x 1 , x 2 , . . . , x n ] to specify the working field Q((x n )) · · · ((x 1 )). Let var = [x i 1 , . . . , x is ] be a list of variables to be eliminated, then the constant term of an Elliott-rational function can be evaluated by the command E OeqW (F, xvar, var) after loading the Ell2 package.
In what follows, we always specify the working field K by letting 0 < x < q < λ < 1. This K includes all the rings, such as Q((q))[[x]], appear below as a subring. Firstly, we shall explain how to realize the PT q and X m,r operators by the constant term operator.
For anti-symmetric Laurent polynomials L(q), we have PT q L(q) = − NT q L(q) ]. It is convenient for us to use The above identities are easily verified for F (x, q) = q i x j and then extended by linearity for arbitrary F (x, q).
Let us redo the m = 3 case for the sake of clarity. The cases n = 3k + 1 and n = 3k + 2 can be done simultaneously. By starting with the formula .

4.3.
The case m = 4. We shall establish the following result. Proof. We start with the formula By constant term extraction, we obtain which do not show the positivity directly. By applying X 4,r for r = 1, 3, we obtain This proves Conjecture 3 b for (m, r) = (4, 1), (4,3).
For the case (m, r) = (4, 2), we need to compute For the case (m, r) = (4, 0), we need to compute Proof. We start with the formula .

4.5.
The cases m ≥ 6. When we calculated the case m ≥ 6 in a similar way, we are not able to prove the positivity in a straightforward way as before. Let us explain the problem by working with the m = 6 case. We start with the formula .
By the constant term method, we obtain where M 6 is a polynomial of many terms that does not show positivity. We can apply X 6,r for r = 1, 5, corresponding to the gcd(m, r) = 1 case. Neither of the two cases shows the positivity directly.
For the gcd(m, r) = 2 case, i.e., r = 2, 4, we need to compute We can apply X 6,r for r = 2, 4. Neither of the two cases shows the positivity directly.
For the gcd(m, r) = 3 case, i.e., r = 3, we need to compute We can apply X 6,3 , and the result does not show the positivity directly.
For the case r = 0, we need to compute We can apply X 6,0 , and the result does not show the positivity directly.

4.
6. An extension. The computation of the case m = 6 suggests that we need to consider different cases for proving Conjecture 4 b . However, we find a possible unified way to attack the conjecture. More precisely, let or equivalently, , and it is easy to verify that: [qx n ]G m (x, q) = [q]N (q 2 − 1)C m,n (q) = 0 only when m, n are both even, [q]G 10 (x, q) = − x 6 q + x 10 q + positive terms.
We summarized all the other cases in the following conjecture.
Conjecture 10. Let G m (x, q) be as in (3). Then G m (x, q) is almost positive for m ≥ 3. More precisely, besides the above formula, we have (1) For every k ≥ 3, [q]G 2k (x, q) has only finitely many negative terms.
A unified approach for m = 4 and r = 1, 3 can be given as follows. We have It follows that which is clearly positive.
Generally, for odd m, we need to show the positivity of G m (x, q). It is possible to decompose G m (x, q) as a sum of trivially positive rational functions. The decomposition is nontrivial even for G 5 (x, q). The even m case needs a minor modification. We succeeded in doing this type of decomposition in [20], and hence decomposition of G m (x, q) for small m, at least for m ≤ 6, should be possible. This suggested us to reconsider the following problem in the near future.
Problem: Given an Elliott rational function Q, how to decompose Q = i Q i with Q i all trivially positive if possible.

5.1.
Combinatorial interpretation of C m,n (q). In this section (m, n) is always a coprime pair of positive integers, unless specified otherwise. The general case gcd(m, n) = d > 1 can be considered but is much more complicated.
We should mention that representation theory maybe suitable for settling our conjectures. For instance, Conjecture 3 can be proved if we can find (usually hard to find) an sl(2) module whose character is q −(m−1)(n−1)/2 C m,n (q). This is based on the following well-known result. See, e.g., [22,Theorem 15].
Let D m,n be the set of Dyck paths in the m × n lattice rectangle, i.e., paths from (0, 0) to (m, n) with unit North step and unit East step, that never go below the diagonal line y = nx/m. The rational q, t Catalan polynomials are defined by where the sum is over Dyck paths in the m × n lattice rectangle, area(D) gives the number of lattice squares between the path and the diagonal, and dinv(D) is a Dyck path statistic that can also be given a relatively simple geometric construction. There is also an equivalent interpretation in terms of simultaneous core partitions. See [5,24].
It seems hard to show the parity unimodality of C m,n (q) by this model, because the dinv statistic is still hard to understand.
A mysterious property of C m,n (q, t) is its symmetry in q and t, i.e., C m,n (q, t) = C m,n (t, q). As a symmetric polynomial, C m,n (q, t) has a Schur expansion where λ can has only two parts, say λ = (λ 1 , λ 2 ), and where [k] q,t = q k−1 + q k−2 t + · · · + t k−1 .
Then the (q, t) Schur positivity of C m,n (q, t) (i.e., c λ ≥ 0 for all λ) implies the unimodality of C m,n (q) with respect to parity.
The symmetry of C m,n (q, t) is a consequence of the rational shuffle conjecture, which can be written as Q m,n (−1) n = H m,n (X; q, t), where H m,n (X; q, t) is the Hikita polynomial that has combinatorial interpretation as a sum over rational parking functions [14], and Q m,n is a symmetric function operator introduced by Gosky and Negut [11]. The rational Shuffle conjecture was proved by Mellit [18]. Detailed definitions are too involved. The reader is referred to [21] for further information on this topic. We should mention that no combinatorial proof of this symmetry is known up to now.
As a symmetric function in X, we can write Then C m,n (q, t) is just [s 1 n ] m,n . From the algebraic side, H m,n (X; q, t) is easily seen to be q, t symmetric, so is its coefficients [s λ ] m,n . It is then natural to conjecture that [s λ ] m,n (q, t) is (q, t) Schur positive. The positivity (though not stated this way) has been proved for the case n = 2 by Leven [15] and for the case n = 3 by Qiu and Remmel [21].

5.2.
Combinatorial interpretation of C n (q). Since C n (q) = C n+1,n (q), we have a combinatorial interpretation of C n (q). Indeed, let D n be short for D n,n . Then we have The second equality follows by the symmetry of C n (q) and by application of the zeta map, which is a bijection from D n to itself that takes dinv to area and area to bounce.
See [24,16]. Usually we think the statistic bounce is simpler than dinv in this case, (while for D ∈ D m,n , the dinv(D) is known but the bounce(D) is not).
Currently the simplest way to compute coarea(D) and bounce(D) might be as follows (see [28]). Firstly, there is a easy way to convert D ∈ D n to a standard Young tableau T (D) of shape (n, n). Then coarea(D) is just the sum of the first row entries minus n 2 , and bounce(D) is the sum of the first row ranks, where the ranks of the entries of T can be computed in a simple way: i) r(1) = 0; ii) r(i) = r(i − 1) if i is in the first row; iii) r(j) = r(i) + 1 if j is under i. For example, Figure 1 illustrates these statistics for the case n = 3. There is a better interpretation found earlier than the above statistics. See, e.g., [23]. where maj(D) is the major index of D, usually defined as the sum of the descent positions. (Here a descent corresponds to a EN turn). The major index is also defined for standard Young tableaux. We only exhibit the major index for C 3 (q). There are also two closely related results. One is the following [22, p. 523].
The other one is the following conjecture. See [8].
Conjecture 13. Write C n (q) = k m n (k)q k . The sequence (m n (1), m n (2), . . . , m n (n(n− 1) − 1)) is unimodal when n is sufficiently large. (Seem to hold for n ≥ 16.) If this conjecture is true for n ≥ 16, then Conjecture 3 is also true because the n ≤ 16 cases are easily verified to be true.