Pattern avoiding permutations with a unique longest increasing subsequence

We investigate permutations and involutions that avoid a pattern of length three and have a unique longest increasing subsequence (ULIS). We prove an explicit formula for 231-avoiders, we show that the growth rate for 321-avoiding permutations with a ULIS is 4, and prove that their generating function is not rational. We relate the case of 132-avoiders to the existing literature, raising some interesting questions. For involutions, we construct a bijection between 132-avoiding involutions with a ULIS and bidirectional ballot sequences. Mathematics Subject Classifications: 05A05, 05A15, 05A16, 05A19


Introduction
Let p = p 1 p 2 · · · p n be a permutation. An increasing subsequence in p is just a subset of entries p j 1 < p j 2 < · · · < p j k so that j 1 < j 2 < · · · < j k . Note that the positions j 1 , j 2 , · · · , j k are not required to be consecutive.
We say that p has a unique longest increasing subsequence, or ULIS, if p has an increasing subsequence that is longer than all other increasing subsequences. For instance, 2314 has a ULIS, namely the sequence 234, but p = 246135 does not, since 246, 245, 235, and 135 are all increasing subsequences of maximal length in p.
Finding the number of all permutations of length n that have a unique longest increasing subsequence appears to be a difficult problem. These numbers are known only for n 15, given by sequence A167995 in the Online Encyclopedia of Integer Sequences [9].
In this paper, we instead consider permutations that avoid a given pattern q of length three that have a ULIS. We say that a permutation p contains the pattern q = q 1 q 2 · · · q k if there is a k-element set of indices i 1 < i 2 < · · · < i k so that p ir < p is if and only if q r < q s . If p does not contain q, then we say that p avoids q. For example, p = 3752416 contains q = 2413, as the first, second, fourth, and seventh entries of p form the subsequence 3726, which is order-isomorphic to q = 2413. A recent survey on permutation patterns can be found in [12] and a book on the subject is [2]. The basic facts that we will simply call well-known are explained in detail in these sources.
If q is any given pattern of length three, then it is well known that the number of all permutations of length n that avoid q is the Catalan number C n = 2n n /(n + 1). As structures counted by the Catalan numbers have been extensively studied, we find it surprising that the questions discussed in this paper prove to be both difficult and unexplored. The diverse nature of the results we prove will also be interesting as we will see that depending on the pattern q, the portion of q-avoiding permutations that have a ULIS may converge to a positive constant, converge to 0 at a subexponential speed, or converge to zero at an exponential speed. On the other hand, if q ∈ {123, 132, 213, 321}, then the number of q-avoiding involutions is n n/2 , while if q ∈ {231, 312}, then the number of q-avoiding involutions is 2 n−1 . We will address q-avoiding involutions with a ULIS in Section 5.
It is straightforward to see that if p = p 1 p 2 · · · p n has a ULIS, then so does its grouptheoretical inverse p −1 , and so does the reverse complement p revc = ((n + 1 − p n ) (n + 1 − p n−1 ) · · · (n + 1 − p 1 )) of p. Therefore, the six possible choices of q can be reduced to four, namely 123, 132, 231, and 321, and one of them, 123, leads to a trivial situation as we will discuss in the last paragraph of this section. The pattern 213 is the reverse complement of 132, and the pattern 312 is the inverse of 231.
We denote by u n (q) the number of permutations of length n that avoid the pattern q and have a ULIS, with u 0 (q) = 1, since the empty permutation does have a ULIS. Similarly, we denote by i n (q) the number of q-avoiding involutions of length n which have a ULIS.
If p avoids 123 and has a ULIS, then that ULIS has to be of length two, if the length of p is more than 1. Permutations with that property are exactly the permutations with one non-inversion. Therefore, u n (123) = n − 1 for n > 1, and u 1 (123) = 1. Such a permutation is an involution if and only if that non-inversion creates two fixed points, yielding that i n (123) = 0 if n > 1 and n is odd, i 1 (123) = 1, and i n (123) = 1 if n is even.

The pattern 231
Let p = p 1 p 2 · · · p n be a permutation that avoids 231. Let p i = n. Let L = p 1 p 2 · · · p i−1 , and let R = p i+1 · · · p n . Note that all entries in L must be smaller than all entries of R. If p has a ULIS, then it is necessary for both L and R to have a ULIS (since increasing sequences in L or R can be extended to increasing sequences in p).
On the other hand, if L and R both have a ULIS, then the only case in which p fails to have a ULIS is when R consists of exactly one entry, (which is necessarily the entry n − 1). Introducing the ordinary generating function F (z) = n 0 u n (231)z n , this leads to the generating function identity This yields showing that the first few numbers u n (231), starting with n = 0 are, 1, 1, 1, 2, 5, 13, 35, 97, 275, 794. This is sequence A082582 in the Online Encyclopedia of Integer Sequences [9]. The dominant singularity of F (z) is close to 0.2956, implying that the exponential growth rate of the sequence u n (231) is the reciprocal of that number, that is, close to 3.383.
The following simple observation will be useful for us one more time in this paper, so we formally announce it. We say that a pattern is sum indecomposable if it cannot be cut into two parts so that every entry before the cut is smaller than every entry after the cut. For instance, q = 3142 is sum indecomposable, but q = 21453 is not as it can be cut between the entries 1 and 4. Proposition 1. Let q be a sum indecomposable pattern. Then the limit lim n→∞ (u n (q)) 1/n exists.
Proof. For any positive m and n, the inequality u m (q)u n (q) u m+n (q) holds. Indeed, if p and r are q-avoiding permutations with a ULIS, then the sum indecomposability of q implies that so is p ⊕ r, that is, the permutation of length m + n that starts with p on the entries {1, 2, · · · , m}, then continues with a r-pattern taken on the entries {m + 1, m + 2, · · · , m + n}. Therefore, by Fekete's Lemma on superadditive sequences, the limit of the sequence lim n→∞ (u n (q)) 1/n exists.
We know that the limit in Proposition 1 is finite, since it is at most as large as the analogously defined limit for all q-avoiding permutations, and that is well-known to be finite.
Proposition 1 and (1) together imply that lim n→∞ (u n (231)) 1/n ≈ 3.383. We could have determined this without Proposition 1, because (1) provided an explicit formula for F (z). In Section 4, we will not have such a form for the relevant generating function, but Proposition 1 will still enable us to compute corresponding limit.

The pattern 132
From our perspective, the pattern 132 behaves in surprising ways. This leads to some intriguing questions, and even in the case when the answer to those questions is known, there is room for simpler proofs.
We say that a permutation p is skew indecomposable if it is not possible to cut p into two parts so that each entry before the cut is larger than each entry after the cut. For instance, p = 3142 is skew indecomposable, but r = 346512 is not as we can cut it into two parts by cutting between entries 5 and 1, to obtain 3465|12.
If p is not skew indecomposable, then there is a unique way to cut p into nonempty skew indecomposable strings s 1 , s 2 , · · · , s of consecutive entries so that each entry of s i is larger than each entry of s j if i < j. We call these strings s i the skew blocks of p. For instance, p = 67|435|2|1 has four skew blocks, while skew indecomposable permutations have one skew block.
There is a well-known bijection ψ from the set of all 132-avoiding permutations of length n to the set of all plane rooted unlabeled trees on n + 1 vertices. If p is a 132avoiding, skew indecomposable permutation of length n, then it necessarily ends in n, so it is of the form Ln. We then define ψ(p) to be the plane tree whose root has only one child, and the subtree rooted at that child is precisely ψ(L). If p is a skew decomposable 132-avoiding permutation of length n, then p = B 1 B 2 · · · B j , where the B i are the skew blocks of p, each of which is necessarily skew indecomposable, and so ends with its largest entry. Therefore, for all i, the tree ψ(B i ) is a tree in which the root has only one child. Then ψ(p) is obtained by taking the sequence ψ(B 1 ), ψ(B 2 ), · · · ψ(B j ) in this order, and contracting the roots of these j trees into one vertex, which will be the root of ψ(p).
It is then easy to see that the number of longest increasing subsequences of p is equal to the number of leaves in ψ(p) that are at maximum distance from the root. Then the following is known.
Theorem 2. Let us select a rooted plane unlabeled tree on n vertices uniformly at random, and let a n,k be the probability that the selected tree has k leaves at maximum distance from the root. Then lim n→∞ a n,k = 2 −k .
A very general theorem, that contains this result, is Theorem 2 in [8]. An earlier, and more specific, reference is [7].
Setting k = 1, we get the result that is equivalent to the question we were interested in. As the proofs of Theorem 2 are probabilistic, it is natural to ask the following question. The sequence of the numbers u n (132) is in the Online Encyclopedia of Integer sequences [9], as sequence A152880. Its first few entries, starting with n = 1, are 1, 1, 3, 8, 23, 71, 229, 759, 2566. This numerical evidence raises two interesting questions.
Question 5. Is it true that u n (132)/C n 0.5 for all n? In other words, is it true that for all n, there are at least as many 132-avoiding permutations of length n that have a ULIS as 132-avoiding permutations of length n that do not have a ULIS?
the electronic journal of combinatorics 27(4) (2020), #P4.44 Needless to say, an injective proof of a positive answer to this question would be particularly interesting. Question 6. Is it true that the sequence u n (132)/C n is monotone decreasing for n 3?
Note that an affirmative answer to Question 6 would imply an affirmative answer to Question 5, because of the convergence result of Corollary 3.

The pattern 321
The case of q = 321 is surprisingly difficult to handle. While we are not able to exhibit an explicit formula for the numbers u n (321), we are able to show that lim n→∞ (u n (321)) 1/n = 4, and that the generating function G(z) = n 0 u n (321)z n is not rational.
In what follows, we assume that the reader is familiar with the basic properties of the Robinson-Schensted (RS) correspondence from the set of all permutations of length n to the set of pairs of standard Young tableaux of the same shape on n boxes. These basic properties, which we will call well-known, are discussed in detail in Sagan's book [10]. It is well-known that the RS bijection maps 321-avoiding permutations into pairs of SYT of the same shape that have at most two rows. Note that with slight abuse of language, we will call the shape of the standard Young tableaux associated to p by the RS correspondance the shape of p.
A left-to-right maximum in a permutation is an entry that is larger than all entries on its left, while a right-to-left minimum is an entry that is smaller than all entries on its right. For instance, in 21354, the left-to-right maxima are 2, 3, and 5, while the right-to-left minima are 1, 3, and 4.
We are going to prove a lower bound for the number u 2m+1 (321). Our main tool is a bijection of Claesson and Kitaev [5]. This bijection, which we will call f , maps the set of all 321-avoiding permutations into the set of sum indecomposable 321-avoiding permutations as follows.
Let p be a 321-avoiding permutation of length n. Locate all left-to-right maxima on the right of the entry 1 that are not right-to-left minima, and underline them. Insert a new maximum entry n+1 immediately to the left of the entry 1, and underline it. Finally, cyclically translate the set of underlined entries one notch to the left so that n + 1 becomes the rightmost underlined entry to get f (p). Proof. First, consider f (p), where p is 321-avoiding. The only way a 321-pattern is created by f is if the entry directly before 1 in f (p) introduces a 321-subsequence. However, if x is the entry of f (p) that lies directly before 1, then it is a left-to-right maximum. So there are no entries before it that are greater than it. If two elements less than x appear after it in descending order, then consider the first of these entries, y. Notice y could not have been an underlined entry, since it is less than x. So, y would have either been a right-to-left minimum in p, which contradicts that there is an entry appearing after y that is smaller, or was not a left-to-right maximum in p, in which case p would have contained the pattern 321. So f preserves 321-avoidance. Now, let us assume that f (p) is decomposable. That means that there exists an i so that the rightmost i entries of f (p) are the largest i entries of f (p), that is, the entries {n − i + 2, n − i + 3, · · · , n + 1}. This is clearly not possible if the entry n + 1 does not get moved into one of these last i positions, so we can assume that it does. However, in that case, one of the last i entries of p gets moved out of one of the last i positions, and that entry x is a left-to-right maximum that is not a right-to-left minimum. So, there is a right-to-left minimum y on the right of x that stays in one of the last i positions. This contradicts the assumption that the rightmost i positions of f (p) contain the i largest entries of f (p).
In order to prove that f is a bijection, it suffices to show that it has an inverse. And that is easy, since the set of underlined entries is easy to recover from f (p), by simply taking the entry immediately on the left of the entry 1, then moving to the right and selecting all left-to-right maxima that are not right-to-left minima.
The rank of an entry is the length of the longest increasing subsequence ending in that entry. It follows that entries of the same rank form a decreasing subsequence. Therefore, in a 321-avoiding permutation, there are at most two entries of each rank.
We will be particularly interested in permutations of shape (m, m). It is well-known that the number of standard Young tableaux of shape (m, m) is the Catalan number C m = 2m m /(m + 1), and therefore, the number of permutations of shape (m, m) is C 2 m . If the permutation p has shape (m, m), then each entry has rank at most m, and there are no decreasing subsequences of length three or more, so p has exactly two entries of each rank i. Let us denote the leftmost entry of rank i by A i and the rightmost entry of rank i by B i . Then A i is a left-to-right maximum, and B i is a right-to-left minimum. For shortness, we will refer to the A i as the large entries and the B i as the small entries.
The following property of the map f is crucial for us.
Proposition 10. Let p be a permutation of shape (m, m). Then f does not increase the rank of any entry. That is, the rank of the entry j in f (p) is at most as large as the rank of the entry j in p.
Proof. First, let us prove the statement for large entries. If A i is a large entry in p, then f (p) either keeps A i fixed, or moves A i to the left. So in either case, no entry leapfrogs A i . So, the set of entries that are on the left of A i that are smaller than A i does not acquire any new elements, therefore all increasing subsequences in f (p) that end in A i are also increasing subsequences in p that end in A i . Now let us prove the statement for small entries. Let us assume that B i is a counterexample to our claim with i minimal. That means that in f (p), there is an increasing subsequence longer than i that ends in B i . As the small entries are not displaced by f , that subsequence must contain a large entry; let A j be the largest such entry in that subsequence. That means that A j is the last element of that subsequence before B i . Then A j must have rank at least i, since B i supposedly has rank larger than i. As the rank of A j is not increased by f , that means that A j has rank i or more in p. That is a contradiction, since A i > B i , so A j > B i , and so A j cannot be part of an increasing subsequence ending in B i . Lemma 11. Let p be a permutation of shape (m, m). Then f (p) has a ULIS.
Proof. Note that the map f does not change the set of right-to-left minima of p, so f (p) has m right-to-left minima. However, f (p) has m + 1 left-to-right maxima, namely the m left-to-right maxima of p, and the new maximum entry 2m + 1. So, the left-to-right maxima form an increasing subsequence of length m + 1 in f (p). We will prove that this sequence is the ULIS of p.
Let us assume the contrary, that is, that there is an increasing subsequence of length m+1 in f (p) that contains at least one small entry. Let B i be the rightmost small entry in such a subsequence. Note that this B i is necessarily (weakly) on the right of 1 as B 1 = 1 is the leftmost small entry. By Proposition 10, the rank of B i in f (p) is at most i. On the other hand, any increasing subsequence of large entries that are on the right of B i in f (p) corresponds to such a subsequence in p, and therefore, is of length at most m − i, since the largest rank of any entry in p is m. (Note that both in p and f (p), large entries on the right of B i are automatically larger than B i .) This gives the chosen sequence in f (p) is of length at most m.
As f is injective, Lemma 11 shows that the number of permutations of shape (m+1, m) that have a ULIS is at least the number of permutations of shape (m, m), that is, where n = 2m + 1. Here we used Stirling's approximation that states that m! ∼ m e m √ 2mπ.
Proof. Proposition 1 shows that the limit L in the statement of the theorem exists. On the other hand, (2) proves that 4 L, while the fact that u n (321) C n proves that L 4.
Even though we do not know the explicit form of the ordinary generating function G(z) = n 0 u n (321)z n , we can use the result of Theorem 12 and a method recently used in [3] to prove that G(z) is not rational.
Theorem 13. The generating function G(z) = n 0 u n (321)z n is not rational.
Proof. A 321-avoiding permutation has a ULIS if and only if all its sum indecomposable blocks have a ULIS. Let u n,1 (321) be the number of sum indecomposable 321-avoiding permutations with a ULIS, and let G 1 (z) = n 1 u n,1 (321)z n . Then the equality holds. Let us assume that G(z) is rational, then so is G 1 (z). Both generating functions have their dominant singularity at z = 1/4, and, as they are both rational functions, those singularities are poles, implying that at the pole z = 1/4, the value of both generating functions goes to infinity. However, as all the coefficients of G 1 (z) are nonnegative, that implies that there is a z 0 ∈ (0, 1/4) so that G 1 (z 0 ) = 1. That means that u 321 (z) has a singular point at z 0 < 1/4, which is a contradiction.
Alternatively, we could argue as follows. Recalling that u n (321) C n for all n, we can compare the values of G(z) and C(z) = n 0 C n z n = 1− at their dominant singularity z 0 = 1/4, to see that Therefore, the point z 0 = 1/4 is a singular point of G(z) that is not a pole, so G(z) is not a rational function.

The pattern 231
If p is a 231-avoiding involution, then p also avoids the inverse of 231, that is, the pattern 312. That means that p is a layered permutation, meaning that it consists of a series of decreasing subsequences of consecutive entries, (the layers), so that the entries decrease within each layer but increase among the layers, as in 32154876. The only way for such a permutation to have a ULIS is by having layers of length one only. That happens if and only if p is the identity permutation, so i n (231) = 1 for all n.

The pattern 132
Consider the usual bijection φ from the set of 132-avoiding permutations of length n to the set of Dyck paths of semilength n. The latter are lattice paths from (0, 0) to (2n, 0) consisting of n steps (1, 1), called U(p) steps, and n steps (1, −1), called D(own) steps that never go below the line y = 0. The bijection φ is constructed inductively. If p is skew decomposable, then φ(p) = φ(B 1 )φ(B 2 ) · · · φ(B k ), where the B k are the skew blocks of p. If p is skew indecomposable, then p is necessarily of the form Ln, and then we set φ(p) = U φ(L)D.
A peak in a Dyck path is a point that is immediately preceded by an U step and is immediately followed by a D step. The height of a peak is its y-coordinate.
The following facts are straightforward to prove by induction.
Proposition 14. Let p be a 132-avoiding permutation. Then p has a ULIS of length k if and only if φ(p) has a unique peak of maximum height k.
Proposition 15. Let p be a 132-avoiding permutation. Then p is an involution if and only if φ(p) is symmetric to the vertical line x = n.
Therefore, if p is a 132-avoiding involution of length n so that φ(p) has a unique peak P of maximum height, then P must be on the vertical line x = n, otherwise it would not be unique.
Corollary 16. Let n be a positive integer. Then the number of 132-avoiding involutions of length n which have a ULIS is equal to the number of lattice paths π consisting of n + 1 steps (each of which are U steps or D steps) so that each prefix and each suffix of π contains strictly more U steps than D steps.
Proof. If p is a 132-avoiding involution of length n, just prepend the first half of φ(p) with an U step.
The lattice paths occurring in Corollary 16 have been studied in numerous papers, such as [4,6,13]. The main term of their asymptotic enumeration can be found in [4], while the most precise enumerative result is proved in [6], where these lattice paths are called bidirectional ballot sequences, and the number of such paths of length n is denoted by B n . The result is that B n 2 n ∼ 1 4n In particular, i n (132) = B n+1 ∼ 2 n+1 4(n + 1) = 2 n−1 n + 1 .
On the other hand, it is well-known that the total number of 132-avoiding involutions of length n is I n (132) = n n/2 ∼ 2 n · 2 πn . Therefore, the probability that a 132-avoiding involution of length n chosen uniformly at random has a ULIS is about π 8n .

The pattern 321
A theorem of Schützenberger [1,11] says that if p is an involution, then the number of fixed points of p is equal to the number of odd columns in the standard Young tableau P (p) into which the RS correspondence maps p. Now if p is 321-avoiding, then the columns of those tableaux are of length at most two, so all odd columns must be of length 1. Finally, note that if p = p 1 p 2 · · · p n is an involution, and p i 1 < p i 2 < · · · < p i j is an increasing subsequence in p, then the sequence i 1 < i 2 < · · · < i j is also an increasing subsequence in p = p −1 . So, it can be shown the only way for the involution p to have a ULIS is by having a ULIS consisting of entirely of fixed points.
Let p be a 321-avoiding involution with a ULIS of length k. Then P (p) has at most two rows, of length k and n − k, with k n − k. In particular, P (p) has k columns. On the other hand, as we proved in the previous paragraph, P (p) must have k columns of length 1. Therefore, all columns of P (p) must be of length 1, so k = n, and p = 12 · · · n. This proves that i n (321) = 1 for all n.