Equipopularity Classes in the Separable Permutations

When two patterns occur equally often in a set of permutations, we say that these patterns are equipopular. Using both structural and analytic tools, we classify the equipopular patterns in the set of separable permutations. In particular, we show that the number of equipopularity classes for length $n$ patterns in the separable permutations is equal to the number of partitions of $n-1$.

For permutations σ and τ of lengths n and m respectively, the direct sum (σ ⊕ τ) and skew sum (σ ⊖ τ) are defined as follows : and describing permutation classes has led to a variety of productive research and deep results relying on the combination of analytic, algebraic, and structural methods. Since classes are closed downward, if β ∈ C then every permutation in C must avoid β. In fact, any class can be defined as the set of permutations avoiding all the minimal (with respect to ≺) elements of its complement. These minimal elements are called the basis of the class.
Since every pattern occurring in a separable permutation is itself separable, the set of all separable permutations forms a permutation class; its basis is easily shown to be {2413, 3142}. Denote the set of separable permutations by S and for any class, C, denote by C n the set of permutations of length n within the class.
A statistic on a set of permutations is just a function from the set to N. This paper is concerned primarily with the statistic of pattern occurrences, that is, the number of times a pattern occurs in a permutation. We formalize this notion below.
Definition 1.1. Let σ be a (permutation) pattern, and let π be any permutation. Define the statistic ν σ on S by defining ν σ (π) to be the number of times the pattern σ occurs in the permutation π.
For example, the number of inversions of π is just ν 21 (π).
When considering a statistic it is a common first step to ask about its mean value, or equivalently, its sum. To that end, for a set X of permutations we define ν σ (X ) to be the total number of occurrences of σ in the set X , that is: In the set of all permutations, the total number of occurrences of a specified pattern depends only on the length of the pattern. To see this, let σ and τ be two permutations of length k and observe that for n ≥ k and any subset X ⊆ {1, 2, . . . , n} of cardinality k we can define a bijection between the set of permutations whose pattern at the indices X is σ and those whose pattern is τ (namely, leave the other elements fixed and rearrange the pattern of the elements on indices X, retaining the set of values). So, if σ has length k and we choose π uniformly at random in S n , then the probability that the pattern of π in any particular set of k indices is equal to σ is just 1/k!. Linearity of expectation then implies that When we restrict to proper permutation classes the situation becomes less trivial, as the bijections used above are not generally available. Given a permutation class C, we say that two patterns σ and τ are equipopular in C (denoted σ ∼ C τ, or just σ ∼ τ when C is clear from context) if the class contains the same number of occurrences of σ and τ at each length. That is: Obviously, ∼ C is an equivalence relation. All permutations not belonging to C are equipopular (since the number of occurrences of any such pattern in permutations of C is 0) and thus we generally restrict its domain to C. We refer to the equivalence classes of C as equipopularity classes. Note that all permutations belonging to an equipopularity class are of the same length, since the least n for which ν σ (C n ) > 0 is n = |σ|.
Motivated by a question of Cooper, Bóna [2,3] showed that, in the class of permutations avoiding the pattern 132, the patterns 213, 231, and 312 are equipopular. Homberger [5] extended this symmetry to show that the number of occurrences of the pattern 231 is identical among the permutations avoiding 123 and those avoiding 132. Rudolph [6] presented a surjection from integer partitions to equipopularity classes, and Chua and Sankar [4] showed that this map is actually a bijection. The class of 132 avoiding permutations is a proper subset of the separable permutations studied here.
This paper enumerates and classifies the equipopularity classes of the separable permutations. In Section 3 we use the structural decomposition of the separables to build a surjection from integer partitions classes onto equipopularity classes, generalizing the results of Rudolph. In Section 4 we show that this surjection is a bijection, generalizing the results of Chua and Sankar and proving that there are exactly partition-many equipopularity classes in the separable permutations.

THE SEPARABLE PERMUTATIONS
The separable permutations exhibit numerous symmetries, and their recursive structure will be a fundamental tool used in this investigation. Since every permutation π ∈ S can be written as a sequence of direct and skew sums of the permutation 1, it follows that π has a (not necessarily unique) decomposition into smaller separable permutations σ and τ in one of two different ways: Definition 2.1. A permutation is sum decomposable if it can be written as a direct sum of two nontrivial permutations, and sum indecomposable otherwise. Skew (in)decomposability is defined analogously.
Since it is clearly impossible for a permutation to be both sum and skew decomposable, it follows then that every sum decomposable separable permutation can be written uniquely as a direct sum of its sum indecomposable parts. That is, for every sum decomposable permutation π, there exists a unique integer k ≥ 2 and a unique sequence of sum indecomposable permutations σ i , for 1 ≤ i ≤ k such that π = σ 1 ⊕ σ 2 ⊕ · · · ⊕ σ k .
A similar decomposition holds for skew decomposable permutations.
The only separable permutation that is both sum and skew indecomposable is 1, so given a separable permutation π which is, say, sum decomposable, its sum indecomposable summands, if not 1, must be skew sums of skew indecomposable permutations. This provides a recursive decomposition of π, and this decomposition can be thought of as being represented by a particular sort of tree.

Definition 2.2.
A decomposition tree is a rooted plane tree in which each non-leaf node is labelled with either ⊕ or ⊖, and has at least two children. The non-leaf children of a node labelled ⊕ are labelled with ⊖, and vice versa.
For example, the tree Γ(π) for π = 215643798 is shown in Figure 2.1, and the full decomposition of π is as follows: These tree representations will be used in Section 3 to investigate and describe the relationships between equipopular patterns. This recursive structure will be used to develop functional relationships between the generating functions for pattern popularity, especially in Section 4. As an illustrative example, we rederive the enumeration of the separable permutations. Let S ⊕ and S ⊖ be the generating functions for the sum and skew decomposable separable permutations, respectively. Since every separable permutation of length at least two is either sum or skew decomposable, and since no permutation can be both, we have that Every sum decomposable separable permutation π can be written as σ ⊕ τ, where τ is separable and σ is sum indecomposable (and hence either skew decomposable or has length one). This leads to the following relationship: Finally, the reverse of a sum decomposable permutation is skew decomposable and vice versa, which implies: Combining equations 2.1-2.3 leads to a functional equation that can then be solved algebraically to obtain the generating function for the Schröder numbers:

EQUIPOPULAR PATTERNS
In this section we show that the number of equipopularity classes for patterns of length n is at most the number of partitions of the integer n. By the the decomposition of separable patterns, we determine sufficient conditions for two patterns to be equipopular (these conditions will be shown to be necessary in the next section). Our primary tool will be the decomposition trees introduced in the previous section. That is, we think of separable permutations and their decomposition trees interchangeably.
Recall that two patterns of a different length can never be equipopular, and that we are only interested in equipopularity for separable patterns (since no other patterns occur at all). Therefore, within the set of all decomposition trees of a fixed size, our goal is to classify those which correspond to equipopular patterns. This classification is obtained by describing structural transformations on trees which lead to equipopular patterns.

Preserving Equipopularity
For a specified pattern σ, define a σ-marked permutation to be a permutation π for which a subset of entries forming a σ pattern have been marked. Note that, for a set X of permutations, the total number of occurrences of a pattern σ in X , i.e., ν σ (X ), is equal to the number of distinct σ-marked permutations π ′ , where the unmarked permutation π lies in X .

Definition 3.1.
For a permutation π ∈ S n , the reverse, complement, and inverse of π, denoted π r , π c , and π −1 respectively, are defined as follows: These three operations respect pattern containment, in the sense that, for each i ∈ {r, c, −1}, we have These three symmetries generate a group of automorphisms of (S, ≺) isomorphic to the dihedral group of order eight, and Smith [9] showed that this is the full group of automorphisms of the Note that if π is separable, then the symmetries of π are also separable. Further, if we apply one of these symmetries f to a σ-marked permutation, we obtain an f (σ)-marked permutation. It follows then that if two patterns are related by a symmetry, then they are equipopular. Considering separable permutations as their decomposition trees, each symmetry corresponds to a tree transformation: complementation equates to inverting the signs at each internal node, reversal to inverting the signs and reversing the order of the children of every node, and inversion reverses the order of the children of each ⊖-node.
Pattern containment can be reconsidered entirely in terms of trees, though the analogous relationship is a bit more technical and requires an additional pair of definitions.

Definition 3.2.
For a decomposition tree T and a set L of leaves, the least common parent of L is the unique node which is furthest from the root and lies above each leaf in L. The skeleton of L is the tree whose vertices are the least common parents of each subset of L, and the ordering is induced from T by transitivity. The reduced skeleton of L is obtained by contracting any two same-sign adjacent nodes of the skeleton into a single node (with the same sign).
An example of the skeleton and reduced skeleton of a set of leaves is shown in Figure 3.1.

Lemma 3.3.
Let σ, π ∈ S. Then σ ≺ π if and only if there is some subset L of leaves of Γ(π) for which the reduced skeleton of L is equal to Γ(σ). Further, the position of the leaves of L within Γ(π) correspond to the position of the occurrence of σ within π.
Proof. This follows from the fact that the positional relationship between any pair of entries of π is determined by the left-to-right order of the corresponding leaves of Γ(π) while the value relationship is determined by the sign of the least common parent of those leaves. That is, if i < j then the leaf corresponding to π i is to the left of the leaf corresponding to π j , and if π i < π j then their least common parent is an ⊕-node, while if π i > π j their least common parent is an ⊖-node. Further, in the reduced skeleton of L, while the least common parent of a pair of leaves may correspond to a contraction of their least common parent in Γ(π) with some ancestral node, all nodes contracted into a single node share a common label.
Define a σ-marked decomposition tree to be a tree with marked leaves, such that the reduced skeleton of the marked leaves is equal to Γ(σ). By the first observation of this section about the correspondence between separable permutations and their decomposition trees and the result above it follows that the number of occurrences of σ in separable permutations of length n is equal to the number of distinct σ-marked trees with n leaves. When we have a σ-marked tree we will often consider the nodes belonging to the skeleton of the marked leaves to be marked as well.
We now seek to describe some transformations of decomposition trees that preserve equipopularity.
Since the labelling of a decomposition tree is uniquely determined by the label of its root, we permit some transformations that might require changing labels in some subtrees to ensure the alternation of ⊕ and ⊖ labels on internal nodes.

Proposition 1.
Let σ be a separable permutation and let v and w be nodes of Γ(σ) such that neither is an ancestor of the other. Let Γ(τ) be the decomposition tree obtained by exchanging the subtrees rooted at v and w in Γ(σ). Then σ and τ are equipopular.
Proof. We describe a bijection between σ-marked trees and τ-marked trees which preserves the number of leaves. Note that by the ancestry condition, both v and w must be internal nodes or leaves, and so have parents v ′ and w ′ respectively.
Let T be any σ-marked tree. In the marked skeleton of T, v ′ may correspond to a set of nodes, V ′ . However, there is a node in V ′ that has a child c v such that all the marked leaves below c v correspond to leaves of the subtree rooted at v (and all such leaves occur below c v ). Correspondingly, among the set of nodes W ′ corresponding to w ′ there is one that has a child c w such that all the marked leaves below c w correspond to leaves of the subtree rooted at w. Form a new marked decomposition tree by exchanging the subtrees rooted at c v and c w in T. This new tree is τ-marked. This process is clearly invertible and preserves the number of leaves. Therefore, σ and τ are equipopular.
See Figure 3.2 for an example.
Using just this proposition, we can repeatedly exchange the rightmost leaf of a tree with any internal node not in the rightmost branch producing, from some arbitrary initial separable permutation σ, a chain of equipopular permutations and culminating in one, τ where all but the rightmost child of any node are leaves. We will see analytically below that any two such τ are equipopular provided that the multisets of degrees of their internal vertices are equal. However, the following result also provides an alternative bijective proof of this fact -thereby allowing in principal the construction of bijections between σ-marked and τ-marked permutations whenever σ ∼ τ in the separable permutations. An alternative description of T ab (and one that accounts for the name) is that it is obtained from T by exchanging all but the rightmost tree of the forests below a and b respectively. See Figure 3.3 for illustration.

Proposition 2.
Let σ be a separable pattern, and let a and b be internal nodes of Γ(σ) for which a forest-exchange is possible. Let τ be the separable pattern such that Γ(τ) is equal to the tree resulting from the forest-exchange. Then σ and τ are equipopular.  Proof. The proof is analogous to that of the preceding proposition. Let a σ-marked tree T be given.
Among the nodes A of T which correspond to node a of Γ(σ) there is an eldest a ′ . Similarly define nodes b ′ and c ′ . Now perform the analog of the ab-exchange in T, but attaching, e.g., T c ′ to T a ′ \ T b ′ as the rightmost child of the rightmost vertex in A (and similarly for the other parts of the exchange). The resulting tree is τ-marked and the operation is invertible, proving the result.

Identifying Equipopular Permutations
The remainder of this section develops a method of identifying the equipopularity classes, and constructing a canonical representative from each. We first build a correspondence between partitions and classes. For every partition λ of an integer n, there exists a decomposition tree with n + 1 leaves having signature λ, as shown in Figure 3.4. Denote this tree by T(λ), and let ω(λ) be the permutation for which Γ(ω(λ)) = T(λ).
This family of trees provides a canonical element for each equipopularity class. Indeed, the following theorem shows that every decomposition tree can be transformed into such a λ-tree through the use of the equipopularity-preserving operations introduced in the previous section. Theorem 3.6. Let σ be a permutation, and suppose that Γ(σ) has signature λ. Then σ is equipopular to ω(λ).
Proof. As noted previously, we can first use Proposition 1 to find a tree equipopular to Γ(σ) with the property that only the rightmost child of any node can be internal. The resulting tree is similar to t(λ) except that the order of the degrees of the internal nodes may differ (the multiset of degrees   is the same). But now we can arbitrarily permute these internal nodes (since they all lie in the rightmost branch) using a series of forest-exchanges and Proposition 2.
Translating back into the language of permutations, we have shown that every separable permutation of length n is equipopular to one in a particular family, and that this family is in bijection with the set of partitions of the integer n − 1. For a partition λ = λ 1 , λ 2 , . . . , λ k , using the definition of the decomposition trees and letting I(n) denote the identity permutation of length n, we have The permutations w(λ) are referred to as the wedge permutations, due to the shape of their plots. See Figure 3.5 for an illustration.

NON-EQUIPOPULAR PATTERNS
In the previous section we proved that if two patterns correspond to the same integer partition, they are equipopular. Here we show that if two patterns correspond to different partitions, they do in fact have different enumerations, showing that the equipopularity classes are in bijection with integer partitions. The methods used here contrast sharply with those used previously: we utilize a variety of analytic techniques to prove this result.
This section is split into two parts. Given the popularity generating function of a wedge permutation, we first show that we can factor it into generating functions for the popularity of the monotone  runs in the pattern (along with some other terms). Next, we show that given the product of several such generating functions, we can identify the lengths of the monotone runs. Together this will show that each popularity generating function is unique to a specific wedge permutation.

Factoring Popularity Generating Functions
For σ ∈ S, let P σ (t) be the generating function for the popularity of the pattern σ. That is, let Recall that we use I(n) to denote the pattern 12 · · · n. Proof. Let P ⊕ I(m)⊕π and P ⊖ I(m)⊕π denote the popularity generating functions of the pattern I(m) ⊕ π within the sum and skew decomposable separable permutations, and let P ⊕,• I(m)⊕π denote the popularity generating function within skew indecomposable permutations. Let S denote the generating function for the separable permutations including the empty permutation, i.e., with constant term one.
We proceed by induction on m, noting that for the base case m = 0 the result is trivial (F 0 = 1).
Suppose that m ≥ 1. Since the length of I(m) ⊕ π is at least two, the permutation of length one does not contain I(m) ⊕ π. Then, since every permutation of S of length at least two is either sum or skew decomposable, we have Suppose that α is an I(m) ⊕ π-marked skew decomposable permutation. Any skew decomposable permutation can be decomposed uniquely into skew indecomposable parts (see Figures 4.1  and 4.2). Since the pattern I(m) ⊕ π is sum decomposable, if a skew decomposable permutation is I(m) ⊕ π marked, then all the marks must occur within a single one of these components (Figure 4.1). Such components are enumerated by P ⊕ I(m)⊕π . The remainder of the permutation is there-  fore unmarked. The portions of the permutation to the left and to the right of the marked component are arbitrary, and are each counted by S. If the permutation itself is skew decomposable, either the left or the right part must be nonempty. This leads to the following relationships The case for marked instances of I(m) ⊕ π in sum decomposable permutations is more complicated. As above, suppose that α is an I(m) ⊕ π-marked sum decomposable permutation. Such a permutation can be decomposed into sum indecomposable parts, and marks can occur in multiple such parts (Figure 4.2). Consider the components from left to right. There may be an arbitrary number of unmarked parts before the first marked one; this segment is counted by S. The first marked part may have i marked entries, where 1 ≤ i ≤ m, or might contain all the marked entries (since π is sum-indecomposable, if any mark of π occurs in a sum indecomposable part, then all the marks must). The remainder of the permutation is then counted by P I(m−i)⊕π , in the first case, or S in the second case. This leads to the following relationship: By our inductive hypothesis, we have that P I(m−i)⊕π = F m−i P π . Combining equations 4.2 and 4.3 yields Rearranging and combining with equation 4.2 gives (4.5) Noting that the first factor in equation 4.5 is independent of π completes the proof.
The functions F m in Lemma 4.1 can be found easily. Considering the pattern I(m) ⊕ 1, the popularity generating function can be computed as P I(m)⊕1 = F m P 1 , or equivalently, In particular, for the wedge permutation w(λ) with λ = λ 1 , λ 2 , . . . , λ k , we can apply Lemma 4.1 iteratively (using the fact that complements are equipopular) to obtain P w(λ) = P I(λ 1 +1) P I(λ 2 +1) · · · P I(λ k +1) It follows that, given the popularity generating function for an unknown pattern, if we can identify the factors of the form F m , we can identify the pattern. We investigate this in the next section. Note also that this means that Proposition 2 is not actually needed to confirm equipopularity of two different decomposition trees having the same internal degrees -but as noted previously it does provide a mechanism for creating explicit bijections between the marked permutations of two different types within an equipopularity class.

Identifying the Partition
This section focuses on the popularity generating functions for the monotone patterns. Our goal is to show that, given an arbitrary product of such functions, the individual factors can always be identified. Define a bivariate generating function P(u, t) = ∑ n≥0 P I(n) (t)u n .
Note that P(u, t) can be thought of as the generating function for marked separable permutations where the marks form an increasing sequence, counted by the number of marks (u) and the length of the permutation (t). As such, we can explicitly compute it using the structure of the separable permutations: Lemma 4.2. The function P(u, t) is given by and r = 1 − 6t + t 2 .
Proof. Let P ⊕ and P ⊖ denote the bivariate generating functions corresponding to P but restricted to the sum and skew decomposable separable permutations respectively. Since the empty permutation cannot be marked, and the permutation of length one can either carry a mark or not, and any longer separable permutation is either sum or skew decomposable, we have: (4.7) A sum decomposable separable permutation can be expressed as the sum of a sum indecomposable separable and any other separable permutation (both of length at least one). Further, a marking of length r in the first part and a marking of length s in the second combine to form a marking of length r + s. Since the generating function for increasing markings within the sum indecomposable separable permutations is P − P ⊕ − 1, we have In a skew decomposable separable permutation, the markings (if any) must lie entirely in one component (since they mark an increasing pattern). Such a permutation must either be entirely free of marks, or consist of a marked skew indecomposable component together with a mark-free separable permutation on either side, at least one of which is nonempty. This leads to the following relationship.
where the first term accounts for the unmarked case, and the first factor in the second term accounts for non-skew decomposable separable permutations containing at least one mark.
Solving the equations 4.7-4.9 yields the explicit generating function.
We can derive this bivariate generating function in a different way, which will yield a connection to the Gegenbauer polynomials, and will ultimately help to prove our main theorem. The Narayana numbers will be useful here. and conventionally N 0,0 = 1.
The generating function for these numbers is given by Now consider a sequence of polynomials q n defined as follows: Note that q 0 (x) = 1/x. These polynomials are described in the OEIS [8] under sequence A174128 and can be expressed in terms of the ordinary hypergeometric function as (4.10) Then P = Q. Equivalently, for all n ≥ 1, Proof. By the definition of the polynomials q n we have that This allows us to express Q in terms of N: After some prodding and coaxing, using the explicit formulas for N and S, a computer algebra system will now verify that this expression matches with the result in Lemma 4.2. Therefore P = Q.
We now turn our attention to the polynomials q n . We show that these polynomials are related to a family of orthogonal polynomials, which will allow us to uniquely identify the factors in an arbitrary product.
They can also be defined by the following recurrence: Lemma 4.6. The polynomials q n satisfy Proof. This follows from known representations of the Gegenbauer polynomials in terms of hypergeometric functions and Equation 4.10, and can be verified by a computer algebra system (Maple, in this case).
Since the Gegenbauer polynomials are orthogonal, it follows that given an arbitrary product of polynomials from the set {q n (x)} n≥0 , we can always recover the factors. Combined with Lemma 4.1, Equation 4.6, and suitable rescalings and changes of variable, this implies that given any popularity generating function of a wedge pattern, we can always recover the lengths of the monotone segments that make up the wedge. Therefore, two wedge patterns have the same popularity generating function if and only if they correspond to the same partition. We summarize in the following theorem.
Theorem 4.7. Two separable patterns are equipopular in the separable permutations if and only if they have the same signature. Therefore, the set of partitions of the integer n − 1 is in bijection with the equipopularity classes for separable patterns of length n in the separable permutations.

SUMMARY AND DISCUSSION
As we have seen, the equipopularity classes among separable permutations of length n are in one-to=one correspondence with partitions of n − 1. This result exactly parallels the corresponding results for the class Av(132) of 132-avoiding permutations obtained in [4,6]. Although Av(132) is a subclass of the separable permutations, neither result immediately implies the other (since the popularities are computed with respect to different universes). However, our techniques can be adapted to Av(132) to obtain new, and arguably more uniform, proofs of those already known results.
There is a wider collection of classes whose permutations correspond to more general decomposition trees -these are the substitution closed classes. Aside from trivial instances, the separable permutations are the smallest of such classes. In the particular case where a substitution closed class has only finitely many simple permutations (see [1] for background material) it seems likely that analogous arguments to those used here could be applied to define invariants which would give upper bounds on the number of equipopularity classes. However, it seems much less likely that the technical and algebraic arguments used here in order to obtain matching lower bounds (i.e., in determining that all classes with differing invariants have differing popularity) would generalize to that broader context.
Concerning those arguments, it is natural to wonder if there is a combinatorial explanation of the appearance of the Narayana numbers and the Gegenbauer polynomials in the popularity generating functions for monotone patterns. Of course the Narayana numbers are close relatives of the Catalan numbers, which in turn belong to the same family as the Schröder numbers so this may be another instance where common sequences arise just because we are looking at simple problems (see [11]).