Pruned inside-out polytopes, combinatorial reciprocity theorems and generalized permutahedra

Generalized permutahedra are a class of polytopes with many interesting combinatorial subclasses. We introduce pruned inside-out polytopes, a generalization of inside-out polytopes introduced by Beck--Zaslavsky (2006), which have many applications such as recovering the famous reciprocity result for graph colorings by Stanley. We study the integer point count of pruned inside-out polytopes by applying classical Ehrhart polynomials and Ehrhart-Macdonald reciprocity. This yields a geometric perspective on and a generalization of a combinatorial reciprocity theorem for generalized permutahedra by Aguiar-Ardila (2017), Billera-Jia-Reiner (2009), and Karaboghossian (2022). Applying this reciprocity theorem to hypergraphic polytopes allows to give a geometric proof of a combinatorial reciprocity theorem for hypergraph colorings by Aval-Karaboghossian-Tanasa (2020). This proof relies, aside from the reciprocity for generalized permutahedra, only on elementary geometric and combinatorial properties of hypergraphs and their associated polytopes.


Introduction
Generalized permutahedra are an interesting class of polytopes containing numerous subclasses of polytopes defined via combinatorial structures, such as graphic zonotopes, hypergraphic polytopes (Minkowski sums of simplices), simplicial complex polytopes, matroid polytopes, associahedra, and nestohedra. Generalized permutahedra themselves are closely related to submodular functions, which have applications in optimization.
A combinatorial reciprocity theorem can be described as a result that relates two classes of combinatorial objects via their enumeration problems (see, e.g., [Sta74,BS18]). For example, the number of proper m-colorings of a graph g = (I, E) agrees with a polynomial χ(g)(m) of degree d = |I| for positive integers m ∈ Z >0 , and (−1) d χ(g)(−m) counts the number of pairs of compatible acyclic orientations and m-colorings of the graph g [Sta73]. For precise definitions see Section 3.4 below.
One of our main results is a combinatorial reciprocity theorem for generalized permutahedra counting integral directions with k-dimensional maximal faces: We will use integer point counting in dissected and dilated cubes to prove this result and comment on further generalizations in Remark 3.5.
The special case of this theorem for k = 0, i.e., generic directions, was obtained by Aguiar and Ardila [AA17], and earlier by Billera, Jia, and Reiner [BJR09] in a slightly different language. The k = 0 case was also recently extended in [Kar22]. As shown for some examples in [AA17, Section 18] the application of such a result to the various subclasses of generalized permutahedra yields already known combinatorial reciprocity theorems for their related combinatorial structures such as matroid polynomials [BJR09], Bergmann polynomials of matroids and Stanley's famous reciprocity theorem for graph colorings [Sta73].
Aguiar and Ardila develop a Hopf monoid structure on the species of generalized permutahedra, work with polynomial invariants defined by characters, and apply their antipode formula to get the combinatorial interpretation of the reciprocity result for generalized permutahedra for k = 0 (Theorem 3.7, below) [AA17,Sections 16,17]. This method is also used in [Kar22]. The approach in [BJR09] is similar to the one by Aguiar and Ardila. Billera, Jia, and Reiner use Hopf algebras of matroids and quasisymmetric functions, as well as a multivariate generating function as isomorphism invariants of matroids. The reciprocity providing ingredient is again the antipode of a Hopf algebra together with Stanley's reciprocity for P -partitions [BJR09, Sections 6 and 9].
We give a different, geometric perspective. In order to prove Theorem 3.4 we apply Ehrhart-Macdonald reciprocity to pruned inside-out polytopes. A pruned inside-out polytope Q \ N co 1 consist of the points that lie inside a polytope Q but not in the codimension one cones N co 1 of a complete polyhedral fan N . This is a generalization of inside-out polytopes introduced by Beck and Zaslavsky [BZ06b]. An inside-out polytope Q\H consists of the points in a polytope Q but off the hyperplanes in the arrangement H. We think of the codimensionone cones N co 1 defining a pruned inside-out polytope as pruned hyperplanes, hence the name.
One of the many applications of inside-out polytopes [BZ06c,BZ06a,BZ10,BS18] is yet a different proof of Stanley's reciprocity result for graph colorings [Sta73].
Aval, Karaboghossian, and Tanasa presented a reciprocity theorem for hypergraph colorings [AKT20], generalizing Stanley's result for graph colorings. A main tool in the paper is a Hopf monoid structure on hypergraphs defined in [AA17, Section 20.1.] and the associated basic polynomial invariant. However, they do not use the antipode as reciprocity inducing element, but rather technical computations involving Bernoulli numbers.
In Section 3.4 we show how the reciprocity theorem for hypergraph colorings in [AKT20] is a consequence of the reciprocity for generalized permutahedra. Our main tool is a vertex description of hypergraphic polytopes in terms of acyclic orientations of hypergraphs (Proposition 3.9). More recent work by Karaboghossian [Kar20,Kar22] presents a more general version of the combinatorial reciprocity result for hypergraphs and an alternative proof with similar techniques as we present in Section 3.4.
As spelled out in [AA17, and [AKT20, Section 4] hypergraphs and hypergraphic polytopes contain a number of interesting combinatorial subclasses such as simple hypergraphs, graphs, simplicial complexes, building sets, set partitions, and paths, together with their associated polytopes such as graphical zonotopes, simplicial complex polytopes, nestohedra, and graph associahedra.
The paper is organized as follows: In Section 2 we introduce the notion of pruned insideout polytopes, define two counting functions on pruned inside-out polytopes, and derive (quasi-)polynomiality and reciprocity results. Section 3 provides two applications of the results in Section 2; first, to generalized permutahedra, giving a new geometric perspective on reciprocity theorems in [BJR09,AA17,Kar22] and, moreover, presenting generalized versions for arbitrary face dimensions (Section 3.2). The relationship between our approach and the polynomial invariants for Hopf monoids is analyzed in Section 3.3. Secondly, we apply the reciprocity theorem for generalized permutahedra to the subclass of hypergraphic polytopes giving an elementary combinatorial and geometric proof of the reciprocity theorem for hypergraph colorings in [AKT20] (Section 3.4). In Appendix A we provide more details about generalized permutahedra. In particular we give a self-contained proof of the well known bijection between generalized permutahedra and submodular set functions. We do not claim the proof to be either new or original, but it is hard to find in the literature.

Pruned inside-out polytopes and Ehrhart theory
In [BZ06b] Beck and Zaslavsky develop the notion of an inside-out polytope, that is, a polytope dissected by hyperplanes. Counting integer point in a polytope but off certain hyperplanes turns out to be a useful tool to derive (quasi-)polynomiality results and reciprocity laws for various applications such as graph colorings and signed graph colorings, composition of integers, nowhere-zero flows on graphs and signed graphs, antimagic labellings, as well as magic, semimagic, and magic latin squares [BZ06c,BZ06a,BZ10]. After reviewing the necessary notions from polytopes and Ehrhart theory (Section 2.1), we introduce a generalization of inside-out polytopes, which we call pruned inside-out polytopes and develop Ehrhart-theoretic results (Section 2.2).
2.1. Preliminaries: Polytopes and Ehrhart theory. First recall some basic notions from polytopes; for more detailed information consult, e.g., [Zie98,Gru03]. A polyhedron P ⊂ R d is the intersection of finitely many halfspaces. If the intersection is bounded it is called a polytope and can equivalently be described as the convex hull of finitely many points in R d . A (polyhedral) cone N is a polyhedron such that for x ∈ N the point λx is again contained in N for every λ ∈ R ≥0 . A supporting hyperplane H of a polyhedron P is a hyperplane such that the polyhedron is contained in one of the closed halfspaces. The intersection of a polyhedron P with a supporting hyperplane H is a face F = H ∩ P of P. The dimension dim(P) (resp. dim(F )) of a polyhedron P (resp. face F ) is the dimension of the affine hull of the polytope P (resp. face F ), 0-dimensional faces are called vertices and (dim(P) − 1)dimensional faces are called facets. The codimension codim(F ) of an polyhedron F is the difference between the dimension of the ambient space and the dimension of the polyhedron dim(F ). A polyhedron P is a rational polyhedron, if all its facet defining hyperplanes H can be described as H = x ∈ R d : a · x = b for some a ∈ Z d and b ∈ Z.
For a polytope Q ⊂ R d and a positive integer t ∈ Z >0 we define the t th dilate of Q as The Ehrhart counting function ehr Q (t) counts the number of integer point in the t th dilate of the polytope Q: See Figure 1 for an example. Recall that a rational (resp. integral) polytope has vertices with rational (resp. integral) coordinates. We call the least common multiple of the denominators of all coordinates of all vertices of a rational polytope the denomina- where c 0 , c 1 , . . . , c d : Z → R are periodic functions. The least common period of c 0 (n), c 1 (n), . . . , c d (n) is the period of q(t).
Theorem 2.1 (Ehrhart's theorem [Ehr62]   For an integer polytope Q Ehrhart's theorem implies that the Ehrhart counting function ehr Q is a polynomial. Therefore it is often called the Ehrhart polynomial. The following reciprocity theorem was conjectured and proved for various special cases by Eugéne Ehrhart and proved by Ian G. Macdonald. It is the foundation for the results in this paper. Theorem 2.2 (Ehrhart-Macdonald reciprocity [Mac71]). Let Q ⊂ R d be a rational polytope and t ∈ Z >0 . Then where Q • is the (relative) interior of the polytope Q.

2.2.
Pruned inside-out polytopes and their counting functions. Let N be a complete fan in R d , that is, a family of polyhedral cones such that (i) every non-empty face of a cone N ∈ N is also contained in N , (ii) the intersection of two cones in N is a face of both cones, (iii) the union of the cones in the fan N covers the ambient space R d , i.e., For an introduction to complete fans consult, e.g., [Zie98, Section 7.1]. A fan is called rational if its cones N ∈ N are generated by rational vectors. For a complete fan N in R d we define the codimension-one fan 1 N co 1 in R d to contain the cones in N with codimension ≥ 1, that is, all but the full-dimensional cones in N : We think of the codimension-one fan as a pruned hyperplane arrangement, since cones of codimension one can be seen as parts of hyperplanes. For a polytope Q ⊂ R d and a complete fan N in R d we call a pruned inside-out polytope and we call the connected components in Q \ N co 1 regions. So, a pruned inside-out polytope Q \ N co 1 is the disjoint union of its regions Q ∩ N • , where N • is an open full-dimensional cone in N . We will mostly consider open pruned inside-out polytopes Q • \ N co 1 , which decompose into disjoint open polytopes, the regions. A pruned inside-out polytope is rational if the topological closures of all its regions are rational polytopes.
Then the codimension-one fan M co 1 = {n 1 , n 2 , n 3 } consists of the three rays See Figure 2(b). The pruned inside-out polytope Figure 2(c). Their topological closures can be described as The open pruned inside-out polytope For a positive integer t ∈ Z >0 we define the inner pruned Ehrhart function as where t · Q \ N co 1 := t · Q \ t · N co 1 := ty ∈ R d : y ∈ Q \ ty ∈ R d : y ∈ N, for some N ∈ N co 1 .

See Figures 3(a) and 3(b) for illustrations.
Lemma 2.4. For a polytope Q ⊂ R d and a complete fan N in R d , where R   Proof. We decompose the pruned inside-out polytope Q\ N co 1 into its regions R 1 , . . . , R k . Then the open pruned inside-out polytope Q The result follows since counting lattice points is a valuation (see, e.g., [BS18, Section 3.4]).
Furthermore, we define a second counting function for pruned inside-out polytopes, the cumulative pruned Ehrhart function cu Q,N co 1 (Z d ), for a positive integer t ∈ Z >0 as cu Q,N co 1 (t) := Lemma 2.5. For a polytope Q ⊂ R d and a complete fan N in R d , where R i are the topological closures of the regions R i of the pruned inside-out polytope Q \ N co 1 .
Proof. The right hand side of the equation counts lattice points in the interior of the regions tR i precisely once and lattice points in the boundaries of the regions once for every closed region the lattice point is contained in. The closed regions are the intersections of the polytope Q with the closed full-dimensional cones in N . Hence every lattice y point in tQ is counted with multiplicity mult (t·Q,t·N co 1 ) (y).
Example 2.6. We compute the counting functions for the pruned inside-out polytopes introduced in Example 2.3: Figure 3 for illustrations.
Theorem 2.7. Let Q\ N co 1 ⊂ R d be a rational pruned inside-out polytope. Then the inner pruned Ehrhart function in Q • ,N co 1 (t) and the cumulative pruned Ehrhart function cu Q,N co 1 (t) agree with quasipolynomials in t of degree d for t ∈ Z >0 and are related by reciprocity: Proof. We first use Lemma 2.4 to get For every i = 1, . . . , k we can apply Ehrhart's Theorem 2.1 to ehr R • i (t), hence the counting function in Q • ,N co 1 (t) is a sum of quasipolynomials, which is again a quasipolynomial.
For the second part of the claim we use Ehrhart-Macdonald reciprocity (Theorem 2.2) and compute where the last equality follows from Lemma 2.5.
Remark 2.8. In the case that the polytope Q and the complete fan intersect such that all the closed regions R = Q ∩ N of the pruned inside-out polytope Q \ N co 1 are integer polytopes, the counting functions in Q • ,N co 1 (t) and cu Q,N co 1 (t) agree with a polynomial of degree d, by Theorem 2.1 and Theorem 2.2. We will use this fact in the proof of Theorem 3.2.
Remark 2.9. One can certainly generalize this setting, e.g., to polyhedral complexes. The framework here is motivated by the applications below.

Applications
After a short introduction to generalized permutahedra (Section 3.1) we show how the tools from Section 2 can be applied to derive known and unknown reciprocity results for generalized permutahedra (Section 3.2). Reciprocity theorems for generalized permutahedra by Ardila and Aguiar ([AA17, Propositions 17.3 and 17.4], see Theorem 3.7) and extended by Karaboghossian ([Kar22, Theorem 2.5 and Theorem 2.8], see Theorem 3.8), were developed by introducing a Hopf monoid structure on the vector species of generalized permutahedra and using their antipode formula to derive polynomial invariants. We give a new interpretation from a discrete-geometric perspective as integer point counting functions. In Section 3.3 we give an explanation on the relation between the results in this paper and prior results developed with Hopf-algebraic tools. Finally we demonstrate why generalized permutahedra are such an interesting class of polytopes by translating the reciprocity result for hypergraphic polytopes to combinatorial statements about hypergraphs (Section 3.4). 3.1. Preliminaries: Generalized permutahedra. We define the standard permutahedron π d as the convex hull of the d! permutations of the point (1, 2, . . . , d), that is, the standard permutahedron π d is defined by 2 x i y(i).
We will also exploit that primal and dual vector spaces are isomorphic.
For a direction y ∈ R d * we define the y-maximal face P y of a polytope P by For a face F of a polytope P define the open and closed normal cone N • P (F ) and N P (F ) to be the set of all direction that (strictly) maximize F in P, that is, See Figure 5 for an example. Note that normal fans of polytopes form complete fans as defined in Section 2.1. The following is straightforward.
Recall that the codimension one fan N co 1 contains all cones of the complete fan N with codimension one. This implies in particular that the codimension-one fan of the normal fan N co 1 (P) of a polytope P defined in (2.1) can be described as The normal fan of the standard permutahedron has a nice description via the braid arrangement B d , the hyperplane arrangement consisting of the finite set of hyperplanes . We therefore denote them by B T 1 ,...,T k . For more details about concepts on hyperplane arrangements see, for example, [Sta07]. The faces of the braid arrangement B d form the braid fan and the normal fan N (π d ) of the standard permutahedron π d is precisely the braid fan (see, for example, [AA17, Section 4]).
We say a fan N is a coarsening of another fan N if every cone in N is the union of some cones in N . A polytope P ⊂ R d is a generalized permutahedron if its normal fan N (P) is a coarsening of the normal fan N (π d ) of the standard permutahedron π d , that is, it is a coarsening of the fan induced by the braid arrangement B d . There are several Figure 6. A generalization P of the standard permutahedron π 3 : here the up-right edge was moved outwards until it degenerated to a vertex. The normal cone of that "new" vertex is the union of the normal cones of the "old" degenerated edge and its adjacent vertices.
3.2. Combinatorial reciprocity theorems for generalized permutahedra. We restate the combinatorial reciprocity result for generalized permutahedra by [AA17, Propositions 17.3 and 17.4] in a slightly different language (see Theorem 3.7 for the original statement) and prove it using Ehrhart theory.
Theorem 3.2. Let P ⊂ R d be a generalized permutahedron and m ∈ Z >0 . Then agrees with a polynomial in m of degree d. Moreover, While we will extend Theorem 3.2 (and our proof) in Theorem 3.4 and Remark 3.5 below, we provide a self-contained proof here to present a flavor of our method. In contrast to [BJR09, AA17, Kar22] we will prove these results without using any Hopf-algebraic method. Our proof gives a geometric point of view by counting integer points in pruned inside-out cubes. That is, we will consider the cube and intersect it with the integer lattice:   Proof of Theorem 3.2. We will argue in the dual space R d * and its integer lattice; to simplify notation we will not always explicitly point that out. Let us recall that y ∈ R d * being P-generic means that the y-maximal face of P is a vertex, that is, y is contained in a fulldimensional cone of the normal fan N (P). So the direction y is not contained in any cone N in the codimension-one fan N (P) co 1 . Hence, where we use in the last line that Lemma 3.3 we know that the unit cube and the normal fan N (P) intersect producing integer regions. Therefore, using Theorem 2.7 and Remark 2.8, polynomiality of χ d (P)(m) follows.
With the above equality and Theorem 2.7 at hand, we compute where we make use of Lemma 3.1.
We can extend Theorem 3.2 above to faces of arbitrary dimension. Before we prove the theorem we extend the notion of codimension-one fans to arbitrary dimensions by defining the codimension-k fan N co k as N co k :={N ∈ N (P) : codim(N ) ≥ k}, that is, for a polytope P, For a polytope Q ⊂ R d and k ≥ 0 we define the k-pruned inside-out polytope as Note this is consistent with the notation in the beginning of this section. As before, for a Proof of Theorem 3.4. We compute The intersection N • ∩ (0, m + 1) d is the relative interior of a polytope. Moreover, since N • is an open cone containing the origin, Using again Lemma 3.3 and Ehrhart's Theorem 2.1 we obtain polynomiality for χ d,k (P)(m).
With Ehrhart-Macdonald reciprocity (Theorem 2.2) we compute Here we use, as in the proof of Theorem 3.2, that the normal fan of a generalized permutahedron is a coarsened braid fan and therefore is invariant under scaling and shifts by λ(1, . . . , 1) for λ ∈ R. So, applying Lemma 3.1 in the last equality.
for coefficients α N . This provides a combinatorial reciprocity result

Theorem 3.4 (and therefore also Theorem 3.2) is a reformulation of this general result with coefficients
Remark 3.6. We observe that we used the following properties of generalized permutahedra in the proofs of Theorem 3.2 and Theorem 3.4 (i) the intersection of the unit cube and the normal fan of a generalized permutahedron form integer pruned inside-out polytopes, (ii) every cone N in the normal fan N (P) of a generalized permutahedron P contains the line L = {λ(1, . . . , 1) : λ ∈ R}.
The first property (i) can be weakened to rational intersections leading to a quasipolynomiality result. Considering normal fans without property (ii) produces similar but more complicated statements, since the shift of the cube [0, m − 1] d to the cube [1, m] d can not be performed in general. Nevertheless, the framework of pruned inside-out polytopes can be applied to generate reciprocity results for generalized permutahedra in other types (see, e.g., [ACEP20]). This will be explored in a future paper.
3.3. Relation to polynomial invariants from Hopf monoids. In this section we compare our results to the polynomial invariants from Hopf monoids developed in [AA17,Kar22]. This paper was motivated by giving a geometric interpretation of the combinatorial reciprocity theorems in [AA17].
In the Hopf-algebraic setting it is convenient to work with vector spaces with unordered base. We briefly introduce the notation, which we also use in Section 3.4. For a non-empty finite set I let RI be the real vector space with distinguished, unordered basis I. The elements i ∈ I with are denoted b i when we want to distinguish the elements i in the set I from the corresponding basis vector b i in the vector space RI. Moreover, we identify an element i∈I x i b i in the vector space RI with the tuple (x i ) i∈I for x i ∈ R. For the disjoint union I = S T of two finite sets S, T the equality RS × RT = RI = RT × RS holds, which is handy in combinatorial contexts. Similarly, the dual vector space (RI) * can be interpreted as (RI) * = R I := {maps y : I → R} .
Recall that the elements y ∈ R I are called directions. They act as linear functionals on elements In the context of this paper those two notations can be used interchangeably.
An introduction to the theory of Hopf monoids can be found in, e.g., [AA17], [AM10] and is omitted here. For a Hopf monoid on the ground set I, a character ζ, and an element x in the Hopf monoid, there is a polynomial invariant where the sum is over all compositions and ∆ denotes the coproduct of the Hopf monoid.
Using the antipode s I of the Hopf monoid one obtains the reciprocity relation # (vertices of P y ) .
This result was obtained earlier but stated differently by Billera, Jia, and Reiner using a similar Hopf-algebraic approach (using the antipode) on quasisymmetric functions and matroids [BJR09, Theorem 9.2. (v)]. We have seen in Section 3.2 how this result can be understood using pruned inside-out cubes. Recently, Theorem 3.7 was generalized in [Kar22]. With a view towards applications our Ehrhart-theoretic approach has some advantages. One strength is that the weights in (3.3) can be chosen arbitrarily, while in the Hopf-theoretic setting the character needs to fulfill certain axioms. This, for example, does not allow to interpret the combinatorial reciprocity result in Theorem 3.4 as an instance of Theorem 3.8. A character taking value one on k-dimensional faces and zero elsewhere would not fulfill compatibility with multiplication in the Hopf monoid of generalized permutahedra. Another advantage will be the extension to generalized permutahedra in other types, as mentioned before (Remark 3.6). This seems to be very hard from the Hopf monoid setting (see, e.g., [AA17, Theorem 6.1], [ACEP20, Section 9]).

3.4.
Hypergraphs and their polytopes. Generalized permutahedra are an especially interesting class of polytopes, due to their many interesting combinatorial subclasses such as graphical zonotopes, matroid polytopes, hypergraphic polytopes, and many more. In this section we illustrate this fruitful connection between combinatorics and geometry proving a combinatorial reciprocity result for hypergraphs, which generalizes Stanley's famous theorem about the chromatic polynomial for graphs. Aval, Karaboghossian, and Tanasa use a Hopftheoretic ansatz similar to that of Ardila and Aguiar to derive the reciprocity theorem for hypergraph colorings [AKT20]. They define a basic polynomial invariant on hypergraphs and give combinatorial interpretations. A general version of this can be found in [Kar22]. For convenience we demonstrate the technique for a special case of orientation, that we call heading. We give another perspective and proof by applying Theorem 3.7 (reciprocity for generalized permutahedra) and exploiting geometric and combinatorial properties of the hypergraph and its associated polytope. This approach is also described as alternative proof for the general case in [Kar22] 3 .
A hypergraph h = (I, E) is a pair of a finite set I of nodes 4 and a finite multiset E of non-empty subsets e ⊆ I called hyperedges. Note that we allow multiple edges and edges consisting of only one node. For simplicity we will often assume without loss of generality that the node set I equals {1, . . . , d} = [d] for d = |I|, since all the claims in this section are invariant under relabeling the set I. In a similar fashion we might switch back and forth between the two vector space notations RI R d and R I R d * (see Section 3.3).
For every hypergraph h we define the corresponding hypergraphic polytope P(h) ⊂ RI as the following Minkowski sum of simplices: There also seems to be a polytopal approach by Alexander Postnikov, mentioned in [AKT20, Acknowledgments] and on http://math.mit.edu/~apost/courses/18.218_2016/ (Lecture 19. W 03/16/2016), but to the best of our knowledge no reference is available. 4 We decided to use the less common term nodes for hypergraphs to distinguish them from the vertices of a polytope. where ∆ e = conv{b i : i ∈ e}, for a hyperedge e ⊆ I and b i are the basis vectors for i ∈ I. An example is depicted in Figure 8. Hypergraphic polytopes have been studied (sometimes as Minkowski sum of simplices) in, e.g., [Agn17,BBM19]. Hypergraphs are in bijection with hypergraphic polytopes and they form a subclass of generalized permutahedra (see Appendix A or, e.g., [Pos09, Proposition 6.3.]). The vertices of graphic polytopes are described by the acyclic orientations of the corresponding graph [Zas91, Corollary 4.2]. We will give an analogous statement and proof for hypergraphic polytopes. In order to do so we need the subsequent definitions following 5 [AKT20]. A heading 6 σ of a hypergraph h = (I, E) is a map σ : E → I such that for every hyperedge e ∈ E we have σ(e) ∈ e. In other words the heading σ picks for every hyperedge e a node i = σ(e) ∈ e within that hyperedge. We will call that node σ(e) the head of the hyperedge e. An oriented cycle in a heading σ of a hypergraph h is a sequence e 1 , . . . , e of hyperedges such that σ(e 1 ) ∈ e 2 \ σ(e 2 ) σ(e 2 ) ∈ e 3 \ σ(e 3 ) . . .
A heading σ of a hypergraph h is called acyclic if it does not contain any oriented cycle. See Figure 9 for some examples. Note that the notions of heading and acyclic here are special cases of the notions in [BBM19, Kar22, RR12, Rus13].
The following description of the vertices of the hypergraphic polytope in terms of acyclic orientations plays a central role in the remainder of this paper and is a particular instance of, e.g., [BBM19, Theorem 2.18.]. Proposition 3.9 was stated without proof in [CF18]. For convenience we give an elementary proof generalizing the proof idea for graphs presented in [CF18].
Proposition 3.9. For a hypergraph h = (I, E) the hypergraphic polytope P(h) can be described as i.e., δ(σ) ∈ RI is the vector of in-degrees of the nodes i ∈ I in the heading σ.
Proof. Since the Minkowski sum of convex hulls of point sets is the same as the convex hull of the Minkowski sum of the points sets, we have Every point in the convex hull on the right-hand side is the vector of in-degrees of the nodes for some heading σ. Indeed, choosing some b i in every summand corresponds to choosing i ∈ e as the head for the hyperedge e, and vice versa. It is left to show that δ(σ) is a vertex of P(h) if and only if the heading σ is acyclic. First, consider a heading containing an oriented cycle e 1 , . . . , e . Then σ(e 1 ) ∈ e 2 \ σ(e 2 ), . . . , σ(e ) ∈ e 1 \ σ(e 1 ) holds. We will construct new headings σ * 1 , . . . , σ * such that their vectors of in-degrees δ(σ * 1 ), . . . , δ(σ * ) convex combine the vector of in-degrees δ(σ) of the original heading σ. We define the new headings σ * j by changing the orientation of the hyperedge e j in the cycle, as depicted in Figure 10 Then Therefore, the vector of in-degrees δ(σ) of a heading σ containing a cycle cannot be a vertex. Now, let σ be an acyclic heading and let us assume there are headings σ * 1 , . . . , σ * l and scalars 0 ≤ λ 1 , . . . , λ l ∈ R such that δ(σ) = j=1 λ j δ(σ * j ) and j=1 λ i = 1. First note that hyperedges e with cardinality |e| = 1 have only one possible heading (the one choosing the only node in the hyperedge as head) and those edges do not appear in oriented cycles. Hence they are irrelevant when it comes to deciding whether an heading is acyclic or not. Therefore we delete all singleton hyperedges and adjust the values in δ(σ) as well as in δ(σ * 1 ), . . . , δ(σ * ).
Since the heading σ is acyclic and we deleted all singleton hyperedges, there exists at least one source s ∈ I with δ(σ) s = 0. From Equation (3.4) it follows that δ(σ * j ) s = 0 for all j = 1, . . . , . So, for the node s the in-degree of all the headings is identical. We proceed by first deleting the source s in all hyperedges, then deleting all hyperedges e with cardinality |e| = 1, and adjusting the entries in δ(σ), δ(σ * 1 ), . . . , δ(σ * l ). After finitely many iterations (the node set I is finite) we get δ(σ) i = δ(σ * j ) i for every node i ∈ I and all j = 1, . . . , and the in-degree vector δ(σ) of the acyclic heading σ cannot be written as a convex combination, that is, δ(σ) is a vertex.
A coloring of a hypergraph h = (I, E) with m colors is a map c : I → [m] that assigns a color c(i) ∈ [m] to every node i ∈ I. A node i ∈ e ∈ E is called a maximal node in the hyperedge e for the coloring c if the color c(i) is maximal among the colors in the hyperedge e, that is c(i) = max j∈e c(j). The color max j∈e c(j) is called the maximal color.   Remark 3.10. Considering usual graphs, the above definitions of (proper) colorings, (acyclic) headings and compatible pairs for hypergraphs specialize to those commonly used for graphs. In the same way the following Theorem 3.11 and Theorem 3.12 generalize Stanley's reciprocity theorem for chromatic polynomials of graphs [Sta73].
which is a polynomial in m of degree d. In particular, the number of acyclic headings of h equals (−1) d χ d (h)(−1).
Note that the colorings do not need to be proper here.
Proof. We follow the same idea as in the previous proof, that is, we use and need to show From Equation (3.7) we can see that a vertex of P(h) y corresponds to choosing for every hyperedge e ∈ E one of the nodes i ∈ e with maximal entry y(i), i.e., maximal color c y (i). This is, by definition, the same as constructing a compatible heading for the coloring c y . We know by Proposition 3.9 that vertices correspond to acyclic headings. Hence, vertices of P(h) y correspond to acyclic headings compatible to the coloring c y . Vice versa, for a coloring c the compatible acyclic headings are those with heads of hyperedges having a maximal coloring. That is, these acyclic headings correspond to those vertices, that are vertices of the maximum face P(h) y c in direction y c .
We compile some useful and nice information about permutahedra, generalized permutahedra and hypergraphic polytopes in this Appendix. This section is not original work, but the proof of Theorem A.1 below might be hard to find in the literature. As explained in Section 3.3 we will use the notations for R d and R I interchangeably.
Recall that we defined the standard permutahedron π d to be the convex hull of all the permutations of the point with entries [d]. The facet description of the standard permutahedron is given by Moreover, every face of the standard permutahedron can be described combinatorially by compositions, for details see, e.g., [AA17, Section 4.1.]. The standard permutahedron can equivalently be described as the Minkowski sum of line segments: where ∆ {i,j} := conv{e i , e j } and e i are standard basis vectors. This implies, in particular, that standard permutahedra are zonotopes.
Recall that generalized permutahedra are those polytopes that have a coarsening of the braid fan as normal fan. Since the normal fan N (P + Q) of the Minkowski sum P + Q of two polytopes P and Q is the common refinement of the two normal fans N (P) and N (Q) [Zie98, Proposition 7.12], generalized permutahedra are the (weak) Minkowski summands of standard permutahedra. That is, P ⊂ R d is a generalized permutahedron if and only if there exists a polytope Q ⊂ R d and a real scalar λ > 0 such that P + Q = λπ [d] .
One picturesque way of defining generalized permutahedra is by deforming standard permutahedra by parallel shifts of facets. This deformation maintains the normal fan until a face degenerates, i.e., at least two vertices are merged into one vertex. In that case the corresponding normal cones of the vertices are glued together. One example can be seen in Figure 6, where the top right edge degenerated and the two corresponding neighboring full-dimensional cones were combined. A formal description of these deformations and a detailed proof of equivalence can be found in [PRW08,Appendix].
Finally, generalized permutahedra can be uniquely described as the base polytopes of sub- We define the base polytope P(z) of a submodular function z : 2 I → R by P(z) := x ∈ RI : i∈I x i = z(I) and i∈A x i ≤ z(A) for all A ⊆ I .
To simplify the proof of Theorem A.1 we will use the following notation x(A) := i∈A x i for A ⊆ I .
With that notation at hand we can write the definition of the base polytopes as P(z) = x ∈ RI : x(I) = z(I) and x(A) ≤ z(A) for all A ⊆ I .
Theorem A.1. A polytope P is a generalized permutahedron if and only if it is the base polytope P(z) of a submodular function z : 2 I → R with z(∅) = 0.
Before we start proving this theorem, we give a description of the faces in terms of set compositions. A composition of a finite set I is an ordered sequence (T 1 , . . . , T k ) of disjoint non-empty subsets T i ⊆ I such that I = T 1 · · · T k . Let 1 T for some T ⊆ I be the 0/1-vector with entries equal to one for indices in the subset T and zero otherwise.
Lemma A.2. The faces of the braid arrangement B I , also called braid cones, can be described uniquely by compositions I = T 1 · · · T k : B T 1 ,...,T k := y ∈ R I : y(i) = y(j) for all i, j ∈ T a , y(i) ≥ y(j) for i ∈ T a , j ∈ T b and a < b where 1 T for some subset T ⊆ I is the 0/1-vector with entries equal to one for indices in the subset T and zero otherwise.
Proof of Theorem A.1. For a submodular function z : 2 I → R we show that P(z) is a generalized permutahedron by showing that every braid cone B T 1 ,...,T k ⊂ R I is contained in a normal cone of P(z). Since P(z) is contained in the hyperplane {x ∈ RI : x(I) = z(I)} the normal cone N P(z) (P(z)) contains the line spanned by 1 I ∈ RI, hence every normal cone of P(z) contains that line. The following part of the proof relies on [FT83]. Fujishige and Tomizawa show under which conditions a greedy-like algorithm gives an optimal solution in the base polytope of a submodular functions on a general distributive lattice. We adapt the proof to our special case.
Then it follows that the braid cone B T 1 ,...,T k is contained in the normal cone N P(z) (F ), where F is a face containingx. For j = 1, . . . , n we computẽ (z(C l ) − z(C l−1 )) = z(C j ), in particular,x(I) = z(I). We show by induction on the cardinality |A| of a subset A ⊆ I thatx(A) ≤ z(A). For the empty set we have 0 =x(∅) = z(∅). For an arbitrary set A ⊆ I let j * be the minimal index such that A ⊆ C j * and define the element i * := A \ C j * −1 ∈ I. We compute using the induction hypothesis, Equation (A.1), and submodularity of z together with A \ {i * } = A ∩ C j * −1 and C j * = A ∪ C j * −1 : Hence,x ∈ P(z). Now, choose an arbitrary direction y ∈ B T 1 ,...,T k . By Lemma A.2 y(i) = y(i ) for i, i ∈ T l so we can setŷ l := y(i) for i ∈ T l and l = 1, . . . , k. Moreover,ŷ l ≥ŷ l+1 . For a point x ∈ P(z) compute: ŷ l x(T 1 · · · T l ) − x(T 1 · · · T l ) −ŷ l x(T 1 · · · T l−1 ) − x(T 1 · · · T l−1 ) = k−1 l=1 (ŷ l −ŷ l+1 ) x(T 1 · · · T l ) − x(T 1 · · · T l ) +ŷ k x(I) − x(I) where we use in the last equality, that the sets T 1 , T 1 T 2 , . . . , T 1 · · · T k are contained in the chain C : ∅ = C 0 ⊂ · · · ⊂ C n = I and that we already knowx(C j ) = z(C j ) for j = 1, . . . , n. Since the computation in A.2 is independent from the actual values of the direction y ∈ B T 1 ,...,T k , the inequality y(x) ≥ y(x) holds for every direction y ∈ B T 1 ,...,T k . So the braid cone B T 1 ,...,T k is contained in the normal cone N P(z) (F ), where F is a face containing x. Hence, P(z) is a generalized permutahedron.
For the opposite implication let P be a generalized permutahedron. We will define a submodular function z P and show that P = P(z P ). Since the generalized permutahedron P is contained in the hyperplane with constant coordinate sum, the following set function is well x i for A ⊆ I .
We can immediately deduce that z(∅) = 0 and P ⊆ P(z P ). First we show that z P is submodular. For arbitrary A, B ⊆ I find a chain C : ∅ = C 0 ⊂ C 1 ⊂ · · · ⊂ C k = I in the Boolean lattice 2 I that contains A ∩ B and A ∪ B. We set T i := C i \ C i−1 for i = 1, . . . , k and consider the braid cone B T 1 ,...,T k = cone{1 T 1 , . . . , 1 T 1 ··· T k−1 } + span{1 I }.
Then there exists a face F of P such that the normal cone N P (F ) contains the braid cone B T 1 ,...,T k and in particular every point x ∈ F is maximal in the direction 1 A∩B , 1 A∪B ∈ B T 1 ,...,T k . Then, and z P is submodular.
Now it is left to show that P ⊇ P(z P ). The main idea for this part of the proof can be found in [DF10]. For the sake of a contradiction, let us assume there is a point u ∈ P(z P ) \ P.
Then there exists a separating hyperplane H t,c := {x ∈ RI : t(x) = c} such that t(u) = i∈I t i u i > c and t(p) = i∈I t i p i ≤ c for all p ∈ P Now choose a braid cone B T 1 ,...,T k such that t ∈ B T 1 ,...,T k and set againt l := t i for i ∈ T l , l = 1, . . . , k. For points q in the t-maximal face F := P t we know by the definition of z that q(T 1 ∪ · · · ∪ T l ) = z(T 1 ∪ · · · ∪ T l ) for l = 1, . . . , k. Using telescoping sums we compute t(u) = i∈I t i u i > c ≥ t · q = i∈I t i q i = k l=1t j q(T l ) =t k q(T 1 ∪ · · · ∪ T k ) + 1 l=k−1 (t l −t l+1 )q(T 1 ∪ · · · ∪ T l ) =t k z(T 1 ∪ · · · ∪ T k ) + 1 l=k−1 (t l −t l+1 )z(T 1 ∪ · · · ∪ T l ) ≥t k u(T 1 ∪ · · · ∪ T k ) + This is a contradiction and completes the proof.
Recall that the hypergraphic polytope P(h) ⊂ RI of a hypergraph h = (I, E) is defined as where ∆ e = conv{b i : i ∈ e}, for a hyperedge e ⊆ I and b i are the basis vectors for i ∈ I. This results in a differing formulation of Proposition A.3 which is nevertheless equivalent.
For an interesting characterization when a submodular function gives rise to a hypergraphic polytope see [AA17,Proposition 19.4.].