Counting results for thin Butson matrices

A partial Butson matrix is a matrix $H\in M_{M\times N}(\mathbb Z_q)$ having its rows pairwise orthogonal, where $\mathbb Z_q\subset\mathbb C^\times$ is the group of $q$-th roots of unity. We investigate here the counting problem for these matrices in the"thin"regime, where $M=2,3,\ldots$ is small, and where $N\to\infty$ (subject to the condition $N\in p\mathbb N$ when $q=p^k>2$). The proofs are inspired from the de Launey-Levin and Richmond-Shallit counting results.


Introduction
A partial Hadamard matrix is a matrix H P M MˆN p˘1q having its rows pairwise orthogonal. These matrices are quite interesting objects, appearing in connection with various questions in combinatorics. The motivating examples are the Hadamard matrices H P M N p˘1q, and their MˆN submatrices, with M ď N. See [9].
A given partial Hadamard matrix H P M MˆN p˘1q can complete or not into an Hadamard matrix r H P M N p˘1q. It is known since Hall [3] and Verheiden [11] that this automatically happens when K " N´M is small, and more precisely when K ď 7.
The structure of such matrices is very simple up to M " 4, where, up to assuming that the first row has 1 entries only, and then permuting the columns, the matrix is: Here a, b P N are subject to the condition a`b " N{4. At M ě 5 no such result is available, and the partial Hadamard matrices give rise to interesting combinatorial structures, related to the Hadamard Conjecture. See Ito [4].
In their breakthrough paper [7], following some previous work in [6], de Launey and Levin proposed a whole new point of view on these matrices, in the asymptotic limit N P 4N, N Ñ 8. Their main result is as follows: Theorem (de Launey-Levin [7]). The probability for a random H P M MˆN p˘1q to be partial Hadamard is p2πNq p M 2 q in the N P 4N, N Ñ 8 limit.
The proof in [7] uses a random walk interpretation of the partial Hadamard matrices, then the Fourier inversion formula, and then some real analysis methods. Importantly, as pointed out there, this method can be probably used for more general situations.
An interesting generalization of the Hadamard matrices are the complex Hadamard matrices H P M N pCq having as entries the roots of unity, introduced by Butson in [2]. The basic example here is the Fourier matrix, F N " pw ij q with w " e 2πi{N : In general, the theory of Butson matrices can be regarded as a "non-standard" branch of discrete Fourier analysis. For a number of results on these matrices, see [10].
We can of course talk about partial Buston matrices: Definition. A partial Butson matrix is a matrix H P M MˆN pZ q q having its rows pairwise orthogonal, where Z q Ă Cˆis the group of q-roots of unity.
Observe that at q " 2 we obtain the partial Hadamard matrices. In general, the interest comes from the Butson matrices H P M N pZ q q, and from their MˆN submatrices.
Let us first discuss the case q " 2 k . At M " 2, up to assuming that the first row has 1 entries only, and then permuting the columns, the matrix must be as follows: Here w " e 2πi{q and a 1 , . . . , a q{2 P N are certain multiplicities, summing up to N{2. Thus counting such objects is the same as counting abelian squares, i.e. length N words of type xx 1 where x 1 is a permutation of x. According now to [8], we have: Theorem (cf. ). For q " 2 k the probability for a randomly chosen H P M 2ˆN pZ q q to be partial Butson is There are actually several proofs of this result, but the one in [8] is remarkably beautiful: based only on the Stirling formula, and on an old idea of Lagrange. Indeed: The point now is that the sum on the right can be estimated by making a clever use of the Stirling formula, and this gives the above result. See [8].
Summarizing, there are several techniques for dealing with the counting problem for partial Butson matrices. In this paper we will try to use and mix these techniques. Our first result here will be an extension of the Richmond-Shallit count: Theorem A. When q " p k is a prime power, the probability for a randomly chosen H P M 2ˆN pZ q q, with N P pN, N Ñ 8, to be partial Butson is: In particular, for q " p prime, P 2 » b p p p2πN q p´1 . When q P N is not a prime power the combinatorics is much more complicated, as shown by Lam and Leung in [5]. Particularly problematic is the case where q has 3 prime factors, because the vanishing sums of q-roots of unity won't necessarily decompose as sums of cycles. Here is such a "tricky" vanishing sum, with w " e 2πi{30 : Our second result will concern the case where q has two prime factors. If we call "dephased" the matrices having the first row consisting of 1 entries only, we have: Theorem B. For q " p k 1 1 p k 2 2 with p 1 , p 2 distinct primes, the dephased partial Butson matrices H P M 2ˆN pZ q q are indexed by matrices A P M p k 1 1ˆp k 2 2 pNq of the following form, , and with B ijy , C jxy P N: In particular at q " 2p with p ě 3 prime, P 2 equals the probability for a random walk on Z p to end up on the diagonal, i.e. at a position of type pt, . . . , tq, with t P Z.
As already mentioned, the general case q " p k 1 1 . . . p ks s is certainly more complicated. One way of avoiding the difficulties would be by imposing the "regularity" assumption from [1]. But the matrices A as above will become s-arrays, and we have no results.
Finally, at M " 3, and when q " p is prime, the partial Butson matrices are related to the matrices A P M p pNq which are "tristochastic", in the sense that the sums on the rows, columns and diagonals are all equal. We will prove the following result: Theorem C. At q " p prime, the dephased partial Butson matrices H P M 3ˆN pZ q q are indexed by the tristochastic matrices A P M p pNq, with sum N{p. In particular at p " 3 we have P 3 » 243 p2πN q 3 , in the N P 3N, N Ñ 8 limit. We can see from the above results that the counting problem depends a lot on q, M. We believe that an extension of [7] should require assuming that q " p is prime.
The paper is organized as follows: 1 is a preliminary section, in 2-4 we state and prove our main results, and 5 contains some further results, and a few concluding remarks.

Partial Hadamard matrices
Let H P M MˆN p˘1q be a partial Hadamard matrix (PHM). We will usually dephase H, i.e. assume that the first row consists of 1 entries only, then put it in "standard form", with the`entries moved to the left as much as possible, by proceeding from top to bottom. Here are some examples, at small values of M: Proof. All the results follow by putting the matrix in standard form, and then writing down the orthogonality equations in terms of the block entries in the last row.
Let us try now to count the partial Hadamard matrices H P M MˆN p˘1q. This is an easy task at M " 2, 3, 4, where the answer is: where the quantities on the right are multinomial coefficients.
Proof. Indeed, the multinomial coefficients at right count the matrices having the first row consisting of 1 entries only, and the 2 N factor comes from this.
At M ě 5 no such simple formula is available, and estimating rather than exactly computing looks like a more reasonable objective. First, we have: The probability for a random H P M MˆN p˘1q to be PHM is Proof. Since there are 2 M N sign matrices of size NˆM, the probability P M in the statement is given by: With this formula in hand, the result follows from Proposition 1.2, by using standard estimates for sums of binomial coefficients (see Lemma 2.4 below).
In general, we have the following result, due to de Launey and Levin: ). The probability for a random H P M MˆN p˘1q to be PHM is Proof. The proof in [7] uses a random walk interpretation of the PHM, then the Fourier inversion formula, and finally a number of quite technical real analysis estimates.

Butson matrices, abelian squares
As mentioned in [7], the method there should apply to more general situations. We discuss in what follows a potential extension to the partial Butson matrices: Definition 2.1. A partial Butson matrix (PBM) is a matrix H P M MˆN pZ q q having its rows pairwise orthogonal, where Z q Ă Cˆis the group of q-roots of unity.
Observe that at q " 2 we obtain the PHM. In general, the interest comes from the Butson matrices H P M N pZ q q, and from their MˆN submatrices. See [2], [10].
Two PBM are called "equivalent" if one can pass from one to the other by permuting the rows and columns, or by multiplying the rows and columns by numbers in Z q .
Up to this equivalence, we can assume that H is dephased, in the sense that its first row consists of 1 entries only. We can also put H in "standard form", as follows: We say that H P M MˆN pZ q q is in standard form if the low powers of w " e 2πi{q are moved to the left as much as possible, by proceeding from top to bottom.
Let us first try to understand the case M " 2. Here a dephased partial Butson matrix H P M 2ˆN pZ q q must look as follows, with λ i P Z q satisfying λ 1`. . .`λ N " 0: . . p ks s , we must have, according to Lam and Leung [5], N P p 1 N`. . .`p s N. Observe however that at s ě 2 this obstruction dissapears at N ě p 1 p 2 .
In this section we restrict attention to the prime power case. First, we have: where w " e 2πi{q and where a 1 , . . . , a q{p P N are multiplicities, summing up to N{p.
Proof. Indeed, it is well-known that for q " p k the solutions of λ 1`. . .`λ N " 0 with λ i P Z q are, up to permutations of the terms, exactly those in the statement.
Our next objective will be to count the matrices in Proposition 2.3. We use: Proof. This is proved by Richmond and Shallit in [8] at p " 2, and the proof in the general case, p P N, is similar. More precisely, let us denote by c sp the sum on the left. By setting n and then by using the various formulae in [8], we obtain: Thus we have obtained the formula in the statement, and we are done. Now with Lemma 2.4 in hand, we can now prove: Theorem 2.5. When q " p k is a prime power, the probability for a randomly chosen M P M 2ˆN pZ q q, with N P pN, N Ñ 8, to be partial Butson is: In particular, for q " p prime, Proof. First, the probability P M for a random M P M MˆN pZ q q to be PBM is: Thus, according to Proposition 2.3, we have the following formula: Now by using the Stirling formula for the left term, and Lemma 2.4 with s " q{p and n " N{p for the right term, we obtain: Thus we have obtained the formula in the statement, and we are done.

Two prime factors, random walks
In this section we discuss the case where M " 2 and q " p k 1 1 p k 2 2 has two prime factors. Let us first examine the simplest such case, namely q " p 1 p 2 , with p 1 , p 2 primes: Proposition 3.1. When q " p 1 p 2 is a product of distinct primes, the standard form of the dephased partial Butson matrices at M " 2 is H "¨1 where w " e 2πi{q , and A P M p 1ˆp2 pNq is of the form A ij " B i`Cj , with B i , C j P N.
Proof. We use the fact that for q " p 1 p 2 any vanishing sum of q-roots of unity decomposes as a sum of cycles. See [5]. Now if we denote by B i , C j P N the multiplicities of the various p 2 -cycles and p 1 -cycles, then we must have A ij " B i`Cj , as claimed.
Regarding the matrices of type A ij " B i`Cj , when taking them over integers, B i , C j P Z, these form a vector space of dimension p 1`p2´1 . Given A P M p 1ˆp2 pZq, the "test" for deciding if we have A ij " B i`Cj or not is A ij`Akl " A il`Ajk .
The problem comes of course from the assumption B i , C j ě 0, which is quite a subtle one. In what follows we restrict attention to the case p 1 " 2. Here we have: For q " 2p with p ě 3 prime, P 2 equals the probability for a random walk on Z p to end up on the diagonal, i.e. at a position of type pt, . . . , tq, with t P Z.
Proof. According to Proposition 3.1, we must understand the matrices A P M 2ˆp pNq which decompose as A ij " B i`Cj , with B i , C j ě 0. But this is an easy task, because depending on A 11 vs. A 21 we have 3 types of solutions, as follows: ere a i ě 0 and t ě 1. Now since cases 2,3 contribute in the same way, we obtain: a 1`t , . . . , a p , a p`tẆ e can write this formula in a more compact way, as follows: a 1 , a 1`| t|, . . . , a p , a p`| t|Ṅ ow since the sum on the right, when rescaled by 1 p2pq N , is exactly the probability for a random walk on Z p to end up at pt, . . . , tq, this gives the result.
According to the above result we have with t P Z is the probability for a random walk on Z p to end up at pt, . . . , tq. Observe that, by using Lemma 2.4 above with s, p, n equal respectively to p, 2, N{2, we obtain: Regarding now the probability P ptq 2 of ending up at pt, . . . , tq, in principle for small t this can be estimated by using a modification of the method in [8]. However, it is not clear on how to compute the full diagonal return probability in Theorem 3.2.
Let us discuss now the exponents q " 3p. The same method as in the proof of Theorem 3.2 works, with the "generic" solution for A being as follows: More precisely, this type of solution, with s, t ě 1, must be counted 6 times, then its s " 0, t ě 1 and s ě 1, t " 0 particular cases must be counted 3 times each, and finally the s " t " 0 case must be counted once. Observe that the s " t " 0 contribution is: Finally, regarding arbitrary exponents with two prime factors, we have: Proposition 3.3. When q " p k 1 1 p k 2 2 has exactly two prime factors, the dephased partial Butson matrices at M " 2 are indexed by the solutions of Proof. We follow the method in the proof of Proposition 3.1. First, according to [5], for q " p k 1 1 p k 2 2 any vanishing sum of q-roots of unity decomposes as a sum of cycles. Let us first work out a simple particular case, namely q " 4p. Here the multiplicity matrices A P M 4ˆp pNq appear as follows: Thus, if we use double binary indices for the elements of t1, 2, 3, 4u, the condition is: The same method works for any exponent of type q " p k 1 1 p k 2 2 , the formula being: But this gives the formula in the statement, and we are done.

Three rows: tristochastic matrices
At M " 3 now, we first restrict attention to the case where q " p is prime. In this case, Proposition 2.3 becomes simply: We call a matrix A P M p pNq "tristochastic" if the sums on its rows, columns and diagonals are all equal. Here, and in what follows, we call "diagonals" the main diagonal, and its p´1 translates to the right, obtained by using modulo p indices.
With this notation, here is now the result at M " 3: It is quite unobvious on how to deal with the tristochastic matrices with bare hands. For the moment, let us just record a few elementary results: Proposition 4.2. For p " 2, 3, the standard form of the dephased PBM at M " 3 is respectively as follows, with w " e 2πi{3 and a`b`c " N{3 at p " 3: Proof. The p " 2, 3 assertions follow from Proposition 4.1, and from the fact that the 2ˆ2 and 3ˆ3 tristochastic matrices are respectively as follows: A "ˆa a a a˙, A "¨a b c b c a c a b‚ Indeed, the p " 2 assertion is clear. Regarding now the p " 3 assertion, consider an arbitary 3ˆ3 bistochastic matrix, written as follows: Here˚" a`b`c`d´n, but we won't use this value, because one of the 3 diagonal equations is redundant anyway. With these notations in hand, the conditions are: b`pn´c´dq`pn´a´dq " n pn´a´bq`d`pn´b´cq " n Now since substracting these equations gives b " d, we obtain the result. Regarding now the last assertion, consider the following pˆp permutation matrix: Since this matrix is tristochastic, for any p ě 3 odd, this gives the result.
As already mentioned, it is not clear on how to understand the tristochastic matrices at p ě 5. Such matrices make sense of course at any p ě 2, not necessarily prime, so we can try to first study them at p " 4. But, the situation at p " 4 is not clear either.
Regarding now the asymptotic count, we have here: For p " 2, 3, the probability for a randomly chosen M P M 3ˆN pZ p q, with N P pN, N Ñ 8, to be partial Butson is respectively: In addition, we have P ppq 3 ą 0 for any N P pN, for any p ě 3 prime.
Proof. According to Proposition 4.2, and then to the Stirling formula, we have: Also, by using Proposition 4.2, and then Lemma 2.4 with s " p " 3, n " N{3: a, b, c, b, c, a, c, a, b" Finally, the last assertion is clear from the last assertion in Proposition 4.2.
For exponents of type q " p k with k ě 2, we obtain as well some kind of tristochastic matrices, but with the tristochastic condition taken blockwise. However, finding a good parametrization of these matrices is quite unobvious, even at q " 4.

Further results, conclusion
We have the following question, which emerges from the above results: Question 5.1. Is there any de Launey-Levin type formula for P M , with M P N arbitrary, at least in the case where q " p is prime?
As a first observation, the beginning of the proof in [7] applies to the general situation q P N. Indeed, by following the idea there, we have: Theorem 5.2. The probability P M for a random H P M MˆN pZ q q to be partial Butson equals the probability for a length N random walk with increments drawn from regarded as a subset Z p M 2 q q , to return at the origin.
Proof. Indeed, with T peq " pe iēj q iăj , a matrix X " re 1 , . . . , e N s P M MˆN pZ q q is partial Butson if and only if T pe 1 q`. . .`T pe N q " 0, and this gives the result.
Observe now that, according to the above result, we have: The problem is to continue the computation in the proof of the inversion formula. More precisely, the next step at q " 2, which is the key one, is as follows: Here D "`M 2˘. The problem is that this formula works when Σξ i is real, as is the case in [7], but not when Σξ i is complex, as is the case in Theorem 5.2.
Yet another problem comes from the fact that the exponent p " 2 used in [7] is quite special, because it forces N P p 2 N, instead of just N P pN. Thus, regardless of the abovementioned real vs. complex issue, the combinatorics in [7] is probably not exactly the p " 2 instance of a "generic" combinatorics, because the generic case would probably require the assumption p ‰ 2. We have no answer so far to these questions.